Audio processing method, electronic equipment, storage medium and chip

Abstract

The invention provides an audio processing method, electronic equipment, a storage medium and a chip. According to the method, the first distance can be acquired, the first distance filter is determined based on the first distance, and the mid-channel audio signal is processed based on the first distance filter, so that the audio distance perceived by a user is the first distance. Since the first distance is smaller than the first threshold (the first threshold can be considered to be a preset threshold based on the distance of the middle sound channel audio perceived by a user in a related scheme), compared with the related scheme, the method and the device can shorten the distance between the middle sound channel audio signal and the user, and improve the user experience. Therefore, the distance of the middle audio signal sensed by the user is relatively short, and the distance sense of the audio is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the terminal field, and in particular to an audio processing method, an electronic device, a storage medium and a chip. BACKGROUND

[0002] With the continuous development of electronic devices with mobile and portable characteristics, good audio-visual experience has become an essential attribute of such electronic devices. For example, for tablet computers, mobile phones, sound boxes and other electronic devices, users have increasingly high requirements for the sound quality of these electronic devices with external audio functions.

[0003] In related solutions, the audio played out by the electronic device has problems such as a far distance between the sound image and the user, which cannot provide better auditory effects for the user. For example, a far distance between the sound image and the user may make the user perceive a weak sense of distance of the audio, affecting the user experience. SUMMARY

[0004] The present application provides an audio processing method, an electronic device, a storage medium and a chip, which can shorten the distance of audio (sound image) perceived by the user, improve the sense of distance of audio perceived by the user, and improve the user experience.

[0005] In a first aspect, an audio processing method is provided. The method includes: in response to a first operation, obtaining a to-be-processed audio signal; in response to a second operation, obtaining a rendering parameter, the rendering parameter including a first distance (the distance refers to a distance corresponding to a sliding bar in a user interface, such as D1 shown in (c) of FIG. 6 The first distance is used to indicate an auditory distance, and the first distance is less than a first threshold (for example, the first threshold can be D2 shown in (c) of FIG. 1 ).

[0006] obtaining a mid-channel audio signal based on the to-be-processed audio signal;

[0007] determining a first distance filter corresponding to the first distance based on the first distance;

[0008] processing the mid-channel audio signal based on the first distance filter to obtain a first audio signal.

[0009] outputting a first output signal based on the first audio signal.

[0010] In the related solution, the audio distance perceived by the user is approximately equal to the first threshold value. In the embodiment of the present application, the electronic device can determine a first distance filter based on the first distance (the first width is less than the first threshold value), and process the center channel audio signal based on the first distance filter, which can make the audio distance perceived by the user be the first distance. Since the first distance is less than the first threshold value, compared with the related solution, the embodiment of the present application can shorten the distance between the center channel audio signal and the user, so that the user perceives the distance of the center audio signal to be close (the distance of the center channel audio signal perceived by the user can be referred to as the center sound image), thereby improving the distance sense of the audio.

[0011] In addition, the embodiment of the present application can obtain the first distance through user operation, and adjust the distance sense of the audio based on the first distance, that is, the user can actively adjust the distance sense of the audio, thereby improving the user experience.

[0012] In combination with the first aspect, in a possible implementation manner of the first aspect, based on the first distance, the first distance filter corresponding to the first distance is determined, including:

[0013] Based on the first distance, M1 near-field head-related transfer functions corresponding to the first near-field distance and M2 far-field head-related transfer functions corresponding to the first far-field distance are determined, M1 and M2 are integers greater than or equal to 1; a first average of the M1 near-field head-related transfer functions and a second average of the M2 far-field head-related transfer functions are determined; and based on the first average and the second average, the first distance filter corresponding to the first distance is determined.

[0014] In the embodiment of the present application, since the head-related transfer function is highly personalized, the embodiment of the present application can determine the first distance filter based on the average of the M1 near-field head-related transfer functions and the average of the M2 far-field head-related transfer functions, which can weaken the influence of the personalized factor on the determined distance filter, thereby improving the accuracy of the determined distance filter.

[0015] In combination with the first aspect, in a possible implementation manner of the first aspect, based on the first distance, M1 near-field head-related transfer functions corresponding to the first near-field distance and M2 far-field head-related transfer functions corresponding to the first far-field distance are determined, including:

[0016] Based on the first distance and the first correspondence relationship, M1 near-field head-related transfer functions corresponding to the first near-field distance and M2 far-field head-related transfer functions corresponding to the first far-field distance are determined;

[0017] The first correspondence includes a correspondence between N distances and N sets of head-related transfer functions, one distance corresponding to one set of head-related transfer functions, one set of head-related transfer functions including at least one near-field head-related transfer function corresponding to a near-field distance and at least one far-field head-related transfer function corresponding to a far-field distance, the N distances including a first distance, one set of head-related transfer functions in the N sets of head-related transfer functions including M1 near-field head-related transfer functions corresponding to a first near-field distance and M2 far-field head-related transfer functions corresponding to a first far-field distance, and N is an integer greater than 1.

[0018] Since the electronic device needs to calculate a large amount of M1 near-field head-related transfer functions and M2 far-field head-related transfer functions in real time, the embodiment of the present application can determine the M1 near-field head-related transfer functions corresponding to the first near-field distance and the M2 far-field head-related transfer functions corresponding to the first far-field distance based on the first distance and the first correspondence, thereby saving the calculation amount and reducing the power consumption of the electronic device.

[0019] In combination with the first aspect, in a possible implementation manner of the first aspect, before the rendering parameter is acquired in response to the second operation, the method includes:

[0020] The first interface includes a first sliding bar, the first sliding bar being used to position a plurality of distances, a maximum distance in the plurality of distances being less than the first threshold value; and the rendering parameter is acquired in response to the second operation, including: in response to the first sliding bar being positioned to a first position, a first distance corresponding to the first position is acquired.

[0021] In the embodiment of the present application, the user can position the first sliding bar to the first position, and the electronic device can acquire the first distance in response to the operation of the user positioning the first sliding bar to the first position, and then process the center channel audio signal based on the first distance, so that the user perceives the center channel audio signal at the first distance, that is, the user can actively adjust the distance of the perceived center channel audio signal through the first sliding bar, thereby improving the use experience.

[0022] In combination with the first aspect, in a possible implementation manner of the first aspect, the rendering parameter further includes a first width, the first width being used to indicate an auditory width, and the first width being greater than a second threshold value; and after the rendering parameter is acquired in response to the second operation, the method further includes:

[0023] The plurality of side channel audio signals are obtained based on a to-be-processed audio signal; each of the plurality of side channel audio signals is rendered based on a first width and a first distance to obtain a plurality of rendering signals, each of the rendering signals including a sound source coordinate of the corresponding side channel audio signal; a plurality of second audio signals are obtained based on the plurality of rendering signals; and a first output signal is output based on the first audio signal, including: the first output signal is output based on the plurality of second audio signals and the first audio signal.

[0024] In related solutions, the audio width perceived by the user is approximately equal to the second threshold. In the embodiments of the present application, the electronic device can perform rendering processing on each of the plurality of side channel audio signals based on the first width (the first width is greater than the second threshold) and the first distance to obtain a plurality of rendering signals, each of the rendering signals including a sound source coordinate of the corresponding side channel audio signal. Since the sound source coordinate of the side channel audio signal is obtained by rendering processing based on the first width, the width of the side channel audio perceived by the user is greater than the second threshold. Compared with related solutions, the width of the side channel audio (the side sound image perceived by the user) can be widened, and the spatial sense of the audio perceived by the user is improved.

[0025] In combination with the first aspect, in a possible implementation manner of the first aspect, the electronic device is configured with P1 virtual loudspeakers, P1 being greater than or equal to 1; and the rendering processing on each of the plurality of side channel audio signals based on the first width and the first distance to obtain the plurality of rendering signals includes:

[0026] The first parameter of each virtual loudspeaker corresponding to each of the plurality of side channel audio signals is determined based on the first width, the first distance, and each of the side channel audio signals, the first parameter including a sound source coordinate of the side channel audio signal corresponding to each virtual loudspeaker, P2 being less than or equal to P1; the first parameter is input into each virtual loudspeaker to obtain an output signal of the P2 virtual loudspeakers corresponding to each of the side channel audio signals; and the plurality of rendering signals are obtained based on the output signal of the P2 virtual loudspeakers corresponding to each of the side channel audio signals.

[0027] In the embodiments of the present application, the electronic device can perform rendering processing on the side channel audio signal through the virtual loudspeaker. Specifically, the electronic device determines the first parameter of each virtual loudspeaker corresponding to each of the plurality of side channel audio signals based on the first width, the first distance, and each of the side channel audio signals, inputs the first parameter into each virtual loudspeaker to obtain the output signal of the P2 virtual loudspeakers corresponding to each of the side channel audio signals, which can improve the clarity and fidelity of the audio.

[0028] With reference to the first aspect, in a possible implementation of the first aspect, before obtaining the plurality of side channel audio signals based on the to-be-processed audio signal, the method further includes:

[0029] displaying a first interface, the first interface including a first sliding bar and a second sliding bar, the first sliding bar being used to position a plurality of distances, a maximum distance in the plurality of distances being less than a first threshold, the second sliding bar being used to position a plurality of widths, a minimum width in the plurality of widths being greater than a second threshold; in response to the first sliding bar being positioned to a first position, obtaining a first distance corresponding to the first position; and in response to the second sliding bar being positioned to a second position, obtaining a first width corresponding to the second position.

[0030] In the embodiment of the application, the user can position the second sliding bar to the second position, and the electronic device can obtain the first width in response to the operation of the user positioning the second sliding bar to the second position, and then process the side channel audio signal based on the first width, so that the width of the side channel audio signal perceived by the user is the first width. That is, the user can actively adjust the width of the perceived side channel audio signal through the second sliding bar, thereby improving the user experience.

[0031] With reference to the first aspect, in a possible implementation of the first aspect, the rendering signal includes a direct sound signal and an early reflection sound signal.

[0032] In the related scheme, the direct sound and the late reverberation sound are rendered when the audio signal is rendered. Since the late reverberation sound can reduce the authenticity of the audio, affect the clarity and purity of the audio, etc., in the embodiment of the application, the direct sound and the early reflection sound are rendered, and the late reverberation sound is not rendered, thereby improving the authenticity, clarity and purity of the audio.

[0033] With reference to the first aspect, in a possible implementation of the first aspect, the plurality of side channel audio signals include a left channel audio signal and a right channel audio signal; or the plurality of side channel audio signals include a left channel audio signal, a right channel audio signal, a left surround channel audio signal and a right surround audio signal.

[0034] With reference to the first aspect, in a possible implementation of the first aspect, based on the plurality of rendering signals, a plurality of second audio signals are obtained, including:

[0035] Based on the plurality of rendering signals, a binaural impulse response is generated; and based on the binaural impulse response, the plurality of side channel audio signals are processed to obtain the plurality of second audio signals.

[0036] The second aspect provides an audio processing method, including:

[0037] In response to a first operation, a to-be-processed audio signal is obtained;

[0038] obtaining a rendering parameter, the rendering parameter comprising a third distance, the third distance being a distance between the electronic device and the user detected by the sensor;

[0039] obtaining a middle channel audio signal based on the to-be-processed audio signal;

[0040] determining a first distance filter based on the third distance, the first distance filter corresponding to a first distance, the first distance being used for indicating a hearing distance, the first distance being smaller than the third distance;

[0041] processing the middle channel audio signal based on the first distance filter to obtain a first audio signal;

[0042] outputting a first output signal based on the first audio signal.

[0043] In the related solution, the audio distance perceived by the user is approximately equal to the third distance. In the embodiment of the present application, the electronic device can determine the first distance filter based on the third distance, the first distance filter corresponding to the first distance, the first distance being used for indicating the hearing distance, the first distance being smaller than the third distance, and then process the middle channel audio signal based on the first distance filter, so that the audio distance perceived by the user is the first distance. Since the first distance is smaller than the third distance, compared with the related solution, the embodiment of the present application can shorten the distance between the middle channel audio signal and the user, so that the distance of the middle audio signal perceived by the user is relatively short (the distance of the middle audio signal perceived by the user can be referred to as the middle sound image), and the distance sense of the audio is improved.

[0044] Moreover, the embodiment of the present application can obtain the distance between the electronic device and the user through the sensor, and can locate the position of the user (the position can be quantified as the third distance). The distance of the middle channel audio signal is accurately adjusted based on the located position of the user, and the user experience is improved.

[0045] With reference to the second aspect, in a possible implementation manner of the second aspect, determining the first distance filter based on the third distance comprises: determining, by the electronic device, M1 near-field head-related transfer functions corresponding to a first near-field distance and M2 far-field head-related transfer functions corresponding to a first far-field distance based on the third distance, M1 and M2 being integers greater than or equal to 1; determining a first average of the M1 near-field head-related transfer functions and a second average of the M2 far-field head-related transfer functions; and determining the first distance filter corresponding to the first distance based on the first average and the second average.

[0046] In the embodiment of the application, the head-related transfer functions are highly personalized, and therefore, the first distance filter can be determined based on the mean of the M1 head-related transfer functions in the near field and the mean of the M2 head-related transfer functions in the far field, so as to weaken the influence of the personalized factors on the determined distance filter and improve the accuracy of the determined distance filter.

[0047] With reference to the second aspect, in a possible implementation of the second aspect, the electronic device determines the M1 head-related transfer functions corresponding to the first near field distance and the M2 head-related transfer functions corresponding to the first far field distance based on the third distance, including:

[0048] determining the M1 head-related transfer functions corresponding to the first near field distance and the M2 head-related transfer functions corresponding to the first far field distance based on the third distance and the second correspondence relationship;

[0049] The second correspondence relationship includes the correspondence relationship between the N groups of head-related transfer functions in the near field and the N groups of head-related transfer functions in the far field, one group of head-related transfer functions in the near field corresponds to one group of head-related transfer functions in the far field, one group of head-related transfer functions in the near field includes at least one head-related transfer function corresponding to one near field distance, one group of head-related transfer functions in the far field includes at least one head-related transfer function corresponding to one far field distance, one group of head-related transfer functions in the N groups of head-related transfer functions in the near field includes the M1 head-related transfer functions corresponding to the first near field distance, and one group of head-related transfer functions in the N groups of head-related transfer functions in the far field includes the M2 head-related transfer functions corresponding to the first far field distance, where N is an integer greater than 1.

[0050] Since the electronic device needs to calculate the M1 head-related transfer functions in the near field and the M2 head-related transfer functions in the far field in real time, the calculation amount is large, and therefore, the M1 head-related transfer functions corresponding to the first near field distance and the M2 head-related transfer functions corresponding to the first far field distance can be determined based on the third distance and the second correspondence relationship, so as to save the calculation amount and reduce the power consumption of the electronic device.

[0051] With reference to the second aspect, in a possible implementation of the second aspect, the rendering parameter further includes a first distance, the first distance is used to indicate the auditory distance, and the first distance filter is determined based on the third distance, including:

[0052] The first distance filter is determined based on the first distance and the third distance.

[0053] The embodiment of the application can obtain the first distance through user operation, obtain the distance between the electronic device and the user through the sensor, and then determine the first distance filter based on the first distance and the third distance. The first distance filter can narrow the audio distance perceived by the user. Therefore, in the embodiment of the application, the first distance obtained through user operation and the third distance obtained through positioning can be combined to narrow the sense of distance of the center channel audio signal, which improves the user experience and accurately adjusts the distance of the center channel audio signal.

[0054] With reference to the second aspect, in a possible implementation of the second aspect, the first distance filter is determined based on the first distance and the third distance, including:

[0055] M1 head-related transfer functions corresponding to the first near-field distance are determined based on the third distance, M2 head-related transfer functions corresponding to the first far-field distance are determined based on the first distance, M1 and M2 are integers greater than or equal to 1; or, M2 head-related transfer functions corresponding to the first far-field distance are determined based on the third distance, M1 head-related transfer functions corresponding to the first near-field distance are determined based on the first distance; and a first average of the M1 head-related transfer functions and a second average of the M2 head-related transfer functions are determined; and the first distance filter is determined based on the first average and the second average.

[0056] With reference to the second aspect, in a possible implementation of the second aspect, the rendering parameter further includes a first width, the first width is used to indicate the auditory width, the first width is greater than the second threshold, and the rendering parameter further includes a first distance, the first distance is used to indicate the auditory distance, and the first distance is less than the third distance; and after the electronic device obtains the rendering parameter, the method further includes:

[0057] The plurality of side channel audio signals are obtained based on the to-be-processed audio signal; each side channel audio signal in the plurality of side channel audio signals is rendered and processed based on the first width and the first distance, to obtain a plurality of rendering signals, each rendering signal including a sound source coordinate of the corresponding side channel audio signal; the plurality of second audio signals are obtained based on the plurality of rendering signals; and the first output signal is output based on the first audio signal, including: the first output signal is output based on the plurality of second audio signals and the first audio signal.

[0058] In a related solution, the audio width perceived by the user is approximately equal to the second threshold. In an embodiment of the present application, the electronic device can perform rendering processing on each of the plurality of side channel audio signals based on the first width (the first width is greater than the second threshold) and the first distance to obtain a plurality of rendering signals, each of which includes the sound source coordinates of the corresponding side channel audio signal. Since the sound source coordinates of the side channel audio signal are obtained by performing rendering processing based on the first width, the width of the side channel audio perceived by the user is greater than the second threshold. Compared with the related solution, the width of the side channel audio (the side sound image perceived by the user) can be widened, and the spatial sense of the audio perceived by the user is improved.

[0059] In combination with the second aspect, in a possible implementation manner of the second aspect, the electronic device is configured with P1 virtual loudspeakers, P1 being greater than or equal to 1; and the rendering processing on each of the plurality of side channel audio signals based on the first width and the first distance to obtain a plurality of rendering signals includes:

[0060] determining, based on the first width, the first distance, and each of the side channel audio signals, a first parameter of each of P2 virtual loudspeakers corresponding to each of the side channel audio signals, the first parameter including sound source coordinates of the side channel audio signal corresponding to each of the virtual loudspeakers, P2 being less than or equal to P1; inputting the first parameter into each of the virtual loudspeakers to obtain an output signal of P2 virtual loudspeakers corresponding to each of the side channel audio signals; and obtaining the plurality of rendering signals based on the output signal of P2 virtual loudspeakers corresponding to each of the side channel audio signals.

[0061] In an embodiment of the present application, the electronic device can perform rendering processing on the side channel audio signal through the virtual loudspeaker. Specifically, the electronic device determines, based on the first width, the first distance, and each of the side channel audio signals, a first parameter of each of P2 virtual loudspeakers corresponding to each of the side channel audio signals, inputs the first parameter into each of the virtual loudspeakers to obtain an output signal of P2 virtual loudspeakers corresponding to each of the side channel audio signals, which can improve the clarity and fidelity of the audio.

[0062] In a third aspect, an electronic device is provided, which is configured to perform the method provided in the first aspect. Specifically, the electronic device can include a processing unit configured to perform any of the possible implementation manners of the first aspect or the second aspect.

[0063] In a fourth aspect, an electronic device is provided, comprising: one or more processors; one or more memories; the one or more memories storing one or more computer programs comprising instructions that, when executed by the one or more processors, cause the electronic device to perform the method in any possible implementation of the first aspect or the second aspect.

[0064] In a fifth aspect, a computer-readable storage medium is provided, comprising computer instructions that, when executed on an electronic device, cause the electronic device to perform the method of the first aspect or the second aspect.

[0065] In a sixth aspect, a chip is provided, comprising a memory for storing instructions; and further comprising a processor for invoking and running the instructions from the memory, so that the electronic device installed with the chip performs the method of the first aspect or the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0066] FIG. 1 is a schematic diagram of a sound image of a related scheme provided by the present application.

[0067] FIG. 2 is a schematic diagram of a sound image provided by an embodiment of the present application.

[0068] FIG. 3 is a schematic diagram of listening to music played by a tablet through a sound box according to an embodiment of the present application.

[0069] FIG. 4 is a structural schematic diagram of an electronic device 100 provided by an embodiment of the present application.

[0070] FIG. 5 is a schematic diagram of a software structure of the electronic device 100 according to an embodiment of the present application.

[0071] FIG. 6 is a schematic diagram of opening a spatial audio according to an embodiment of the present application.

[0072] FIG. 7 is a schematic diagram of opening a spatial audio according to another embodiment of the present application.

[0073] FIG. 8 is a schematic diagram of opening a spatial audio according to another embodiment of the present application.

[0074] FIG. 9 is a schematic diagram of a processing process of an audio algorithm module according to an embodiment of the present application.

[0075] FIG. 10 is a schematic diagram of a processing process of an audio algorithm module according to another embodiment of the present application.

[0076] FIG. 11 FIG. 1 is a schematic diagram of an implementation process of a mid sound image control module according to an embodiment of the present application.

[0077] FIG. 12 FIG. 2 is an example diagram of a spatial position of a sound source according to an embodiment of the present application.

[0078] FIG. 13 FIG. 3 is an example diagram of an audio playing device and a measuring body according to an embodiment of the present application.

[0079] FIG. 14 FIG. 4 is an example diagram of a user interface according to an embodiment of the present application.

[0080] FIG. 15 FIG. 5 is an example diagram of a distance filter according to an embodiment of the present application.

[0081] FIG. 16 FIG. 6 is a schematic diagram of an implementation process of another mid sound image control module according to an embodiment of the present application.

[0082] FIG. 17 FIG. 7 is a schematic diagram of an implementation process of another mid sound image control module according to an embodiment of the present application.

[0083] FIG. 18 FIG. 8 is a schematic diagram of an implementation process of another mid sound image control module according to an embodiment of the present application.

[0084] FIG. 19 FIG. 9 is an example diagram of an implementation process of a side sound image control module according to an embodiment of the present application.

[0085] FIG. 20 FIG. 10 is an example diagram of an array of virtual loudspeakers according to an embodiment of the present application.

[0086] FIG. 21 FIG. 11 is an example diagram of generating a binaural impulse response according to an embodiment of the present application.

[0087] FIG. 22 FIG. 12 is a schematic diagram of an audio processing method according to an embodiment of the present application.

[0088] FIG. 23 FIG. 13 is a schematic diagram of another audio processing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0089] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

[0090] With the continuous development of portable electronic devices, a good audio-visual experience has become an essential attribute for these devices. For example, users have increasingly higher requirements for the sound quality of electronic devices with external audio output capabilities, such as tablets, mobile phones, and speakers.

[0091] These portable electronic devices are typically small and lightweight, which limits the internal layout of their components. Consequently, when playing audio aloud, these limitations result in issues such as a flat sound image and a perceived distance between the sound image and the user, hindering the optimal listening experience. For example, a flat sound image may weaken the user's perception of spatial depth, while a distant sound image may weaken the user's perception of distance, negatively impacting the user experience.

[0092] Sound image: This refers to the spatial positioning of sound in a musical work. Sound image is typically used to describe the perceived location of different sound sources in music, including their left-right position on the horizontal plane (stereo image), their high-low position on the vertical plane, and their sense of depth in terms of front-back position. The clarity and accuracy of the sound image are crucial for creating a sense of presence and immersion in music.

[0093] For example, please refer to FIG. 1 , FIG. 1 This is a schematic diagram of the sound image of externally played audio in one of the related solutions provided. For example, FIG. 1 The electronic device shown in (a), taking a tablet as an example, may include eight physical speakers. In other embodiments, it may include more or fewer physical speakers. Of these eight physical speakers, two physical speakers may be provided in areas A and B at the bottom of the tablet and areas G and H at the top, and two physical speakers may be provided in areas C and D and E and F on the left and right sides of the tablet, respectively. FIG. 1 When playing music through a flat panel as shown in (a), the sound image formed by playing music through a flat panel in the relevant scheme can be referenced. FIG. 1 The acoustic images 1 in (b) and (c) are shown. FIG. 1 (b) is a top-down view of the tablet and the user. FIG. 1 (c) in the diagram is a side view of the tablet and the user. FIG. 1 As shown in (b) and (c), sound images can include center sound images and side sound images. Side sound images include left and right sound images. For example, in sound image 1, the user's perceived position of the human voice is the center sound image, the user's perceived positions of instrument 1, instrument 3, instrument 5, and instrument 7 are the left sound images, and the user's perceived positions of instrument 2, instrument 4, instrument 6, and background sounds are the right sound images. Of course, sound images can also include the positions of other instruments or sounds perceived by the user. FIG. 1(b) and (c) are merely illustrative examples and do not constitute a limitation on the audio-visual content.

[0094] from FIG. 1 As shown in (b) of the relevant scheme, the spatial arrangement of the left and right side sound images and the center sound image in the user-perceived sound image 1 is relatively compact. The width of the left and right side sound images is approximately equal to the distance between the left and right speakers. For example, the width of the left and right side sound images is... FIG. 1 The width W1 shown in (b) results in a relatively flat overall sound image, weakening the user's perception of spatial audio. From FIG. 1 As shown in (c), in the relevant user perception scheme, the sound image formed by the tablet's external music playback is relatively far from the user, for example... FIG. 2 The distance D1 shown in (c) results in a weaker perceived distance for the user in the audio.

[0095] To address the aforementioned issues, this application provides an audio processing method that can broaden the width of the sound image, enhancing the user's perception of audio spatiality, and also bring the sound image closer to the user, further enhancing the user's perception of audio distance, thereby providing a better immersive audio experience. For example, please refer to... FIG. 2 , FIG. 2 This is a schematic diagram of an audio-visual representation provided in an embodiment of this application. For example, through... FIG. 2 When playing music through the tablet as shown in (a) of this application embodiment, the sound image formed by playing music through the tablet can be referenced. FIG. 2 The sound image 2 in (b) and (c) is shown. FIG. 2 (b) is a top-down view of the tablet and the user. FIG. 2 (c) in the diagram is a side view of the tablet and the user.

[0096] Depend on FIG. 2 As can be seen from (b) in this embodiment, the spatial arrangement of the left and right side sound images and the center sound image perceived by the user is relatively wide, and the width of the left and right side sound images is... FIG. 1 The width W2 shown in (b) is relative to the acoustic image formed by the relevant scheme (e.g., FIG. 2 The sound image shown in (b) is widened (width W2 is greater than width W1), which enhances the user's perception of the spatial sense of the audio.

[0097] Depend on FIG. 2 As can be seen from (c) in this embodiment, the sound image formed by the music played from the tablet is located around the user's ear, and the distance of the sound image perceived by the user is... FIG. 1 The distance D2 shown in (c) is relative to the acoustic image formed by the relevant scheme (e.g., FIG. 2 The sound image shown in (c) is closer to the user (distance D2 is less than D1), which enhances the user's perception of the distance of the audio.

[0098] The above FIG. 3 The application scenario shown in (a) is merely illustrative and does not constitute a limitation on the application scenarios of the embodiments of this application. For example, the application scenarios applicable to the embodiments of this application may be: Please refer to FIG. 3 , FIG. 3 This is a schematic diagram illustrating listening to music played on a tablet via a speaker, according to an embodiment of this application. FIG. 4 The tablet can be connected to an external speaker, allowing users to listen to music played from the tablet. Alternatively, the application scenarios applicable to this embodiment can also include users listening to audio played from external speakers in cinemas, home theaters, vehicles, and other similar settings.

[0099] The audio processing method provided in this application is applied to electronic devices with external audio playback capabilities. Electronic devices can also be referred to as terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Electronic devices can be mobile phones, smart TVs, tablets, laptops, desktop computers, computers with wireless transceiver capabilities, smart speakers, audio equipment, virtual reality (VR) electronic devices, augmented reality (AR) electronic devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on. The embodiments of this application do not limit the specific technologies or device forms used in the electronic devices.

[0100] To better understand the embodiments of this application, the structure of the electronic device of the embodiments of this application is described below.

[0101] For example, FIG. 4A structural diagram of the electronic device 100 is shown. The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 130, an antenna 1, an antenna 2, a mobile communication module 140, a wireless communication module 150, an audio module 160, a speaker 160A, a receiver 160B, a microphone 160C, a headset interface 160D, a sensor module 170, a camera 180, a display screen 190, and the like. The sensor module 170 can include a pressure sensor 170A, a touch sensor 170B, and the like.

[0102] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0103] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), and the like. Different processing units can be independent devices, or can be integrated into one or more processors.

[0104] In the embodiments of the present application, the processor 110 is configured to widen the width of the sound image and shorten the distance between the sound image and the user.

[0105] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 140, the wireless communication module 150, the modem processor, and the baseband processor, and the like.

[0106] The mobile communication module 140 can provide a solution including 2G / 3G / 4G / 5G wireless communication applied to the electronic device 100.

[0107] The wireless communication module 150 can provide solutions for wireless communication including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, and the like, which are applied on the electronic device 100.

[0108] In some embodiments, the antenna 1 and the mobile communication module 140 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 150 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology.

[0109] The electronic device 100 implements a display function through a GPU, a display screen 190, and an application processor, and the like.

[0110] The display screen 190 is used to display images, videos, and the like. The electronic device 100 can implement a photographing function through an ISP, a camera 180, a video codec, a GPU, a display screen 190, and an application processor, and the like.

[0111] The internal memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to extend the storage capacity of the electronic device 100.

[0112] The internal memory 130 can be used to store computer executable program codes, which include instructions. The processor 110 executes various function applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 130.

[0113] The electronic device 100 can implement an audio function through an audio module 160, a speaker 160A, a receiver 160B, a microphone 160C, an earphone interface 160D, and an application processor, and the like. For example, music playing, recording, and the like.

[0114] The audio module 160 is used to convert digital audio information into an analog audio signal output, and is also used to convert an analog audio input into a digital audio signal. The audio module 160 can also be used to encode and decode audio signals. In some embodiments, the audio module 160 can be disposed in the processor 110, or part of the function modules of the audio module 160 can be disposed in the processor 110.

[0115] Speaker 160A, also called "loudspeaker", is used to convert audio electrical signals into sound signals.

[0116] Receiver 160B, also called "earpiece", is used to convert audio electrical signals into sound signals.

[0117] Microphone 160C, also called "microphone", "sound transducer", is used to convert sound signals into electrical signals.

[0118] Earphone interface 160D is used to connect wired earphones.

[0119] Pressure sensor 170A is used to sense pressure signals, and can convert pressure signals into electrical signals. In some embodiments, pressure sensor 170A can be disposed on display screen 190. When a touch operation is applied to display screen 190, electronic device 100 detects the intensity of the touch operation according to pressure sensor 170A. Electronic device 100 can also calculate the position of the touch according to the detection signal of pressure sensor 170A.

[0120] Touch sensor 170B, also called "touch panel". Touch sensor 170B can be disposed on display screen 190, and touch sensor 170B and display screen 190 together form a touch screen, also called "touch panel". Touch sensor 170B is used to detect touch operations applied thereto or in the vicinity thereof.

[0121] The hardware structure of electronic device 100 is introduced here, and it can be understood that, FIG. 4 The components included in the hardware structure shown do not constitute a specific limitation on electronic device 100. Electronic device 100 can have more or fewer components than those shown in the figure, two or more components can be combined, or different component configurations can be used. FIG. 5 The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0122] In addition, an operating system runs on the above components. For example, iOS operating system, Android open source operating system, Windows operating system, and Harmony operating system. Application programs can be installed and run on the operating system.

[0123] The operating system of electronic device 100 can use a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take the Android operating system with a layered architecture as an example to illustrate the operating system of electronic device 100.

[0124] FIG. 5is a software structure block diagram of the electronic device 100 of the embodiments of the present application. The layered architecture divides the software into several layers, each of which has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and the system library, and the kernel layer. The application layer can include a series of application packages.

[0125] As shown in FIG. 5 , the application layer package camera, gallery, calculator, memo, instant messaging, music 1, video 1, and other applications. Among them, music 1 refers to one of the music application programs, and the user can listen to music through the music 1 application program. Video 1 refers to one of the video application programs, and video 1 can be a long video application program, a short video application program, and a live application program, etc., which is not limited by the embodiments of the present application, and the user can watch videos through the video 1 application program.

[0126] The application framework layer provides the application programming interface (application programming interface, API) and the programming framework for the application programs of the application layer. The application framework layer includes some pre-defined functions.

[0127] As shown in FIG. 5 , the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager module, etc.

[0128] The window manager is used to manage the window program. The window manager can obtain the display screen size, judge whether there is a status bar, lock the screen, and intercept the screen, etc.

[0129] The content provider is used to store and obtain data, and make these data accessible to the application program. The data can include video, image, audio, dialing and answering phone, browsing history and bookmark, phone book, etc.

[0130] The view system includes visual controls, such as controls that display text, controls that display pictures, etc. The view system can be used to build an application program. The display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view that displays text and a view that displays pictures.

[0131] The phone manager is used to provide the communication function of the electronic device 100. For example, the management of the call state (including connection, hang up, etc.).

[0132] The resource manager provides applications access to various resources, such as localized strings, icons, pictures, layout files, video files, and so on.

[0133] The notification manager enables applications to display notification information in the status bar, which can be used to convey informational type messages that can automatically disappear after a brief stay without user interaction.

[0134] The Android Runtime includes the core libraries and the Dalvik virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0135] The core libraries contain two parts: one part is the function functions that the java language needs to call, and the other part is the core library of Android.

[0136] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to manage the life cycle of objects, stack management, thread management, security and exception management, and garbage collection and other functions.

[0137] The system library can include multiple functional modules. For example: surface manager, media libraries, three-dimensional graphics processing library (such as: OpenGL ES), 2D graphics engine (such as: SGL), audio algorithm module, etc.

[0138] The surface manager is used to manage the display subsystem and provides 2D and 3D layer fusion for multiple applications.

[0139] The media library supports multiple commonly used audio, video format playback and recording, and static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0140] The three-dimensional graphics processing library is used to realize three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc.

[0141] The 2D graphics engine is a drawing engine for 2D drawing.

[0142] The audio algorithm module is used to process multiple commonly used audios supported by the media library, and is used to enhance the spatial sense and distance sense of the audio. As shown in the figure, the audio algorithm module includes a pre-gain module, an upmix or pass-through module, a mid sound image control module, a first side sound image control module, a second side sound image control module, a first crosstalk cancellation module, a second crosstalk cancellation module, an equalization module, a downmix module, a post-gain module, etc. FIG. 5 ​

[0143] The pre-gain module is used to adjust the gain of the audio signal before it enters the upmix or pass-through module for processing. The gain adjustment helps ensure that the signal is at an appropriate level before entering the subsequent processing devices, preventing signal overload or noise issues.

[0144] The upmix or pass-through module determines whether the audio signal is a two-channel or multi-channel signal. If it is determined to be a two-channel signal (which can be stereo), meaning it includes a left channel electrical signal and a right channel electrical signal, the upmix or pass-through module processes the audio signal by upmixing it into a three-channel audio signal. The three-channel audio signal is then passed through to the center image control module and the first side image control module for processing.

[0145] It should be understood that upmixing refers to converting an audio signal from a smaller number of channels to a larger number of channels. For example, upmixing stereo (two-channel) audio signals into three-channel, or 5.1-channel, or 7.1-channel, or 9.1-channel surround sound. This process typically involves using algorithms to create additional channels that do not exist in the original signal. Upmixing can increase the spatial and immersive qualities of the audio, making it more suitable for playback in a multi-channel sound system.

[0146] If it is determined that the audio signal is a multi-channel signal, such as a 5.1-channel, 7.1-channel, or 9.1-channel surround sound signal, the upmix or pass-through module passes the audio signal directly to the center image control module, the first side image control module, and the second side image control module for processing.

[0147] It should be understood that 5.1-channel, 7.1-channel, and 9.1-channel are configuration standards for audio systems, designed to provide an immersive listening experience. These numbers represent the number of channels, where the first number represents the number of full-range speakers, and the numbers after the decimal point represent the number of subwoofers. Here is a brief explanation of each channel configuration:

[0148] 5.1-channel includes five full-range speakers and one subwoofer. The specific distribution is: front left, front right, center, surround left, and surround right, along with a central subwoofer.

[0149] 7.1-channel: Based on the 5.1-channel configuration, it adds two surround speakers, namely surround left back and surround right back, providing a more immersive surround sound experience. The configuration is: front left, front right, center, surround left, surround right, surround left back, and surround right back, along with a central subwoofer.

[0150] 9.1 Channel: This configuration further adds two speakers, typically front left center and front right center, or in some cases, two additional surround speakers, providing a wider sound field and more nuanced positioning effects. The configuration can include: front left, front right, front center, front left center, front right center, center, surround left, surround right, surround left back, and surround right back, as well as a central subwoofer.

[0151] Of course, in other embodiments, the multi-channel audio signal can also include 11.1 channel, 13.1 channel audio signal, etc., which are not limited in the embodiments of the present application.

[0152] In the embodiments of the present application, a channel refers to a channel of an audio signal, and the channel can also be referred to as a channel, an audio channel, etc., which are not limited in the embodiments of the present application.

[0153] The center image control module is used to shorten the distance between the center vocal and the user, and improve the sense of distance of the audio signal perceived by the user. The implementation process of the center image control module can be referred to in the following embodiments, which will not be described here.

[0154] The first side image control module and the second side image control module are used to widen the width of the sound image and improve the spatial sense of the audio signal perceived by the user. The implementation process of the first side image control module and the second side image control module can be referred to in the following embodiments, which will not be described here.

[0155] The audio signal can include a multi-channel audio signal, such as stereo sound including left channel signal and right channel signal, and there will be crosstalk between the channels, thereby affecting the audio quality. The first crosstalk cancellation module and the second crosstalk cancellation module are used to cancel the crosstalk between the channels and improve the audio quality.

[0156] Equalizer (EQ): used to adjust the frequency components of the audio signal to improve the quality and effect of the sound. For example, certain frequency ranges in the audio signal can be enhanced or attenuated to help balance the sound. For example, the equalizer can help shape the tone of an instrument or sound by adjusting different frequencies, making it more distinct or soft. For example: the equalizer can create a wider or more compact sound field effect by adjusting high and low frequencies.

[0157] Downmix module: used for converting audio signals from a larger number of channels to a smaller number of channels. For example, downmixing 5.1 channel surround sound to stereo sound. In the downmixing process, the information in the original multi-channel signal is combined or redistributed to a smaller number of channels. This usually involves adjusting the volume level and balancing the relative intensity between channels to ensure a good listening experience in a smaller number of channels.

[0158] The post-gain module is used for gain adjustment after an audio signal is processed by some processing procedure, such as an equalizer, a compressor or an effecter. The post-gain can be used for adjusting the final volume of the processed signal to ensure the consistency and balance of the output signal, and it allows fine tuning of the final loudness of the signal to adapt to different playing environments or audience needs.

[0159] The kernel layer is a layer between hardware and software. The kernel layer at least includes display drivers, camera drivers, audio drivers and sensor drivers.

[0160] It should be understood that FIG. 6 The illustrated software and hardware architecture diagram is only illustrative and should not be construed as limiting the embodiments of the present application.

[0161] The above embodiments introduce the hardware structure and software architecture of the electronic device 100. It should be understood that before the audio signal played by the electronic device is processed by the audio processing method provided by the embodiments of the present application, the electronic device needs to be in an external audio mode and the spatial audio function needs to be turned on. The following embodiments introduce the electronic device in the external audio mode and the spatial audio function turned on.

[0162] The electronic device in the external audio mode can be that when the electronic device is not connected to an audio output device such as an earphone, the electronic device can be defaulted to be in the external audio mode.

[0163] It can also be that when the electronic device is connected to an audio output device such as an earphone, the earphone output mode can be changed to the external audio mode. The embodiments of the present application do not limit the way of changing the earphone output mode to the external audio mode.

[0164] The way of turning on the spatial audio of the electronic device can be that please refer to FIG. 6 , FIG. 6 is a schematic diagram of turning on the spatial audio provided by the embodiments of the present application. It should be understood that FIG. 6 is a schematic diagram of turning on the spatial audio provided by the embodiments of the present application. It should be understood that

[0165] Please refer to the interface 610 shown in (a) of FIG. 6 , which is the main interface of the tablet. The interface includes an icon 61, which is the icon corresponding to the "Settings" application, which can be clicked by the user. In response to the operation of the user clicking the icon 61, the electronic device displays the interface 620 shown in (b) of FIG. 6 . The interface includes a parent menu 621 and a child menu 622. The parent menu 621 includes multiple setting categories, such as the setting category 623 of audio and sound effects. The child menu 622 includes more specific setting options in a certain setting category of the parent menu 621, for example, the child menu corresponding to the setting category 623 isFIG. 6 (b) shows the submenu 622. The user can click on "Settings Category" 623, and in response, the electronic device displays the submenu 622. The submenu 622 includes controls 624 for spatial audio, 625 for orientation rendering, 626 for width, and 627 for distance, etc. Before the user activates spatial audio, such as... FIG. 7 As shown in (b), the controls 624 corresponding to spatial audio and 625 corresponding to directional rendering in submenu 622 are in a closed state. At this time, the user cannot adjust the width parameter via control 626 or the distance parameter via control 627. The user can click or slide controls 624 and 625. In response to the user's click or slide operation of controls 624 and 625, the electronic device can display as shown in (b). FIG. 7 The interface 630 shown in (c) has the control 631 corresponding to spatial audio and the control 632 corresponding to directional rendering in the interface 630 in the open state. The user can adjust the width parameter through the control 633 to adjust the sound image width of the external audio, and adjust the distance parameter through the control 634 to adjust the distance of the audio perceived by the user.

[0166] It should be understood that the above-described method of opening the settings interface by clicking the settings icon 61 in the interface 610 is merely an exemplary description. In other embodiments, users may also open the settings interface through specific gestures or shortcuts, and this application embodiment does not limit this.

[0167] Other ways for electronic devices to enable spatial audio include: Please refer to... FIG. 7 , FIG. 7 This is a schematic diagram illustrating another method of enabling spatial audio according to an embodiment of this application. It should be understood that... FIG. 7 This illustrates the process of enabling spatial audio through the task center of an electronic device (such as a tablet).

[0168] Please refer to FIG. 7 The interface 710 shown in (a) is the main interface of the tablet. Users can perform a swipe-down operation from the upper right side of this interface; in response to the swipe-down operation, the electronic device displays... FIG. 7 The interface 720 shown in (b) displays a drop-down menu 721 corresponding to the control center, which includes audio and sound effects options 722. The user can click on this option 722, and the electronic device responds to this click by displaying... FIG. 7 The interface 730 shown in (c) includes a control 731 for spatial audio, a control 732 for orientation rendering, a control 733 for width, and a control 734 for distance. Before the user activates spatial audio, as shown in (c)...FIG. 8 As shown in (c) of FIG. 7A, the control 731 corresponding to the spatial audio and the control 732 corresponding to the azimuth rendering are in the closed state, and at this time, the user cannot adjust the width parameter through the control 733 and cannot adjust the distance parameter through the control 734. The user can click or slide the control 731 and the control 732, and in response to the operation of the user clicking or sliding the control 731 and the control 732, the electronic device can display an interface 740 as shown in (d) of FIG. 7A. FIG. 8 As shown in (d) of FIG. 7A, the interface 740 in which the control 741 corresponding to the spatial audio and the control 742 corresponding to the azimuth rendering are in the open state, and the user can adjust the width parameter through the control 743 to adjust the width of the sound image of the playback audio, and adjust the distance parameter through the control 744 to adjust the distance of the audio perceived by the user.

[0169] It should be understood that the above-mentioned manner of opening the task center by performing a downward sliding operation on the upper right side of the interface 710 is only an exemplary illustration, and in other embodiments, the user can also open the task center through a specific gesture or button, which is not limited in the embodiments of the present application. It should also be understood that the user can also open the task center on other interfaces in addition to the interface 710, which is not limited in the embodiments of the present application.

[0170] The manner in which the electronic device opens the spatial audio can also be that please refer to FIG. 8 , FIG. 8 is another schematic diagram provided by the embodiments of the present application for opening the spatial audio. It should be understood that FIG. 8 The schematic diagram shows that when the electronic device plays audio, it is detected that the electronic device is in the playback audio mode, and the user is prompted to open the spatial audio.

[0171] FIG. 8 The interface 810 shown in (a) of FIG. 8 is an interface in which the electronic device plays music through a music application. The interface 810 includes a capsule 811, which is a form of an icon corresponding to a spatial audio function. Assuming that the user listens to music through the interface 810, the electronic device is connected to earphones, and the electronic device is in an earphone output mode, then the capsule 811 is in a closed state as shown in (a) of FIG. 8. Assuming that the user changes the earphone output mode to the playback audio mode, such as the user disconnects the electronic device from the earphones, then the electronic device can detect that it is in the playback audio mode, and after detecting that it is in the playback audio mode, the electronic device displays an interface 820 as shown in (b) of FIG. 8. FIG. 8 FIG. 6 to FIG. 8 ​Interface 820 shown in (b) of FIG. 8. Capsule 821 in interface 820 is in an open state, and interface 820 further includes a prompt pop-up window 822 for prompting the user whether to open the spatial audio function, which includes a control 823 for instructing to open the spatial audio function and a control 824 for instructing not to open the spatial audio function. The user can click control 823, and in response to the user's operation of clicking control 823, the electronic device displays an interface 830 as shown in (c) of FIG. 8. FIG. 5 Interface 830 includes a pop-up window 831, which includes a control 832 and a control 833, the user can adjust the width parameter through control 832 to adjust the width of the sound image of the loudspeaker audio, and adjust the distance parameter through control 833 to adjust the distance of the audio perceived by the user.

[0172] It should be understood that the electronic device can detect whether it is in the loudspeaker audio mode in the following manner, for example: the electronic device uses the isSpeakerphoneOn() method of AudioManager to check whether the loudspeaker is on, and if true is returned, it indicates that the current is in the loudspeaker audio mode. For another example: the electronic device can obtain the current audio mode through the getMode() method of AudioManager, and then judge whether it is AudioManager.MODE_NORMAL. If it is, and isSpeakerphoneOn() also returns true, it can be considered that the electronic device is in the loudspeaker audio mode, etc., which is not limited in the embodiments of the present application.

[0173] Of course, in other embodiments, the user can also open the spatial audio function of the electronic device in a manner other than FIG. 9 which is not limited in the embodiments of the present application.

[0174] The above embodiments introduce the operation process of the electronic device in the loudspeaker audio mode and having opened the spatial audio function, and the following embodiments introduce the implementation process of the audio processing method provided by the embodiments of the present application in combination with FIG. 10 the audio algorithm module shown in FIG. 9.

[0175] It should be understood that the channel type of the audio signal includes dual-channel and multi-channel, and the explanation of dual-channel and multi-channel has been stated in the above embodiments, which will not be repeated here. When the channel type of the audio signal is different, the implementation process of the audio processing method provided by the embodiments of the present application is slightly different. In the following embodiments, FIG. 9 the implementation process of the audio processing method provided by the embodiments of the present application is introduced when the channel type of the audio signal is dual-channel (stereo). FIG. 9 the implementation process of the audio processing method provided by the embodiments of the present application is introduced when the channel type of the audio signal is multi-channel.

[0176] Please refer to FIG. 9 , FIG. 1 is a schematic diagram of a processing procedure of an audio algorithm module provided by an embodiment of the present application. FIG. 11 to FIG. 18 The numbers in the circles represent steps of the processing procedure, which include S91-S99.

[0177] S91, the audio algorithm module acquires an audio signal A1 and inputs the audio signal A1 to a pre-emphasis gain module.

[0178] It should be understood that the audio signal A1 refers to an original audio signal that has not been processed by the audio algorithm module. The original audio signal can refer to a signal of an audio file in a music application or a video application downloaded in an electronic device before the signal is decoded and converted into an analog signal that can be played by a loudspeaker of the electronic device. The audio file is in a digital format such as MP3, AAC, FLAC, etc. when stored. When a user plays the audio through the music application or the video application, the original audio signal is decoded into an original digital audio signal.

[0179] The original audio signal can also be a sound signal captured by a microphone of the electronic device. Such a signal is an analog signal, i.e., a continuously varying electrical signal, which reflects the real-time changes of a sound waveform. The original audio signal is not subjected to any digital processing or compression before being processed by the electronic device.

[0180] S92, the pre-emphasis gain module processes the audio signal A1 to obtain an audio signal A2 and sends the audio signal A2 to an upmix or pass-through module.

[0181] It should be understood that the pre-emphasis gain module processes the audio signal A1 to improve the level of the audio signal, so that it is suitable for further processing or transmission.

[0182] In implementation, for example, the pre-emphasis gain module can adjust the amplitude of the audio signal A1 through a variable gain amplifier. The pre-emphasis gain module can remove unwanted frequency components in the audio signal A1 through a low-pass filter, a high-pass filter, or a band-pass filter. The pre-emphasis gain module can also automatically adjust the gain according to the intensity of the audio signal A1 to keep the output audio signal A2 within a certain dynamic range. The pre-emphasis gain module can also reduce noise components in the audio signal A1 through a noise reduction algorithm. The pre-emphasis gain module can also adjust the signal waveform through a specific circuit or algorithm to adapt to specific transmission or processing requirements.

[0183] S93, the upmix or pass-through module determines the channel type of the audio signal A2. After determining that the channel type of the audio signal A2 is a dual-channel, the upmix processing is performed on the audio signal A2 to obtain the audio signals A21, A22 and A23, and the audio signal A21 is sent to the middle image control module, and the audio signals A22 and A23 are sent to the first side image control module.

[0184] It should be understood that the audio signal A1 can refer to a dual-channel audio signal including a left channel signal and a right channel signal, and thus the audio signal A2 obtained after processing by S92 also includes a dual-channel audio signal. When the upmix or pass-through module determines the channel type of the audio signal A2, it can be determined that the channel type of the audio signal A2 is a dual-channel.

[0185] In implementation, for example, the audio signal A1 obtained by S91 includes attribute information of the audio signal, and the attribute information of the audio signal includes the number of channels of the audio signal. The upmix or pass-through module can determine whether the audio signal is a dual-channel or a multi-channel by the number of channels included in the attribute information of the audio signal. For another example, some audio encoding formats such as Dolby Digital and Digital Theater Systems (DTS) are designed for multi-channel audio, and some audio encoding formats such as Pulse Code Modulation (PCM) and MPEG-1 Audio Layer 3 (MP3) are designed for dual-channel audio. The upmix or pass-through module can determine whether the audio signal is a dual-channel or a multi-channel by the audio encoding format of the audio signal. The implementation of the embodiment of the present application for determining the channel type of the audio signal is not limited.

[0186] It should also be understood that upmix processing refers to converting an audio signal from a smaller number of channels to a larger number of channels. In the embodiment of the present application, the upmix or pass-through module can obtain a center channel audio signal, a left channel audio signal and a right channel audio signal after upmix processing on the audio signal. For ease of description, the center channel audio signal after upmix processing is referred to as the audio signal A21, the left channel audio signal after upmix processing is referred to as the audio signal A22, and the right channel audio signal after upmix processing is referred to as the audio signal A23. Among them, the center channel audio signal mainly includes vocals, and the left channel audio signal and the right channel audio signal after upmix processing mainly include environmental sound, which refers to accompaniment, noise, etc. in the audio signal.

[0187] In implementation, for example, the upmix or pass-through module can perform upmix processing on the audio signal A2 by the following formula one:

[0188]

[0189] wherein, is a representation of the eigenvector of the audio signal A2 input to the upmix or pass-through module, may be is a representation of the eigenvector of the left channel audio signal included in the audio signal A2, may also be is a representation of the eigenvector of the right channel audio signal included in the audio signal A2; is the principal eigenvector, when is , is the principal eigenvector of is , is the principal eigenvector of is the transpose of the principal eigenvector. is a representation of the eigenvector of the center channel audio signal (audio signal A21), may be is a representation of the eigenvector of the center channel audio signal (audio signal A21) based on the left channel audio signal, may be is a representation of the eigenvector of the center channel audio signal (audio signal A21) based on the right channel audio signal.

[0190] is a representation of the eigenvector of the ambient sound, may be is a left channel audio signal (audio signal A22) after upmix processing, may be obtained by , may be is a right channel audio signal (audio signal A23) after upmix processing, may be obtained by .

[0191] S94: The center image control module processes the audio signal A21 to obtain an audio signal A211 and sends the audio signal A211 to the equalization module.

[0192] It should be understood that the audio signal A21 refers to the center channel audio signal obtained after upmix processing by S93, which mainly includes vocals. After the center image control module processes the audio signal A21, it can pull the vocals closer to the listener. FIG. 1The distance between the vocal included in the middle sound image 1 and the user is shown, and the distance sense of the vocal perceived by the user is improved.

[0193] The specific implementation process of the middle sound image control module processing the audio signal A21 to obtain the audio signal A211 can be referred to below FIG. 19 The specific implementation process of the middle sound image control module processing the audio signal A21 to obtain the audio signal A211 can be referred to below

[0194] S95, the first side sound image control module processes the audio signal A22 and the audio signal A23 to obtain the audio signal A221 and the audio signal A231, and sends the audio signal A221 and the audio signal A231 to the first crosstalk elimination module.

[0195] It should be understood that the audio signal A22 refers to the left channel audio signal after the upmix processing of S93, and the audio signal A23 refers to the right channel audio signal after the upmix processing of S93. The left channel audio signal and the right channel audio signal can also be referred to as side sound image audio signals. After the first side sound image control module processes the audio signal A22 and the audio signal A23, the width of the side sound image in the sound image 1 can be widened, and the spatial sense of the side sound image perceived by the user can be improved. FIG. 10 It should be understood that the audio signal A22 refers to the left channel audio signal after the upmix processing of S93, and the audio signal A23 refers to the right channel audio signal after the upmix processing of S93. The left channel audio signal and the right channel audio signal can also be referred to as side sound image audio signals. After the first side sound image control module processes the audio signal A22 and the audio signal A23, the width of the side sound image in the sound image 1 can be widened, and the spatial sense of the side sound image perceived by the user can be improved.

[0196] The specific implementation process of the first side sound image control module processing the audio signal A22 and the audio signal A23 to obtain the audio signal A221 and the audio signal A231 can be referred to below FIG. 10 The specific implementation process of the first side sound image control module processing the audio signal A22 and the audio signal A23 to obtain the audio signal A221 and the audio signal A231 can be referred to below

[0197] S96, the first crosstalk elimination module processes the audio signal A221 and the audio signal A231 to obtain the audio signal A2211 and the audio signal A2311, and sends the audio signal A2211 and the audio signal A2311 to the equalization module.

[0198] It should be understood that the audio signal A221 refers to the left channel audio signal after the processing of S95, and the audio signal A231 refers to the right channel audio signal after the processing of S95.

[0199] It should also be understood that since the audio signal obtained after processing by S95 includes the left channel audio signal and the right channel audio signal, crosstalk will exist between each channel, thereby affecting the audio quality. Therefore, the first crosstalk elimination module is used to eliminate the crosstalk between the left channel audio signal and the right channel audio signal.

[0200] In the implementation, the first crosstalk cancellation module can identify and cancel the unwanted crosstalk signals in the audio signal A221 and the audio signal A231 through a filter, and can also reduce the crosstalk between the audio signal A221 and the audio signal A231 by setting a delay value, and the embodiments of the present application are not limited in this regard.

[0201] S97, the equalization module processes the audio signal A211 to obtain an audio signal A2111, processes the audio signal A221 to obtain an audio signal A2211, processes the audio signal A231 to obtain an audio signal A2311, and sends the audio signal A2111, the audio signal A2211 and the audio signal A2311 to the downmix module.

[0202] It should be understood that the audio signal A211 is a center channel audio signal obtained after processing by S94, the audio signal A2211 refers to a left channel audio signal obtained after processing by S96, and the audio signal A2311 refers to a right channel audio signal obtained after processing by S96.

[0203] It should also be understood that the equalization module can adjust the frequency components of the audio signal A211, the audio signal A2211 and the audio signal A2311 to improve the quality and effect of the sound.

[0204] In the implementation, the equalization module can balance the overall audio by compressing the audio signal A211, the audio signal A2211 and the audio signal A2311 of different frequency bands, can also adjust the frequency response of the audio signal A211, the audio signal A2211 and the audio signal A2311 through a mathematical algorithm, and can also perform equalization processing on the audio signal A211, the audio signal A2211 and the audio signal A2311 through a special hardware device such as an equalizer, an audio processor, etc., and the embodiments of the present application are not limited in this regard.

[0205] S98, the downmix module processes the audio signal A2111, the audio signal A2211 and the audio signal A2311 to obtain an audio signal A2 and an audio signal A3, and sends the audio signal A2 and the audio signal A3 to the post-gain module.

[0206] It should be understood that the audio signal A2111 is a center channel audio signal obtained after processing by S97, the audio signal A2211 refers to a left channel audio signal obtained after processing by S97, and the audio signal A2311 refers to a right channel audio signal obtained after processing by S97.

[0207] It should also be understood that the downmix module is used to convert the audio signal from a larger number of channels to a smaller number of channels. It should be understood that some electronic devices do not support playing multi-channel audio, only support playing two-channel audio, such as earphones, portable speakers, etc., and the downmix module can ensure that the audio signal can also maintain good sound quality and balance in a smaller number of channels. Therefore, assuming that the audio processing method provided in the embodiments of the present application is applied to an electronic device that does not support playing multi-channel audio, the audio signal can be downmixed by S98. Assuming that the audio processing method provided in the embodiments of the present application is applied to an electronic device that supports playing multi-channel audio, S98 can also not be executed, and S99 is executed after S97 is executed.

[0208] In the implementation, the downmix module can combine the audio signal A2111, the audio signal A22111 and the audio signal A2311 into a smaller number of channels through simple linear operation to obtain the audio signal A2 and the audio signal A3, where the audio signal A2 refers to the left channel audio signal obtained after downmix processing, and the audio signal A3 refers to the right channel audio signal obtained after downmix processing. The audio signal A2111, the audio signal A22111 and the audio signal A2311 can also be downmixed to obtain the audio signal A2 and the audio signal A3 by selectively retaining or removing certain frequency components through analysis of the loudness and spectral distribution of the audio signal. The embodiments of the present application are not limited in this regard.

[0209] S99, the post-gain module processes the audio signal A2 and the audio signal A3 to obtain the audio signal A4 and the audio signal A5, and plays the audio signal A4 and the audio signal A5 through the speaker of the electronic device.

[0210] It should be understood that the post-gain module is used to adjust the final volume of the audio signal obtained after S97 or S98 processing to ensure the consistency and balance of the output signal. The audio signal A4 can refer to the left channel audio signal obtained after processing by the post-gain module, and the audio signal A5 can refer to the right channel audio signal obtained after processing by the post-gain module.

[0211] In the implementation, the post-gain module can adjust the amplitude of the audio signal to the desired volume level. It can also control the dynamic range of the audio signal to improve the loudness without sacrificing the sound quality. The embodiments of the present application are not limited in this regard.

[0212] It should also be understood that the number of speakers of the electronic device is at least two, for example, when the number of speakers is two, one of them can play the audio signal A4 and the other can play the audio signal A5. For example, when the number of speakers is eight, four of them can play the audio signal A4 and the other four can play the audio signal A5.

[0213] In the embodiment of the present application, the audio algorithm module can process the dual-channel audio signal through the pre-emphasis gain module, which can improve the level of the audio signal to make it suitable for further processing or transmission. The audio algorithm module can also perform upmix processing on the dual-channel audio signal through the upmix or pass-through module to obtain a center channel audio signal, a left channel audio signal and a right channel audio signal, and then process the center channel audio signal through the center image control module to reduce the distance between the human voice included in the center channel audio signal and the user and improve the distance perception of the perceived human voice. The audio algorithm module can also process the left channel audio signal and the right channel audio signal after upmix processing through the first side image control module to widen the width of the side image and improve the spatial perception of the side image perceived by the user. The audio algorithm module can also eliminate crosstalk between the signals of each channel through the first crosstalk cancellation module to improve the audio quality. The audio algorithm module can also adjust the frequency components of the audio signal through the equalization module to improve the quality and effect of the sound. The audio algorithm module can also perform downmix processing on the audio signal through the downmix module to ensure that the audio signal can maintain good sound quality and balance in fewer channels. The audio algorithm module can also adjust the final volume of the audio signal through the post-emphasis gain module to ensure the consistency and balance of the output signal.

[0214] Please refer to FIG. 10 , FIG. 11 to FIG. 18 is a schematic diagram of a processing process of an audio algorithm module provided in the embodiment of the present application. FIG. 19 The numbers in the circles represent the steps of the processing process, which include S101 to S1011.

[0215] S101, the audio algorithm module obtains an audio signal B1 and inputs the audio signal B1 into a pre-emphasis gain module.

[0216] It should be understood that the audio signal B1 refers to an original audio signal that has not been processed by the audio algorithm module. The definition of the original audio signal can be referred to the above embodiment, which will not be described here again.

[0217] The difference between the audio signal B1 and the audio signal A1 in the above embodiment is that the channel type of the audio signal A1 is dual-channel, and the channel type of the audio signal B1 is multi-channel. The multi-channel audio signal B1 can refer to the 5.1 channel, 7.1 channel, 9.1 channel and the like mentioned in the above embodiment, and the embodiment of the present application is exemplarily described by taking the 5.1 channel.

[0218] S102, the pre-emphasis gain module processes the audio signal B1 to obtain an audio signal B2 and sends the audio signal B2 to an upmix or pass-through module.

[0219] The implementation process of this step can be referred to S92, which will not be described here again.

[0220] S103, the upmix or pass-through module determines the channel type of the audio signal B2, and after determining that the channel type of the audio signal B2 is multi-channel, sends the audio signal B21 to the mid image control module, sends the audio signal B22 and the audio signal B23 to the first side image control module, and sends the audio signal B24 and the audio signal B5 to the second side image control module.

[0221] It should be understood that the audio signal B1 can refer to a multi-channel audio signal including a mid channel signal, a left channel signal, a right channel signal, a left surround channel signal, and a right surround signal, etc., so that the audio signal B2 obtained after processing by S102 also includes a multi-channel audio signal. When the upmix or pass-through module determines the channel type of the audio signal B2, it can be determined that the channel type of the audio signal B2 is multi-channel.

[0222] For ease of description, the embodiments of the present application can refer to the mid channel signal as the audio signal B21, the left channel signal as the audio signal B22, the right channel signal as the audio signal B23, the left surround channel signal as the audio signal B24, and the right surround channel signal as the audio signal B25.

[0223] For the implementation of determining the channel type of the audio signal, please refer to S93, which will not be described here.

[0224] It should also be understood that after the upmix or pass-through module determines that the channel type of the audio signal B2 is multi-channel, the audio signal B2 can be passed through to the next module, i.e., the audio signal B21 is sent to the mid image control module, the audio signal B22 and the audio signal B23 are sent to the first side image control module, and the audio signal B24 and the audio signal B5 are sent to the second side image control module.

[0225] S104: The mid image control module processes the audio signal B21 to obtain the audio signal B211, and sends the audio signal B211 to the equalization module.

[0226] For the implementation of this step, please refer to the embodiments shown in the following FIG. 19 description, which will not be described here.

[0227] S105, the first side image control module processes the audio signal B22 and the audio signal B23 to obtain the audio signal B221 and the audio signal B231, and sends the audio signal B221 and the audio signal B231 to the first crosstalk cancellation module.

[0228] It should be understood that the first side image control module refers to a module for processing the left channel signal (audio signal B22) and the right channel signal (audio signal B23) in the multi-channel signal.

[0229] For the specific implementation of this step, refer to the following FIG. 9 The embodiments shown here will not be repeated.

[0230] S106, the second side image control module processes the audio signal B24 and the audio signal B25 to obtain the audio signal B241 and the audio signal B251, and sends the audio signal B241 and the audio signal B251 to the second crosstalk cancellation module.

[0231] It should be understood that the second side image control module refers to a module for processing the left surround channel signal (audio signal B24) and the right surround channel signal (audio signal B24) in the multi-channel signal.

[0232] It should also be understood that the embodiments of the present application are exemplarily described with the audio signal B1 being a 5.1 channel audio signal. When the audio signal B1 is a multi-channel audio signal other than the 5.1 channel, for example, when the audio signal B1 is a 7.1 channel audio signal, the 7.1 channel audio signal is additionally provided with a left rear surround channel signal and a right rear surround channel signal on the basis of the channel signals included in the 5.1 channel, and the redundant channel signals can also be processed by other side image control modules, for example, the left rear surround channel signal and the right rear surround channel signal are processed by a third side image control module to widen the width of the side image and improve the user's perception of the spatial sense of the side image.

[0233] For the specific implementation of this step, refer to the following FIG. 10 The embodiments shown here will not be repeated.

[0234] S107, the first crosstalk cancellation module processes the audio signal B221 and the audio signal B231 to obtain the audio signal B2211 and the audio signal B2311, and sends the audio signal B2211 and the audio signal B2311 to the equalization module.

[0235] It should be understood that the first crosstalk cancellation module is a module for processing the audio signal output by the first side image control module.

[0236] For the implementation of this step, refer to the implementation of S96 in the above embodiments, which will not be repeated here.

[0237] S108, the second crosstalk cancellation module processes the audio signal B241 and the audio signal B251 to obtain the audio signal B2411 and the audio signal B2511, and sends the audio signal B2411 and the audio signal B2511 to the equalization module.

[0238] It should be understood that the second crosstalk cancellation module is a module for processing the audio signal output by the second side image control module.

[0239] It should also be understood that when the audio signal B1 is a multi-channel audio signal other than 5.1 channels, for example, when the audio signal B1 is a 7.1 channel audio signal, the embodiments of the present application can also include a third side image control module and a third crosstalk cancellation module, the third crosstalk cancellation module being a module for processing the audio signal output by the third side image control module.

[0240] For the implementation of this step, reference can be made to the implementation of S96 in the above embodiments, which will not be repeated here.

[0241] S109, the equalization module processes the audio signal B211 to obtain an audio signal B2111, processes the audio signal B2211 to obtain an audio signal B22111, processes the audio signal B2311 to obtain B23111, processes the audio signal B2411 to obtain an audio signal B24111, processes the audio signal B2511 to obtain B25111, and sends the audio signal B2111, the audio signal B22111, the audio signal B23111, the audio signal B24111 and the audio signal B25111 to the downmix module.

[0242] For the implementation of this step, reference can be made to the implementation of S97 in the above embodiments, which will not be repeated here.

[0243] S1010, the downmix module processes the audio signal B2111, the audio signal B22111, the audio signal B23111, the audio signal B24111 and the audio signal B25111 to obtain an audio signal B2 and an audio signal B3, and sends the audio signal B2 and the audio signal B3 to the post-gain module.

[0244] For the implementation of this step, reference can be made to the implementation of S98 in the above embodiments, which will not be repeated here.

[0245] S1011, the post-gain module processes the audio signal B2 and the audio signal B3 to obtain an audio signal B4 and an audio signal B5, and plays the audio signal B4 and the audio signal B5 through the loudspeakers of the electronic device.

[0246] For the implementation of this step, reference can be made to the implementation of S99 in the above embodiments, which will not be repeated here.

[0247] It should be understood that the number of loudspeakers of the electronic device is at least 2, for example, when the number of loudspeakers is 2, one of them can be used to play the audio signal B4 and the other to play the audio signal B5. For example, when the number of loudspeakers is 8, four of them can be used to play the audio signal B4 and the other four to play the audio signal B5.

[0248] In the embodiments of the present application, the audio algorithm module can process the multi-channel audio signals through the pre-emphasis gain module, which can improve the level of the audio signals to make them suitable for further processing or transmission. The audio algorithm module can also process the center channel audio signals in the multi-channel through the center image control module, which can shorten the distance between the human voice included in the center channel audio signals and the user and improve the distance sense of the perceived human voice. The audio algorithm module can also process the left channel audio signals and the right channel audio signals in the multi-channel through the first side image control module and process the left surround channel audio signals and the right surround channel audio signals in the multi-channel through the second side image control module, which can widen the width of the side image and improve the spatial sense of the perceived side image. The audio algorithm module can also eliminate the crosstalk between the signals of each channel through the first crosstalk cancellation module and the second crosstalk cancellation module, which improves the audio quality. The audio algorithm module can also adjust the frequency components of the audio signals through the equalization module to improve the quality and effect of the sound. The audio algorithm module can also perform downmix processing on the audio signals through the downmix module, which can ensure that the audio signals can maintain good sound quality and balance in fewer channels. The audio algorithm module can also adjust the final volume of the audio signals through the post-emphasis gain module to ensure the consistency and balance of the output signals.

[0249] The following embodiments are described with reference to FIG. 11 to FIG. 17 and FIG. 18 The specific implementation process of the center image control module in the embodiments shown is introduced.

[0250] It should be understood that the center image control module mainly processes the audio signals through the distance filter, which can be obtained based on the pre-measured head-related transfer function or the head-related transfer function calculated in real time by the electronic device. In the following embodiments FIG. 11 In the specific implementation process of the center image control module introduced, the distance filter is obtained based on the pre-measured head-related transfer function. FIG. 11 In the specific implementation process of the center image control module introduced, the distance filter is obtained based on the head-related transfer function calculated in real time by the electronic device. The distance filter obtained based on the pre-measured head-related transfer function can be obtained in three ways, and the distance filter obtained based on the head-related transfer function calculated in real time by the electronic device can be obtained in one way. For ease of description, the four ways are referred to as way A, way B, way C, and way D in the embodiments of the present application.

[0251] Way A:

[0252] Please refer to FIG. 9 , FIG. 10is a schematic diagram of an implementation process of a mid sound image control module provided in an embodiment of the present application. The mid sound image control module determines a distance filter based on a pre-measured head-related transfer function and obtains a distance parameter based on a user operation in a user interface during processing of an audio signal. The implementation process in this embodiment includes S941 to S945.

[0253] S941, the mid sound image control module receives the mid channel audio signal sent by the upmix or pass-through module, and obtains a distance parameter 1.

[0254] It should be understood that the mid channel audio signal can refer to FIG. 2 The audio signal A21 shown can also refer to FIG. 6 The audio signal B21 shown can also refer to a mid channel audio signal in a multi-channel audio signal other than 5.1 channels, such as a mid channel audio signal in a 7.1 channel audio signal.

[0255] It should also be understood that the distance parameter 1 is used to indicate the distance between the audio perceived by the user and the user himself / herself, and the distance parameter 1 can also be referred to as an auditory distance. For example, the distance can refer to FIG. 7 D2 shown in (c) of FIG. 6. In implementation, the mid sound image control module can obtain the distance parameter 1 after the user adjusts the sliding bar corresponding to the distance parameter through the user interface, or use a default distance parameter as the distance parameter 1 when the user does not adjust the sliding bar corresponding to the distance parameter in the user interface. For example, the user can adjust the sliding bar corresponding to the distance parameter through the control 634 shown in (c) of FIG. 6, or adjust the sliding bar corresponding to the distance parameter through the control 744 shown in (d) of FIG. 7, or adjust the sliding bar corresponding to the distance parameter through the control 833 shown in (c) of FIG. 8. In response to the user operation of adjusting the distance parameter, the mid sound image control module can obtain the distance parameter 1. For another example: when the user enables the spatial audio function of the electronic device but does not adjust the distance parameter included in the spatial audio function, the distance parameter refers to a default distance parameter. For example, the user enables the spatial audio function through the control 631 shown in (c) of FIG. 6, but does not adjust the control 634, at this time, the default distance parameter can refer to the distance parameter corresponding to the middle position of the sliding bar shown in (c) of FIG. 6. FIG. 8 FIG. 6 FIG. 6 FIG. 1 FIG. 6

[0256] ​​​​​It can also be understood that the purpose of the middle sound image control module obtaining the distance parameter is to: based on the distance parameter, the middle sound channel audio signal is processed, which can shorten the distance between the included vocal sound in the middle sound channel audio signal and the user, and improve the distance sense of the perceived vocal sound. Therefore, the numerical value of the distance parameter corresponding to each position where the sliding bar in the user interface moves is less than the distance threshold, and the distance threshold can be FIG. 6 D1 shown in (c) in FIG. 1C. For example, FIG. 6 The leftmost position of the distance parameter corresponding to the sliding bar in (c) in FIG. 1C corresponds to a distance parameter less than the distance threshold D1.

[0257] It should also be understood that the user can further improve the distance sense of the perceived vocal sound by adjusting the distance parameter corresponding to the sliding bar, for example, the user can slide the distance parameter corresponding to the sliding bar from left to right in the user interface shown in (c) in FIG. 1C to further improve the distance sense of the perceived vocal sound. Therefore, FIG. 12 The numerical value of the distance parameter gradually decreases when the distance parameter sliding bar shown in (c) in FIG. 1C is slid from the left to the right. FIG. 12

[0258] S942, the middle sound image control module determines data set 1 and data set 2 corresponding to distance parameter 1 based on distance parameter 1, data set 1 includes a plurality of near-field head-related transfer functions corresponding to distance parameter 1, and data set 2 includes a plurality of far-field head-related transfer functions corresponding to distance parameter 1.

[0259] It should be understood that the head-related transfer function (Head-Related Transfer Function, HRTF) is a function that describes the frequency response characteristics of sound propagation to the human ear in three-dimensional space. It involves acoustic phenomena such as reflection, refraction, diffraction, and scattering of sound on the head, ears, shoulders, and other body parts. HRTF can help the listener determine the location of the sound source in space. The listener can perceive the distance of the sound source through characteristics such as sound attenuation and echo. The listener can determine the direction of the sound source through the difference between the sound in both ears. HRTF can be divided into near-field HRTF and far-field HRTF. Near-field HRTF describes the characteristics of low-frequency sound (usually below 1500Hz) propagating near the human head and pinna, which takes into account the effects of sound reflection, absorption, and diffraction around the head and pinna. Near-field HRTF is usually used to simulate sound near the head, such as whisper or very close sound sources. Far-field HRTF describes the characteristics of high-frequency sound (usually above 1500Hz) propagating away from the human head and pinna, which mainly considers the propagation attenuation and phase change of sound in the air, and less considers the direct influence of the head and pinna. Far-field HRTF is used to simulate sound propagating at a distance, such as distant conversations or environmental sounds. ​

[0260] The HRTF can be expressed by Formula Two as follows:

[0261]

[0262] wherein H(r, θ, Φ, f) represents the head-related transfer function, P(r, θ, Φ, f) represents the complex sound pressure at the center of the human head when the sound source is present, P0(r, θ) represents the complex sound pressure at the center of the human head when the human head is not present, r represents the distance between the sound source and the center of the human head, θ represents the azimuth angle, Φ represents the elevation angle, and f represents the frequency of the sound source. For example, refer to FIG. 12 , FIG. 12 is an example diagram of the spatial position of a sound source provided by an embodiment of the present application. FIG. 13 In the diagram, the sphere represents the human head, the position of point O is the center of the human head, the position of point S is the position of the sound source, and the coordinate system is with point O as the origin. The distance r, the azimuth angle θ, and the elevation angle Φ are as shown in FIG. 13 .

[0263] In the implementation, the center image control module can query data set 1 and data set 2 corresponding to the distance parameter 1 based on the distance parameter 1 in the pre-measured HRTF data set, to determine data set 1 and data set 2 corresponding to the distance parameter 1.

[0264] It should be understood that the pre-measured HRTF data set refers to the HRTF data set obtained by measuring a specific artificial head or a real person before the electronic device is shipped. The artificial head refers to a device that simulates the human ear to receive sound. The method of pre-measuring the HRTF data set is described below.

[0265] For example, first, a plurality of measuring bodies are placed at different distances in the direction 1 of the audio playback device. The measuring bodies can be real persons or artificial heads. The measuring bodies need to wear microphones, and the microphones are used to obtain sound pressure signals. The audio playback device is used to play the center channel audio signal.

[0266] It should be understood that, according to Formula Two of the HRTF, the HRTF is determined by parameters such as the distance r between the sound source and the center of the human head, the azimuth angle θ, and the elevation angle Φ. When measuring the HRTF, each of the above parameters needs to be changed to accurately measure the HRTF. For example, the distance r between the real person or the artificial head and the audio playback device is constantly changed. At each distance r, the azimuth angle needs to be changed from 0 degrees to 360 degrees, and at each distance r, the elevation angle needs to be changed from 0 degrees to 360 degrees. Thus, the engineering quantity and the calculation quantity are large. In order to reduce the engineering quantity and the calculation quantity, in the measurement, the measuring bodies are placed in the direction 1 of the audio playback device, that is, the elevation angle and the azimuth angle are kept unchanged, and only the distance between the sound source and the center of the human head is changed.

[0267] For example, refer to FIG. 13 , FIG. 13 is an example diagram of the audio playback device and the measurement bodies provided in an embodiment of the present application. FIG. 13 The measurement bodies shown in the figure are artificial heads, and there are multiple artificial heads including two, such as artificial head 1 and artificial head 2. Assuming that the position of the audio playback device is the coordinate origin, the artificial head 1 and the artificial head 2 can be placed in the direction 1 of the audio playback device, FIG. 13 The direction 1 shown is the positive direction of the X-axis of the coordinate system of the audio playback device, that is, the azimuth and elevation angles of the artificial head 1 and the audio playback device are the same as the azimuth and elevation angles of the artificial head 2 and the audio playback device, such as, from FIG. 13 It can be seen that the azimuth angles are both 90 degrees, and the elevation angles are both 0 degrees. The distance between the artificial head 1 and the audio playback device is r1, and the distance between the artificial head 2 and the audio playback device is r2.

[0268] It should also be understood that the HRTF is highly personalized, and the HRTF of each person is unique because it is related to the individual's anatomy. Therefore, when measuring, multiple real people or artificial heads can be placed at different distances from the audio playback device, and the number of real people or artificial heads is not limited by the embodiments of the present application. For example, 100 artificial heads are placed at the distance r1 shown, FIG. 14 100 artificial heads are placed at the distance r2 shown, and the HRTF is measured based on the 100 artificial heads, respectively. FIG. 14 100 artificial heads are placed at the distance r2 shown, and the HRTF is measured based on the 100 artificial heads, respectively.

[0269] It should also be understood that generally, when the distance r between the sound source and the center of the human head is less than 1.2m, the measured HRTF is the near-field HRTF, and when the distance r is greater than or equal to 1.2m, the measured HRTF is the far-field HRTF. Embodiments of the present application can place measurement bodies at multiple positions where r is less than 1.2m to measure the near-field HRTF. Measurement bodies can be placed at multiple positions where r is greater than or equal to 1.2m to measure the far-field HRTF.

[0270] It can also be understood that the HRTF data set measured by the embodiments of the present application is to process the center channel audio signal based on the data set to bring the included vocals in the center channel audio signal closer to the user, so when measuring, the audio playback device can play the center channel audio signal.

[0271] Secondly, based on the sound pressure signals obtained by the microphone worn by each measurement body, the HRTF data set A is determined.

[0272] In one implementation, for example, the sound pressure signals obtained by each measurement body can be substituted into the above Formula Two to determine the HRTF data set A.

[0273] Since the sound pressure signal obtained by each measurement body is substituted into the above formula two, the sound pressure signal needs to be measured twice and involves frequency domain division, therefore, in another implementation, the binaural impulse response (Head-Impulse Response, HRIR) can be obtained based on the sound pressure signal obtained by each measurement body, and then the HRTF is obtained through Fourier transform. The HRIR is a description of the HRTF in the time domain, and the HRIR and the HRTF are a Fourier transform pair of each other.

[0274] It should be understood that only the key steps in the above method of measuring the HRTF data set are introduced, and in other embodiments, in order to accurately measure the HRTF, other steps should also be taken to ensure the accuracy of the data, which are not limited by the embodiments of the present application. For example, the measurement should be carried out in an anechoic chamber, and the measurement body should be kept from being moved during the measurement.

[0275] For example, the HRTF data set A obtained by the above measurement method can refer to Table 1 below, for example:

[0276] Table 1

[0277]

[0278] In Table 1: For example: when measuring the near-field HRTF, the measurement body 1, the measurement body 2 and the measurement body 3 can be placed at a distance of 0.32m from the audio playback device, and the measured near-field HRTF is HRTF11, HRTF21 and HRTF31. The measurement body 4, the measurement body 5 and the measurement body 6 are placed at a distance of 0.6m from the audio playback device, and the measured near-field HRTF is HRTF41, HRTF51 and HRTF61. The measurement body 7, the measurement body 8 are placed at a distance of 0.72m from the audio playback device, and the measured near-field HRTF is HRTF71, HRTF81.

[0279] When measuring the far-field HRTF, the measurement body 1, the measurement body 2 and the measurement body 3 can be placed at a distance of 1.6m from the audio playback device, and the measured far-field HRTF is HRTF12, HRTF22 and HRTF32. The measurement body 4, the measurement body 5 and the measurement body 6 are placed at a distance of 1.5m from the audio playback device, and the measured far-field HRTF is HRTF42, HRTF52 and HRTF62. The measurement body 7, the measurement body 8 are placed at a distance of 1.2m from the audio playback device, and the measured far-field HRTF is HRTF72, HRTF82. The above HRTF data set A is only an exemplary description and cannot constitute a limitation on the HRTF data set A.

[0280] It should be noted that, for ease of subsequent calculations, when performing measurements, a near-field HRTF dataset obtained based on a near-field distance measurement needs to correspond to a far-field HRTF dataset obtained based on a far-field distance measurement. For example, in Table 1, the near-field HRTF dataset measured at a near-field distance of 0.32m corresponds to the far-field HRTF dataset measured at a far-field distance of 1.6m, the near-field HRTF dataset measured at a near-field distance of 0.6m corresponds to the far-field HRTF dataset measured at a far-field distance of 1.5m, and the near-field HRTF dataset measured at a near-field distance of 0.72m corresponds to the far-field HRTF dataset measured at a far-field distance of 1.2m.

[0281] It should also be understood that users can adjust the perceived distance between themselves and the audio by modifying the distance parameters in the user interface, for example, please refer to... FIG. 14 , FIG. 14 This is an example diagram of a user interface provided in an embodiment of this application. Assuming... FIG. 14 The slider corresponding to the distance parameters shown can be slid to three positions: the position corresponding to control 1410, the position corresponding to control 1420, and the position corresponding to control 1430. For example, the distance parameter corresponding to control 1410 is 0.6m, which means the user perceives the audio distance to themselves as 0.6m; the distance parameter corresponding to control 1420 is 0.4m, which means the user perceives the audio distance to themselves as 0.4m; and the distance parameter corresponding to control 1430 is 0.2m, which means the user perceives the audio distance to themselves as 0.2m. Of course, in other embodiments, the slider corresponding to the distance parameters can be slid to more or fewer positions than three, and this application embodiment does not limit this. FIG. 14 It can be seen that when a user slides the slider from left to right, the audio perceived by the user gets closer and closer to themselves.

[0282] In order to achieve the goal of passing FIG. 14 To adjust the perceived audio distance, it's necessary to map the distance parameter at each position on the distance parameter slider to the data in the HRTF dataset A mentioned above, thus obtaining the correspondence between HRTF dataset A and the distance parameters. For example, please refer to Table 2:

[0283] Table 2

[0284]

[0285] for example, FIG. 14The distance parameter corresponding to control 1410 in the interface shown is 0.6m. This 0.6m corresponds to the data in the fourth column of Table 2. For example, this 0.6m corresponds to the near-field distance of 0.72 and the far-field HRTF data of the near-field distance of 0.72 in Table 2, and the far-field HRTF data corresponding to the far-field distance of 1.2m. FIG. 14 The distance parameter corresponding to control 1420 in the interface shown is 0.4m, which corresponds to the data in the third column of Table 2. FIG. 15 The distance parameter corresponding to control 1430 in the interface shown is 0.2m, which corresponds to the data in the second column of Table 2.

[0286] It should be understood that the correspondence between HRTF dataset A and distance parameters can be stored in the memory of the electronic device. When the center-mounted audio-visual control module executes S942, the correspondence between HRTF dataset A and distance parameters can be retrieved from the memory. To save storage resources of the electronic device, the correspondence between HRTF dataset A and distance parameters can also be stored in a cloud server or a physical server. When the center-mounted audio-visual control module executes S942, the correspondence between HRTF dataset A and distance parameters can be retrieved from the cloud server or a physical server.

[0287] In implementation, the center-mounted audio-visual control module can determine data set 1 and data set 2 corresponding to distance parameter 1 based on the correspondence between distance parameter 1 and HRTF dataset A and distance parameters. For example: assuming distance parameter 1 is the distance between the user and the target location... FIG. 15 In the interface, the distance obtained by sliding the slider to the position of control 1420 is 0.4m. The center-mounted audio-visual control module can then use this 0.4m to query the HRTF datasets for the near-field and far-field distances corresponding to 0.4m in Table 2. For example, the HRTF datasets for the near-field distances corresponding to 0.4m in Table 2 are HRTF41, HRTF51, and HRTF61. In this embodiment, these near-field HRTF datasets can be referred to as Data Set 1. The HRTF datasets for the far-field distances corresponding to 0.4m in Table 2 are HRTF42, HRTF52, and HRTF62. In this embodiment, these far-field HRTF datasets can be referred to as Data Set 2.

[0288] S943, the central audio-visual control module performs mean processing on data set 1 to obtain mean 1; and performs mean processing on data set 2 to obtain mean 2.

[0289] It should also be understood that HRTF is highly personalized, and each person's HRTF is unique because it is related to individual anatomy. Therefore, in order to mitigate the influence of personalized factors, the embodiments of this application require averaging the data in datasets 1 and 2.

[0290] In an implementation, the mid sound image control module can substitute the data in the data set 1 into the following Formula Three to obtain the mean 1:

[0291]

[0292] wherein, refers to the mean 1, HRTF ni refers to any one of the near-field HRTFs in the data set 1, and N refers to the number of the data of the multiple near-field HRTFs in the data set 1. For example, assuming that the data set 1 includes the HRTF41, the HRTF51 and the HRTF61 in Table 1, the HRTF41, the HRTF51 and the HRTF61 and the number 3 can be substituted into the above formula to obtain the mean 1.

[0293] In an implementation, the mid sound image control module can substitute the data in the data set 2 into the following Formula Four to obtain the mean 2:

[0294]

[0295] wherein, refers to the mean 2, HRTF fi refers to any one of the far-field HRTFs in the data set 2, and N refers to the number of the data of the multiple far-field HRTFs in the data set 2. For example, assuming that the data set 2 includes the HRTF42, the HRTF52 and the HRTF62 in Table 1, the HRTF42, the HRTF52 and the HRTF62 and the number 3 can be substituted into the above formula to obtain the mean 2.

[0296] S944, the mid sound image control module obtains the distance filter 1 based on the mean 1 and the mean 2.

[0297] In an implementation, the mid sound image control module substitutes the following Formula Five to obtain the distance filter 1:

[0298]

[0299] wherein, DF refers to the distance filter, refers to the mean 1, refers to the mean 2.

[0300] It should be understood that the distance filter is obtained based on the pre-measured HRTF data set A, and noise can be generated in the measurement process. In order to avoid the influence of the noise, after the distance filter 1 is obtained through S944, the distance filter 1 can be denoised, such as smoothing the distance filter 1, etc. The embodiments of the present application are not limited thereto, which can weaken the noise color of the waveform graph of the distance filter 1, and retain the main wave peaks and wave troughs. The noise color refers to the power spectrum of the noise.

[0301] For example, the mid sound image control module can perform denoising processing on the distance filter 1 through the following formula six:

[0302] DF ‘ = smooth (DF).

[0303] Wherein, DF ‘ denotes the distance filter 1 after denoising processing, and DF denotes the distance filter.

[0304] For example, the waveform diagram of the distance filter 1 can refer to FIG. 9 . FIG. 10 The horizontal coordinate represents the frequency, and the vertical coordinate represents the amplitude.

[0305] S945, the mid sound image control module performs convolution processing on the mid channel audio signal based on the distance filter 1 to obtain the audio signal A211 or B211.

[0306] It should be understood that assuming that the mid sound image control module receives the mid channel audio signal shown as the audio signal A21, then after the execution of S945, the mid sound image control module outputs the audio signal A211. Assuming that the mid sound image control module receives the mid channel audio signal shown as the audio signal B21, then after the execution of S945, the mid sound image control module outputs the audio signal B211. FIG. 14 FIG. 14

[0307] In implementation, the mid sound image control module can perform convolution processing on the distance filter 1 and the mid channel audio signal through segmented convolution, fast convolution, uniform segmentation convolution, etc. to obtain the audio signal A211 or the audio signal B211, and the application embodiment does not limit the convolution manner.

[0308] It should also be understood that the HRTF can not only simulate the direction of the sound, but also simulate the distance of the sound, i.e. the propagation distance of the sound, which helps to create a more realistic auditory experience. Based on the above characteristics of the HRTF, by performing convolution on the distance filter 1 (as known from formula five in the above embodiment, the distance filter is essentially the ratio of the average of the near-field HRTF to the average of the far-field HRTF) and the mid channel audio signal, the distance between the mid channel audio signal and the user can be made closer, and the distance perception of the human voice included in the mid channel audio signal perceived by the user can be improved.

[0309] ​​In this embodiment, the center channel audio control module can obtain distance parameter 1 through the user interface. Then, based on distance parameter 1 and the correspondence between the HRTF dataset A obtained based on the pre-measured head correlation transfer function and the distance parameter, distance filter 1 is determined. By convolving distance filter 1 with the center channel audio signal, the distance between the center channel audio signal and the user can be made closer, thereby improving the user's perception of the distance of human voices included in the center channel audio signal.

[0310] Example 1: In related solutions, the distance of the center channel audio signal perceived by the user is approximately equal to the distance between the electronic device and the user. For example, assuming the near-field distance of 0.72m in Table 2 is a distance threshold, this distance threshold represents the distance between the electronic device and the user. In this application embodiment, the user can... FIG. 14 The slider in the user interface is shown to be moved to position 1410. After that, the center channel imaging control module can process the center channel audio signal based on steps S941 to S945. In S941, the distance parameter 1 is 0.6m. In S942, the data set 1 is the near-field HRTF corresponding to the near-field distance of 0.72m, and the data set 2 is the far-field HRTF of 1.2m. At this time, the electronic device considers the near-field distance of 0.72m corresponding to distance parameter 1 of 0.6m as the distance between itself and the user. In the related scheme, the distance of the center channel audio signal perceived by the user is approximately equal to 0.72m. However, in this embodiment, after the center channel audio signal is convolved by distance filter 1 in S945, since distance filter 1 is based on the ratio of the mean of the near-field HRTF corresponding to 0.72m to the mean of the far-field HRTF corresponding to 1.2m, the distance of the center channel audio signal perceived by the user is approximately equal to 0.72 / 1.2 = 0.6m. Thus, after the center channel audio control module processes the center channel audio signal through S945, compared with the related scheme (the distance of the center channel audio signal perceived by the user in the related scheme is 0.72m, while in this application it is 0.6m), the distance between the center channel audio signal and the user can be reduced, improving the sense of audio distance.

[0311] In this embodiment of the application, users can further enhance the sense of distance of the center channel audio signal by adjusting the position of the slider in the user interface.

[0312] Example 2: When the user will FIG. 14 After the slider shown is moved to position 1410, assuming the user wants to further enhance the sense of distance in the center channel audio signal, they can... FIG. 14 The slider shown is moved to position 1420 or 1430. For example, the user will... FIG. 14When the slider moves to 1420, the distance parameter 1 obtained in S941 is 0.4 m, and the data set 1 determined in S942 is the near-field HRTF corresponding to the near-field distance 0.6 m and the far-field HRTF corresponding to the far-field distance 1.5 m. After the middle sound image control module convolves the middle sound channel audio signal based on the distance filter 1 in S945, since the distance filter 1 is obtained based on the ratio of the average of the near-field HRTF corresponding to 0.6 m to the average of the far-field HRTF corresponding to 1.5 m, the distance of the middle sound channel audio signal perceived by the user is approximately equal to 0.6 / 1.5=0.4 m. In Example One, the user will FIG. 16 When the slider moves to 1410, the distance of the middle sound channel audio signal perceived by the user is approximately equal to 0.72 / 1.2=0.6 m, and in Example Two, the user will FIG. 16 When the slider moves to 1420, the distance of the middle sound channel audio signal perceived by the user is approximately equal to 0.6 / 1.5=0.4 m, and the distance between the middle sound channel audio signal and the user can be further shortened, improving the distance sense of the audio.

[0313] Method B:

[0314] For reference FIG. 11 , FIG. 11 is another implementation process of the middle sound image control module provided by the embodiment of the present application. It is illustrated that in the process of processing the audio signal, the distance filter determined by the middle sound image control module is obtained based on the pre-measured head-related transfer function, and the distance parameter obtained by the middle sound image control module is detected by the sensor. The implementation process in this embodiment includes S946 to S9410.

[0315] S946, the middle sound image control module receives the middle sound channel audio signal sent by the upmixing or transparent transmission module, and obtains the distance parameter 2.

[0316] The meaning of the middle sound channel audio signal has been explained in the above embodiment and will not be repeated here.

[0317] It should be understood that the distance parameter 2 is used to indicate the distance between the position of the electronic device and the user.

[0318] In implementation, the middle sound image control module can call the sensor to detect the distance between the user and the electronic device to obtain the distance parameter 2. The sensor can be a distance sensor or a camera.

[0319] S947, the middle sound image control module determines the data set 3 and the data set 4 corresponding to the distance parameter 2 based on the distance parameter 2, the data set 3 includes a plurality of near-field head-related transfer functions corresponding to the distance parameter 2, and the data set 4 includes a plurality of far-field head-related transfer functions corresponding to the distance parameter 1.

[0320] In an implementation, the middle sound image control module determines the data set 3 and the data set 4 corresponding to the distance parameter 2 based on the distance parameter 2 and the HRTF data set A mentioned in the above embodiment. The HRTF data set A is obtained based on a pre-measured head-related transfer function, and the data in the HRTF data set A can refer to the data in Table 1 in the above embodiment.

[0321] It should be understood that the near-field HRTF data set in Table 1 based on one near-field distance measurement corresponds to the far-field HRTF data set based on one far-field distance measurement. For example, in Table 1, the near-field HRTF data set measured based on the near-field distance 0.32 m corresponds to the far-field HRTF data set measured based on the far-field distance 1.6 m, the near-field HRTF data set measured based on the near-field distance 0.6 m corresponds to the far-field HRTF data set measured based on the far-field distance 1.5 m, and the near-field HRTF data set measured based on the near-field distance 0.72 m corresponds to the far-field HRTF data set measured based on the far-field distance 1.2 m. Thus, in an implementation, the distance parameter 2 can be queried in Table 1 to determine whether the distance parameter 2 is a near-field distance or a far-field distance. If it is determined that the distance parameter 2 is a near-field distance, the HRTF data set of the near-field distance is taken as the data set 3, and then the HRTF data set of the far-field distance corresponding to the near-field distance is taken as the data set 4. If it is determined that the distance parameter 2 is a far-field distance, the HRTF data set of the far-field distance is taken as the data set 4, and then the HRTF data set of the near-field distance corresponding to the far-field distance is taken as the data set 3. For example, when the distance parameter 2 is 1.2 m, it is determined through Table 1 that the distance parameter 2 is a far-field distance, and then the far-field HRTF data set of the far-field distance 1.2 m is taken as the data set 4, and the near-field HRTF data set of the near-field distance 0.72 m corresponding to the far-field distance 1.2 m is taken as the data set 3.

[0322] In another implementation, for example, the sound image control module first determines whether the distance parameter 2 is greater than or equal to a threshold 1 (for example, the threshold 1 can be 1.2 m). Assuming that the distance parameter 2 is greater than or equal to the threshold 1, the distance parameter 2 is determined as a far-field distance, and then the data set of the far-field HRTF corresponding to the distance parameter 2 is queried in the HRTF data set A (Table 1), and the data set is determined as data set 4. For example, the distance parameter 2 is 1.2 m which is greater than the threshold 1, and the far-field HRTF data set corresponding to 1.2 m is HRTF72 and HRTF82, which is determined as data set 4. In addition, the data set of the near-field HRTF corresponding to the near-field distance of the far-field HRTF data set (data set 4) is queried in the HRTF data set A (Table 1), and the data set is determined as data set 3. For example, the near-field distance corresponding to 1.2 m is 0.72 m, and the near-field HRTF data set of the near-field distance 0.72 m includes HRTF41, HRTF51 and HRTF61.

[0323] Assuming that the distance parameter 2 is less than the threshold 1, the data set of the near-field HRTF corresponding to the distance parameter A1 is queried in Table 1, and the data set is determined as data set 1. Then the distance parameter 2 is determined as a near-field distance, and then the data set of the near-field HRTF corresponding to the distance parameter 2 is queried in the HRTF data set A (Table 1), and the data set is determined as data set 3. For example, the distance parameter 2 is 0.72 m which is less than the threshold 1, and the near-field HRTF corresponding to 0.72 m is queried from Table 1. The near-field HRTF corresponding to 0.72 m can be referred to as data set 1 in the present application embodiment, for example, data set 1 includes HRTF71 and HRTF81 in Table 1. In addition, the far-field distance corresponding to the near-field HRTF data set (data set 3) and the data set of the far-field HRTF of the far-field distance are queried in Table 1, and the far-field HRTF data set is determined as data set 4. For example, the far-field distance corresponding to 0.72 m in Table 1 is 1.2 m, and the far-field HRTF data set of 1.2 m includes HRTF42, HRTF52 and HRTF62.

[0324] It should be understood that the HRTF data set A can be stored in the memory of the electronic device, and the HRTF data set A can be called from the memory when the sound image control module performs S947. In order to save the storage resources of the electronic device, the HRTF data set A can be stored in a cloud server or a server, and the HRTF data set A can be called from the cloud server or the server when the sound image control module performs S947.

[0325] S948, the central audio-visual control module performs mean processing on data set 3 to obtain mean 3; and performs mean processing on data set 4 to obtain mean 4.

[0326] The implementation method for this step can be referred to the above embodiment, and will not be repeated here.

[0327] S949, the center-mounted audio-visual control module obtains distance filter 2 based on mean 3 and mean 4.

[0328] The implementation method for this step can be referred to the above embodiment, and will not be repeated here.

[0329] S9410, the center channel audio control module performs convolution processing on the center channel audio signal based on distance filter 2 to obtain audio signal A211 or B211.

[0330] The implementation method for this step can be referred to the above embodiment, and will not be repeated here.

[0331] In this embodiment, the center channel imaging control module can acquire distance parameter 2 through a sensor. Then, based on distance parameter 2 and the HRTF dataset A obtained from a pre-measured head correlation transfer function, a distance filter 2 is determined. By convolving the distance filter 2 with the center channel audio signal, the distance between the center channel audio signal and the user can be made closer, improving the user's perceived distance of the human voice included in the center channel audio signal. Furthermore, relative to... FIG. 14 The illustrated embodiment ( FIG. 17 In the illustrated embodiment, the audio distance perceived by the user is: for example FIG. 17 The distance parameter is shown at each position on the slider corresponding to the distance parameter in the user interface. Since the distance parameter 2 in this embodiment refers to the distance between the electronic device and the user detected by the sensor, the center-mounted audio-visual control module in this embodiment can locate the user's position (which can be quantified as distance parameter 2) and accurately adjust the distance of the human voice based on the located user's position, thereby improving the user experience.

[0332] For example, in the related scheme, the distance of the middle channel audio signal perceived by the user is approximately equal to the distance between the electronic device and the user. For example, assuming that the distance between the electronic device and the user is 0.72 m, the distance of the middle channel audio signal perceived by the user is approximately equal to 0.72 m. In an embodiment of the present application, when the distance between the electronic device and the user is 0.72 m, the middle image control module can locate the position of the user as the distance parameter 2 (0.72 m) through the sensor in S946. Then, the middle image control module can process the middle channel audio signal based on the steps of S947 to S9410. In S947, the determined data set 3 is the near-field HRTF data set of the near-field distance 0.72 m in Table 1, and the data set 4 is the data set of the far-field HRTF of the far-field distance 1.2 m. After the convolution processing of the middle channel audio signal based on the distance filter 2 in S945, since the distance filter 2 is obtained based on the ratio of the average of the near-field HRTF corresponding to 0.72 m to the average of the far-field HRTF corresponding to 1.2 m, the distance of the middle channel audio signal perceived by the user is approximately equal to 0.72 / 1.2 = 0.6 m. Thus, after the processing of the middle channel audio signal by the middle image control module through S9410, the distance between the middle channel audio signal and the user can be shortened compared with the related scheme (in the related scheme, the distance of the middle channel audio signal perceived by the user is approximately equal to 0.72 m, and in the present application, it is 0.6 m), thereby improving the distance sense of the audio.

[0333] When the distance between the user and the electronic device changes, for example, the distance between the user and the electronic device is reduced from 0.72 m to 0.6 m, the distance of the center channel audio signal perceived by the user in the related solution is approximately equal to 0.6 m, and the center image control module in the embodiment of the present application can locate the position of the user as the distance parameter 2 (0.6 m) in S946. Then, the center image control module can process the center channel audio signal based on the steps of S947 to S9410. In S947, the determined data set 3 is the near-field HRTF data set of the near-field distance 0.6 m in Table 1, and the data set 4 is the data set of the far-field HRTF of the far-field distance 1.5 m. After the convolution processing of the center channel audio signal based on the distance filter 2 in S945, since the distance filter 2 is obtained based on the ratio of the average of the near-field HRTF corresponding to 0.6 m to the average of the far-field HRTF corresponding to 1.5 m, the distance of the center channel audio signal perceived by the user is approximately equal to 0.6 / 1.5=0.4 m. Thus, after the center image control module processes the center channel audio signal through S9410, the distance between the center channel audio signal and the user can be pulled closer compared with the related solution (in the related solution, the distance of the center channel audio signal perceived by the user is approximately equal to 0.6 m, and in the present application, it is 0.4 m), thereby improving the distance sense of the audio. Moreover, when the distance between the user and the electronic device changes, the distance of the center channel audio signal can be accurately adjusted based on the repositioned position of the user (for example, the distance between the user and the electronic device is reduced from 0.72 m to 0.6 m, the user is at a distance of 0.72 m from the electronic device, and the perceived distance of the center channel audio signal is 0.6 m, and the user is at a distance of 0.6 m from the electronic device, and the perceived distance of the center channel audio signal is 0.4 m), thereby improving the user experience.

[0334] Mode C:

[0335] Please refer to FIG. 11 , FIG. 11 is another schematic diagram of an implementation process of a center image control module provided in the embodiment of the present application. It is shown that in the process of processing the audio signal by the center image control module, the distance filter determined by the center image control module is obtained based on the pre-measured head-related transfer function, and the distance parameter obtained by the center image control module includes two parts, one part is detected by a sensor, and the other part is obtained based on the operation of the user in the user interface. The implementation process in this embodiment includes S9411 to S9415.

[0336] In S9411, the center image control module receives the center channel audio signal sent by the upmix or transparent transmission module, and obtains the distance parameter 1 and the distance parameter 2.

[0337] It should be understood that the distance parameter 1 is used to indicate the distance of the audio perceived by the user from the user himself. The distance parameter 2 is used to indicate the distance between the position of the electronic device and the user. The manner of obtaining the distance parameter 1 and the distance parameter 2 can refer to the above embodiments, which will not be described here.

[0338] In S9412, the sound image control module determines whether the distance parameter 2 is greater than a threshold 1. In a case where it is determined that the distance parameter 2 is greater than or equal to the threshold 1, the distance parameter 2 is determined as a far-field distance. Then, based on the distance parameter 2 and the correspondence between the HRTF data set A and the distance parameter, the data set 5 is determined, and based on the distance parameter 1, the data set 6 is determined. In a case where the distance parameter 2 is less than the threshold 1, the distance parameter 2 is determined as a near-field distance. Then, based on the distance parameter 2 and the correspondence between the HRTF data set A and the distance parameter, the data set 6 is determined, and based on the distance parameter 1, the data set 5 is determined. The data set 5 includes a plurality of near-field head-related transfer functions corresponding to the distance parameter 2, and the data set 6 includes a plurality of far-field head-related transfer functions corresponding to the distance parameter 1.

[0339] It should be understood that the correspondence between the HRTF data set A and the distance parameter can refer to the data in Table 2 above, which will not be described here. The threshold 1 refers to a threshold for distinguishing between the near-field distance and the far-field distance. The distance greater than or equal to the threshold 1 is the far-field distance, and the distance less than the threshold 1 is the near-field distance.

[0340] In implementation, for example, after obtaining the distance parameter 1 and the distance parameter 2, the middle sound image control module determines whether the distance parameter 2 is greater than or equal to a threshold 1 (for example, the threshold 1 can be 1.2 m). Assuming that it is determined that the distance parameter 2 is greater than or equal to the threshold 1, the distance parameter 2 is determined as a far field distance. Then, the data set of the far field HRTF corresponding to the distance parameter 2 is queried in Table 2, and the data set is determined as data set 6. For example, the distance parameter 2 obtained by the middle sound image control module is 1.2 m, which is greater than the threshold 1. The far field HRTF corresponding to 1.2 m is queried in Table 2. In this application, the far field HRTF corresponding to 1.2 m can be referred to as data set 6. For example, data set 6 includes HRTF72 and HRTF82 in Table 2. Then, the data set of the near field HRTF corresponding to the distance parameter 1 is queried in Table 2, and the data set is determined as data set 5. For example, the distance parameter 1 obtained by the middle sound image control module is 0.6 m. The near field distance corresponding to the distance parameter 0.6 m is 0.72 m, which is queried in Table 2. The near field HRTF corresponding to the near field distance 0.72 m is referred to as data set 5. For example, data set 5 includes HRTF71 and HRTF81 in Table 2. Of course, when the distance parameter 2 obtained by the middle sound image control module is 1.2 m, the distance parameter 1 obtained may also be 0.2 m or 0.4 m shown in Table 1. Then, data set 5 can be determined based on the near field distance 0.32 m corresponding to the distance parameter 0.2 m or the near field distance 0.6 m corresponding to the distance parameter 0.4 m.

[0341] Assuming that it is determined that the distance parameter 2 is less than the threshold 1, the distance parameter 2 is determined as a near field distance. Then, the data set of the near field HRTF corresponding to the distance parameter 2 is queried in Table 2, and the data set is determined as data set 6. For example, the distance parameter 2 is 0.72 m, which is less than the threshold 1. The near field HRTF corresponding to 0.72 m is queried in Table 2. In this application, the near field HRTF corresponding to 0.72 m can be referred to as data set 5. For example, data set 5 includes HRTF71 and HRTF81 in Table 2. Then, the data set of the far field HRTF corresponding to the distance parameter 1 is queried in Table 2, and the data set is determined as data set 6. For example, the distance parameter 1 obtained by the middle sound image control module is 0.6 m. The far field distance corresponding to the distance parameter 0.6 m is 1.2 m, which is queried in Table 2. The far field HRTF corresponding to the far field distance 1.2 m is referred to as data set 6. For example, data set 6 includes HRTF72 and HRTF82 in Table 2. Of course, when the distance parameter 2 obtained by the middle sound image control module is 1.2 m, the distance parameter 1 obtained may also be 0.2 m or 0.4 m shown in Table 1. Then, data set 5 can be determined based on the far field distance 0.32 m corresponding to the distance parameter 0.2 m or the far field distance 0.6 m corresponding to the distance parameter 0.4 m.

[0342] S9413, the middle sound image control module performs mean value processing on the data set 5 to obtain a mean value 5, and performs mean value processing on the data set 6 to obtain a mean value 6.

[0343] The implementation of this step can refer to the foregoing embodiments, and details are not described herein.

[0344] S9414, the middle sound image control module obtains the distance filter 3 based on the mean value 5 and the mean value 6.

[0345] The implementation of this step can refer to the foregoing embodiments, and details are not described herein.

[0346] S9415, the middle sound image control module performs convolution processing on the middle sound channel audio signal based on the distance filter 3 to obtain the audio signal A211 or B211.

[0347] The implementation of this step can refer to the foregoing embodiments, and details are not described herein.

[0348] In the embodiments of the present application, the middle sound image control module can obtain the distance parameter 2 through the sensor, obtain the distance parameter 1 through the user interface, and then determine the distance filter 3 based on the distance parameter 1, the distance parameter 2, and the correspondence between the HRTF data set A and the distance parameter based on the pre-measured head-related transfer function. Convolution of the distance filter 3 and the middle sound channel audio signal can make the distance between the middle sound channel audio signal and the user closer, and improve the distance perception of the human voice included in the middle sound channel audio signal perceived by the user. And relative to FIG. 14 The embodiments shown in the figure FIG. 16 In the embodiments shown in the figure, the audio distance perceived by the user is, for example FIG. 14 In the user interface shown in the figure, each position on the sliding bar corresponding to the distance parameter, because the distance parameter 2 in the embodiments of the present application refers to the distance between the electronic device and the user detected through the sensor, the middle sound image control module in the embodiments of the present application can locate the position of the user (which can be quantified as the distance parameter 2), and accurately adjust the distance of the middle sound channel audio signal based on the located position of the user, thereby improving the user experience. And relative to FIG. 14 In the embodiments shown in the figure, in the embodiments of the present application, the user can also adjust the distance parameter 1 by adjusting the position of the sliding bar in the user interface to further improve the distance perception of the middle sound channel audio signal based on the adjusted distance parameter 1.

[0349] Example three, in the related scheme, the distance of the middle channel audio signal perceived by the user is approximately equal to the distance between the electronic device and the user. For example, assuming that the distance between the electronic device and the user is 0.72 m, the distance of the middle channel audio signal perceived by the user is approximately equal to 0.72 m. In an embodiment of the present application, when the distance between the electronic device and the user is 0.72 m, the middle image control module can locate the position of the user as the distance parameter 2 (0.72 m) through the sensor in S911, and the distance parameter 1 is assumed to be FIG. 14 The control 1410 shown corresponds to the distance parameter 0.6 m. Then, in S9412, the middle image control module determines the data set 5 based on the distance parameter 2 as the near-field HRTF data set of the near-field distance 0.72 m in Table 2, and determines the data set 6 based on the distance parameter 1 as the far-field HRTF data set of the far-field distance 1.2 m in Table 2. After the convolution processing of the middle channel audio signal based on the distance filter 3 in S9415, since the distance filter 3 is obtained based on the ratio of the average of the near-field HRTF corresponding to 0.72 m to the average of the far-field HRTF corresponding to 1.2 m, the distance of the middle channel audio signal perceived by the user is approximately equal to 0.72 / 1.2 = 0.6 m. In this way, after the middle image control module processes the middle channel audio signal through S9415, the distance between the middle channel audio signal and the user can be shortened compared with the related scheme (in the related scheme, the distance of the middle channel audio signal perceived by the user is 0.72 m, and in the present application, it is 0.6 m), thereby improving the distance sense of the audio.

[0350] In an embodiment of the present application, the user can further improve the distance sense of the middle channel audio signal by adjusting the position of the sliding bar in the user interface. Example four, after the user moves the sliding bar to the position of 1410 FIG. 14 After the sliding bar shown is moved to the position of 1410, the distance sense of the audio is improved through the above example three, and assuming that the user wants to further improve the distance sense of the middle channel audio signal, the sliding bar shown can be moved to the position of 1420 or 1430. For example, the user moves the sliding bar to the position of 1420 FIG. 14 After the sliding bar shown is moved to the position of 1410, the distance sense of the audio is improved through the above example three, and assuming that the user wants to further improve the distance sense of the middle channel audio signal, the sliding bar shown can be moved to the position of 1420 or 1430. For example, the user moves the sliding bar to the position of 1420 FIG. 18After the sliding bar is moved to 1420, the distance parameter 2 obtained by the mid sound image control module is still 0.72 m in the above example three, and the distance parameter 1 is changed from 0.6 m to 0.4 m. Then, the data set 5 determined by the mid sound image control module based on the distance parameter 2 in S9412 is the near-field HRTF data set of the near-field distance 0.72 m in Table 2, and the data set 6 determined based on the distance parameter 1 is the far-field HRTF data set of the far-field distance 1.5 m in Table 2. After the convolution processing of the mid channel audio signal based on the distance filter 3 in S9415, since the distance filter 3 is obtained based on the ratio of the average of the near-field HRTF corresponding to 0.72 m to the average of the far-field HRTF corresponding to 1.5 m, the distance of the mid channel audio signal perceived by the user is approximately equal to 0.72 / 1.5=0.48 m. Thus, after the mid sound image control module processes the mid channel audio signal in S9415, the distance of the mid channel audio signal to the user can be further reduced compared to the above example three (the distance of the mid channel audio signal perceived by the user in the above example three is 0.6 m, and the distance of the mid channel audio signal perceived by the user in the above example four is 0.48 m), thereby improving the distance sense of the audio.

[0351] The embodiments of the present application can also accurately adjust the distance of the mid channel audio signal based on the position of the located user. In example five, when the distance between the user and the electronic device changes, for example, the distance between the user and the electronic device is reduced from 0.72 m in example three to 0.6 m, in the related scheme, the distance of the mid channel audio signal perceived by the user is 0.6 m. In the embodiments of the present application, the mid sound image control module can locate the position of the user as the distance parameter 2 (0.6 m) in S911, and the obtained distance parameter 1 is assumed to be FIG. 18The control 1420 corresponds to the distance parameter 0.4 m. Then, the center image control module determines, in S9412, the data set 5 based on the distance parameter 2 as the near-field HRTF data set of the near-field distance 0.6 m in Table 2, and the data set 6 based on the distance parameter 1 as the far-field HRTF data set of the far-field distance 1.5 m in Table 2. After the convolution processing of the center channel audio signal based on the distance filter 3 in S9415, since the distance filter 3 is obtained based on the ratio of the average of the near-field HRTF corresponding to 0.6 m to the average of the far-field HRTF corresponding to 1.5 m, the distance of the center channel audio signal perceived by the user is approximately equal to 0.6 / 1.5=0.4 m. Thus, after the processing of the center channel audio signal by the center image control module in S9415, the distance of the center channel audio signal relative to the user can be shortened compared to the related solution (in the related solution, the distance of the center channel audio signal perceived by the user is 0.6 m, and in this application, it is 0.4 m), thereby improving the distance sense of the audio. Moreover, when the distance between the user and the electronic device changes, the distance of the center channel audio signal can be accurately adjusted based on the repositioned position of the user (for example, the distance between the user and the electronic device is reduced from 0.72 m to 0.6 m, in Example Three, the user is at a distance of 0.72 m from the electronic device, and in Example Five, the user is at a distance of 0.6 m from the electronic device, and the perceived distance of the center channel audio signal is 0.4 m), thereby improving the user experience.

[0352] Mode D:

[0353] Please refer to FIG. 11 , FIG. 16 is another schematic diagram of an implementation process of a center image control module provided in an embodiment of the present application. It is shown that in the process of processing the audio signal by the center image control module, the distance filter determined by the center image control module is obtained based on the head-related transfer function calculated by the electronic device in real time, and the distance parameter obtained by the center image control module is obtained based on the operation of the user in the user interface. The implementation process in this embodiment includes S9416 to S9419.

[0354] S9416, the center image control module receives the center channel audio signal sent by the upmix or transparent transmission module, and obtains the distance parameter 2.

[0355] It should be understood that the distance parameter 2 is used to indicate the distance between the position of the electronic device and the user.

[0356] S9417, the center image control module determines the distance parameter 3 based on the distance parameter 2, and the value of the distance parameter 3 is less than the distance parameter 2.

[0357] In implementation, after obtaining the distance parameter 2, the middle sound image control module can determine any value less than the distance parameter 2 as the distance parameter 3.

[0358] S9418, the middle sound image control module determines the distance filter 4 based on the distance parameter 3, the azimuth angle 1 and the elevation angle 1.

[0359] It should be understood that the azimuth angle refers to the azimuth angle of the sound source (audio signal) relative to the user, and the elevation angle refers to the elevation angle of the sound source (audio signal) relative to the user. The explanation of the azimuth angle and the elevation angle can refer to the above embodiments, which will not be repeated here.

[0360] It should also be understood that the azimuth angle 1 and the elevation angle 1 refer to the preset azimuth angle and the preset elevation angle. The middle sound image control module processes the audio signal played by the middle channel. The middle channel is part of the audio system of the electronic device. The middle channel is usually used to play dialogue and other important central sound elements. The middle channel is usually user-oriented. Therefore, based on the above characteristics of the middle channel, the preset azimuth angle and the preset elevation angle can be set so that the audio played by the middle channel of the electronic device is user-oriented. For example, the preset azimuth angle 1 can be around 90 degrees, such as 80 degrees to 100 degrees, and can be 90 degrees. The elevation angle is around 0 degrees, such as -10 degrees to 10 degrees, and can be 0 degrees.

[0361] In implementation, the middle sound image control module can solve the wave equation based on the distance parameter 3, the azimuth angle 1 and the elevation angle 1, that is, the distance filter 4 can be obtained.

[0362] It should be understood that the distance filter is essentially an HRTF. The distance filter 4 can be understood as the HRTF corresponding to the distance parameter 3. As can be seen from Formula Two of the HRTF in the above embodiments, the HRTF is determined by parameters such as distance r, azimuth angle θ and elevation angle Φ. Therefore, the embodiments of the present application determine the distance filter by parameters such as distance r, azimuth angle θ and elevation angle Φ. Among them, the distance r can be the distance parameter 3, and the azimuth angle and the elevation angle can be preset values.

[0363] It should also be understood that the middle sound image control module essentially determines the HRTF corresponding to the distance parameter 3 by determining the distance filter 4 in real time. Real-time determination of the HRTF usually involves a series of complex steps. In the embodiments of the present application, only some key steps are described. In other embodiments, real-time determination of the HRTF also needs to establish a user model based on the geometric parameters of the user's head, torso, auricle, define the boundary conditions and initial conditions of the model, and perform frequency domain analysis and parameterized scanning. The way of real-time determination of the HRTF in the embodiments of the present application is not limited.

[0364] It can also be understood that FIG. 17 , FIG. 9 and FIG. 10The distance filter in the corresponding embodiment is obtained based on a pre-measured head-related transfer function. The distance filter in the embodiment of the application is obtained based on a head-related transfer function calculated in real time by the electronic device. The difference lies in that the way of obtaining the distance filter based on the pre-measured head-related transfer function does not require the electronic device to consume much computing resource to obtain the distance filter, but the pre-measured head-related transfer function fails to fully consider the influence of the individualized characteristics of the user (such as the geometric parameters of the head, pinna, and torso of the user) on the measurement result, so that the obtained distance filter is relatively incorrect. Although the distance filter obtained based on the head-related transfer function calculated in real time by the electronic device is relatively accurate, it requires higher computing power of the electronic device and has a larger amount of calculation.

[0365] S9419, the mid sound image control module convolves the mid sound channel audio signal based on the distance filter 4 to obtain the audio signal A211 or B211.

[0366] The meaning of the convolution manner and the audio signal A211 or B211 can be referred to the above embodiment, which will not be described here again.

[0367] For example, in the related scheme, the distance of the mid sound channel audio signal perceived by the user is approximately equal to the distance between the electronic device and the user. For example, the distance parameter 2 obtained by the mid sound image control module is used to indicate the distance between the position of the electronic device and the user. After the mid sound image control module obtains the distance parameter 2, the electronic device can consider that the distance between itself and the user is the distance parameter 2, and the distance of the mid sound channel audio signal perceived by the user is approximately equal to the distance parameter 2.

[0368] In the embodiment of the application, the mid sound image control module can determine the distance parameter 3 based on the distance parameter 2, the distance parameter 3 is smaller than the distance parameter 2, then determine the distance filter 4 based on the distance parameter 3, the azimuth angle 1 and the elevation angle 1, the distance filter 4 refers to the distance filter corresponding to the distance parameter 3, finally, convolve the mid sound channel audio signal based on the distance filter 4. Since the distance filter 4 is obtained based on the distance parameter 3, the distance of the mid sound channel audio signal perceived by the user is approximately equal to the distance parameter 3. Compared with the related scheme (in the related scheme, the distance of the mid sound channel audio signal perceived by the user is the distance parameter 2, and the numerical value of the distance parameter 3 is smaller than that of the distance parameter 2), the distance between the mid sound channel audio signal and the user can be shortened, and the distance sense of the audio is improved. Moreover, the distance filter 4 is determined based on the distance parameter 3, the azimuth angle 1 and the elevation angle 1 in real time in the embodiment of the application, so that the distance filter can be determined more accurately.

[0369] The above embodiments introduce FIG. 9 and FIG. 10The implementation process of the center image control module in the embodiment is described as follows. FIG. 19 and FIG. 19 The implementation process of the side image control module in the embodiment is described as follows.

[0370] Please refer to FIG. 9 , FIG. 10 is an example diagram of the implementation process of the side image control module. The implementation process includes S1051-S1054.

[0371] S1051, the side image control module receives the side channel audio signal sent by the upmix or transparent transmission module, and obtains distance parameter 1 and width parameter 1.

[0372] It should be understood that when the channel type of the audio signal is two channels, the side image control module is FIG. 2 The first side image control module shown in the embodiment receives the side channel audio signal sent by the upmix or transparent transmission module, including audio signal A22 and audio signal A23. When the channel type of the audio signal is multi-channel, such as 5.1 channels, the side image control module is FIG. 6 The first side image control module and the second side image control module shown in the embodiment, wherein the first side image control module receives the side channel audio signal sent by the upmix or transparent transmission module, including audio signal B22 and audio signal B23, and the second side image control module receives the side channel audio signal sent by the upmix or transparent transmission module, including audio signal B24 and audio signal B25.

[0373] Distance parameter 1 is used to indicate the distance between the user and the audio perceived by the user. The way to obtain distance parameter 1 can refer to the above embodiment, which will not be repeated here.

[0374] Width parameter 1 is used to indicate the width of the audio perceived by the user, and width parameter 1 can also be called auditory width. For example, the width can be W2 shown in (b) of FIG. 7 In the implementation, the side image control module can obtain width parameter 1 after the user adjusts the sliding bar corresponding to the width parameter through the user interface, or use the default width parameter as width parameter 1 when the user does not adjust the sliding bar corresponding to the width parameter in the user interface. For example, the user can adjust the sliding bar corresponding to the width parameter through the control 633 shown in (c) of FIG. 8 or adjust the sliding bar corresponding to the width parameter through the control 743 shown in (d) of FIG. 6 or adjust the sliding bar corresponding to the width parameter through the control 743 shown in (d) of FIG. 6The control 832 shown in (c) of FIG. 8 adjusts the width parameter corresponding to the sliding bar, and the mid sound image control module can obtain the width parameter 1 in response to the operation of the user adjusting the width parameter. For another example, the user opens the spatial audio function of the electronic device, but does not adjust the width parameter included in the spatial audio function, and the width parameter refers to the default width parameter. For example, the user opens the spatial audio function of the electronic device through the control 631 shown in (c) of FIG. 6, and does not adjust the control 633, and at this time, the default width parameter can refer to the width parameter corresponding to the middle position of the sliding bar shown in (c) of FIG. 6. FIG. 1 The control 631 shown in (c) of FIG. 6 opens the spatial audio function, but does not adjust the control 633, and at this time, the default width parameter can refer to the width parameter corresponding to the middle position of the sliding bar shown in (c) of FIG. 6. FIG. 6 The middle position of the sliding bar corresponding to the width parameter shown in (c) of FIG. 6.

[0375] It can be understood that the purpose of the side sound image control module obtaining the width parameter is to process the side channel audio signal based on the width parameter, which can widen the width of the sound image and improve the user's perception of the spatial sense of the audio. Therefore, the numerical value of the width parameter corresponding to each position to which the sliding bar moves in the user interface is greater than the width threshold, and the width threshold can be W1 shown in (b) of FIG. 6. FIG. 6 The leftmost position of the width parameter corresponding to the sliding bar shown in (c) of FIG. 6 is greater than W1. FIG. 6

[0376] It should also be understood that the user can further improve the width of the audio perceived by the user by adjusting the sliding bar corresponding to the width parameter. For example, the user can slide the sliding bar corresponding to the width parameter from left to right in the user interface shown in (c) of FIG. 6 to further improve the width of the audio perceived by the user. FIG. 20 The width parameter corresponding to the sliding bar shown in (c) of FIG. 6 is gradually increased when the width parameter sliding bar is slid from the left side to the right side. FIG. 20

[0377] S1052, the side sound image control module renders and processes the side channel audio signal based on the distance parameter 1 and the width parameter 1 to obtain a rendered side channel audio signal.

[0378] It should be understood that the rendering of the audio signal refers to the process of converting the audio signal into sound that can be heard by the human ear. The rendering in the embodiments of the present application involves spatial audio processing, such as placing sound in a specific location in a virtual environment, and adjusting the direction and distance of the sound according to the head position and direction of the listener. This rendering can generally simulate the sound propagation and reflection in the real world.

[0379] ​​The rendering in the embodiments of the present application includes direct sound rendering and early reflection sound rendering. It should be understood that the direct sound refers to the sound signal in the side channel audio signal that directly reaches the listener and is less affected by the acoustic environment of the surrounding environment. The early reflection sound refers to the sound signal that is first reflected to the listener's ear after encountering obstacles such as room walls, floors, and ceilings in the side channel audio signal. These reflected sounds usually arrive within 50ms to 80ms after the direct sound, which helps to enhance the spatial and depth perception of the sound. Direct sound rendering refers to the process of simulating the arrival of direct sound from a real sound source to the human ear, so that the listener can perceive the position, distance, and other information of the real direct sound in space. Early reflection sound rendering refers to the process of simulating the arrival of early reflection sound from a real sound source to the human ear, so that the listener can perceive the position, distance, and other information of the real early reflection sound in space.

[0380] The side image control module in the embodiments of the present application can perform rendering processing through the driving equation of each virtual speaker in the virtual speaker array.

[0381] It should be understood that the virtual speaker can use digital signal processing technology to simulate the sound field effect of a real speaker to provide a wider and more natural listening experience for the user. The output signal of the virtual speaker can be regarded as a kind of "rendering" of the audio signal, which is processed by an algorithm so that the listener can perceive the sound effect similar to a multi-channel speaker system without multiple physical speakers.

[0382] It should also be understood that the array of virtual speakers can be a three-dimensional array, a two-dimensional array, etc. The two-dimensional array refers to the arrangement of speakers in one plane, usually a horizontal plane, and the three-dimensional array refers to the arrangement of speakers not only in one plane but also possibly with different heights in the vertical direction. The embodiments of the present application do not limit this. The array of virtual speakers can also be a circular array, an arc-shaped array, a linear array, etc. The embodiments of the present application do not limit this. For examples, please refer to FIG. 20 , FIG. 20 is an example diagram of an array of virtual speakers provided by the embodiments of the present application. FIG. 20 The array of virtual speakers configured in the electronic device in FIG. 8 is a circular array, and the array includes eight virtual speakers. FIG. 20 S1 to S8 shown in FIG. 8.

[0383] It can also be understood that in the case where only the direct sound rendering is considered in the output signal of the virtual speaker, the driving equation of each virtual speaker output signal can be represented by the following formula seven:

[0384]

[0385] wherein D1 z.5D(x0, ω) represents the output signal of any one of the plurality of virtual loudspeakers, which can be considered as the signal after the direct sound rendering, x0 represents the position information of any one of the plurality of virtual loudspeakers (usually the position information can be quantified as a coordinate), ω represents the amplitude of the audio signal, w(x0) represents the window function, the application of the window function in the virtual loudspeaker driving equation mainly reflects the reduction of the spectral leakage of the signal and the improvement of the time domain characteristics of the signal, i represents the imaginary unit, c represents the sound speed, x ref represents the reference point coordinate, usually refers to the position information of the user, S refers to the position information of the simulated sound source (usually the position information can be quantified as a coordinate), refers to the distance between any one of the plurality of virtual loudspeakers and the center of the user's head.

[0386] In the case where the output signal of the virtual loudspeaker considers the direct sound rendering and the reflected sound rendering, the driving equation of each virtual loudspeaker output signal can be represented by the following formula eight:

[0387]

[0388] where D2 2.5D (x0, ω) represents the output signal of any one of the plurality of virtual loudspeakers, which can be considered as the signal after the direct sound rendering and the early reflected sound rendering, and refers to the reflection coefficient of the four walls in the room, q, k is used to represent whether the sound source is reflected after reaching the wall, q, k ∈ {0, 1}, q and k are 0, which means that the sound source is not reflected after reaching the wall, q and k are 1, which means that the sound source is reflected after reaching the wall, m x , m y represents the number of reflections, -N≤m x , m z ≤N, N is an integer greater than or equal to 1, the meanings of other parameters in formula eight can be referred to formula seven, which will not be described here. Of course, in other embodiments, formula eight can also include more or less wall reflection coefficients, and the embodiments of the present application are exemplified with four reflection coefficients.

[0389] It should be understood that the above formula eight is based on formula seven. Specifically, for any one of the virtual loudspeakers, the position of the direct sound in the environment can be simulated, that is, the position of the direct sound in the environment simulated by a virtual loudspeaker can be considered as the position of a point sound source in the environment, for any point sound source, it can be represented as:

[0390]

[0391] Wherein, the meaning of each parameter in the above formula can refer to the embodiment, which will not be repeated here.

[0392] For any virtual loudspeaker simulated direct sound point source, in the case of known room geometry information, wall material information, the mirror image source set of each direct sound corresponding early reflection sound can be obtained by mirror method, ray tracing method or numerical solution method. For each direct sound point source corresponding mirror image source set, it can be expressed as:

[0393]

[0394] Wherein, x s , y s and z s represent the coordinates of the direct sound point source, L x , L y and L z represent the room geometry information, such as when the room is approximately a cuboid, L x , L y and L z represent the length, width and height of the room, q,j,k is used to represent whether the sound source reflects after reaching the wall, q,j,k∈{0,1}, q,j,k is 0, which represents that the sound source does not reflect after reaching the wall, q,j,k is 1, which represents that the sound source does not reflect after reaching the wall, m x , m y , m z represent the number of reflections, -N≤m x , m y , m z ≤N, N is an integer greater than or equal to 1, q,j,k and m x , m y , m z are determined by the material information of the wall of the room.

[0395] After determining each mirror image source set, based on each mirror image source set, room geometry information and formula seven, formula eight can be obtained, which is not limited by the embodiment of the application.

[0396] It should be understood that in the implementation, the side sound image control module can render and process a side channel audio signal through mode E and mode F. Mode E is to render and process a side channel audio signal through all virtual loudspeakers in the virtual loudspeaker array, and mode F is to render and process a side channel audio signal through part of the virtual loudspeakers in the virtual loudspeaker array. The following describes the two modes:

[0397] Mode E includes steps E1 and E2:

[0398] Step E1: The side sound image control module can determine the parameters in the driving equation (Equation Eight) of each virtual speaker in the array of virtual speakers in the electronic device based on the distance parameter 1, the width parameter 1, and the side channel audio signal.

[0399] It should be understood that some parameters in the driving equation (Equation Eight) of the virtual speaker are preset, such as: and are preset reflection coefficients, where, in order to facilitate calculation, m x ,m z and q, k can be set to 1, c is a preset sound speed, π is a preset circular constant. The window function w(x0) is also preset. The position information x0 of the virtual speaker is also preset, for example, the position information of each virtual speaker has been set based on the coordinate system of the electronic device before the electronic device is shipped. For example, FIG. 20 , the origin of the coordinate system of the electronic device is O1, the coordinates of the virtual speaker S1 are (0, 2), the coordinates of S5 are (0, -2), the coordinates of S3 are (-2, 0), the coordinates of S7 are (2, 0), the coordinates of S2 are (-1, 1), the coordinates of S4 are (-1, -1), the coordinates of S6 are (1, -1), and the coordinates of S8 are (1, 1).

[0400] Some parameters in Equation Eight are real-time calculated, such as A(ω), ω, x s , x ref and , etc.

[0401] , where A(ω) and ω can be obtained according to the side channel audio signal. For example, when the side channel audio signal is the audio signal A22, the amplitude A(ω) and the frequency ω of the audio signal A22 can be obtained by calculation, and when the side channel audio signal is the audio signal A23, the amplitude A(ω) and the frequency ω of the audio signal A23 can be obtained by calculation. The method for calculating the amplitude and frequency of the audio signal is not limited in the embodiments of the present application. It should be understood that when the audio signals are the same, the amplitude A(ω) and the frequency ω of the driving equation of each virtual speaker in the array of virtual speakers are the same.

[0402] , where the reference point coordinates x refGenerally refers to the position information of the user. The reference point coordinate can be based on real-time detection or a preset value. For example, the side sound image control module can detect the position information of the user based on a direction sensor, a global positioning system, etc. configured on the electronic device. For example, the coordinates of the user's head can be detected by the direction sensor, the global positioning system, etc. If there is a user in the scene of the external audio, the detected head of the user can be taken as the reference point coordinate x ref If there is no user in the scene of the external audio, the head coordinates of the user cannot be detected by the direction sensor, the global positioning system, etc. At this time, the preset value can be taken as the reference point coordinate. The preset value is not limited in the embodiments of the present application.

[0403] wherein the parameter refers to the distance between any one of the plurality of virtual loudspeakers and the center of the user's head, and thus the reference point coordinate x ref obtained above and the simulated sound source coordinate x S determined below can be used to calculate i.e.

[0404]

[0405] wherein the position information x s of the simulated sound source can be obtained based on the distance parameter 1 and the width parameter 1.

[0406] It should be understood that the position of the simulated point sound source in the driving equation of each virtual loudspeaker in the array of virtual loudspeakers is different, and thus the value of x s in each driving equation is different when calculating the driving equation of each virtual loudspeaker. For example, FIG. 20 In the above equation, the positions of the eight virtual loudspeakers S1 to S8 simulate the positions of the eight point sound sources V1 to V8. When calculating the driving equation of the virtual loudspeaker S1, the position information of V1 needs to be determined as the parameter x s in the driving equation of S1. When calculating the virtual loudspeaker S2, the position information of V2 needs to be determined as the parameter x s in the driving equation of S2. When calculating other driving equations, the parameter x s also needs to take the position information of the corresponding simulated point sound source as x s .

[0407] In the implementation, the side sound image control module can obtain x s based on the distance parameter 1 and the width parameter 1. For example, FIG. 20The side sound image control module can determine the distance parameter 1 as the horizontal coordinate of V1 in the simulated sound source, and determine the half of the positive value of the width parameter 1 as the vertical coordinate of V1, i.e., the coordinates of V1 can be determined as (ra, rb / 2). The side sound image control module can determine the distance parameter 1 as the horizontal coordinate of V5 in the simulated sound source, and determine the half of the negative value of the width parameter 1 as the vertical coordinate of V5, i.e., the coordinates of V5 can be determined as (ra, -rb / 2). The side sound image control module can determine the difference between the distance parameter 1 and the width parameter 1 as the horizontal coordinate of V7 in the simulated sound source, and determine the value 0 as the vertical coordinate of V7, i.e., the coordinates of V7 can be determined as (ra-rb, 0). The side sound image control module can determine the sum of the distance parameter 1 and the width parameter 1 as the horizontal coordinate of V3 in the simulated sound source, and determine the value 0 as the vertical coordinate of V3, i.e., the coordinates of V3 can be determined as (ra+rb, 0). The side sound image control module can determine the average of the horizontal coordinate of V1 and the horizontal coordinate of V7 as the horizontal coordinate of V8, and determine the average of the vertical coordinate of V1 and the vertical coordinate of V7 as the vertical coordinate of V8, i.e., the coordinates of V7 can be determined as ((2ra-rb) / 2, rb / 4). The side sound image control module can determine the average of the horizontal coordinate of V5 and the horizontal coordinate of V7 as the horizontal coordinate of V6, and determine the average of the vertical coordinate of V5 and the vertical coordinate of V7 as the vertical coordinate of V6, i.e., the coordinates of V6 can be determined as ((2ra-rb) / 2, -rb / 4). The side sound image control module can determine the average of the horizontal coordinate of V3 and the horizontal coordinate of V1 as the horizontal coordinate of V2, and determine the average of the vertical coordinate of V3 and the vertical coordinate of V1 as the vertical coordinate of V2, i.e., the coordinates of V2 can be determined as ((2ra+rb) / 2, rb / 4). The side sound image control module can determine the average of the horizontal coordinate of V3 and the horizontal coordinate of V5 as the horizontal coordinate of V4, and determine the average of the vertical coordinate of V3 and the vertical coordinate of V5 as the vertical coordinate of V4, i.e., the coordinates of V4 can be determined as ((2ra+rb) / 2, -rb / 4).

[0408] The set of coordinates of V1 to V8 shown can be regarded as the sound source coordinates of the simulated one side channel audio signal. FIG. 20 The set of coordinates of V1 to V8 shown can be regarded as the sound source coordinates of the simulated one side channel audio signal.

[0409] It should be understood that FIG. 20 The simulated sound source shown is obtained after processing one side channel audio signal (such as the left channel audio signal) through the virtual loudspeaker array, and the way of processing other side channel audio signals through the virtual loudspeaker array can refer to FIG. 20 The way shown, which will not be described here.

[0410] It should also be understood that the coordinates involved in Formula Eight can refer to coordinates in a coordinate system of the electronic device, which is a coordinate system with the center position of the electronic device as the origin of the coordinate system. In other embodiments, the coordinates involved in Formula Eight can also refer to coordinates in a user coordinate system, which is a coordinate system with the center position of the line connecting the two ears of the user as the origin of the coordinate system. When Formula Eight is calculated in the user coordinate system, the coordinates need to be converted from the coordinate system of the electronic device to the user coordinate system.

[0411] Step E2: The side sound image control module inputs the parameter into the driving equation of each virtual loudspeaker in the array of virtual loudspeakers to obtain an output signal of each virtual loudspeaker, and the sum of the output signals of all virtual loudspeakers is the rendered side channel audio signal.

[0412] For example, for the audio signal A22 received by the side sound image control module, the side sound image control module can determine each parameter in Formula Eight according to the manner described in the above embodiments, and then input the parameter into the driving equation of each virtual loudspeaker in the array of virtual loudspeakers (Formula Eight) to obtain an output signal of each virtual loudspeaker. FIG. 20 The sum of the eight output signals is determined as the rendered audio signal A22. For other side channel audio signals received by the side sound image control module, such as the audio signal A23, the audio signal B22, etc., they can also be processed in the above manner to obtain the rendered side channel audio signal, which will not be described here.

[0413] The rendered side channel audio signal can be represented by the following Formula Nine:

[0414]

[0415] where P is the number of all virtual loudspeakers in the array of virtual loudspeakers, D2 2.5D (x0, ω) represents the output signal of any virtual loudspeaker in the array of virtual loudspeakers.

[0416] Method F, comprising steps F1 and F2:

[0417] Step F1: The side sound image control module can determine the parameters in the driving equation of each virtual loudspeaker in the array of virtual loudspeakers in the electronic device (Formula Eight) based on the distance parameter 1, the width parameter 1, and the side channel audio signal.

[0418] For example, the array of virtual loudspeakers includes FIG. 21The 8 virtual speakers shown, the side sound image control module can render a left channel audio signal through 4 of the 8 virtual speakers, and render a right channel audio signal through the other 4 of the 8 virtual speakers when rendering a stereo sound including the left channel audio signal and the right channel audio signal. For example, the side sound image control module can render a left channel audio signal, a right channel audio signal, a left surround channel audio signal and a right surround channel audio signal through 2 virtual speakers respectively when rendering a 5.1 sound. The rendering manner for other multi-channel audio signals can refer to the rendering manner for the 5.1 sound.

[0419] In the implementation, when determining the parameters in the driving equation of the virtual speaker, a stereo sound is taken as an example. For example, for a left channel audio signal, the side sound image control module can determine the parameters in the driving equation of each of the 4 virtual speakers in the array of virtual speakers in the electronic device based on the distance parameter 1, the width parameter 1 and the left channel audio signal. For a right channel audio signal, the side sound image control module can determine the parameters in the driving equation of each of the 4 virtual speakers in the array of virtual speakers in the electronic device based on the distance parameter 1, the width parameter 1 and the right channel audio signal.

[0420] In this implementation, the determination manner of the parameters in the driving equation (Formula Eight) of the virtual speaker can refer to the manner in the manner E.

[0421] The manners F and E differ in the manner of simulating the sound source coordinates of a side channel audio signal by the virtual speaker array. In the manner F, the coordinate set that all the virtual speakers in the virtual speaker array need to simulate is regarded as the sound source coordinates of a side channel audio signal. For example, FIG. 21 The coordinate set of V1 to V8 shown can be regarded as the sound source coordinates of a side channel audio signal. In the manner E, the coordinate set that part of the virtual speakers in the virtual speaker array need to simulate is regarded as the sound source coordinates of a side channel audio signal. For example, for a left channel audio signal, the coordinate set of V1 to V4 shown can be regarded as the sound source coordinates of the left channel audio signal. FIG. 21 The coordinate set of the point sound sources V1, V2, V3 and V4 simulated by the virtual speakers S1, S2, S3 and S4 shown can be regarded as the sound source coordinates of the left channel audio signal. For a right channel audio signal, the coordinate set of the point sound sources V5, V6, V7 and V8 simulated by the virtual speakers S5, S6, S7 and S8 shown can be regarded as the sound source coordinates of the right channel audio signal. FIG. 21 The coordinate set of the point sound sources V1, V2, V3 and V4 simulated by the virtual speakers S1, S2, S3 and S4 shown can be regarded as the sound source coordinates of the left channel audio signal. For a right channel audio signal, the coordinate set of the point sound sources V5, V6, V7 and V8 simulated by the virtual speakers S5, S6, S7 and S8 shown can be regarded as the sound source coordinates of the right channel audio signal.

[0422] Step F2: the side sound image control module inputs the parameter into a driving equation of each virtual speaker in the partial virtual speaker, to obtain an output signal based on each virtual speaker, and a sum of the output signals of the partial virtual speakers is a rendered side channel audio signal.

[0423] In this implementation, the rendered side channel audio signal can also be represented by Formula Nine, except that P in Formula Nine represents the number of partial virtual speakers in the virtual speaker array in this mode F.

[0424] Some related schemes render an audio signal by using an HRTF filter, a digital-to-analog converter, etc., while the embodiments of the present application render the audio signal by using a driving equation of a virtual speaker in a virtual speaker array. The rendered audio signal of the embodiments of the present application is clearer and more realistic than that of the related schemes, and the embodiments of the present application can improve the clarity and realism of the audio.

[0425] Some other related schemes render an audio signal by rendering direct sound and late reverberation sound. The late reverberation sound refers to sound formed by reflection and scattering again after early reflection sound. Since the late reverberation sound can reduce the authenticity of the audio and affect the clarity and purity of the audio, etc., the embodiments of the present application render direct sound and early reflection sound, and do not render late reverberation sound, thereby improving the authenticity, clarity and purity of the audio. In addition, when rendering, the direct sound and the late reverberation sound need to be rendered respectively, so that the electronic device needs to allocate more computing resources for rendering processing, which has a large amount of calculation and high power consumption. However, the embodiments of the present application can render the direct sound and the late reverberation sound simultaneously based on the driving equation of the virtual speaker, so that the electronic device can allocate less computing resources for rendering processing, thereby saving the amount of calculation and reducing the power consumption.

[0426] S1053: The side sound image control module generates a binaural impulse response based on the rendered side channel audio signal.

[0427] It should be understood that the binaural impulse response (Binaural Impulse Response, BIR) refers to a pressure change caused by air propagation to the two ears of a person when a pulse signal is emitted at a certain position in space. The audio signal can be considered as a pulse signal.

[0428] In the implementation, the side sound image control module can substitute the output signal obtained by each virtual speaker in the rendered side channel audio signal into Formula Ten to calculate the binaural impulse response.

[0429]

[0430] wherein I L denotes the left ear impulse response, I R denotes the right ear impulse response, D2 2.5D (x0, ω) denotes the output signal of any one of the plurality of virtual speakers in the rendered side channel audio signal obtained by Equation Eight. h L denotes the left ear HRTF, h R denotes the right ear HRTF.

[0431] For example, refer to FIG. 21 , FIG. 21 is an example diagram for generating binaural impulse responses provided by an embodiment of the present application. FIG. 21 is an example illustration taking the stereo (two-channel) sound as an example, FIG. 22 The virtual speaker array in the above-mentioned way E is rendered and processed, FIG. 21 The audio signal A22 in the above-mentioned way E is the left channel audio signal, and the audio signal A23 is the right channel audio signal. The side image control module inputs the audio signals A22 and A23 into the virtual speaker array, respectively, FIG. 21 The virtual speaker array in the above-mentioned way E includes four virtual speakers, S1 to S4. Each virtual speaker calculates an output signal based on the above-mentioned Equation Eight. For example, the audio signal A22 is input into the virtual speaker S1. The virtual speaker S1 obtains FIG. 21 the output signal 1 shown in the above-mentioned way E. The other virtual speakers obtain the output signals in the same way as the virtual speaker S1 obtains the output signal 1. The other audio signals obtain the output signals in the same way as the audio signal A22 obtains the output signals. Here, no further description is given. For example, FIG. 21 The audio signal A22 shown in the above-mentioned way E can obtain the output signals 1, 2, 3 and 4 through the four virtual speakers in the virtual speaker array. The audio signal A23 shown in the above-mentioned way E can obtain the output signals 5, 6, 7 and 8 through the four virtual speakers in the virtual speaker array. Then, each output signal is convoluted with the left ear HRTF and the right ear HRTF by Equation Ten. The sum of the convolutions of all the output signals with the left ear HRTF is taken as the left ear impulse response, and the sum of the convolutions of all the output signals with the right ear HRTF is taken as the right ear impulse response. It should be understood that FIG. 9 The above-mentioned way E takes the stereo sound as an example. When the audio signal received by the side image control module is a multi-channel audio signal, the implementation process of generating binaural impulse responses can refer to FIG. 9 . FIG. 9In the implementation process of generating the binaural impulse response, the virtual loudspeaker array is rendered by the above-mentioned manner E, and when the virtual loudspeaker is rendered by the above-mentioned manner F, the implementation process of generating the binaural impulse response can refer to FIG. 9 .

[0432] It should also be understood that h L and h R in formula ten are obtained in the following manner one and manner two.

[0433] Manner one:

[0434] First, after the side sound image control module determines the position information of the simulated sound source of the virtual loudspeaker and the coordinates of the reference point in S1052, based on the position information of the simulated sound source of the virtual loudspeaker and the coordinates of the reference point, the azimuth angle 2 and the elevation angle 2 of the simulated sound source relative to the reference point are determined.

[0435] In the implementation, for example, it is assumed that the coordinates of the position information of the simulated sound source are (x1, y1, z1), and the coordinates of the reference point are (0, y2, 0), then the azimuth angle 2 can be obtained based on the following formula:

[0436]

[0437] Wherein, θ2 refers to the azimuth angle 2 of the simulated sound source relative to the reference point.

[0438] The elevation angle 2 can be obtained based on the following formula:

[0439]

[0440] Wherein, Φ2 refers to the elevation angle 2 of the simulated sound source relative to the reference point.

[0441] Secondly, the side sound image control module determines the h L and h R corresponding to the virtual loudspeaker in the preset HRTF data set B based on the azimuth angle 2, the elevation angle 2 and the distance parameter 1.

[0442] It should be understood that the HRTF data set B can be stored in the memory of the electronic device, and when the side sound image control module executes S1053, the HRTF data set B can be called from the memory. In order to save the storage resources of the electronic device, the HRTF data set B can be stored in the cloud server or the server, and when the side sound image control module executes S1053, the HRTF data set B can be called from the cloud server or the server.

[0443] It should also be understood that the HRTF dataset B can be an open source dataset, which is stored in the memory of the electronic device before the electronic device is shipped, or stored in a cloud server or a server.

[0444] The left ear HRTF and the right ear HRTF of the measurement body are measured when the measurement body included in the HRTF dataset B is located at different distances, different azimuth angles and different elevation angles of the audio playback device. It should also be understood that the data in the HRTF dataset B is obtained based on the audio playback device playing a monophonic audio signal, for example, the left ear HRTF is obtained based on the audio playback device playing a left channel audio signal, and the right ear HRTF is obtained based on the audio playback device playing a right channel audio signal. For example, the HRTF dataset B can refer to Table 3 below:

[0445] Table 3

[0446]

[0447] In an implementation, for example, the azimuth angle 2 refers to the azimuth angle C1 in Table 3, the elevation angle 2 refers to the elevation angle C2 in Table 3, and the distance parameter 1 refers to the distance C1 in Table 3. Then, the side sound image control module can query h L and h R in Table 3 corresponding to the azimuth angle 2, the elevation angle 2 and the distance C1, for example, the queried h L is the left ear HRTF-C2, and the queried h R is the right ear HRTF-C2. Then, the left ear HRTF-C2 and the right ear HRTF-C2 are taken as h L and h R corresponding to a virtual loudspeaker. In this way, h L and h R corresponding to each virtual loudspeaker are obtained. Then, h L and h R corresponding to each virtual loudspeaker are substituted into Formula 10 to calculate the binaural impulse response.

[0448] Method two: first, after the side sound image control module determines the position information of the sound source simulated by a virtual loudspeaker in S1052, the azimuth angle 2 and the elevation angle 2 of the simulated sound source relative to the electronic device, and the distance parameter 1 of the simulated sound source from the electronic device are determined based on the position information of the simulated sound source and the coordinates of the reference point.

[0449] Secondly, the side sound image control module can solve the wave equation based on the azimuth angle 2, the elevation angle 2 and the distance parameter 1, that is, h L and h R corresponding to a virtual loudspeaker are obtained. In this way, hL and h R .

[0450] It should be understood that the second mode is to calculate the left ear HRTF and the right ear HRTF in real time, and the implementation of the second mode can refer to S9418 in the above embodiments.

[0451] It should also be understood that the first mode is to determine the left ear HRTF and the right ear HRTF based on the preset HRTF data set B, and the second mode is to calculate the left ear HRTF and the right ear HRTF in real time, the difference lies in that: the electronic device does not need to spend more computing resources to determine the left ear HRTF and the right ear HRTF in the first mode, but the left ear HRTF and the right ear HRTF obtained by the preset HRTF data set B are relatively incorrect because the influence of the individualized characteristics of the user (such as the geometric parameters of the head, pinna, and torso of the user) is not fully considered. While the left ear HRTF and the right ear HRTF obtained based on the second mode are relatively accurate, the computing power requirement of the electronic device is relatively high, and the amount of calculation is relatively large.

[0452] It should be noted that the audio signal can include audio signals of multiple channels. Taking left and right channels as an example, in an ideal state, it is expected that the left channel audio is received by the left ear of the person, and the right channel audio is received by the right ear of the person, but in the actual propagation environment, the audio signal will diffuse in space due to the propagation characteristics of air, objects and other media during propagation, which means that the left channel audio will also be received by the right ear of the person, and the right channel audio will also be received by the left ear of the person. Therefore, when generating the binaural impulse response by S1053, the left ear impulse response I L includes the left ear impulse response I 1L received by the left ear of the person, and the left ear impulse response I 2L received by the right ear of the person. And the right ear impulse response I R includes the right ear impulse response I 1R received by the right ear of the person, and the right ear impulse response I 2R received by the left ear of the person.

[0453] In the implementation, for example, the side sound image control module, after obtaining the left ear impulse response I l by formula ten, inverts, attenuates, and delays the left ear impulse response I L to generate an equal-amplitude inverse signal of the right ear impulse response I R , and superimposes the equal-amplitude inverse signal and the right ear impulse response I R , so that the right ear impulse response I 1R received by the right ear of the person can be obtained, and then I R is subtracted from the right ear impulse response I 1R received by the left ear of the person.The right ear impulse response I 2R For example, the side sound image control module obtains the left ear impulse response I R The right ear impulse response I R is reversed, attenuated and delayed to generate an equal-amplitude anti-phase signal of the left ear impulse response I L , and the equal-amplitude anti-phase signal is superimposed with the left ear impulse response I L , so that the left ear impulse response I 1L received by the left ear of the person is obtained. Then, the left ear impulse response I L is subtracted by I 1L , so that the left ear impulse response I 2L received by the right ear of the person is obtained.

[0454] S1054, the side sound image control module convolves the side channel audio signal based on the binaural impulse response to obtain the audio signals A221 and A231, or the audio signals B221, B231, B241 and B251.

[0455] In the implementation, the side sound image control module can perform the convolution processing through the following formula eleven:

[0456]

[0457] Wherein, L out is the left side channel audio signal output through S1054, and the left side channel audio signal includes a left channel, a left surround channel and the like, R out is the right side channel audio signal output through S1054, and the right side channel audio signal includes a right channel, a right surround channel and the like, L in is the left side channel audio signal input into the side sound image control module, R in is the right side channel audio signal input into the side sound image control module, I 1L is the left ear impulse response received by the left ear of the person, I 2L is the left ear impulse response received by the right ear of the person, I 1R is the right ear impulse response received by the right ear of the person, and I 2R is the right ear impulse response received by the left ear of the person.

[0458] For example, assuming that the channel type of the audio signal received by the side sound image control module is stereo, such as FIG. 9 the audio signal A22 (left channel) and the audio signal A23 (right channel) shown in the figure, then the output signal obtained based on the formula eleven is represented as:

[0459]

[0460] wherein L 1out denotes FIG. 10 the audio signal A221, R 1out denotes FIG. 10 the audio signal A231, L 1in denotes FIG. 10 the audio signal A22, R 1in denotes FIG. 10 the audio signal A23.

[0461] Suppose the channel type of the audio signal received by the side image control module is multi-channel, such as 5.1 channel. Then, the audio signal received by the side image control module includes, for example, FIG. 10 the audio signal B22 (left channel), the audio signal B23 (right channel), the audio signal B24 (left surround channel), and the audio signal B25 (right surround channel). Then, the output signal obtained based on Formula Eleven is represented as:

[0462]

[0463] wherein L 2out denotes FIG. 10 the audio signal B221, R 2out denotes FIG. 10 the audio signal B231, L 3our denotes FIG. 10 the audio signal B241, R 3out denotes FIG. 1 the audio signal B251, L 2in denotes FIG. 22 the audio signal B22, R 2in denotes FIG. 22 the audio signal B23, L 3in denotes FIG. 4 the audio signal B24, R 3in denotes FIG. 5 the audio signal B25.

[0464] In the related scheme, the sound image formed by the electronic device playing out music is relatively flat as a whole, and the user's perception of the spatial sense of the audio is weak. Please refer to FIG. 5 (b) shows the sound image 1, the overall width of the sound image 1 is W1, the sound image as a whole is relatively flat, and thus the user's perception of the spatial sense of the audio is weak.

[0465] It should be understood that the side-channel audio control module provided in this application embodiment can obtain the width parameter 1 and the distance parameter 1 through the user interface, and then perform rendering processing on the side channel audio signal based on the distance parameter 1 and the width parameter 1 through S1052 to obtain the rendered side channel audio signal. During the rendering process, the position information of the sound source (side channel audio signal) can be simulated, and this position information is obtained based on the width parameter 1 corresponding to the width of the audio perceived by the user. Moreover, the width parameter 1 is greater than the width threshold (for example, the width threshold is W1). Therefore, the width of the simulated sound source is approximately equal to the width parameter 1. Thus, after the side-channel audio control module executes S1053 and S1054, since the width of the sound source in the simulated sound source is approximately equal to the width parameter 1, compared with the related scheme (the width of the sound image in the related scheme is the width threshold W1, while the width of the sound source (sound image) in this application embodiment is the width parameter 1, and the value of the width parameter 1 is greater than the width threshold W1), the width of the side-channel audio image can be widened, thereby improving the user's sense of spatial perception of the audio.

[0466] To further enhance the user's perception of the spatiality of the audio, the user can slide the slider corresponding to the width from left to right through the user interface. This increases the value of the width parameter 1 obtained by the side image control module, and the width of the simulated sound source during the rendering process increases accordingly. This further widens the width of the side image and enhances the user's perception of the spatiality of the audio.

[0467] The following embodiments describe the implementation process of an audio processing method provided in this application.

[0468] Please refer to FIG. 9 , FIG. 10 This is a schematic diagram of an audio processing method provided in an embodiment of this application. The method can be... FIG. 6 The method described herein can be executed by the electronic device 100, or by a processor or chip within the electronic device 100; this application embodiment does not impose any limitations. For ease of description, the method is illustrated using an electronic device as an example. The method includes steps S2210 to S2260.

[0469] S2210, the electronic device responds to the first operation by acquiring the audio signal to be processed.

[0470] It should be understood that the first operation refers to the user's action of playing audio, such as the user clicking... FIG. 6 The image shows a playback control for a song in the Music 1 application within the application layer, for example, when a user clicks... FIG. 9 The image shows a playback control for a video in the video 1 application within the application layer. The first operation can be a click operation, or it can be a voice input, gesture, or other operation; this application does not limit this to any particular operation.

[0471] It should also be understood that the audio signal to be processed refers to the original audio signal, such as... FIG. 10 The audio signal A1 shown is... FIG. 9 The audio signal B1 shown is described in the example above. For an introduction to the original audio signal, please refer to the above embodiment.

[0472] S2220, the electronic device responds to the second operation by acquiring rendering parameters, the rendering parameters including a first distance, the first distance being used to indicate an auditory distance, the first distance being less than a first threshold.

[0473] It should be understood that user actions can refer to the user's movement on the slider corresponding to the distance swiped in the audio rendering interface, such as when the user... FIG. 10 (c) shows the slider corresponding to the sliding distance in interface 630. User operation can also be user voice input, or the user can fill in rendering parameters on the user interface of the electronic device, etc., which are not limited in this embodiment.

[0474] Rendering parameters refer to parameters related to audio rendering, such as distance parameters and width parameters. The first distance refers to one of several distance parameters; for example, the first distance could be distance parameter 1 mentioned in the previous example. For instance, a user could... FIG. 9 (c) shows a slider in interface 630 corresponding to the sliding distance. Each position on the slider can correspond to a distance parameter. The first distance refers to the distance parameter corresponding to a position on the slider. Auditory distance refers to the distance between the user-perceived audio and the user.

[0475] The first threshold is a preset parameter of the electronic device, and the first threshold refers to the distance threshold in the above embodiment. The first threshold can be obtained through testing and data analysis. For example, testers can test multiple products in the relevant solution and obtain multiple test data. During the test, the product is in external audio mode, and each test data represents the audio distance perceived by the tester. Data analysis of multiple test data reveals that the value of the test data is approximately equal to the distance between the product and the tester. Therefore, it can be considered that the first threshold is related to the distance between the electronic device and the user. Thus, when presetting the first threshold in the electronic device of this application embodiment, the minimum value among multiple test data can be used as the first threshold.

[0476] In this embodiment of the application, the maximum value of the multiple distance parameters included in the rendering parameters is less than a first threshold. Thus, the obtained first distance (the first distance is one of the multiple distance parameters) is less than the first threshold.

[0477] The implementation method of electronic devices obtaining rendering parameters in response to user operations can be referred to the above embodiments, and will not be repeated here.

[0478] S2230, the electronic device obtains the mid-channel audio signal based on the to-be-processed audio signal.

[0479] It should be understood that the mid-channel audio signal mainly includes a human voice signal, such as, FIG. 10 The audio signal A21 shown, FIG. 9 The audio signal B21 shown.

[0480] For the implementation of this step, please refer to the above embodiments, which will not be repeated here.

[0481] S2240, the electronic device determines a first distance filter corresponding to the first distance based on the first distance.

[0482] It should be understood that from the above embodiments, it can be known that the distance filter is essentially an HRTF, and the HRTF is related to the distance (auditory distance) between the sound source and the user. Therefore, the embodiments of the present application can determine the distance filter corresponding to the distance based on the first distance.

[0483] In the implementation, the electronic device can determine the first distance filter based on the first distance and a pre-measured HRTF data set, and can also determine the first distance filter based on the first distance and a real-time calculated HRTF, which is not limited by the present application.

[0484] S2250, the electronic device processes the mid-channel audio signal based on the first distance filter to obtain a first audio signal.

[0485] It should be understood that the first audio signal can refer to FIG. 10 The audio signal A211 shown, and can also refer to FIG. 14 The audio signal B211 shown.

[0486] In the implementation, the electronic device convolves the mid-channel audio signal based on the first distance filter to obtain the first audio signal, and the manner of convolution processing is not limited by the embodiments of the present application.

[0487] S2260, the electronic device outputs a first output signal based on the first audio signal.

[0488] It should be understood that the first output signal refers to FIG. 14 The audio signal A4 and the audio signal A5 shown, or FIG. 1 The audio signal A4 or the audio signal A5 shown.

[0489] In the implementation, after obtaining the first audio signal, the electronic device needs to further process the first audio signal, such as through FIG. 14 And FIG. 6 The crosstalk module, the equalization module, the post-gain module and the like shown are processed to output the first output signal.

[0490] In related solutions, the audio distance perceived by the user is approximately equal to a first threshold. In this embodiment, the electronic device can determine a first distance filter based on a first distance (the first width is less than the first threshold), and process the center channel audio signal based on the first distance filter. This allows the audio distance perceived by the user to be the first distance. Since the first distance is less than the first threshold, compared to related solutions, this embodiment can shorten the distance between the center channel audio signal and the user, making the center channel audio signal perceived by the user closer (the distance perceived by the user as the center channel audio signal can be called the center image), thus improving the sense of audio distance.

[0491] Furthermore, in this embodiment of the application, a first distance can be obtained through user operation, and the perceived distance of the audio can be adjusted based on the first distance. That is, the user can actively adjust the perceived distance of the audio, thereby improving the user experience.

[0492] In this embodiment of the application, the electronic device determines a first distance filter corresponding to a first distance based on a first distance, including:

[0493] The electronic device determines M1 near-field head correlation transfer functions corresponding to the first near-field distance and M2 far-field head correlation transfer functions corresponding to the first far-field distance based on the first distance, where M1 and M2 are integers greater than or equal to 1; determines the first mean of the M1 near-field head correlation transfer functions and the second mean of the M2 far-field head correlation transfer functions; and determines the first distance filter corresponding to the first distance based on the first mean and the second mean.

[0494] It should be understood that since HRTF is highly personalized and each person's HRTF is unique, in order to weaken the influence of personalized factors, the embodiments of this application can determine multiple near-field HRTFs corresponding to near-field distances related to the first distance based on the first distance, and determine multiple far-field HRTFs corresponding to far-field distances related to the first distance based on the first distance.

[0495] For ease of description, in this embodiment of the application, the multiple near-field HRTFs corresponding to the near-field distance related to the first distance are referred to as the M1 near-field HRTFs corresponding to the first near-field distance, and in this embodiment of the application, the multiple far-field HRTFs corresponding to the far-field distance related to the first distance are referred to as the M2 far-field HRTFs corresponding to the first far-field distance.

[0496] In an implementation, the electronic device can determine, based on the first distance and a preset dataset (data in the preset dataset can refer to Table 2 in the foregoing embodiment), M1 near-field HRTFs corresponding to the first near-field distance related to the first distance and M2 far-field HRTFs corresponding to the first far-field distance related to the first distance, the preset dataset including M1 near-field HRTFs corresponding to near-field distances related to the first distance and M2 far-field HRTFs corresponding to far-field distances related to the first distance. The first near-field distance related to the first distance and the first far-field distance related to the first distance have a ratio of the first near-field distance to the first far-field distance that is approximately equal to the first distance. The implementation process of this part can refer to the implementation process of S942 in the foregoing embodiment, which will not be repeated here.

[0497] In another implementation, the electronic device can determine, based on the first distance, the first near-field distance and the first far-field distance related to the first distance, for example, a near-field distance that has a ratio to a far-field distance approximately equal to the first distance can be determined as the first near-field distance and the first far-field distance, and then M1 near-field HRTFs are calculated in real time based on the first near-field distance and M2 far-field HRTFs are calculated in real time based on the first far-field distance.

[0498] After the electronic device determines the M1 near-field HRTFs corresponding to the first near-field distance and the M2 far-field HRTFs corresponding to the first far-field distance, the electronic device can determine a ratio of a mean value of the M1 near-field HRTFs to a mean value of the M2 far-field HRTFs as the first distance filter. The implementation process of this part can refer to the implementation process of S943 and S944, which will not be repeated here.

[0499] In the embodiments of the present application, since the head-related transfer function is highly personalized, the first distance filter can be determined based on the mean value of the M1 near-field head-related transfer functions and the mean value of the M2 far-field head-related transfer functions, which can weaken the influence of the personalized factor on the determined distance filter and improve the accuracy of the determined distance filter.

[0500] In the embodiments of the present application, the electronic device determines, based on the first distance, the M1 near-field head-related transfer functions corresponding to the first near-field distance and the M2 far-field head-related transfer functions corresponding to the first far-field distance, including:

[0501] The electronic device determines, based on the first distance and the first correspondence, the M1 near-field head-related transfer functions corresponding to the first near-field distance and the M2 far-field head-related transfer functions corresponding to the first far-field distance;

[0502] The first correspondence includes a correspondence between N distances and N sets of head-related transfer functions, one distance corresponding to one set of head-related transfer functions, one set of head-related transfer functions including at least one near-field head-related transfer function corresponding to a near-field distance and at least one far-field head-related transfer function corresponding to a far-field distance, the N distances including a first distance, one set of head-related transfer functions in the N sets of head-related transfer functions including M1 near-field head-related transfer functions corresponding to a first near-field distance and M2 far-field head-related transfer functions corresponding to a first far-field distance, and N is an integer greater than 1.

[0503] It should be understood that the first correspondence can refer to the correspondence between the HRTF dataset A and the distance parameter shown in Table 2 in the above embodiment. For example, the correspondence between N distances (3 distance parameters in the second row of Table 2) and N sets of head-related transfer functions (data in columns 2, 3 and 4 of Table 2) is shown in Table 2.

[0504] It should also be understood that the first correspondence can be stored in the memory of the electronic device. In order to save the storage resources of the electronic device, the first correspondence can also be stored in the cloud server or the server.

[0505] In the implementation, the electronic device can query M1 near-field HRTFs and M2 far-field HRTFs corresponding to the first distance in Table 2. For example, the first distance is 0.4m, the distance parameter in Table 2, and the data in the third column of Table 2 is M1 near-field HRTFs and M2 far-field HRTFs. The distance implementation process of this step can refer to the implementation process of S942.

[0506] Since the electronic device has a large amount of calculation for real-time calculation of M1 near-field head-related transfer functions and M2 far-field head-related transfer functions, the embodiment of the present application can determine M1 near-field head-related transfer functions corresponding to the first near-field distance and M2 far-field head-related transfer functions corresponding to the first far-field distance based on the first distance and the first correspondence, thereby saving the calculation amount and reducing the power consumption of the electronic device.

[0507] In some embodiments, before the electronic device acquires the rendering parameter in response to the second operation, the method includes:

[0508] The electronic device displays a first interface, the first interface including a first sliding bar for positioning a plurality of distances, the maximum distance in the plurality of distances being less than a first threshold; and in response to the second operation, acquiring the rendering parameter includes: in response to the first sliding bar being positioned to a first position, acquiring a first distance corresponding to the first position.

[0509] It should be understood that the first interface refers to an audio rendering interface, and the first sliding bar refers to a sliding bar in the first interface for positioning a plurality of distances. For example, the first interface can refer to FIG. 9The interface shown, the first sliding bar refers to the distance corresponding to the sliding bar, FIG. 10 The middle controls 1410, 1420 and 1430 can all position a distance, for example, the control 1410 is used to position 0.6 m, the control 1420 is used to position 0.4 m, and the control 1430 is used to position 0.2 m. When the sliding bar slides from the left to the right, the value of the positioned distance gradually decreases. Assuming that the first threshold is FIG. 9 D1 shown in (c) in the figure, then the distance positioned by the leftmost sliding bar (the maximum distance in the plurality of distances) is less than the first threshold. The first position can be the position after the user slides the first sliding bar, or a default position, which can be, for example FIG. 10 The middle position of the first sliding bar shown.

[0510] In the embodiments of the present application, the user can position the first sliding bar to the first position, and the electronic device can acquire the first distance in response to the operation of the user positioning the first sliding bar to the first position, and then process the center channel audio signal based on the first distance, so that the user perceives the center channel audio signal at the first distance. That is, the user can actively adjust the perceived distance of the center channel audio signal through the first sliding bar, thereby improving the user experience.

[0511] In the embodiments of the present application, the rendering parameter further includes a first width, the first width is used to indicate an auditory width, and the first width is greater than a second threshold; and after the electronic device acquires the rendering parameter in response to the second operation, the method further includes:

[0512] The electronic device obtains a plurality of side channel audio signals based on the to-be-processed audio signal; performs rendering processing on each side channel audio signal in the plurality of side channel audio signals based on the first width and the first distance to obtain a plurality of rendering signals, each rendering signal including a sound source coordinate of the corresponding side channel audio signal; obtains a plurality of second audio signals based on the plurality of rendering signals; and outputs a first output signal based on the first audio signal, including: outputting the first output signal based on the plurality of second audio signals and the first audio signal.

[0513] It should be understood that the auditory width is used to indicate the width of the audio perceived by the user. The rendering parameter can further include a width parameter, and the first width refers to one of the plurality of width parameters, for example, the user can slide the width corresponding sliding bar in the interface 630 shown in (c) in the figure, each position on the sliding bar can correspond to a width parameter, and the first width refers to the width parameter corresponding to one position on the sliding bar. FIG. 9

[0514] ​The second threshold is a parameter preset by the electronic device, and the second threshold refers to the width threshold in the above embodiments. The second threshold can be obtained through testing and data analysis. For example, a tester can test multiple products in a related scheme to obtain multiple test data, and the products are in the external audio mode during testing. Each test data represents the audio width perceived by the tester. Through data analysis on the multiple test data, it is found that the numerical value of the test data is related to the layout of the physical loudspeakers of the electronic device. For example, when the electronic device is arranged with two physical loudspeakers at the bottom of the screen, the audio width perceived by the tester is approximately equal to the distance between the two physical loudspeakers. For another example, when the electronic device is arranged with two physical loudspeakers at the bottom of the screen, one physical loudspeaker is arranged at the left side of the screen and one physical loudspeaker is arranged at the right side of the screen, the audio width perceived by the tester is the distance between the two physical loudspeakers, and thus it is considered that the second threshold is related to the layout of the physical loudspeakers of the electronic device, so that when the second threshold is preset in the electronic device in the embodiments of the present application, the maximum value in the multiple test data can be taken as the second threshold.

[0515] In the multiple width parameters included in the rendering parameter in the embodiments of the present application, the minimum value in the multiple width parameters is greater than the first threshold, so that the obtained first width (the first width is one of the multiple width parameters) is greater than the second threshold.

[0516] It should also be understood that the multiple side channel audio signals can refer to the left channel audio signal and the right channel audio signal in stereo sound, such as FIG. 10 the audio signals A22 and A23 shown. It can also refer to the left channel audio signal, the right channel audio signal, the left surround channel audio signal, and the right surround channel audio in 5.1 sound, such as FIG. 9 the audio signals B22, B23, B24, and B25 shown. It can also refer to the side channel audio signal in other multi-channel sound, such as 7.1 sound, 9.1 sound, etc.

[0517] The rendering signal refers to the side channel audio signal obtained after rendering processing. The second audio signal can refer to FIG. 10 the audio signals A221 and A231 shown, and can also refer to FIG. 9 the audio signals B221, B231, B241, and B251 shown. The first audio signal can refer to FIG. 10 the audio signal A211 shown, and can also refer to FIG. 20 the audio signal B211 shown. The first output signal refers to FIG. 9 the audio signal A4 and the audio signal A5 shown, or FIG. 10 the audio signal A4 or the audio signal A5 shown.

[0518] The implementation manner in which the electronic device obtains the multiple side channel audio signals based on the to-be-processed audio signal can be referred toFIG. 14 The implementation of S93 shown can also refer to FIG. 1 The implementation of S103 shown.

[0519] The implementation of the electronic device obtaining the plurality of rendering signals can be that the electronic device can perform rendering processing on each of the plurality of side channel audio signals based on the first width and the first distance by an HRTF filter, a digital-to-analog converter, or the like, to obtain the plurality of rendering signals. The plurality of rendering signals can also be obtained by performing rendering processing on each of the plurality of side channel audio signals based on the first width and the first distance by a driving equation of a virtual loudspeaker. The embodiments of the present application do not limit this.

[0520] For example, the plurality of side channel audio signals include a left channel audio signal and a right channel audio signal. The electronic device can perform rendering processing on the left channel audio signal based on the first width and the first distance to obtain a rendered left channel audio signal, and the rendered left channel audio signal includes the sound source coordinates of the left channel audio signal. The electronic device can perform rendering processing on the right channel audio signal based on the first width and the first distance to obtain a rendered right channel audio signal, and the rendered left channel audio signal includes the sound source coordinates of the right channel audio signal. The sound source coordinates of the right channel audio signal and the sound source coordinates of the left channel audio signal can refer to FIG. 14 The coordinates of V1 to V8 shown can be referred to as the sound source coordinates of one side channel audio signal. Or part of the coordinates of V1 to V8, such as the coordinates of V1 to V4, can be referred to as the sound source coordinates of one side channel audio signal.

[0521] The implementation of the electronic device obtaining the plurality of second audio signals based on the plurality of rendering signals can refer to the implementation of S1053 and S1054 in the above embodiments, which will not be repeated here.

[0522] The implementation of the electronic device outputting the first output signal can be that the electronic device can perform processing on the plurality of second audio signals and the first audio signal by FIG. 23 Or FIG. 23 The implementation of the crosstalk module, the equalization module, the downmix module, and the post-gain module shown can perform processing on the plurality of second audio signals and the first audio signal to output the first output signal. The specific implementation can refer to the above embodiments, which will not be repeated here.

[0523] In the related scheme, the audio width perceived by the user is approximately equal to the second threshold. In the embodiments of the present application, the electronic device can perform rendering processing on each of the plurality of side channel audio signals based on the first width (the first width is greater than the second threshold) and the first distance to obtain a plurality of rendering signals, each of which includes the sound source coordinates of the corresponding side channel audio signal. Since the sound source coordinates of the side channel audio signal are obtained by rendering processing based on the first width, the width of the side channel audio perceived by the user is greater than the second threshold. Compared with the related scheme, the width of the side channel audio (the side sound image perceived by the user) can be widened, and the spatial sense of the audio perceived by the user is improved.

[0524] In some embodiments, the electronic device is configured with P1 virtual loudspeakers, P1 being greater than or equal to 1; and performing rendering processing on each of the plurality of side channel audio signals based on the first width and the first distance to obtain a plurality of rendering signals, including:

[0525] The electronic device determines, based on the first width, the first distance, and each side channel audio signal, a first parameter of each virtual loudspeaker of P2 virtual loudspeakers corresponding to each side channel audio signal, the first parameter including the sound source coordinates of the side channel audio signal corresponding to each virtual loudspeaker, P2 being less than or equal to P1; inputs the first parameter into each virtual loudspeaker to obtain an output signal of the P2 virtual loudspeakers corresponding to each side channel audio signal; and obtains the plurality of rendering signals based on the output signal of the P2 virtual loudspeakers corresponding to each side channel audio signal.

[0526] It should be understood that when P2 is equal to P1, the electronic device performs rendering processing on the side channel audio signal through the driving equation of all virtual loudspeakers of the P1 virtual loudspeakers. When P2 is less than P1, the electronic device performs rendering on the side channel audio signal through the driving equation of part of the virtual loudspeakers of the P1 virtual loudspeakers. The driving equation of the virtual loudspeaker can refer to Formula Eight described above.

[0527] It should also be understood that the first parameter can include preset parameters, which have been described in the implementation manner of the above-mentioned mode E, and the first parameter further includes some real-time calculated parameters, such as A(ω), ω, x s , x ref and which need real-time calculation. The way to calculate the above-mentioned parameters in real time can refer to the way described in mode E, which will not be described here.

[0528] The implementation manner in which the electronic device renders the side channel audio signals through all virtual speakers of the P1 virtual speakers can refer to the manner E in the foregoing embodiments. The sound source coordinates of the side channel audio signals in the manner E refer to a coordinate set that all virtual speakers (P1 virtual speakers) in the virtual speaker array need to simulate x s .

[0529] The implementation manner in which the electronic device renders the side channel audio signals through part of the virtual speakers of the P1 virtual speakers can refer to the manner F in the foregoing embodiments. The sound source coordinates of the side channel audio signals in the manner E refer to a coordinate set that part of the virtual speakers (P2 virtual speakers) in the virtual speaker array need to simulate x s .

[0530] In the embodiments, the electronic device can render the side channel audio signals through the virtual speakers. Specifically, the electronic device determines, based on the first width, the first distance, and each side channel audio signal, a first parameter of each virtual speaker in the P2 virtual speakers corresponding to each side channel audio signal, inputs the first parameter into each virtual speaker, and obtains an output signal of the P2 virtual speakers corresponding to each side channel audio signal, which can improve the clarity and fidelity of the audio.

[0531] In some embodiments, before the electronic device obtains the plurality of side channel audio signals based on the to-be-processed audio signal, the method further includes:

[0532] The electronic device displays a first interface, the first interface including a first sliding bar and a second sliding bar, the first sliding bar being used to position a plurality of distances, a maximum distance in the plurality of distances being less than a first threshold, and the second sliding bar being used to position a plurality of widths, a minimum width in the plurality of widths being greater than a second threshold; in response to the first sliding bar being positioned to a first position, obtaining a first distance corresponding to the first position; and in response to the second sliding bar being positioned to a second position, obtaining a first width corresponding to the second position.

[0533] It should be understood that the first interface refers to an audio rendering interface, the first sliding bar refers to a sliding bar in the first interface used to position the plurality of distances, and the second sliding bar refers to a sliding bar in the first interface used to position the plurality of widths. For example, the first interface can refer to the interface shown in FIG. 4 , the first sliding bar refers to a width corresponding sliding bar, and when the sliding bar is slid from the left side to the right side, the value of the positioned distance gradually increases. Assuming that the second threshold is W1 shown in (b) in FIG. 5 , then the width positioned at the leftmost side of the sliding bar (the minimum width in the plurality of widths) is greater than the second threshold W1. The second position can be a position after the user slides the second sliding bar, or a default position. The default position can be, for example, the middle position of the second sliding bar shown in FIG. 5 .

[0534] In the embodiment, the user can position the second sliding bar to the second position, and the electronic device can obtain the first width in response to the operation of the user positioning the second sliding bar to the second position, and then process the side channel audio signals based on the first width, so that the width of the side channel audio signals perceived by the user is the first width. That is, the user can actively adjust the width of the perceived side channel audio signals through the second sliding bar, thereby improving the user experience.

[0535] In some embodiments, the rendering signals include direct sound signals and early reflection sound signals.

[0536] The explanation of the direct sound signals and the early reflection sound signals can refer to the above embodiments, which will not be repeated here.

[0537] In the related scheme, the direct sound and the late reverberation sound are rendered when the audio signals are rendered. Since the late reverberation sound can reduce the authenticity of the audio, affect the clarity and purity of the audio, etc., in the embodiment of the present application, the direct sound and the early reflection sound are rendered, and the late reverberation sound is not rendered, thereby improving the authenticity, clarity and purity of the audio.

[0538] In some embodiments, the plurality of side channel audio signals includes a left channel audio signal and a right channel audio signal; or the plurality of side channel audio signals includes a left channel audio signal, a right channel audio signal, a left surround channel audio signal and a right surround audio signal.

[0539] The plurality of side channel audio signals in the embodiment of the present application can refer to the left channel audio signal and the right channel audio signal in stereo sound, or can refer to the left channel audio signal, the right channel audio signal, the left surround channel audio signal and the right surround audio signal in 5.1 sound, and of course can also refer to other multi-channel side channel audio signals, such as 7.1 sound, 9.1 sound, etc.

[0540] In some embodiments, the electronic device obtains a plurality of second audio signals based on the plurality of rendering signals, including:

[0541] The electronic device generates a binaural impulse response based on the plurality of rendering signals, and processes the plurality of side channel audio signals based on the binaural impulse response to obtain a plurality of second audio signals.

[0542] The implementation manner of the embodiment of the present application can refer to S1053 and S1054, which will not be repeated here.

[0543] The following embodiments introduce the implementation process of another audio processing method provided by the embodiment of the present application.

[0544] Please refer to ​ ,​ is a schematic diagram of an audio processing method provided by an embodiment of the present application. The method can be executed by ​ the electronic device 100 shown in the figure, or can be executed by a processor or a chip in the electronic device 100, and the embodiments of the present application do not make any limitation. For the convenience of description, the method is described in detail taking the electronic device as an example. The method includes S2310 to S2360.

[0545] S2310, the electronic device acquires a to-be-processed audio signal in response to a first operation.

[0546] It should be understood that the first operation refers to an operation of playing audio by a user, for example, the user clicks ​ the play control of a song in the music 1 application in the application layer shown in the figure, and for another example, the user clicks ​ the play control of a video in the video 1 application in the application layer shown in the figure. In addition to the click operation, the first operation can also be a voice input, a gesture, etc., and the present application does not make any limitation.

[0547] It should also be understood that the to-be-processed audio signal refers to an original audio signal, and the introduction of the original audio signal can be referred to the above embodiments.

[0548] S2320, the electronic device acquires a rendering parameter, and the rendering parameter includes a third distance, which is the distance between the electronic device and the user detected by the sensor.

[0549] It should be understood that the electronic device is configured with a sensor, which can be a distance sensor, a camera, etc. The third distance can refer to the distance parameter 2 in the above embodiments.

[0550] S2330, the electronic device obtains a mid-channel audio signal based on the to-be-processed audio signal.

[0551] The implementation manner of this step can refer to the above embodiments, which will not be described here again.

[0552] S2340, the electronic device determines a first distance filter based on the third distance, the first distance filter corresponds to a first distance, and the first distance is used to indicate an auditory distance, and the first distance is smaller than the third distance.

[0553] It should be understood that from the above embodiments, it can be known that the distance filter is essentially an HRTF, and the HRTF is related to the distance (auditory distance) between the sound source and the user.

[0554] The third distance refers to a distance between the electronic device and the user. In related solutions, the audio distance perceived by the user is approximately equal to the third distance. Therefore, in order to shorten the audio distance perceived by the user, an HRTF of a first distance (which is smaller than the third distance) can be determined, so that the audio distance perceived by the user can be shortened through the HRTF of the first distance.

[0555] In implementations, the electronic device can determine the first distance filter based on the third distance and the pre-measured HRTF dataset, and can also determine the first distance based on the third distance, and then calculate the HRTF in real time based on the first distance to determine the first distance filter. The present application does not limit this.

[0556] In S2350, the electronic device processes the mid-channel audio signal based on the first distance filter to obtain a first audio signal.

[0557] The implementation of this step can refer to the above embodiments, which will not be repeated here.

[0558] In S2360, the electronic device outputs a first output signal based on the first audio signal.

[0559] The implementation of this step can refer to the above embodiments, which will not be repeated here.

[0560] In related solutions, the audio distance perceived by the user is approximately equal to the third distance. In the present application, the electronic device can determine the first distance filter based on the third distance, the first distance filter corresponds to the first distance, the first distance is used to indicate the auditory distance, and the first distance is smaller than the third distance. Then, the mid-channel audio signal is processed based on the first distance filter, which can make the audio distance perceived by the user be the first distance. Since the first distance is smaller than the third distance, compared with related solutions, the present application can shorten the distance between the mid-channel audio signal and the user, so that the user perceives the mid-channel audio signal to be closer (the distance of the mid-channel audio signal perceived by the user can be referred to as the mid sound image), and the distance sense of the audio is improved.

[0561] Furthermore, the present application can obtain the distance between the electronic device and the user through the sensor, and can locate the position of the user (which can be quantified as the third distance). The distance of the mid-channel audio signal is accurately adjusted based on the located position of the user, and the user experience is improved.

[0562] In an embodiment of the present application, the electronic device determines the first distance filter based on the third distance, including: the electronic device determines M1 head-related transfer functions corresponding to the first near-field distance and M2 head-related transfer functions corresponding to the first far-field distance based on the third distance, M1 and M2 are integers greater than or equal to 1; determining the first mean of the M1 head-related transfer functions and the second mean of the M2 head-related transfer functions; determining the first distance filter corresponding to the first distance based on the first mean and the second mean.

[0563] It should be understood that, since the HRTF is highly personalized, the HRTF of each person is unique, and thus, in order to weaken the influence of the personalized factor, the embodiment of the present application can determine a plurality of near-field HRTFs corresponding to the first near-field distance and a plurality of far-field HRTFs corresponding to the first far-field distance based on the third distance.

[0564] In one implementation, the electronic device can determine M1 near-field HRTFs corresponding to the first near-field distance and M2 far-field HRTFs corresponding to the first far-field distance based on the third distance and a preset data set (the data in the preset data set can refer to Table 1 in the above embodiment). The implementation process of this part can refer to the implementation process of S947 in the above embodiment, which will not be described here.

[0565] In another implementation, the electronic device can first determine the first near-field distance and the first far-field distance based on the third distance, for example, the near-field distance corresponding to the ratio of the near-field distance to the far-field distance equal to the first distance can be determined as the first near-field distance and the first far-field distance, and then M1 near-field HRTFs are calculated in real time based on the first near-field distance, and M2 far-field HRTFs are calculated in real time based on the first far-field distance.

[0566] In an embodiment of the present application, since the head-related transfer function is highly personalized, the embodiment of the present application can determine the first distance filter based on the mean of the M1 head-related transfer functions and the mean of the M2 head-related transfer functions, which can weaken the influence of the personalized factor on the determined distance filter and improve the accuracy of the determined distance filter.

[0567] In some embodiments, the electronic device determines M1 head-related transfer functions corresponding to the first near-field distance and M2 head-related transfer functions corresponding to the first far-field distance based on the third distance, including:

[0568] The electronic device determines M1 head-related transfer functions corresponding to the first near-field distance and M2 head-related transfer functions corresponding to the first far-field distance based on the third distance and the second correspondence;

[0569] The second correspondence includes a correspondence between N groups of near-field head-related transfer functions and N groups of far-field head-related transfer functions, one group of near-field head-related transfer functions corresponds to one group of far-field head-related transfer functions, one group of near-field head-related transfer functions includes at least one near-field head-related transfer function corresponding to one near-field distance, one group of far-field head-related transfer functions includes at least one far-field head-related transfer function corresponding to one far-field distance, one group of near-field head-related transfer functions in the N groups of near-field head-related transfer functions includes M1 near-field head-related transfer functions corresponding to the first near-field distance, one group of far-field head-related transfer functions in the N groups of far-field head-related transfer functions includes M2 far-field head-related transfer functions corresponding to the first far-field distance, and N is an integer greater than 1.

[0570] It should be understood that the second correspondence can refer to the HRTF dataset A shown in Table 1 in the above embodiments.

[0571] It should also be understood that the second correspondence can be stored in the memory of the electronic device. In order to save the storage resources of the electronic device, the second correspondence can also be stored in the cloud server or the server.

[0572] In the implementation, the electronic device can query M1 near-field HRTFs and M2 far-field HRTFs corresponding to the third distance in Table 1 to determine the M1 near-field HRTFs and the M2 far-field HRTFs. The distance implementation process of this step can refer to the implementation process of S948.

[0573] Since the electronic device calculates M1 near-field head-related transfer functions and M2 far-field head-related transfer functions in real time, the calculation amount is large, and therefore, the embodiments of the present application can determine M1 near-field head-related transfer functions corresponding to the first near-field distance and M2 far-field head-related transfer functions corresponding to the first far-field distance based on the third distance and the second correspondence, which can save the calculation amount and reduce the power consumption of the electronic device.

[0574] In other embodiments, the rendering parameter further includes a first distance, the first distance being used to indicate a hearing distance; and determining a first distance filter based on the third distance, including:

[0575] The electronic device determines the first distance filter based on the first distance and the third distance.

[0576] It should be understood that the explanation and acquisition method of the first distance can refer to the above embodiments, which will not be described here.

[0577] It should also be understood that the first distance is acquired based on user operation, the third distance is acquired based on the sensor, and the first distance filter can shorten the audio distance perceived by the user.

[0578] In the implementation, the electronic device can determine the first distance filter based on the third distance, the first distance, and the preset HRTF dataset. The HRTF can also be calculated in real time based on the third distance and the first distance to determine the first distance filter.

[0579] The embodiment of the application can obtain the first distance through user operation, obtain the distance between the electronic device and the user through the sensor, and then determine the first distance filter based on the first distance and the third distance. The first distance filter can narrow the audio distance perceived by the user. Therefore, in the embodiment of the application, the first distance obtained through user operation and the third distance obtained through positioning can be combined to narrow the sense of distance of the center channel audio signal, which improves the user experience and accurately adjusts the distance of the center channel audio signal.

[0580] In some embodiments, the electronic device determines the first distance filter based on the first distance and the third distance, including:

[0581] The electronic device determines M1 near-field head-related transfer functions corresponding to the first near-field distance based on the third distance, determines M2 far-field head-related transfer functions corresponding to the first far-field distance based on the first distance, M1 and M2 are integers greater than or equal to 1, or determines M2 far-field head-related transfer functions corresponding to the first far-field distance based on the third distance, determines M1 near-field head-related transfer functions corresponding to the first near-field distance based on the first distance, and determines the first average of the M1 near-field head-related transfer functions and the second average of the M2 far-field head-related transfer functions, and determines the first distance filter based on the first average and the second average.

[0582] The implementation of this step can refer to the implementation process of S9412 to S9415, which will not be described here.

[0583] In some embodiments, the rendering parameter further includes a first width, the first width is used to indicate an auditory width, the first width is greater than a second threshold, and the rendering parameter further includes a first distance, the first distance is used to indicate an auditory distance, the first distance is less than the third distance, and after the electronic device obtains the rendering parameter, the method further includes:

[0584] The electronic device obtains a plurality of side channel audio signals based on the to-be-processed audio signal, performs rendering processing on each side channel audio signal in the plurality of side channel audio signals based on the first width and the first distance to obtain a plurality of rendering signals, each rendering signal includes a sound source coordinate of the corresponding side channel audio signal, obtains a plurality of second audio signals based on the plurality of rendering signals, and outputs a first output signal based on the first audio signal, including: outputting the first output signal based on the plurality of second audio signals and the first audio signal.

[0585] In some embodiments, the electronic device is configured with P1 virtual speakers, P1 is greater than or equal to 1; and the rendering processing of each of the plurality of side channel audio signals based on the first width and the first distance to obtain a plurality of rendering signals comprises:

[0586] The electronic device determines, based on the first width, the first distance, and each of the side channel audio signals, a first parameter of each of the P2 virtual speakers corresponding to each of the side channel audio signals, the first parameter comprising a sound source coordinate of the side channel audio signal corresponding to each of the virtual speakers, P2 being less than or equal to P1; inputs the first parameter into each of the virtual speakers to obtain an output signal of the P2 virtual speakers corresponding to each of the side channel audio signals; and obtains the plurality of rendering signals based on the output signal of the P2 virtual speakers corresponding to each of the side channel audio signals.

[0587] It should be noted that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0588] The embodiments of the present application provide a computer program product, when the computer program product is run on an electronic device, so that the electronic device executes the technical solutions in the above embodiments. The implementation principle and technical effects are similar to the above method related embodiments, and will not be repeated here.

[0589] The embodiments of the present application provide a readable storage medium, the readable storage medium contains instructions, when the instructions are run on an electronic device, so that the electronic device executes the technical solutions of the above embodiments. The implementation principle and technical effects are similar, and will not be repeated here.

[0590] The embodiments of the present application provide a chip, the chip is used for executing instructions, when the chip is run, executes the technical solutions in the above embodiments. The implementation principle and technical effects are similar, and will not be repeated here.

[0591] In the embodiments described above, all or some of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or some of the embodiments can be implemented in the form of a computer program product storing computer instructions. When the computer instructions are loaded into and executed by a computer, all or some of the processes or functions according to the embodiments described in the specification are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatuses. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as high-density digital video disc (digital video disc, DVD)), or semiconductor media (such as solid state disk (solid state disk, SSD)) and the like.

[0592] It should be understood that the "embodiments" mentioned throughout the specification mean that the specific features, structures or characteristics related to the embodiments are included in at least one embodiment of the present application. Therefore, the various embodiments throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that the size of the sequence number of each process described above does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0593] Those of ordinary skill in the art can understand that the various numbers such as first, second, etc. involved in the present application are only for differentiation for convenience of description, and do not limit the scope of the embodiments of the present application, nor represent the order of sequence.

[0594] In the present application, the element expressed by a singular form is intended to represent "one or more", rather than "one and only one", unless otherwise specified. In the present application, "at least one" is intended to represent "one or more" and "a plurality" is intended to represent "two or more" unless otherwise specified.

[0595] The term "and / or" used in the present text is used to connect two or more elements with the meaning that each of the elements can be present alone or in any combination with the other elements. For example, the phrase "A and / or B" means that only A can be present, only B can be present, or both A and B can be present.

[0596] The term "at least one of" used in the present text means a combination of all or any of the listed items. For example, the phrase "at least one of A, B and C" means that A can be present alone, B can be present alone, C can be present alone, A and B can be present together, B and C can be present together, A and C can be present together, or A, B and C can be present together.

[0597] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed in the present text can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0598] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0599] The same or similar parts among various embodiments in the present application can be mutually referred. In the present application, the terms and / or descriptions among different embodiments, and among various implementation manners / implementation methods / realization methods in each embodiment have consistency and can be mutually referred if there is no special description and no logical conflict. The technical features among different embodiments, and among various implementation manners / implementation methods / realization methods in each embodiment can be combined to form new embodiments, implementation manners, implementation methods or realization methods according to their inherent logical relationship. The above-mentioned implementation manners of the present application do not constitute limitation to the protection scope of the present application.

[0600] The above-mentioned is merely specific implementation manners of the present application, but the protection scope of the present application is not limited to this. Any modification or replacement within the technical range disclosed by the present application can be easily thought by any person skilled in the art, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims. In summary, the above-mentioned is merely the preferred implementation manners of the technical solutions of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. An audio processing method, characterized in that, Applied to electronic devices, the method includes: In response to the first operation, acquire the audio signal to be processed; In response to the second operation, rendering parameters are obtained, the rendering parameters including a first distance, the first distance being used to indicate auditory distance, the first distance being less than a first threshold; The center channel audio signal is obtained based on the audio signal to be processed; Based on the first distance, determine the first distance filter corresponding to the first distance; The center channel audio signal is processed based on the first distance filter to obtain the first audio signal; The first output signal is output based on the first audio signal.

2. The method according to claim 1, characterized in that, The step of determining the first distance filter corresponding to the first distance based on the first distance includes: Based on the first distance, determine M1 near-field head-related transfer functions corresponding to the first near-field distance and M2 far-field head-related transfer functions corresponding to the first far-field distance, where M1 and M2 are integers greater than or equal to 1; Determine the first mean of the M1 near-field head-correlation transfer functions and the second mean of the M2 far-field head-correlation transfer functions; Based on the first mean and the second mean, determine the first distance filter corresponding to the first distance.

3. The method according to claim 2, characterized in that, The step of determining the M1 near-field head correlation transfer functions corresponding to the first near-field distance and the M2 far-field head correlation transfer functions corresponding to the first far-field distance based on the first distance includes: Based on the first distance and the first correspondence, the M1 near-field head-related transfer functions corresponding to the first near-field distance and the M2 far-field head-related transfer functions corresponding to the first far-field distance are determined; The first correspondence includes N distances and N sets of head correlation transfer functions. One distance corresponds to one set of head correlation transfer functions. One set of head correlation transfer functions includes at least one near-field head correlation transfer function corresponding to a near-field distance and at least one far-field head correlation transfer function corresponding to a far-field distance. The N distances include the first distance. One set of head correlation transfer functions in the N sets of head correlation transfer functions includes the M1 near-field head correlation transfer functions corresponding to the first near-field distance and the M2 far-field head correlation transfer functions corresponding to the first far-field distance. N is an integer greater than 1.

4. The method according to any one of claims 1 to 3, characterized in that, Before obtaining rendering parameters in response to the second operation, the method includes: The first interface is displayed, which includes a first slider. The first slider is used to locate multiple distances, and the maximum distance among the multiple distances is less than the first threshold. And, in response to the second operation, obtaining rendering parameters includes: In response to the first slider being positioned at the first position, the first distance corresponding to the first position is obtained.

5. The method according to any one of claims 1 to 4, characterized in that, The rendering parameters also include a first width, which is used to indicate the auditory width, and the first width is greater than a second threshold. And, after obtaining the rendering parameters in response to the second operation, the method further includes: Multiple side channel audio signals are obtained based on the audio signal to be processed; Based on the first width and the first distance, each of the multiple side channel audio signals is rendered to obtain multiple rendered signals, each of which includes the sound source coordinates of the corresponding side channel audio signal; Based on the multiple rendering signals, multiple second audio signals are obtained; and, The step of outputting a first output signal based on the first audio signal includes: The first output signal is output based on multiple second audio signals and the first audio signal.

6. The method according to claim 5, characterized in that, The electronic device is configured with P1 virtual speakers, where P1 is greater than or equal to 1; and the rendering process is performed on each of the multiple side channel audio signals based on the first width and the first distance to obtain multiple rendered signals, including: Based on the first width, the first distance, and each of the side channel audio signals, determine the first parameter of each of the P2 virtual speakers corresponding to each of the side channel audio signals. The first parameter includes the sound source coordinates of the side channel audio signal corresponding to each virtual speaker, and P2 is less than or equal to P1. The first parameter is input into each of the virtual speakers to obtain the output signals of P2 virtual speakers corresponding to each side channel audio signal; Based on the output signals of the P2 virtual speakers corresponding to each side channel audio signal, a plurality of rendering signals are obtained.

7. The method according to claim 5 or 6, characterized in that, Before obtaining multiple side channel audio signals based on the audio signal to be processed, the method further includes: The first interface is displayed, which includes a first slider and a second slider. The first slider is used to locate multiple distances, the maximum distance of which is less than a first threshold. The second slider is used to locate multiple widths, the minimum width of which is greater than the second threshold. In response to the first slider being positioned at a first position, the first distance corresponding to the first position is obtained; and... In response to the second slider being positioned at the second position, the first width corresponding to the second position is obtained.

8. The method according to any one of claims 5 to 7, characterized in that, The rendering signal includes direct sound signal and early reflected sound signal.

9. The method according to any one of claims 5 to 8, characterized in that, The plurality of side channel audio signals include a left channel audio signal and a right channel audio signal; or, the plurality of side channel audio signals include the left channel audio signal, the right channel audio signal, the left surround channel audio signal, and the right surround audio signal.

10. The method according to any one of claims 5 to 9, characterized in that, The process of obtaining multiple second audio signals based on multiple rendering signals includes: Based on the plurality of rendering signals, a binaural impulse response is generated; Based on the binaural impulse response, multiple side channel audio signals are processed to obtain multiple second audio signals.

11. An audio processing method, characterized in that, Applied to an electronic device, the electronic device being configured with a sensor; and the method comprising: In response to the first operation, acquire the audio signal to be processed; Obtain rendering parameters, including a third distance, which is the distance between the electronic device and the user detected by the sensor; The center channel audio signal is obtained based on the audio signal to be processed; Based on the third distance, a first distance filter is determined, the first distance filter corresponds to a first distance, the first distance is used to indicate the auditory distance, and the first distance is less than the third distance; The center channel audio signal is processed based on the first distance filter to obtain the first audio signal; The first output signal is output based on the first audio signal.

12. The method according to claim 11, characterized in that, The step of determining the first distance filter based on the third distance includes: Based on the third distance, M1 near-field head-related transfer functions corresponding to the first near-field distance and M2 far-field head-related transfer functions corresponding to the first far-field distance are determined, where M1 and M2 are integers greater than or equal to 1; Determine the first mean of the M1 near-field head-correlation transfer functions and the second mean of the M2 far-field head-correlation transfer functions; Based on the first mean and the second mean, determine the first distance filter corresponding to the first distance.

13. The method according to claim 12, characterized in that, The step of determining the M1 near-field head correlation transfer functions corresponding to the first near-field distance and the M2 far-field head correlation transfer functions corresponding to the first far-field distance based on the third distance includes: Based on the third distance and the second correspondence, the M1 near-field head-related transfer functions corresponding to the first near-field distance and the M2 far-field head-related transfer functions corresponding to the first far-field distance are determined; The second correspondence includes the correspondence between N sets of near-field head-related transfer functions and N sets of far-field head-related transfer functions. Each set of near-field head-related transfer functions corresponds to a set of far-field head-related transfer functions. Each set of near-field head-related transfer functions includes at least one near-field head-related transfer function corresponding to a near-field distance, and each set of far-field head-related transfer functions includes at least one far-field head-related transfer function corresponding to a far-field distance. Among the N sets of near-field head-related transfer functions, one set of near-field head-related transfer functions includes the M1 near-field head-related transfer functions corresponding to the first near-field distance, and one set of far-field head-related transfer functions among the N sets of far-field head-related transfer functions includes the M2 far-field head-related transfer functions corresponding to the first far-field distance. N is an integer greater than 1.

14. The method according to claim 11, characterized in that, The rendering parameters further include a first distance, which indicates the auditory distance; and determining the first distance filter based on the third distance includes: The first distance filter is determined based on the first distance and the third distance.

15. The method according to claim 14, characterized in that, Determining the first distance filter based on the first distance and the third distance includes: Based on the third distance, determine M1 near-field head-related transfer functions corresponding to the first near-field distance; Based on the first distance, determine M2 far-field head-related transfer functions corresponding to the first far-field distance, where M1 and M2 are integers greater than or equal to 1; or, Based on the third distance, determine the M2 far-field head-related transfer functions corresponding to the first far-field distance; Based on the first distance, determine M1 near-field head-related transfer functions corresponding to the first near-field distance; as well as, Determine the first mean of the M1 near-field head-correlation transfer functions and the second mean of the M2 far-field head-correlation transfer functions; The first distance filter is determined based on the first mean and the second mean.

16. The method according to any one of claims 11 to 15, characterized in that, The rendering parameters also include a first width, which is used to indicate the auditory width, and the first width is greater than a second threshold; and the rendering parameters also include a first distance, which is used to indicate the auditory distance, and the first distance is less than the third distance. Furthermore, after obtaining the rendering parameters, the method further includes: Multiple side channel audio signals are obtained based on the audio signal to be processed; Based on the first width and the first distance, each of the multiple side channel audio signals is rendered to obtain multiple rendered signals, each of which includes the sound source coordinates of the corresponding side channel audio signal; Based on the multiple rendering signals, multiple second audio signals are obtained; and, The step of outputting a first output signal based on the first audio signal includes: The first output signal is output based on multiple second audio signals and the first audio signal.

17. The method according to claim 16, characterized in that, The electronic device is configured with P1 virtual speakers, where P1 is greater than or equal to 1; and the rendering process is performed on each of the multiple side channel audio signals based on the first width and the first distance to obtain multiple rendered signals, including: Based on the first width, the first distance, and each of the side channel audio signals, determine the first parameter of each of the P2 virtual speakers corresponding to each of the side channel audio signals. The first parameter includes the sound source coordinates of the side channel audio signal corresponding to each virtual speaker, and P2 is less than or equal to P1. The first parameter is input into each of the virtual speakers to obtain the output signals of P2 virtual speakers corresponding to each side channel audio signal; Based on the output signals of the P2 virtual speakers corresponding to each side channel audio signal, a plurality of rendering signals are obtained.

18. An electronic device, characterized in that, include: One or more processors; One or more memory units; The one or more memories store one or more computer programs, the one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform the method as claimed in claims 1 to 10, or any one of claims 11 to 17.

19. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as claimed in claims 1 to 10, or any one of claims 11 to 17.

20. A chip, characterized in that, The chip includes: Memory, used to store instructions; A processor for retrieving and executing the instructions from the memory, causing an electronic device on which the chip is mounted to perform the method as claimed in claims 1 to 10, or any one of claims 11 to 17.

Images