Audio processing method and device

Abstract

The embodiment of the invention provides an audio processing method and device, and relates to the technical field of terminals, and the method comprises the steps that terminal equipment displays a first interface; the terminal device receives a first operation for a first control in the plurality of controls; in response to the first operation, the terminal device obtains a first target parameter corresponding to the first control; wherein the first target parameter comprises one or more of the following: interaural time difference, interaural sound pressure level or interaural coherence; the terminal equipment obtains a first audio signal; the terminal equipment processes the first audio signal by using the first target parameter to obtain a second audio signal; and the terminal equipment sends the second audio signal to the earphone. Therefore, the terminal equipment can select the head feature according to the user and match the appropriate target parameter, so that the audio signal output through the target parameter can adapt to the head feature of the user, personalized processing of the audio signal is realized, and the user obtains better audio listening experience.

Description

Audio processing method and device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on August 19, 2022, with application number 202211001656.6 and application name “Audio Processing Method and Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to an audio processing method and device. Background Art

[0003] With the popularization and development of terminal devices, the number of users listening to audio through headphones has increased dramatically. When the terminal device is connected to headphones to play audio signals, the original stereo material or multi-channel audio will have an obvious "head-centered feeling" when it is downmixed and presented through headphones in a traditional way. This head-centered feeling can be understood as the sound emitted being perceived as emanating from the center of the head to the left and right sides of the ear.

[0004] Typically, a terminal device processes the original audio signal using appropriate binaural room impulse response (BRIR) parameters, so that the processed audio signal produces a better spatial effect when played through headphones. The BRIR describes the sound transmission process from the sound source to the sound receiving location (e.g., the user's ears).

[0005] However, the above audio processing method cannot enable users to achieve a better audio listening experience.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide an audio processing method and apparatus, so that a terminal device can match appropriate target parameters based on the user's selected head features, so that the audio signal output through the target parameters can adapt to the user's head features, realize personalized processing of the audio signal, and enable the user to obtain a better audio listening experience.

[0008] In a first aspect, an embodiment of the present application provides an audio processing method, which is applied to a terminal device, wherein the terminal device establishes a communication connection with a headset, and the method includes: the terminal device displays a first interface; the first interface includes multiple controls for selecting head features; the terminal device receives a first operation on a first control among the multiple controls; in response to the first operation, the terminal device obtains a first target parameter corresponding to the first control; wherein the first target parameter includes one or more of the following: interaural time difference, interaural sound pressure level, or interaural coherence; the terminal device obtains a first audio signal; the terminal device processes the first audio signal using the first target parameter to obtain a second audio signal; and the terminal device sends the second audio signal to the headset. In this way, the terminal device can match the appropriate target parameters according to the head features selected by the user, so that the audio signal output through the target parameters can adapt to the user's head features, realize personalized processing of the audio signal, and enable the user to obtain a better audio listening experience.

[0009] The first interface may be the interface shown in b in FIG. 6 , and the first control may be a control corresponding to any head feature in the interface shown in b in FIG. 6 .

[0010] In one possible implementation, the first interface further includes: a second control for selecting a spatial rendering type, and the method further includes: the terminal device receives a second operation on the second control; in response to the second operation, the terminal device obtains a second target parameter corresponding to the second control; wherein the second target parameter includes one or more of the following: a BRIR direct-mix ratio adjustment parameter, echo density, target spectrum peaks and valleys, room size, or reverberation time; the terminal device processes the first audio signal using the first target parameter, including: the terminal device processes the first audio signal using the first target parameter and the second target parameter to obtain a second audio signal. In this way, the terminal device can obtain the second target parameter corresponding to the spatial rendering type selected by the user, and the terminal device processes the original initial audio signal according to the second target parameter, so that the processed audio signal can simulate the effect when played in different spaces, so that the user can obtain a better audio listening experience.

[0011] The second control for selecting the spatial rendering type may be a control corresponding to any rendering type in the interface shown in a of FIG. 7 (or b of FIG. 7 ).

[0012] In one possible implementation, in response to a second operation, the terminal device obtains a second target parameter corresponding to the second control, including: in response to the second operation, the terminal device displays a progress bar corresponding to the second control; the progress bar is used to adjust the distance between the second audio signal and the terminal device when the earphone plays the second audio signal; the terminal device receives a third operation on the progress bar; and in response to the third operation, the terminal device obtains the second target parameter corresponding to the target position of the progress bar. This allows the user to not only simulate listening to audio in different spaces but also flexibly adjust the distance between the audio and the user, further improving the user's listening experience.

[0013] The progress bar corresponding to the second control may be the progress bar 802 shown in b in FIG. 8 .

[0014] In one possible implementation, the method further includes: the terminal device displaying a second interface; the second interface including a third control for spatially rendering the first audio signal; the terminal device receiving a fourth operation on the third control; and the terminal device displaying the first interface, including: in response to the fourth operation, the terminal device displaying the first interface. In this way, the terminal device can perform spatial rendering of the audio signal based on the user triggering the third control, thereby improving the user's listening experience when listening to the audio.

[0015] Among them, the second interface can be the interface shown in a in Figure 6, and the third control can be the opening control shown in a in Figure 6.

[0016] In one possible implementation, the method further includes: the terminal device inputting the first binaural room impulse response (BRIR) parameter and the first target parameter into a first target neural network model, and outputting a second BRIR parameter; wherein the order of the first BRIR parameter is N1, the order of the second BRIR parameter is N2, and N1 is greater than N2; the loss function of the first target neural network model is related to one or more parameters of the first target parameters; and the terminal device processing the first audio signal using the first target parameter, including: the terminal device processing the first audio signal using the second BRIR parameter. In this way, the terminal device can use the first target neural network model to reduce the order of the first BRIR parameter, thereby simplifying the complexity of audio processing.

[0017] Among them, the first BRIR parameter can be the N1-order BRIR parameter described in the embodiment of the present application; the second BRIR parameter can be the N2-order BRIR parameter described in the embodiment of the present application; the first audio signal can be the initial audio signal described in the embodiment of the present application; and the second audio signal can be the target audio signal described in the embodiment of the present application.

[0018] In one possible implementation, a terminal device inputs a first BRIR parameter and a first target parameter into a first target neural network model and outputs a second BRIR parameter, including: the terminal device inputs the first BRIR parameter, the first target parameter, and the second target parameter into the first target neural network model and outputs the second BRIR parameter; wherein the loss function is related to one or more of the target parameters; and the target parameters include the first target parameter and the second target parameter. In this way, the terminal device can use the first target parameter and the second target parameter to adjust the BRIR parameter in the first target neural network model so that the second BRIR parameter output by the first target neural network model meets various user requirements for audio signals.

[0019] In a possible implementation, the target parameter further includes: subjective loudness, or a value corresponding to the bark spectrum.

[0020] In one possible implementation, the method further includes: the terminal device inputting the first BRIR parameter into a second target neural network model to output a third BRIR parameter; the order of the first BRIR parameter is N1, the order of the third BRIR parameter is N2, and N1 is greater than N2; the terminal device adjusting the third BRIR parameter using one or more parameters in the first target parameters to obtain a second BRIR parameter; the order of the second BRIR parameter is N2; and the terminal device processing the first audio signal using the first target parameter, including: the terminal device processing the first audio signal using the second BRIR parameter. In this way, the terminal device can use the second target neural network model to reduce the order of the first BRIR parameter, thereby simplifying the complexity of audio processing.

[0021] In a possible implementation, the terminal device adjusts the third BRIR parameter using one or more parameters in the first target parameters to obtain the second BRIR parameter, including: the terminal device adjusts the third BRIR parameter using one or more parameters in the first target parameters to obtain a fourth BRIR parameter; the order of the fourth BRIR parameter is N2; and the terminal device adjusts the fourth BRIR parameter using one or more parameters in the second target parameters to obtain the second BRIR parameter.

[0022] In one possible implementation, a terminal device processes a first audio signal using a second BRIR parameter, including: obtaining a target rotation angle from the earphones; the target rotation angle is the rotation angle detected by the earphones when the earphones are worn; obtaining a target BRIR parameter corresponding to the target rotation angle from the second BRIR parameter; the order of the target BRIR parameter is N2; and processing the first audio signal using the target BRIR parameter. In this way, the terminal device can process the audio signal in real time based on the target rotation angle during playback, thereby improving the user's sense of spatial perception when listening to the audio.

[0023] In one possible implementation, the second interface further includes: a fourth control for adjusting the second audio signal to rotate with the earphones when the second audio signal is played; the method further includes: the terminal device receiving a fifth operation on the fourth control; in response to the fifth operation, the terminal device sending instruction information to the earphones for obtaining a target rotation angle; and the terminal device obtaining the target rotation angle from the earphones includes: the terminal device obtaining the target rotation angle from the earphones based on the instruction information. In this way, the user can determine whether to enable audio to rotate with the earphones based on their audio playback needs, thereby improving the user's audio listening experience.

[0024] Among them, the fourth control may be the head-following control in the interface shown in a of FIG6 .

[0025] In a second aspect, an embodiment of the present application provides an audio processing device, wherein a terminal device establishes a communication connection with a headset, and the method includes: a display unit for displaying a first interface; the first interface includes multiple controls for selecting head features; a processing unit for receiving a first operation for a first control among the multiple controls; in response to the first operation, the processing unit is also used to obtain a first target parameter corresponding to the first control; wherein the first target parameter includes one or more of the following: interaural time difference, interaural sound pressure level, or interaural coherence; the processing unit is also used to obtain a first audio signal; the processing unit is also used to process the first audio signal using the first target parameter to obtain a second audio signal; and a communication unit is used to send the second audio signal to the headset.

[0026] In one possible implementation, the first interface also includes: a second control for selecting a spatial rendering type, a processing unit, and a processing unit for receiving a second operation for the second control; in response to the second operation, the processing unit is further used to obtain a second target parameter corresponding to the second control; wherein the second target parameter includes one or more of the following: BRIR direct mixing ratio adjustment parameter, echo density, target spectrum peaks and valleys, room size, or reverberation time; the processing unit is further used to process the first audio signal using the first target parameter and the second target parameter to obtain a second audio signal.

[0027] In one possible implementation, in response to the second operation, the display unit is used to display a progress bar corresponding to the second control; the progress bar is used to adjust the distance between the second audio signal and the terminal device when the headset plays the second audio signal; the processing unit is also used to receive a third operation on the progress bar; in response to the third operation, the processing unit is also used to obtain the second target parameter corresponding to the target position of the progress bar.

[0028] In one possible implementation, the display unit is also used to display a second interface; the second interface includes a third control for spatially rendering the first audio signal; the processing unit is also used to receive a fourth operation for the third control; in response to the fourth operation, the display unit is also used to display the first interface.

[0029] In one possible implementation, the processing unit is further used to input the first binaural room impulse response (BRIR) parameter and the first target parameter into the first target neural network model, and output a second BRIR parameter; wherein the order of the first BRIR parameter is N1, the order of the second BRIR parameter is N2, and N1 is greater than N2; the loss function of the first target neural network model is related to one or more parameters of the first target parameter; the processing unit is further used to process the first audio signal using the first target parameter, including: the terminal device processes the first audio signal using the second BRIR parameter.

[0030] In one possible implementation, the processing unit is further used to input the first BRIR parameter, the first target parameter, and the second target parameter into the first target neural network model, and output the second BRIR parameter; wherein the loss function is related to one or more parameters of the target parameters; and the target parameters include the first target parameter and the second target parameter.

[0031] In a possible implementation, the target parameter further includes: subjective loudness, or a value corresponding to the bark spectrum.

[0032] In one possible implementation, the processing unit is further used to input the first BRIR parameter into the second target neural network model and output a third BRIR parameter; the order of the first BRIR parameter is N1, the order of the third BRIR parameter is N2, and N1 is greater than N2; the processing unit is further used to adjust the third BRIR parameter using one or more parameters in the first target parameters to obtain a second BRIR parameter; the order of the second BRIR parameter is N2; and the processing unit is further used to process the first audio signal using the second BRIR parameter.

[0033] In one possible implementation, the processing unit is further configured to adjust the third BRIR parameter using one or more parameters in the first target parameters to obtain a fourth BRIR parameter; the order of the fourth BRIR parameter is N2; and the processing unit is further configured to adjust the fourth BRIR parameter using one or more parameters in the second target parameters to obtain a second BRIR parameter.

[0034] In one possible implementation, the communication unit is further configured to obtain a target rotation angle from the earphone; wherein the target rotation angle is a rotation angle detected by the earphone when the earphone is worn; the processing unit is further configured to obtain a target BRIR parameter corresponding to the target rotation angle from the second BRIR parameter; the order of the target BRIR parameter is N2; and the processing unit is further configured to process the first audio signal using the target BRIR parameter.

[0035] In one possible implementation, the second interface also includes: a fourth control for adjusting the second audio signal as the earphone rotates when playing the second audio signal; a processing unit for receiving a fifth operation for the fourth control; in response to the fifth operation, a communication unit for sending indication information for obtaining a target rotation angle to the earphone; and the communication unit for obtaining the target rotation angle from the earphone according to the indication information.

[0036] In a third aspect, an embodiment of the present application provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the terminal device executes the audio processing method as described in the first aspect or any one of the implementations of the first aspect.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed, the computer executes the audio processing method described in the first aspect or any implementation of the first aspect.

[0038] In a fifth aspect, a computer program product includes a computer program. When the computer program is executed, the computer executes the audio processing method as described in the first aspect or any one of the implementations of the first aspect.

[0039] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] FIG1 is a schematic diagram of a scenario provided by an embodiment of the present application;

[0041] FIG2 is a schematic diagram of the hardware structure of a terminal device provided in an embodiment of the present application;

[0042] FIG3 is a schematic diagram of the hardware structure of a headset provided in an embodiment of the present application;

[0043] FIG4 is a schematic diagram of a software structure of a system architecture provided in an embodiment of the present application;

[0044] FIG5 is a flow chart of an audio processing method provided in an embodiment of the present application;

[0045] FIG6 is a schematic diagram of an interface for enabling spatial audio rendering according to an embodiment of the present application;

[0046] FIG7 is a schematic diagram of another interface for enabling spatial audio rendering provided by an embodiment of the present application;

[0047] FIG8 is a schematic diagram of an interface for enabling spatial audio rendering according to another embodiment of the present application;

[0048] FIG9 is a schematic diagram of another interface for enabling spatial audio rendering provided by an embodiment of the present application;

[0049] FIG10 is a schematic diagram of a process for generating BRIR parameters according to an embodiment of the present application;

[0050] FIG11 is a schematic diagram of the structure of a neural network model provided in an embodiment of the present application;

[0051] FIG12 is a schematic diagram of the structure of another neural network model provided in an embodiment of the present application;

[0052] FIG13 is a schematic diagram of another process for generating BRIR parameters according to an embodiment of the present application;

[0053] FIG14 is a schematic structural diagram of an audio processing device provided in an embodiment of the present application;

[0054] FIG15 is a schematic diagram of the hardware structure of another terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. For example, the first value and the second value are merely used to distinguish different values ​​and do not limit their order. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit different values.

[0056] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0057] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, c can be single or multiple.

[0058] For example, Figure 1 is a schematic diagram of a scenario provided by an embodiment of the present application. In the embodiment corresponding to Figure 1, a mobile phone is used as an example for illustration, and this example does not constitute a limitation on the embodiment of the present application.

[0059] As shown in Figure 1, the scenario may include a terminal device 101 and headphones 102 worn by the user. The headphones 102 may be headphones as shown in Figure 1, or the headphones 102 may be true wireless stereo (TWS), or wired headphones, etc. The type of the headphones 102 is not limited in the embodiments of the present application.

[0060] For example, when the terminal device 101 establishes a communication connection with the headset 102, the user can use the terminal device 101 to process the initial audio signal and play the processed target audio signal through the headset 102. For example, the terminal device can process the initial audio signal using the BRIR parameter so that the processed target audio signal can produce a better spatial effect when played through the headset.

[0061] However, since everyone listens to sounds in their environment through their body, head, and ears in real life, a personalized head transfer function is also important for spatial positioning. However, professional head transfer function measurement methods are very complex, time-consuming, and labor-intensive. Furthermore, due to different head characteristics of different users, such as the distance between their ears, ear size, ear position, and head size, these different head characteristics can affect the audio signal they hear, making it difficult for each user to achieve a good audio listening experience.

[0062] In view of this, an embodiment of the present application provides an audio processing method, wherein a terminal device displays a first interface; the first interface includes multiple controls for selecting head features; when the terminal device receives a first operation directed to a target control among the multiple controls, the terminal device can match appropriate target parameters for the target control; so that a first audio signal outputted through the target parameters can be adapted to the user's head features, achieving personalized processing of the audio signal, and providing the user with a better audio listening experience. The target parameters include one or more of the following: interaural time difference, interaural sound pressure level, or interaural coherence.

[0063] It is understandable that the above-mentioned terminal devices may also be referred to as terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. The terminal devices may be mobile phones, smart TVs, wearable devices, tablet computers (Pad), computers with wireless transceiver functions, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc. Among them, the wearable devices may include: smart watches, smart bracelets, smart belts, etc. The embodiments of the present application do not limit the specific technologies and specific device forms adopted by the terminal devices and wearable devices.

[0064] Therefore, in order to better understand the embodiment of the present application, the structure of the terminal device of the embodiment of the present application is introduced below. For example, Figure 2 is a schematic diagram of the structure of a terminal device provided by the embodiment of the present application.

[0065] The terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, an indicator 192, a camera 193, and a display screen 194, etc.

[0066] It is understood that the structures illustrated in the embodiments of the present application do not constitute specific limitations on the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0067] The processor 110 may include one or more processing units. The different processing units may be independent devices or integrated into one or more processors. The processor 110 may also be provided with a memory for storing instructions and data.

[0068] USB interface 130 is an interface that complies with USB standards and specifications, and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. USB interface 130 can be used to connect a charger to charge the terminal device, or to transfer data between the terminal device and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect to other terminal devices, such as AR devices.

[0069] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. The power management module 141 is used to connect the charging management module 140 to the processor 110.

[0070] The wireless communication function of the terminal device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0071] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. The antennas in a terminal device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0072] Mobile communication module 150 can provide wireless communication solutions for terminal devices, including 2G / 3G / 4G / 5G. It can include at least one filter, switch, power amplifier, and low-noise amplifier (LNA). Mobile communication module 150 receives electromagnetic waves from antenna 1, filters and amplifies the received electromagnetic waves, and transmits them to the modem processor for demodulation.

[0073] The wireless communication module 160 can provide wireless communication solutions applied to terminal devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), etc.

[0074] The terminal device implements display functions through a GPU, display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering.

[0075] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the terminal device may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0076] The terminal device can realize the shooting function through the ISP, camera 193, video codec, GPU, display 194 and application processor.

[0077] The camera 193 is used to capture static images or videos. In some embodiments, the terminal device may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0078] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0079] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 can include a program storage area and a data storage area.

[0080] The terminal device can implement audio functions, such as audio playback or recording, through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0081] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. Speaker 170A, also known as a "horn," is used to convert audio electrical signals into sound signals. The terminal device includes at least one speaker 170A. The terminal device can listen to music or make hands-free calls through speaker 170A. Receiver 170B, also known as a "handset," is used to convert audio electrical signals into sound signals. When the terminal device receives a call or voice message, the voice can be heard by holding receiver 170B close to the human ear.

[0082] In the embodiment of the present application, the terminal device may be provided with multiple playback devices, which may include: a speaker 170A and / or a receiver 170B. In the scenario where the terminal device plays a video, at least one speaker 170A and / or at least one receiver 170B plays an audio signal simultaneously.

[0083] The headphone jack 170D is used to connect wired headphones. The microphone 170C, also known as a "microphone" or "microphone," is used to convert sound signals into electrical signals. In the embodiment of the present application, the terminal device can receive the sound signal used to wake up the terminal device based on the microphone 170C and convert the sound signal into an electrical signal that can be subsequently processed, such as the voiceprint data described in the embodiment of the present application. The terminal device can have at least one microphone 170C.

[0084] The sensor module 180 may include one or more of the following sensors, such as: a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, or a bone conduction sensor (not shown in FIG2 ).

[0085] Keys 190 include a power button, a volume button, and other buttons. Keys 190 can be mechanical or touch-sensitive. The terminal device can receive key inputs and generate key signal inputs related to user settings and function control of the terminal device. Indicator 192 can be an indicator light that can be used to indicate charging status, battery level changes, messages, missed calls, notifications, and the like.

[0086] The software system of the terminal device can adopt a layered architecture, event-driven architecture, micro-kernel architecture, microservice architecture, or cloud architecture, etc., which will not be described here.

[0087] For example, FIG3 is a schematic diagram of the hardware structure of a headset provided in an embodiment of the present application.

[0088] As shown in Figure 3, the headset includes one or more processors 310, one or more memories 320, a communication interface 330, an audio acquisition circuit, and an audio playback circuit. The audio acquisition circuit may further include at least one microphone 340 and an analog-to-digital converter (ADC) 350. The audio playback circuit may further include a speaker 360 and a digital-to-analog converter (DAC).

[0089] The headset may further include one or more sensors 380 , such as an inertial measurement unit (IMU), a proximity sensor, a motion sensor, etc. These hardware components may communicate on one or more communication buses.

[0090] In an embodiment of the present application, the IMU can be used to measure the motion posture of the headset. For example, the IMU can be used to determine the rotation angle of the user's head when wearing the headset. The IMU can be provided with a gyroscope sensor and an acceleration sensor.

[0091] The processor 310 is the control center of the headset. The processor may also be called a control unit, controller, microcontroller or other appropriate terms. The processor 310 uses various interfaces and lines to connect the various components of the headset. In a possible embodiment, the processor 310 may also include one or more processing cores. In a possible embodiment, the processor 310 may be integrated with a main control unit and a signal processing module. The main control unit (MCU) is used to receive data collected by the sensor 380 or monitoring signals from the signal processing module or control signals from the terminal (such as a mobile phone APP), and finally control the headset through comprehensive judgment and decision-making.

[0092] The memory 320 can be coupled to the processor 310 or connected to the processor 310 via a bus to store various software programs and / or multiple sets of instructions and data. The memory 320 can also store a communication program that can be used to communicate with the terminal. In one example, the memory 320 can also store data / program instructions, and the processor 310 can be used to call and execute the data / program instructions in the memory 320. Exemplarily, the memory 320 can store multiple sets of noise reduction parameters. Optionally, the memory 320 can be a memory external to the MCU or a storage unit built into the MCU.

[0093] The communication interface 330 is used to communicate with the terminal, and the communication method can be wired or wireless. When the communication method is wired communication, the communication interface 330 can be connected to the terminal via a cable. When the communication method is wireless communication, the communication interface 330 is used to receive and send radio frequency signals. The wireless communication method supported by it can be, for example, Bluetooth communication, wireless fidelity (Wi-Fi) communication, infrared communication, or cellular 2 / 3 / 4 / 5 generation (2G / 3G / 4G / 5G) communication, etc.

[0094] Microphone 340 can be used to collect sound signals (or audio signals, which are analog signals). Analog-to-digital converter 350 is used to convert the analog signals collected by microphone 340 into digital signals and send the digital signals to processor 310 for processing. In specific embodiments, the digital signals can be sent to the signal processing module for processing. The signal processing module can transmit the processed signals (e.g., mixed audio signals) to digital-to-analog converter 370. Digital-to-analog converter 370 can convert the received signals into analog signals, which are then transmitted to speaker 360. The speaker is used to play the analog signals so that the user can hear the sounds.

[0095] In this embodiment of the present application, the communication interface 330 can be used to send the head rotation angle detected by the IMU to the terminal device, so that the terminal device can use the head rotation angle to match the appropriate BRIR parameters. In addition, the communication interface 330 can also be used to receive audio signals sent by the terminal device.

[0096] It is understandable that the above-mentioned headphones may also be called earplugs, headsets, walkmans, audio players, media players, head-mounted receivers, earphone devices or other appropriate terms, and the embodiments of the present application are not limited to this.

[0097] The embodiment of the present application takes the layered architecture Android system as an example to illustrate the software structure of the terminal device. For example, FIG4 is a schematic diagram of the software structure of a system architecture provided by the embodiment of the present application.

[0098] As shown in Figure 4, the system architecture may include a terminal device and a headset. A layered architecture may be displayed in the terminal device, with each layer having a clear role and division of labor. Layers communicate with each other via software interfaces. In some embodiments, the Android system is divided from top to bottom into an application layer, an application framework layer, and a hardware abstraction layer (HAL). It is understood that the layered architecture may include other layers depending on the actual scenario, and this will not be described in detail in the embodiments of the present application.

[0099] The application layer may include a series of applications. For example, the application layer may include a music application and a headphone settings application. The music application may be used to control the playback and pause of audio signals, and the headphone settings application may be used to configure the audio playback effects when playing audio through headphones. Both the music application and the headphone settings application may be system applications in the terminal device, or may be applications provided by a third party, which is not limited in the embodiments of the present application.

[0100] It is understandable that the application layer may also include: camera, calendar, phone, map, phone, music, mailbox, video, social and other applications (not shown in Figure 4), which is not limited in the embodiments of the present application.

[0101] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0102] The application framework layer may include: an audio playback module, and a head-following algorithm module. The audio playback module is used to instruct the audio playback HAL to call the hardware interface for audio playback based on the instruction information sent by the music application; the audio playback module can also be used to process and forward the received audio signal. The head-following algorithm module can be used to determine the relative angle between the user's head and the terminal device based on the head rotation angle; alternatively, the head-following algorithm module can also be used to send the acquired head rotation angle to the spatial audio algorithm module based on the instruction sent by the headphone setting application.

[0103] In a possible implementation, the application framework layer may further include: a window manager, a content provider, a resource manager, a view system, and a notification manager (not shown in FIG. 4 ).

[0104] For example, the window manager manages windowed applications. It can determine the display size, determine whether a status bar exists, lock the screen, handle touches, drag the screen, and take screenshots. Content providers store and retrieve data and make it accessible to applications. This data can include video, images, audio, incoming and outgoing calls, browsing history and bookmarks, and the phone book. The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images. The resource manager provides applications with various resources, such as localized strings, icons, images, layout files, and video files. The notification manager enables applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically, requiring no user interaction. For example, the notification manager is used to notify users of download completions and message reminders. The notification manager can also display notifications in the system's top status bar, such as icons or scrolling text, for applications running in the background. It can also display notifications in the form of dialog windows. For example, a text message may be displayed in the status bar, a prompt tone may be emitted, the terminal device may vibrate, an indicator light may flash, etc.

[0105] The purpose of the HAL layer is to abstract the hardware and provide a unified interface for querying hardware devices for upper-layer applications, or to provide data storage services or data processing services for upper-layer applications. In the embodiment of the present application, the HAL layer may include: an audio playback HAL and a spatial audio algorithm module.

[0106] The audio playback HAL is used to forward the audio signal. For example, the audio playback HAL is used to play the audio signal sent from the audio playback module through hardware.

[0107] The spatial audio algorithm module is used to determine the N2-order BRIR parameters using the target parameters and the N1-order BRIR parameters, and to spatially render the audio signal using the N2-order BRIR parameters. Alternatively, the spatial audio algorithm can also be used to send the spatially rendered audio signal to the audio playback module for processing. The target parameters may include the first target parameter, the second target parameter, the subjective loudness, and the bark spectrum values ​​described in the embodiments of this application.

[0108] As shown in Figure 4, the headset may include: a Bluetooth module, a head posture algorithm module, an audio playback module, and an IMU, etc. Among them, the Bluetooth module can be used to receive the audio signal sent by the terminal device, and the Bluetooth module can also be used to forward the head rotation angle calculated by the head posture algorithm module; the audio playback module can be used to play the audio signal received by the Bluetooth module; the head posture module is used to calculate the head rotation angle based on the motion data collected by the IMU; the IMU is used to collect the motion data of the user's head when wearing the headset. Among them, the motion data may include: gyroscope data, acceleration data, etc.

[0109] The following specific embodiments are used to describe in detail the technical solution of the present application and how the technical solution of the present application solves the above technical problems. The following specific embodiments can be implemented independently or in combination with each other. For the same or similar concepts or processes, some embodiments may not be described in detail.

[0110] For example, Figure 5 is a flowchart of an audio processing method provided in an embodiment of the present application. In the embodiment corresponding to Figure 5, the audio processing method may involve a terminal device and a headset.

[0111] As shown in FIG5 , the audio processing method may include the following steps:

[0112] S501: The terminal device establishes a communication connection with the headset.

[0113] For example, the terminal device and the headset can establish a communication connection via a wired manner; alternatively, the terminal device and the headset can establish a communication connection via a wireless manner. For example, the terminal device and the headset can establish a communication connection via Bluetooth, WIFI, or wireless means such as connecting to the same cloud account. The embodiment of the present application does not limit the method for establishing a communication connection between the terminal device and the headset.

[0114] S502: When the terminal device receives an operation from the user to turn on the spatial audio rendering switch in the headset setting interface, the terminal device displays multiple controls for selecting target head features in the spatial audio rendering function.

[0115] Among them, the spatial audio rendering function can be understood as simulating the effect of listening to audio in different spaces, processing the initial audio signal so that the processed audio signal has a sense of space and three-dimensionality.

[0116] For example, Figure 6 is a schematic diagram of an interface for enabling spatial audio rendering provided by an embodiment of the present application. In the embodiment corresponding to Figure 6, a mobile phone is used as an example for illustration, and this example does not constitute a limitation on the embodiment of the present application.

[0117] When the terminal device receives a trigger operation from the user for a function for setting up the headphones, the terminal device may display an interface as shown in a of FIG6 , which may be a headphone setting interface. As shown in a of FIG6 , the interface may include: a control 601 for turning on spatial audio rendering (or referred to as an on control 601), a control 602 for turning off spatial audio rendering (or referred to as an off control 602), a control 603 for indicating that the audio can follow the head movement during the spatial audio rendering process (or referred to as a follow-head movement control 603), and a control 604 for indicating that the audio can not follow the head movement during the spatial audio rendering process (or referred to as a not-follow-head-movement control 604). The function for setting up the headphones may be a function in a headphone setting application.

[0118] In a possible implementation, the interface shown in a in Figure 6 may also include: a control for exiting the headphone settings, a control for turning on more functions of the headphone, an indicator for displaying the connection status of the headphone and the battery level of the left and right earphones and the headphone box, a control for noise reduction, a control for turning off noise reduction, a control for transparent transmission, and a control for setting up dual-device connection, etc. The content displayed in the interface shown in a in Figure 6 is not specifically limited in the embodiment of the present application.

[0119] In the interface shown in a of Figure 6 , when the terminal device receives a trigger operation from the user to turn on control 601, the terminal device may display an interface as shown in b of Figure 6 . As shown in b of Figure 6 , the interface may display sub-functions corresponding to spatial audio rendering, such as a head feature selection function. The interface where the head feature selection function is located may display multiple head features: such as head feature 605, head feature 606, head feature 607, head feature 608, head feature 609, head feature 610, head feature 611, and head feature 612. In the embodiment of the present application, there is no limitation on the number of head features and the style of head features set in the terminal device. The interface shown in b of Figure 6 may also include prompt information. For example, the prompt information may be displayed as: If you do not select any head feature, a default head feature will be configured for you.

[0120] It is understandable that different users have different head features, for example, the width and height of the user's head, the distance between the user's ears, the size of the ears, the position of the ears, and other head features may all be different. As shown in b of Figure 6, the ear width of head feature 606 is smaller than the ear width of head feature 605; the ear size of head feature 607 is smaller than the ear size of head feature 605 and smaller than the ear size of head feature 606; the ear position of head feature 608 is lower; the head feature length of head feature 609 is smaller than the head feature length of head feature 605, and the ear size of head feature 609 is larger than the ear size of head feature 605; the head feature length of head feature 610 is smaller than the head feature length of head feature 605, and the ear size of head feature 610 is smaller than the ear size of head feature 605; the head feature length of head feature 611 is longer than the head feature length of head feature 605, and the ear size of head feature 611 is smaller than the ear size of head feature 605; the head feature length of head feature 612 is longer than the head feature length of head feature 605, and the ear size of head feature 612 is larger than the ear size of head feature 605.

[0121] In this way, the terminal device can be pre-configured with multiple head features, and the user can select the head feature that is most similar to his or her own from the pre-configured multiple head features.

[0122] S503: When the terminal device receives an operation on a control corresponding to a target head feature, the terminal device obtains a first target parameter.

[0123] Among them, the control corresponding to the target head feature can be a control corresponding to any head feature in the interface shown in b in Figure 6; the first target parameter can be a parameter related to the target head feature, for example, the first target parameter can include one or more of the following: interaural time difference (ITD), interaural sound pressure level (ILD), or interaural coherence (IC).

[0124] In a possible implementation, the first target parameter may also include other parameters related to head features, which is not limited in the embodiments of the present application.

[0125] For example, as shown in the interface b in Figure 6, when the terminal device receives a trigger operation from the user for any head feature, for example, when the terminal device receives a trigger operation from the user for head feature 605, the terminal device can obtain the first target parameter corresponding to the head feature 605.

[0126] S504: When the terminal device receives a control for enabling audio playback to follow head movement, the terminal device sends instruction information for obtaining a head rotation angle to the headset.

[0127] The instruction information for obtaining the head rotation angle can be sent to the headset via Bluetooth. For example, as shown in the interface a of Figure 6, when the terminal device receives a user trigger operation for the follow head movement control 603, the terminal device can send the instruction information for obtaining the head rotation angle to the headset via Bluetooth.

[0128] S505: The headset sends the head rotation angle to the terminal device.

[0129] For example, the IMU in the headset can obtain the movement posture of the headset when the user wears the headset, and determine the head rotation angle based on the head posture algorithm module in the headset; further, the headset can send the head rotation angle to the terminal device based on Bluetooth.

[0130] In a possible implementation, when the headset and the terminal device establish a communication connection based on the step shown in S501, the terminal device may also set the audio to follow the head movement by default when playing the audio signal, thereby avoiding multiple triggering by the user. In the scenario where the terminal device sets the audio to follow the head movement by default when playing the audio signal, the headset may also periodically send the head rotation angle to the terminal device.

[0131] S506: The terminal device obtains an N2-order BRIR parameter based on the N1-order BRIR parameter and the first target parameter.

[0132] The N1-order BRIR parameter may be preset in the terminal device or the cloud server.

[0133] In one implementation, the terminal device may input the N1-order BRIR parameter and the first target parameter into a neural network model, and output the N2-order BRIR parameter.

[0134] In the embodiments of the present application, the neural network model can be set in a cloud server, so that the terminal device can input the N1-order BRIR parameter and the first target parameter into the neural network model in the cloud server, and the N2-order BRIR parameter is generated in the cloud server. Alternatively, the neural network model can be set in the terminal device, so that the terminal device can input the N1-order BRIR parameter and the first target parameter into the local neural network model to generate the N2-order BRIR parameter. The N1-order BRIR parameter can be set in the terminal device or in the cloud server, and this is not limited in the embodiments of the present application.

[0135] The neural network model described in this implementation is used to reduce the order of the BRIR parameter and to adjust the neural network model according to the first target parameter; N1 is greater than N2.

[0136] In another implementation, the terminal device may input the N1-order BRIR parameter into a neural network model to output an N2-order BRIR intermediate parameter; and adjust the N2-order BRIR intermediate parameter at least once using at least one parameter from the first target parameter to obtain a final N2-order BRIR parameter. The neural network model described in this other implementation is used to reduce the order of the BRIR parameter.

[0137] It is understandable that since the complexity of processing audio signals using high-order BRIR parameters is high and occupies more memory space, the terminal device can use a neural network model to reduce the order of the BRIR parameters and simplify the complexity of audio processing.

[0138] S507: The terminal device processes the initial audio signal using the N2-order BRIR parameter and the head rotation angle to obtain a target audio signal.

[0139] The head rotation angle may include: horizontal angle, tilt angle, and pitch angle. The horizontal angle may range from [0°-360°], the pitch angle may range from [-90°-90°], and the tilt angle may range from [-90°-90°]. The horizontal angle may be understood as the rotation angle of the user's head in the horizontal direction; the pitch angle may be understood as the rotation angle of the user's head in the vertical direction, such as the angle corresponding to when the user raises his head or the angle corresponding to when the user lowers his head; the tilt angle may be the angle between the tilt position of the head and the horizontal position when the user tilts his head.

[0140] The initial audio signal (or first audio signal) may include: an initial left channel audio signal and an initial right channel audio signal; the N2-order BRIR parameter may also include an N2-order BRIR parameter corresponding to the left channel and an N2-order BRIR parameter corresponding to the right channel.

[0141] It can be understood that there is a correspondence between the head rotation angle and the N2-order BRIR parameter. For example, the terminal device can obtain the N2-order target BRIR parameter corresponding to any head rotation angle, and use the left channel N2-order target BRIR parameter in the N2-order target BRIR parameter to convolve the initial left channel audio signal, and use the right channel N2-order target BRIR parameter in the N2-order target BRIR parameter to convolve the initial right channel audio signal to obtain the target audio signal (or called the second audio signal).

[0142] Among them, when the head rotation angle includes: horizontal angle, tilt angle, and pitch angle, the terminal device can store the correspondence between one or more of the horizontal angle, tilt angle, and pitch angle and the N2-order BRIR parameter; then, when using the N2-order BRIR parameter to process the initial audio signal, the N2-order BRIR parameter corresponding to one or more of the horizontal angle, tilt angle, and pitch angle is obtained, and the N2-order BRIR parameter corresponding to the angle is used to process the audio signal.

[0143] S508: The terminal device sends the target audio signal to the headset.

[0144] S509: The earphone plays the target audio signal through the speaker.

[0145] It can be understood that the target audio signal may include a target audio signal corresponding to the left channel and a target audio signal corresponding to the right channel, so that the headset can play the target audio signal corresponding to the left channel through the left speaker and play the target audio signal corresponding to the right channel through the right speaker.

[0146] Based on this, the terminal device can not only use the neural network model to reduce the order of the BRIR parameters, thereby reducing the complexity of the audio processing process; but also, the terminal device can match the appropriate first target parameters based on the user's selection of the target head features, so that the terminal device can process the initial audio signal through the N2-order BRIR parameters related to the first target parameters, thereby realizing personalized processing of the audio signal, allowing the user to obtain a better audio listening experience.

[0147] It can be understood that the steps S506-S507 in the embodiment corresponding to Figure 5 can be executed in the terminal device, or in the processor of the headset, or in the cloud server, and this is not limited in the embodiments of the present application.

[0148] Based on the embodiment corresponding to FIG. 5 , the terminal device may also obtain a second target parameter based on a user's selection of a rendering type for the spatial audio rendering function. Furthermore, the terminal device may input the second target parameter into the neural network model in step S506 to output an N2-order BRIR parameter.

[0149] In this scenario, the step shown in S506 may be: the terminal device inputs the N1-order BRIR parameter, the first target parameter, and the second target parameter into the neural network model, and outputs the N2-order BRIR parameter.

[0150] The second target parameter may include one or more of the following, for example: a BRIR direct-to-mix ratio adjustment parameter, echo density, specific spectrum peaks and valleys, room size, or reverberation time.

[0151] It can be understood that the second target parameter can also be used to adjust the neural network.

[0152] For example, FIG7 is a schematic diagram of another interface for enabling spatial audio rendering provided in an embodiment of the present application.

[0153] When the terminal device receives the user's operation of sliding the screen upward in the interface shown in b in Figure 6, the terminal device can display the interface shown in a in Figure 7. As shown in the interface in a in Figure 7, the interface may include: multiple spatial rendering types corresponding to the spatial audio rendering function, for example: control 701 corresponding to the concert hall rendering type, control 702 corresponding to the outdoor rendering type (or referred to as outdoor control 702), control 703 corresponding to the indoor rendering type (or referred to as indoor control 703), and controls corresponding to other rendering types. The interface shown in a in Figure 7 may also include prompt information, for example, the prompt information may be displayed as: If you do not select a spatial rendering type, a default idle rendering type will be configured for you.

[0154] In a possible implementation, when the terminal device receives a trigger operation from the user on the control 701 corresponding to the concert hall rendering type, the terminal device can obtain the second target parameter corresponding to the control 701 corresponding to the concert hall rendering type, so that the second target parameter can be used to adjust the N1-order BRIR parameter in the neural network model.

[0155] In one implementation, in the interface shown in FIG. 7 a, when the terminal device receives a user trigger operation on the outdoor control 702, the terminal device displays the interface shown in FIG. The interface shown in FIG. 7 b may include outdoor rendering types such as ocean rendering type, forest rendering type, and mountain rendering type. The specific content of the outdoor rendering types is not limited in this embodiment of the present application. Any of these outdoor rendering types may correspond to a set of second target parameters.

[0156] In another implementation, in the interface shown in FIG7 a, when the terminal device receives a user trigger operation on the indoor control 703, the terminal device displays the interface shown in FIG7 c. The interface shown in FIG7 c may include indoor rendering types corresponding to: a church rendering type, a stadium rendering type, and a bedroom rendering type, etc. The specific content of the indoor rendering type is not limited in this embodiment of the application. Each indoor rendering type may correspond to a set of second target parameters.

[0157] In this way, the terminal device can obtain the second target parameter corresponding to the spatial rendering type based on the user's selection of the spatial rendering type. The terminal device processes the original initial audio signal based on the second target parameter, so that the processed audio signal can simulate the effect of playing in different spaces, so that the user can obtain a better audio listening experience.

[0158] In a possible implementation, the terminal device may also support adjustment of the distance between the sound source and the user (the user may also be understood as the sound source listening position, such as the user's ears) in any spatial rendering type. The distance between the sound source and the user corresponds to the BRIR direct mix ratio adjustment parameter (and / or echo density) in the second target parameter.

[0159] In an embodiment of the present application, the terminal device may also support two methods for adjusting the distance between the sound source and the user. For example, the terminal device may support adjusting the distance between the sound source and the user by using a progress bar (see the embodiment corresponding to FIG8 ); or the terminal device may support adjusting the distance between the sound source and the user by inputting a distance value between the sound source and the user in a text box (see the embodiment corresponding to FIG9 ).

[0160] In one implementation, the terminal device may support adjusting the distance between the sound source and the user through a progress bar.

[0161] Exemplarily, FIG8 is a schematic diagram of an interface for enabling spatial audio rendering provided in an embodiment of the present application.

[0162] The terminal device displays an interface as shown in a of FIG8 . The content displayed in the interface can be similar to the interface shown in b of FIG7 , and will not be repeated here. As shown in the interface of a of FIG8 , when the terminal device receives a trigger operation of the user for the control 801 corresponding to the concert hall rendering type, the terminal device can display an interface as shown in b of FIG8 . As shown in the interface of b of FIG8 , the interface may include: a progress bar 802 for adjusting the distance between the sound source and the user. The progress bar 802 can be in a default position, such as the default position can be displayed as 0. The other content displayed in the interface can be similar to the interface shown in a of FIG8 , and will not be repeated here again. Among them, the default position can be understood as the shortest distance between the sound source and the user, such as the shortest distance can be 0.2 meters, and the range of the distance between the sound source and the user can be in the range of 0.2 meters to 20 meters.

[0163] In the interface shown in b of FIG8 , when the terminal device receives a signal from the user sliding the progress bar, causing the distance between the adjusted sound source and the user to increase by 10 meters, the terminal device can obtain the BRIR direct mixing ratio adjustment parameter (and / or echo density) corresponding to the 10-meter increase in the distance between the sound source and the user, and display the interface shown in c of FIG8 , so that the terminal device can input the BRIR direct mixing ratio adjustment parameter (and / or echo density) corresponding to the 10-meter increase in the distance between the user and the user into the neural network model described in step S506 to adjust the neural network model. The interface shown in c of FIG8 may include: a progress bar 803, which indicates that the distance between the sound source and the user has increased by 10 meters, and the number 10 may be displayed around the progress bar 803.

[0164] It is understandable that the progress bar can be set to multiple fixed positions, and any fixed position can correspond to the change in the distance between the sound source and the user. For example, 0, 5, 10, 15, and 20 can be set in the progress bar.

[0165] In a possible implementation, when the terminal device receives a trigger operation from the user for any rendering type in the interface shown in b in Figure 7, the terminal device can display a progress bar corresponding to any rendering type. The function of this progress bar is the same as that of the progress bar 802 in the interface shown in b in Figure 8, and will not be repeated here.

[0166] In a possible implementation, the distance between the sound source and the user may also correspond to the second target parameter, such as the BRIR direct-mix ratio adjustment parameter echo density, target spectrum peaks and valleys, room size, and / or reverberation time. The process of determining the second target parameter based on the distance between the sound source and the user is not further described here.

[0167] In another implementation, the terminal device may support adjusting the distance between the sound source and the user by inputting a distance value between the sound source and the user in a text box.

[0168] Exemplarily, FIG9 is a schematic diagram of another interface for enabling spatial audio rendering provided in an embodiment of the present application.

[0169] The terminal device displays an interface as shown in a in Figure 9 . The content displayed in the interface may be similar to the interface shown in b in Figure 7 and will not be described in detail here. As shown in the interface in a in Figure 9 , when the terminal device receives a trigger operation from the user on the control 901 corresponding to the concert hall rendering type, the terminal device may display an interface as shown in b in Figure 9 . As shown in the interface in b in Figure 9 , the interface may include: a text box 902 for inputting the distance between the sound source and the user, and the text box 902 may display 0 by default. Other content displayed in the interface may be similar to the interface shown in a in Figure 9 and will not be described in detail here.

[0170] In the interface shown in b of Figure 9 , when the terminal device receives an operation in which a user enters a numerical value, for example, 10, into text box 902, the terminal device may display the interface shown in c of Figure 9 , and the terminal device obtains the BRIR direct-mix ratio adjustment parameter (and / or echo density) corresponding to the entered numerical value 10. The interface shown in c of Figure 9 may include a text box 903, in which the value 10 may be displayed.

[0171] In a possible implementation, the distance range that can be entered in the text box 903 can be 0-20, and the interface shown in c in Figure 9 can also display indication information for indicating the distance range that can be entered in the text box. This is not limited in the embodiment of the present application.

[0172] Based on the embodiment corresponding to Figure 8 and the embodiment corresponding to Figure 9, the user can not only simulate listening to audio in different spaces, but also flexibly adjust the distance between the audio and the user, further improving the user's auditory experience when listening to audio.

[0173] Based on the embodiment corresponding to FIG5 , in a possible implementation, the method in which the terminal device outputs the N2-order BRIR parameter using the neural network model in step S506 can be referred to the embodiments corresponding to FIG10 and FIG11 .

[0174] For example, FIG10 is a flow chart of a method for generating BRIR parameters according to an embodiment of the present application. As shown in FIG10 , the method for generating BRIR parameters may include the following steps:

[0175] S1001. The terminal device inputs the N1-order BRIR parameter, the first target parameter, the BRIR reverberation length, and the second target parameter into the neural network model, and outputs the N2-order BRIR parameter.

[0176] The BRIR reverberation length may indicate an order of a BRIR parameter. For example, the BRIR reverberation length may be a preset value. For example, the BRIR reverberation length may be N2.

[0177] In an embodiment of the present application, the first target parameter, BRIR reverberation length and the second target parameter can be used to adjust the N1-order BRIR parameter in the neural network model. The method for obtaining the first target parameter can refer to the embodiment corresponding to Figure 6, and the method for obtaining the second target parameter can refer to the embodiments corresponding to Figures 7-9.

[0178] Among them, the neural network model can be: a convolutional recurrent neural network (CRNN) model, a deep convolutional recurrent neural network (DCNRR) model, a Transformer model, a sequence-to-sequence generation model, etc., which is not limited in the embodiments of the present application.

[0179] For example, FIG11 is a schematic diagram of the structure of a neural network model provided in an embodiment of the present application.

[0180] As shown in Figure 11, the terminal device can partition the N1-order BRIR parameters. For example, the terminal device can divide the N1-order BRIR parameters into a direct response area, an early reflection area, and a late reverberation area based on the space size and / or reverberation time; the N1-order BRIR parameters corresponding to each area are respectively subjected to short-time Fourier transform (STFT) processing to obtain the STFT-processed N1-order BRIR parameters corresponding to each area.

[0181] The terminal device inputs the N1-order BRIR parameters corresponding to each area into the neural network model to obtain the N2-order BRIR parameters corresponding to each area; and performs STFT processing on the N2-order BRIR parameters corresponding to each area to obtain the STFT-processed N2-order BRIR parameters corresponding to each area.

[0182] Furthermore, the terminal device calculates the loss function using the N1-order BRIR parameters after STFT processing corresponding to each zone and the N2-order BRIR parameters after STFT processing corresponding to each zone, and outputs the final N2-order BRIR parameters when the loss function converges.

[0183] In a possible implementation, the input of the neural network model may also include: an N1-order BRIR parameter, a blank N2-order BRIR parameter, a first target parameter, a BRIR reverberation length, and a second target parameter. In a scenario where the terminal device inputs the N1-order BRIR parameter and the blank N2-order BRIR parameter into the neural network model, the neural network model library may also be: a generative adversarial network (GAN) model, a variational auto-encoder (VAE) model, etc., which is not limited in the embodiments of the present application.

[0184] For example, FIG12 is a schematic diagram of the structure of another neural network model provided in an embodiment of the present application.

[0185] As shown in Figure 12, the terminal device can partition the blank N2-order BRIR parameters and the N1-order BRIR parameters respectively, and perform STFT processing on the two parameters after partitioning, and then input them into the neural network model to obtain the output results corresponding to the blank N2-order BRIR parameters and the output results corresponding to the N1-order BRIR parameters.

[0186] The terminal device inputs the output result corresponding to the blank N2-order BRIR parameter into the N1-N2-order fully connected neural network to obtain the N1-N2 neural network output result. The N1-N2-order fully connected neural network is used to reduce the order of the blank N2-order BRIR parameter, for example, reducing the order of the output result corresponding to the blank N2-order BRIR parameter from N1 to N2.

[0187] Furthermore, the terminal device can use the output results of the N1-N2 fully connected neural network and the output results corresponding to the N1-order BRIR parameters to calculate the loss function, and when the loss function converges, perform an inverse short-time Fourier transform (ISTFT) on the output results of the N1-N2 fully connected neural network to obtain the final N2-order BRIR parameters.

[0188] S1002: The terminal device processes the N2-order BRIR parameters with a resolution of M1 into N2-order BRIR parameters with a resolution of M2 using an interpolation method.

[0189] In the embodiment of the present application, M1 is greater than M2, and the resolution of the BRIR parameter can be used to indicate the BRIR parameters corresponding to different angles.

[0190] For example, when the resolution of the BRIR parameter is 5°, an N2-order BRIR parameter can be set every 5°, and 360° can correspond to 72 N2-order BRIR parameters. For example, the terminal device can set the N2-order BRIR parameters at 0°, 5°, 10°, 15°, and 20°, respectively.

[0191] Alternatively, when the resolution of the BRIR parameter is 1°, an N2-order BRIR parameter may be set for every 1°, and 360° may correspond to 360 N2-order BRIR parameters.

[0192] Exemplarily, the interpolation method may include triangular interpolation or linear interpolation, etc., which is not limited in the embodiments of the present application.

[0193] It is understandable that the terminal device can increase the number of BRIR parameters by interpolating the BRIR parameters, so that the terminal device can match appropriate BRIR parameters for the user at different rotation angles, thereby improving the accuracy of audio processing.

[0194] Furthermore, the terminal device can match the N2-order BRIR parameter corresponding to the head rotation angle according to the head rotation angle, so that the audio signal can be adjusted at any time according to the change of the user's rotation angle, so that the user can obtain a better audio listening experience.

[0195] For example, FIG13 is a schematic diagram of another process for generating BRIR parameters according to an embodiment of the present application. As shown in FIG13 , the method for generating BRIR parameters may include the following steps:

[0196] S1301. The terminal device inputs the N1-order BRIR parameter into the neural network model and outputs the N2-order BRIR parameter.

[0197] In a possible implementation, the terminal device may also input the N1-order BRIR parameter and the blank N2-order BRIR parameter into the neural network model, and output the N2-order BRIR parameter.

[0198] S1302: The terminal device adjusts the N2-order BRIR parameters using the ITD and / or ILD to obtain the N2-order BRIR parameters adjusted by the ITD and / or ILD.

[0199] Among them, the ITD and ILD can be preset by the terminal device, or the ITD and ILD can also be obtained by the terminal device based on the embodiment corresponding to Figure 6, which is not limited in the embodiments of the present application.

[0200] In a possible implementation, the terminal device may also use ITD, ILD and / or IC to adjust the N2-order BRIR parameters to obtain the N2-order BRIR parameters after ITD and / or ILD adjustment.

[0201] S1303: The terminal device uses an interpolation method to process the N2-order BRIR parameters with a resolution of M1 and after ITD and / or ILD adjustment into N2-order BRIR parameters with a resolution of M2.

[0202] S1304: The terminal device adjusts the N2-order BRIR parameter with a resolution of M2 based on the preset BRIR reverberation length to obtain an N3-order BRIR parameter.

[0203] The preset BRIR reverberation length may be N3, where N3 is smaller than N2.

[0204] S1305: The terminal device adjusts the N3-order BRIR parameter using the BRIR direct-mix ratio adjustment parameter and / or the echo density to obtain a final N3-order BRIR parameter.

[0205] In a possible implementation, the terminal device may also adjust the N3-order BRIR parameters using the BRIR direct-to-mix ratio adjustment parameter, echo density, target spectrum peaks and valleys, room size, and / or reverberation time to obtain the final N3-order BRIR parameters.

[0206] Based on the description in the embodiment corresponding to FIG. 13 , it can also be understood that the terminal device adjusts the N2-order BRIR parameter output by the neural network model using one or more of the first and second target parameters to obtain a final BRIR parameter. The order of the final BRIR parameter can be N2 or N3, which is not limited in the embodiments of the present application.

[0207] Among them, the BRIR direct-to-mix ratio adjustment parameters, echo density, target spectrum peaks and valleys, room size and / or reverberation time can be preset by the terminal device, or can also be obtained by the terminal device based on the embodiments corresponding to Figures 7-9, which is not limited in the embodiments of the present application.

[0208] Based on this, the terminal device can not only generate BRIR parameters related to user needs in the neural network model based on the embodiment corresponding to Figure 10; it can also generate BRIR parameters related to user needs by multiple adjustments to the BRIR parameters output by the neural network model based on the embodiment corresponding to Figure 13. The method of generating BRIR parameters in the embodiments of the present application is not specifically limited.

[0209] Based on the embodiment corresponding to Figure 10, the loss function of the neural network module described in the embodiment corresponding to Figure 10 can be related to one or more of the following parameters, such as: ITD, ILD, IC, BRIR direct-to-mix ratio parameters, echo density, specific spectral peaks and valleys, subjective loudness, and the numerical value corresponding to the bark spectrum, etc.

[0210] For example, the predicted BRIR parameters after STFT processing and the actual BRIR parameters after STFT processing can be obtained, and the corresponding ITD, ILD, and IC can be obtained. 1left -E 1right )-(E 2left -E 2right )|

[0211] Among them, E 1left -E 1right The ILD, E corresponding to the real BRIR parameters after STFT processing 2left -E 2right It can be the ILD corresponding to the predicted BRIR parameter after STFT processing. 1left -T 1right )-(T 2left -T 2right )|

[0212] Among them, T 1left -T 1right It can be the ITD corresponding to the real BRIR parameters after STFT processing, T 2left -T 2right It can be the ITD corresponding to the predicted BRIR parameter after STFT processing. F3=|Γ1-Γ2|

[0213] Wherein, Γ1 may be the IC corresponding to the true BRIR parameter after STFT processing, and Γ2 may be the IC corresponding to the predicted BRIR parameter after STFT processing.

[0214] When λ is the number of frames and μ is the frequency, IC can be defined as:

[0215] Among them, P XIXI (λ,μ) can be the autopower spectrum of XI(λ,μ), P XRXR (λ,μ) can be the autopower spectrum of XR(λ,μ), P XIXR (λ,μ) can be the cross power spectrum between XI(λ,μ) and XR(λ,μ). XIXI (λ,μ)=γP XIXI (λ,μ)+(1-γ)|XI(λ,μ)| 2 P XIXR(λ,μ)=γP XIXR (λ,μ)+(1γ)XI(λ,μ)XR * (λ,μ)

[0216] Where * represents conjugate calculation, and γ can be a smoothing factor between adjacent frames.

[0217] Furthermore, the loss function can be determined as: Loss1 = α1×F1+α2×F2+α3×F3

[0218] Among them, α1, α2, and α3 may be preset parameters.

[0219] In a possible implementation, one or more of F1, F2, and F3 can also be independently used as the loss function of the neural network model, which is not limited in the embodiments of the present application.

[0220] For example, the predicted BRIR parameters after STFT processing of different regions after partitioning and the real BRIR parameters after STFT processing of different regions can be obtained. Further, the difference F4 between the direct mixing ratios (wherein the direct mixing ratio can be (the sum of the energy of the late reverberation and the early reflection) / the energy of the direct response), the difference F5 between the echo densities (wherein the echo density can be the ratio of the energy of the late reverberation to the total energy), and the difference F6 between specific spectral peaks and spectral valleys (wherein the specific spectral peaks and spectral valleys can be the spectral peaks and spectral valleys in the preset frequency band) are calculated. F4 = |(E early-rev1 +E late-rev1 ) / E dir1 -(E early-rev2 +E late-rev2 ) / E dir1 |

[0221] in,( arly-rev1 +E late-rev1 ) / E dir1 The direct mixing ratio corresponding to the real BRIR parameters after STFT processing in different regions (E early-rev2 +E late-rev2 ) / E dir2 It can be the direct mixing ratio corresponding to the predicted BRIR parameters after STFT processing in different regions. late-rev1 / E dir1 -E late-rev2 / E dir2 |

[0222] Among them, E late-rev1 / E dir1 The echo density corresponding to the real BRIR parameters after STFT processing in different regions, E late-rev2 / E dir2is the echo density corresponding to the predicted BRIR parameters after STFT processing in different regions. 1top -E 1bottom )-(E 2top -E 2bottom )|

[0223] Among them, E 1top -E 1bottom It can be the difference between the spectral peaks and valleys corresponding to the real BRIR parameters after STFT processing in different regions, E 2top -E 2bottom It can be the difference between the peaks and valleys of the spectrum corresponding to the predicted IR parameters after STFT processing in different regions. Loss2 = α4×F4+α5×F5+α6×F6

[0224] Among them, α4, α5, and α6 may be preset parameters.

[0225] In a possible implementation, one or more of F4, F5, and F6 can also be independently used as the loss function of the neural network model, which is not limited in the embodiments of the present application.

[0226] For example, the predicted BRIR parameters after STFT processing of different regions after partitioning and the actual BRIR parameters after STFT processing of different regions can be obtained, and the corresponding subjective loudness can be obtained. Loss3 = |LK1-LK2|

[0227] Among them, LK1 can be the actual BRIR parameters corresponding to the subjective loudness of different regions after STFT processing; LK2 can be the predicted BRIR parameters corresponding to the subjective loudness of different regions after STFT processing.

[0228] Specifically, the subjective loudness LK of any signal can be: LK = -0.0691 + 10log 10 (L+R)

[0229] Among them, L can be the average value calculated by summing the squares of the results of the left channel audio signal being processed by multiple filters, and R can be understood as the average value calculated by summing the squares of the results of the right channel audio signal being processed by multiple filters.

[0230] For example, the predicted BRIR parameters after STFT processing of different regions after partitioning and the actual BRIR parameters after STFT processing of different regions can be obtained, and the corresponding bark spectrum values ​​can be obtained. Loss4=|(E 1B_dir +E 1B_early_rev +E 1B_late_rev )-(E 2B_dir +E 2B_early_rev +E2B_late_rev )|

[0231] Among them, E 1B_dir +E 1B_early_rev +E 1B_late_rev The bark spectrum values ​​corresponding to the real BRIR parameters after STFT processing in different regions, E 2B_dir +E 2B_early_rev +E 2B_late_rev It can be the bark spectrum value corresponding to the predicted BRIR parameters after STFT processing in different regions.

[0232] Among them, any bark spectrum value B can be:

[0233] Here, f can be understood as the frequency of the signal.

[0234] Furthermore, the terminal device can also determine the loss function Loss of the neural network model total Can be: Loss total =β1×Loss1+β2×Loss2+β3×Loss3+β4×Loss4

[0235] Among them, β1, β2, β3, and β4 can all be preset parameters.

[0236] It is understandable that the loss function in the neural network model can also be related to other parameters, which is not limited in the embodiments of the present application.

[0237] It will be understood that the interface described in the embodiments of the present application is merely an example and does not constitute a limitation on the embodiments of the present application.

[0238] The method provided in the embodiment of the present application is described above in conjunction with Figures 5-13. The device for executing the above method provided in the embodiment of the present application is described below. As shown in Figure 14, Figure 14 is a structural schematic diagram of an audio processing device provided in the embodiment of the present application. The audio processing device can be a terminal device in the embodiment of the present application, or a chip or chip system within the terminal device.

[0239] As shown in FIG14 , an audio processing device 1400 can be used in a communication device, circuit, hardware component, or chip. The audio processing device includes a display unit 1401, a processing unit 1402, and a communication unit 1403. Display unit 1401 is used to support the display steps performed by audio processing device 1400; processing unit 1402 is used to support the information processing steps performed by audio processing device 1400; and communication unit 1403 is used to instruct audio processing device 1400 to perform steps such as sending and receiving data. Communication unit 1403 can be an input or output interface, pin, or circuit.

[0240] Specifically, an embodiment of the present application provides an audio processing device 1400, in which a terminal device establishes a communication connection with a headset, and the method includes: a display unit 1401, for displaying a first interface; the first interface includes multiple controls for selecting head features; a processing unit 1402, for receiving a first operation for a first control among multiple controls; in response to the first operation, the processing unit 1402, is also used to obtain a first target parameter corresponding to the first control; wherein the first target parameter includes one or more of the following: interaural time difference, interaural sound pressure level, or interaural coherence; the processing unit 1402, is also used to obtain a first audio signal; the processing unit 1402, is also used to process the first audio signal using the first target parameter to obtain a second audio signal; and a communication unit 1403, is used to send the second audio signal to the headset.

[0241] In a possible embodiment, the audio processing device may further include a storage unit 1404. The processing unit 1402 and the storage unit 1404 are connected via a circuit. The storage unit 1404 may include one or more memories, which may be devices in one or more devices or circuits for storing programs or data. The storage unit 1404 may exist independently and be connected to the processing unit 1402 of the audio processing device via a communication line. The storage unit 1404 may also be integrated with the processing unit 1402.

[0242] The storage unit 1404 can store computer-executable instructions for the method in the terminal device, so that the processing unit 1402 executes the method in the above embodiment. The storage unit 1404 can be a register, a cache, or a RAM, etc., and the storage unit 1404 can be integrated with the processing unit 1402. The storage unit 1404 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions. The storage unit 1404 can be independent of the processing unit 1402.

[0243] Figure 15 is a schematic diagram of the hardware structure of another terminal device provided in an embodiment of the present application. As shown in Figure 15, the terminal device includes a processor 1501, a communication line 1504 and at least one communication interface (communication interface 1503 is exemplified in Figure 15).

[0244] The processor 1501 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0245] Communications link 1504 may include circuitry that transmits information between the aforementioned components.

[0246] The communication interface 1503 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, wireless local area networks (WLAN), etc.

[0247] Possibly, the terminal device may further include a memory 1502 .

[0248] The memory 1502 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be independent and connected to the processor via a communication line 1504. The memory may also be integrated with the processor.

[0249] The memory 1502 is used to store computer-executable instructions for executing the solution of the present application, and the execution is controlled by the processor 1501. The processor 1501 is used to execute the computer-executable instructions stored in the memory 1502, thereby implementing the method provided by the embodiment of the present application.

[0250] Possibly, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, and the embodiments of the present application do not specifically limit this.

[0251] In a specific implementation, as an embodiment, the processor 1501 may include one or more CPUs, such as CPU0 and CPU1 in FIG15 .

[0252] In a specific implementation, as an example, a terminal device may include multiple processors, such as processor 1501 and processor 1505 in FIG15 . Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0253] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. For example, the available medium can include magnetic media (e.g., floppy disk, hard disk or tape), optical media (e.g., digital versatile disc (DVD)), or semiconductor media (e.g., solid state disk (SSD)).

[0254] The present application also provides a computer-readable storage medium. The methods described in the above embodiments can be implemented in whole or in part via software, hardware, firmware, or any combination thereof. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one location to another. The storage medium can be any target medium that can be accessed by a computer.

[0255] As one possible design, computer-readable media may include compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM or other optical disc storage; computer-readable media may include magnetic disk storage or other magnetic disk storage devices. Moreover, any connecting line may also be appropriately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically using lasers.

[0256] The above combinations are also included within the scope of computer-readable media. The above are only specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An audio processing method, characterized in that: Applied to a terminal device, the terminal device establishes a communication connection with a headset, and the method includes: The terminal device displays a first interface; the first interface includes a plurality of controls for selecting head features; The terminal device receives a first operation on a first control among the multiple controls; In response to the first operation, the terminal device obtains a first target parameter corresponding to the first control; wherein the first target parameter includes one or more of the following: interaural time difference, interaural sound pressure level, or interaural coherence; The terminal device obtains a first audio signal; The terminal device processes the first audio signal using the first target parameter to obtain a second audio signal; The terminal device sends the second audio signal to the headset.

2. The method according to claim 1, characterized in that The first interface further includes: a second control for selecting a spatial rendering type, and the method further includes: The terminal device receives a second operation on the second control; In response to the second operation, the terminal device obtains a second target parameter corresponding to the second control; wherein the second target parameter includes one or more of the following: a BRIR direct-to-mix ratio adjustment parameter, an echo density, a target spectrum peak and valley, a room size, or a reverberation time; The terminal device processes the first audio signal using the first target parameter, including: the terminal device processes the first audio signal using the first target parameter and the second target parameter to obtain a second audio signal.

3. The method according to claim 2, characterized in that The step of obtaining, by the terminal device, a second target parameter corresponding to the second control in response to the second operation includes: In response to the second operation, the terminal device displays a progress bar corresponding to the second control; the progress bar is used to adjust the distance between the second audio signal and the terminal device when the headset plays the second audio signal; The terminal device receives a third operation on the progress bar; In response to the third operation, the terminal device obtains the second target parameter corresponding to the target position of the progress bar.

4. The method according to any one of claims 2 or 3, characterized in that The method further comprises: The terminal device displays a second interface; the second interface includes a third control for spatial rendering of the first audio signal; The terminal device receives a fourth operation on the third control; The terminal device displays a first interface, comprising: in response to the fourth operation, the terminal device displays the first interface.

5. The method according to claim 4, characterized in that The method further comprises: The terminal device inputs the first binaural room impulse response BRIR parameter and the first target parameter into a first target neural network model, and outputs a second BRIR parameter; wherein the order of the first BRIR parameter is N1, the order of the second BRIR parameter is N2, and N1 is greater than N2; and the loss function of the first target neural network model is related to one or more parameters of the first target parameters; The terminal device processes the first audio signal using the first target parameter, including: the terminal device processes the first audio signal using the second BRIR parameter.

6. The method according to claim 5, characterized in that The terminal device inputs the first BRIR parameter and the first target parameter into a first target neural network model, and outputs a second BRIR parameter, including: The terminal device inputs the first BRIR parameter, the first target parameter, and the second target parameter into the first target neural network model, and outputs the second BRIR parameter; wherein the loss function is related to one or more parameters of the target parameters; and the target parameters include the first target parameter and the second target parameter.

7. The method according to claim 6, characterized in that The target parameters also include: subjective loudness, or a value corresponding to the bark spectrum.

8. The method according to claim 4, characterized in that The method further comprises: The terminal device inputs the first BRIR parameter into the second target neural network model and outputs a third BRIR parameter; the order of the first BRIR parameter is N1, the order of the third BRIR parameter is N2, and N1 is greater than N2; The terminal device adjusts the third BRIR parameter using one or more parameters in the first target parameters to obtain a second BRIR parameter; the order of the second BRIR parameter is N2; The terminal device processes the first audio signal using the first target parameter, including: the terminal device processes the first audio signal using the second BRIR parameter.

9. The method according to claim 8, characterized in that The terminal device adjusts the third BRIR parameter by using one or more parameters in the first target parameters to obtain a second BRIR parameter, including: The terminal device adjusts the third BRIR parameter using one or more parameters in the first target parameters to obtain a fourth BRIR parameter; the order of the fourth BRIR parameter is N2; The terminal device adjusts the fourth BRIR parameter using one or more parameters in the second target parameters to obtain the second BRIR parameter.

10. The method according to claim 5 or 8, characterized in that The terminal device processes the first audio signal using the second BRIR parameter, including: The terminal device obtains a target rotation angle from the headset; wherein the target rotation angle is a rotation angle detected by the headset when the headset is worn; The terminal device obtains a target BRIR parameter corresponding to the target rotation angle from the second BRIR parameter; the order of the target BRIR parameter is N2; The terminal device processes the first audio signal using the target BRIR parameter.

11. The method according to claim 10, characterized in that The second interface further includes: a fourth control for adjusting the second audio signal as the earphone rotates when the second audio signal is played, and the method further includes: The terminal device receives a fifth operation on the fourth control; In response to the fifth operation, the terminal device sends instruction information for acquiring the target rotation angle to the headset; The terminal device obtains the target rotation angle from the headset, comprising: the terminal device obtains the target rotation angle from the headset according to the indication information.

12. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the terminal device is caused to perform the method according to any one of claims 1 to 11.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the computer is caused to perform the method according to any one of claims 1 to 11.

14. A computer program product, characterized in that The invention comprises a computer program which, when being executed, causes a computer to execute the method according to any one of claims 1 to 11.

Images