Method and apparatus for processing audio

Abstract

The application discloses a processing method and device of audio, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a first audio to be played and a first playing angle of the first audio, wherein the first playing angle is an angle of a sound source playing the first audio relative to a target user on a horizontal plane in a virtual surround scene; correcting the first playing angle based on playing angle correction information corresponding to the target user to obtain a corrected playing angle of the first audio, wherein the playing angle correction information is a difference value between a playing angle of a test audio and a listening angle feedback by the target user in a process of playing the test audio; and playing the first audio based on the corrected playing angle.

Description

Technical Field

[0001] This application belongs to the field of audio processing technology, specifically relating to an audio processing method and apparatus. Background Technology

[0002] In the application of virtual reality glasses, in order to provide users with an immersive virtual experience, it is necessary to combine audio and visual effects. This requires that, with only two speakers placed on both sides of the user's ears, the user can perceive that the sound comes from different directions and distances, thus creating a sense of spatial sound effects for the user.

[0003] Currently, HRTF (Head-Related Transfer Function) data is typically used to create the spatial effect of sound effects. However, HRTF data is measured and modeled based on an individual's head and ear structure, and individual differences are a significant factor affecting the HRTF data. Therefore, the use of standardized HRTF data to create the spatial effect of sound effects ignores individual differences, resulting in virtual sound effects that are not audibly realistic and cannot accurately reproduce the effect of sound coming from different directions and distances. Summary of the Invention

[0004] The purpose of this application is to provide an audio processing method and apparatus that can solve the technical problem that existing virtual sound effects are not realistic enough in terms of auditory perception due to individual differences.

[0005] In a first aspect, embodiments of this application provide an audio processing method, the method comprising:

[0006] Obtain the first audio to be played and the first playback angle of the first audio, wherein the first playback angle is the angle of the sound source playing the first audio on the horizontal plane relative to the target user in a virtual surround scene;

[0007] Based on the playback angle correction information corresponding to the target user, the first playback angle is corrected to obtain the corrected playback angle of the first audio. The playback angle correction information is the difference between the playback angle of the test audio and the listening angle reported by the target user during the playback of each test audio.

[0008] The first audio is played based on the corrected playback angle.

[0009] Secondly, embodiments of this application provide an audio processing apparatus, the apparatus comprising:

[0010] The acquisition module is used to acquire the first audio to be played and the first playback angle of the first audio. The first playback angle is the angle of the sound source playing the first audio on the horizontal plane relative to the target user in a virtual surround scene.

[0011] The correction module is used to correct the first playback angle based on the playback angle correction information corresponding to the target user, so as to obtain the corrected playback angle of the first audio. The playback angle correction information is the difference between the playback angle of the test audio and the listening angle fed back by the target user during the playback of each test audio.

[0012] The playback module is used to play the first audio based on the corrected playback angle.

[0013] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method provided in the first aspect.

[0014] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method provided in the first aspect.

[0015] Fifthly, embodiments of this application provide a chip, which includes a processor and a communication interface, the communication interface and the processor being coupled together, the processor being used to run programs or instructions to implement the method provided in the first aspect.

[0016] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method as provided in the first aspect.

[0017] In the audio processing method and apparatus of this application, a first audio to be played and a first playback angle of the first audio on a horizontal plane can be obtained. Based on the playback angle correction information corresponding to the target user, the first playback angle is corrected to obtain a corrected playback angle of the first audio. The playback angle correction information is the difference between the playback angle of the test audio and the perceived listening angle reported by the target user during the playback of each test audio. The first audio is then played based on the corrected playback angle. This application utilizes the perceived listening angle of the test audio reported by the user and the original playback angle of the test audio during testing of the target user to obtain playback angle correction information. This information is then used to correct the first playback angle of each first audio to be played, resulting in a corrected playback angle. The first audio is then played based on the corrected playback angle. This audio processing method can provide a more accurate, personalized, and realistic sound spatial effect for each user, making it more suitable for the head and ear shape of the target user, and providing a more realistic virtual sound effect experience. Attached Figure Description

[0018] Figure 1 This is a schematic flowchart of an audio processing method provided in one embodiment of this application;

[0019] Figure 2 This is a schematic diagram of an audio testing interface for an audio processing method provided in one embodiment of this application.

[0020] Figure 3 This is a schematic diagram of an audio test interface for an audio processing method provided in another embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the structure of an audio processing apparatus provided in another embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0025] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0026] In related technologies, HRTF (Head-Related Transfer Function) is usually used to create a sense of space in sound effects. By using the acquired HRTF data, the position parameters of sound sources in space at different directions and distances can be simulated in the headphones and applied to sound effect processing to create a sense of space in the sound effects.

[0027] However, HRTF data is measured and modeled based on an individual's head and ear structure, and individual differences are a significant factor affecting HRTF data. Therefore, related technologies use standardized HRTF data to create a sense of space in sound effects, ignoring individual differences. This results in virtual sound effects that are not realistic enough to accurately reproduce the effects of sound coming from different directions and distances.

[0028] To address the aforementioned technical problems, this application provides an audio processing method. The audio processing method provided by this application will be described in detail below with reference to the accompanying drawings and specific embodiments and scenario information.

[0029] like Figure 1 As shown, Figure 1 This is a flowchart illustrating an audio processing method provided in one embodiment of this application. This embodiment of the application provides an audio processing method, which may include:

[0030] S101, obtain the first audio to be played and the first playback angle of the first audio, wherein the first playback angle is: the angle of the sound source playing the first audio on the horizontal plane relative to the target user in a virtual surround scene;

[0031] In this embodiment, the first audio signal can be an audio signal to be played on an XR (Extended Reality) device. The XR device can be a VR (Virtual Reality) device, an AR (Augmented Reality) device, or a MR (Mixed Reality) device.

[0032] XR devices typically output audio and video signals to the user. Video signals are output through a head-mounted display, while audio signals are output through two speakers positioned near the user's ears. This allows the user to experience changes in perspective within a virtual surround scene.

[0033] To achieve audio-visual synchronization in XR devices and create a more realistic sense of spatial sound, the XR device needs to obtain the position parameters of the first audio source within the virtual surround scene beforehand, and then simulate the effect of the first audio being played from that source based on these position parameters. The first audio source is a virtual sound source within the virtual surround scene.

[0034] The location parameters of the sound source include the first playback angle of the first audio, which is the angle between the center of the target user and the sound source of the first audio, and the angle between the target user's facing direction and the horizontal plane in the virtual surround scene. The first playback angle is used to characterize the orientation of the sound source of the first audio relative to the target user on the horizontal plane.

[0035] S102, based on the playback angle correction information corresponding to the target user, the first playback angle is corrected to obtain the corrected playback angle of the first audio, wherein the playback angle correction information is: the difference between the playback angle of the test audio and the listening angle fed back by the target user during the playback of each test audio.

[0036] In this embodiment, since each person's head shape and ear structure are unique, these individual differences will affect the frequency response and phase change of sound during propagation, thereby affecting the listener's sound localization and spatial perception. Therefore, different individuals will have different listening experiences of the first audio played.

[0037] However, the first audio signal is a standardized audio signal, and the first playback angle represents the perceived location of the first audio relative to the target user by users with average head and ear shapes within a certain range. However, due to individual differences, the first playback angle needs to be corrected.

[0038] In this embodiment, at least one test audio can be played to the target user using an XR device. During the playback of at least one test audio, the target user is asked to provide feedback on the perceived angle of each test audio. The perceived angle of each test audio is then compared with the playback angle. The difference between the playback angle of the test audio and the perceived angle provided by the target user is used as playback angle correction information to obtain the playback angle correction information of the first audio. The playback angle correction information is then used to correct the first playback angle of each first audio, thereby ensuring that the played first audio is more in line with the target user's listening experience.

[0039] S103, Play the first audio based on the corrected playback angle.

[0040] In this application, the head and ear structure of the target user can be modeled by modifying positional parameters such as playback angle to obtain the head model of the target user. The first audio is then processed and played based on the head model of the target user. In this way, the processed first audio can simulate the effect of the sound source of the first audio when the target user perceives it through binaural hearing.

[0041] In this application, a first audio to be played and a first playback angle of the first audio on a horizontal plane can be obtained. Based on the playback angle correction information corresponding to the target user, the first playback angle is corrected to obtain a corrected playback angle for the first audio. The playback angle correction information is the difference between the playback angle of the test audio and the perceived listening angle reported by the target user during the playback of each test audio. The first audio is then played based on the corrected playback angle. This application utilizes the perceived listening angle of the test audio reported by the user and the original playback angle of the test audio during testing to obtain playback angle correction information. This information is then used to correct the first playback angle of each first audio to be played, resulting in a corrected playback angle. The first audio is then played based on the corrected playback angle. This audio processing method provides a more accurate, personalized, and realistic spatial sound effect for each user, making it more suitable for the head and ear shape of the target user, and providing a more realistic virtual sound experience.

[0042] In some embodiments, prior to S102, the method further includes:

[0043] Display the audio test interface;

[0044] During the sequential playback of the at least two test audios, at least two target inputs corresponding to the at least two test audios are received in the audio test interface, wherein different test audios have different playback angles, each target input is the input of the target user when playing its corresponding test audio, and the target input is used to indicate the target user's listening angle.

[0045] The playback angle correction information is determined based on the playback angles of the at least two test audios and the perceived listening angles indicated by the at least two target inputs.

[0046] In this embodiment, the audio test interface is a virtual interface displayed by the XR device. This virtual interface can be a virtual reality interface, an augmented reality interface, or a mixed reality interface.

[0047] The audio testing interface can display at least two test audio sources, each capable of playing one test audio. These at least two test audios can be played sequentially in a specific order. Each time the target user hears a test audio, they can input a target information into the audio testing interface, indicating the perceived angle of the heard test audio. Thus, each test audio corresponds to a perceived angle, and each test audio has a playback angle. The perceived angle characterizes the horizontal position of the test audio source relative to the target user. The playback angle of the test audio characterizes the horizontal position of its corresponding test audio source relative to the target user.

[0048] If there is a difference between the playback angle and the perceived listening angle corresponding to the same test audio, it means that the current audio playback method is not entirely suitable for the target user. The first playback angle of each first audio to be played needs to be corrected. The playback angle correction information can be determined by the playback angle of at least two test audios and the perceived listening angle indicated by the at least two target inputs, and the playback angle correction information is applied to correct the first playback angle.

[0049] For example, such as Figure 2 As shown, the audio test interface can be a virtual, visualized spatial audio room. This spatial audio room is equipped with multiple virtual surround speakers 201 as multiple test sound sources. These surround speakers and the target user's head 202 are on the same horizontal plane, and the multiple surround speakers are evenly arranged on an arc centered on the midpoint of the target user. The user can view the speaker arrangement in this spatial audio room through VR glasses.

[0050] Then, these surround sound speakers can be played sequentially in a clockwise or counterclockwise direction. During each playback, the target user can use the controller corresponding to the VR glasses in the virtual room to emit a virtual ray in the spatial audio room to point at the sound source that the target user perceives, thereby inputting the target into the audio test interface and recording each target input and test sound source.

[0051] As an alternative embodiment, such as Figure 3 As shown, taking the target user's orientation as 0° as an example, the playback position of a certain test audio is located at 90° from the target user. Therefore, the playback angle of the test audio is 90°. The user's judgment position indicated by the target user's input is at 80°, so the perceived angle of the test audio is 80°. Therefore, the playback angle and perceived angle of the test audio differ by 10°.

[0052] In this embodiment, since each person's head and ear structure is unique, standardized audio signal playback angle correction information is obtained using personalized test audio data. This provides a more accurate, personalized, and realistic sense of sound space, making it more suitable for the target user's head and ear shape.

[0053] In some embodiments, determining the playback angle correction information based on the playback angles of the at least two test audios and the perceived listening angles indicated by the at least two target inputs includes:

[0054] An angle matrix is ​​generated based on the playback angle of the at least two test audios and the perceived angle indicated by the at least two target inputs;

[0055] The angle matrix is ​​fitted to obtain the playback angle correction equation, which is the playback angle correction information.

[0056] In this embodiment, after obtaining at least two playback angles of test audio and at least two listening angles indicated by the user, a listening matrix can be generated using the at least two playback angles and at least two listening angles, where the at least two playback angles and at least two listening angles are elements in the matrix.

[0057] Then, the angle matrix can be fitted to obtain the playback angle correction equation, which can be used to correct the first playback angle of each first audio file to be played. For example, a linear equation can be found using linear regression or least squares, and then the data in the angle matrix can be fitted using this linear equation to obtain the playback angle correction equation.

[0058] By fitting the angle matrix, the relationship between the first playback angle and the corrected playback angle for each first audio file of the target user can be predicted through the description and relationship analysis of the data in the angle matrix. Thus, each first playback angle can be corrected into a corrected playback angle.

[0059] Fitting the angle matrix involves determining a model that approximates the data within it. For example, in linear fitting, the angle matrix can be [(x1,y1),(x2,y2),...,(xn,yn)], where x1, x2, ..., xn represent the perceived angles of the test audio, and y1, y2, ..., yn represent the playback angles of the test audio. The perceived angles and playback angles within the same parentheses correspond to the same test audio, leading to a linear equation:

[0060] y = mx + k

[0061] Where y is the dependent variable, x is the independent variable, m is the slope, and k is the y-intercept. Then, the angle matrix can be linearly fitted using the least squares method to determine the slope m and the intercept k, so that the fitted value is as close as possible to the data points in the angle matrix.

[0062] In some embodiments, the playback angle correction equation includes:

[0063] D = a*θ + b

[0064] Where D is the first playback angle, θ is the corrected playback angle, a is the first coefficient, and b is the second coefficient.

[0065] In this embodiment, the playback angle correction equation can be a linear equation, where D is the first playback angle, θ is the corrected playback angle, a is the first coefficient representing the slope of the straight line, and b is the second coefficient representing the y-intercept.

[0066] After determining the playback angle correction equation, for each first audio to be played and its first playback angle, the first playback angle can be substituted into the above playback angle correction equation. Since a and b are known quantities, the corrected playback angle corresponding to the first playback angle can be calculated.

[0067] In some embodiments, S103 includes:

[0068] The frequency of the first audio, the pitch angle of the sound source of the first audio, and the distance between the sound source and the center point of the target user's head are obtained.

[0069] Based on the corrected playback angle, the frequency of the first audio, the pitch angle of the first audio source, and the distance between the sound source and the center point of the target user's head, a first transfer function from the sound source to the left ear of the target user and a second transfer function from the sound source to the right ear of the target user are generated.

[0070] The first audio is played based on the first transfer function and the second transfer function.

[0071] In this embodiment, after obtaining the first audio and the first playback angle of the first audio, the first playback angle can be corrected to a corrected playback angle using a playback angle correction equation. After that, the frequency of the first audio, the pitch angle of the first audio source relative to the center point of the target user's head, and the distance between the first audio source and the center point of the target user's head can also be obtained.

[0072] Based on the obtained corrected playback angle, the frequency of the first audio, the pitch angle of the first audio source, and the distance between the source and the center point of the target user's head, a first transfer function and a second transfer function can be constructed. Then, the first audio and the first transfer function can be convolved and played, so that the target user's left ear can hear the first audio with realistic audio spatial sense and stereo effect. The first audio and the second transfer function can be convolved and played, so that the target user's right ear can hear the first audio with realistic audio spatial sense and stereo effect.

[0073] Specifically, the first transfer function can be:

[0074]

[0075] Among them, H L This represents the transfer function of the first audio signal from its source to the target user's left ear. It describes the attenuation of the frequency and phase responses of the first audio signal as it passes through the target user's head and left ear. r is the distance between the sound source and the center point of the target user's head, θ is the corrected playback angle of the first audio signal, φ is the pitch angle of the sound source, f is the frequency of the first audio signal, and P... L P1 represents the sound pressure level at the left ear of the target user, and P2 represents the sound pressure level at the center of the target user's head when the target user is not present.

[0076] Similarly, the second transfer function can be:

[0077]

[0078] H RThis represents the transfer function of the first audio signal from its source to the target user's right ear. It describes the attenuation of the frequency and phase responses of the first audio signal as it passes through the target user's head and right ear. r is the distance between the sound source and the center point of the target user's head, θ is the corrected playback angle of the first audio signal, φ is the pitch angle of the sound source, f is the frequency of the first audio signal, and P... L P1 represents the sound pressure level at the target user's right ear, and P2 represents the sound pressure level at the center of the target user's head when the target user is not present.

[0079] Figure 4 This is a schematic diagram of the structure of an audio processing device provided in another embodiment of this application, as shown below. Figure 4 As shown, the audio processing device may include:

[0080] The acquisition module 401 is used to acquire the first audio to be played and the first playback angle of the first audio. The first playback angle is the angle of the sound source playing the first audio on the horizontal plane relative to the target user in a virtual surround scene.

[0081] The correction module 402 is used to correct the first playback angle based on the playback angle correction information corresponding to the target user, so as to obtain the corrected playback angle of the first audio. The playback angle correction information is the difference between the playback angle of the test audio and the listening angle fed back by the target user during the playback of each test audio.

[0082] The playback module 403 is used to play the first audio based on the corrected playback angle.

[0083] In this application, a first audio to be played and a first playback angle of the first audio on a horizontal plane can be obtained. Based on the playback angle correction information corresponding to the target user, the first playback angle is corrected to obtain a corrected playback angle for the first audio. The playback angle correction information is the difference between the playback angle of the test audio and the perceived listening angle reported by the target user during the playback of each test audio. The first audio is then played based on the corrected playback angle. This application utilizes the perceived listening angle of the test audio reported by the user and the original playback angle of the test audio during testing to obtain playback angle correction information. This information is then used to correct the first playback angle of each first audio to be played, resulting in a corrected playback angle. The first audio is then played based on the corrected playback angle. This audio processing method provides a more accurate, personalized, and realistic sound effect spatial sense for each user, making it more suitable for the head and ear shape of the target user, and providing a more realistic virtual sound effect experience.

[0084] In another alternative example, the audio processing device includes:

[0085] The display module is used to display the audio test interface;

[0086] The receiving module is configured to receive at least two target inputs corresponding to the at least two test audios in the audio test interface during the sequential playback of the at least two test audios, wherein different test audios have different playback angles, each target input is the input of the target user when playing its corresponding test audio, and the target input is used to indicate the target user's listening angle.

[0087] The determining module is used to determine the playback angle correction information based on the playback angles of the at least two test audios and the perceived listening angles indicated by the at least two target inputs.

[0088] In another alternative example, the determining module also includes:

[0089] A construction unit is configured to generate an angle matrix based on the playback angles of the at least two test audios and the perceived angles indicated by the at least two target inputs;

[0090] A fitting unit is used to fit the angle matrix to obtain a playback angle correction equation, wherein the playback angle correction equation is the playback angle correction information.

[0091] In another optional example, the playback angle correction equation includes:

[0092] D = a*θ + b

[0093] Where D is the first playback angle, θ is the corrected playback angle, a is the first coefficient, and b is the second coefficient.

[0094] In another alternative example, playback module 403 also includes:

[0095] The acquisition unit is used to acquire the frequency of the first audio, the pitch angle of the sound source of the first audio, and the distance between the sound source and the center point of the target user's head;

[0096] The generation unit is configured to generate a first transfer function from the sound source to the left ear of the target user and a second transfer function from the sound source to the right ear of the target user based on the corrected playback angle, the frequency of the first audio, the pitch angle of the sound source of the first audio, and the distance between the sound source and the center point of the target user's head.

[0097] A playback unit is configured to play the first audio based on the first transfer function and the second transfer function.

[0098] The audio processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific type of device.

[0099] The audio processing device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0100] The audio processing apparatus provided in this application embodiment can achieve Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0101] Optionally, such as Figure 5 As shown in the illustration, this application also provides an electronic device 100, including a processor 110 and a memory 119. The processor 110 includes a central processing unit (CPU) 1101 and a GPU compression processing unit 1102. A program or instruction stored in the memory 119 and executable on the processor 110 implements the various processes of the audio processing method embodiments described above when executed by the processor 110, and achieves the same technical effects. To avoid repetition, further details are omitted here.

[0102] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0103] Please refer to the following: Figure 6 , Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. The electronic device 100 includes, but is not limited to, components such as: a radio frequency unit 121, a network module 122, an audio output unit 123, an input unit 124, a sensor 125, a display unit 126, a user input unit 127, an interface unit 128, a memory 129, and a processor 100.

[0104] Those skilled in the art will understand that the electronic device 120 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 120 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0105] The processor 120 is used to acquire the first audio to be played and the first playback angle of the first audio. The first playback angle is the angle of the sound source playing the first audio on the horizontal plane relative to the target user in a virtual surround scene.

[0106] The processor 120 is configured to correct the first playback angle based on playback angle correction information corresponding to the target user, thereby obtaining a corrected playback angle for the first audio, wherein the playback angle correction information is the difference between the playback angle of the test audio and the perceived listening angle reported by the target user during the playback of each test audio.

[0107] The audio output unit 123 is used to play the first audio based on the corrected playback angle.

[0108] In this application, a first audio to be played and a first playback angle of the first audio on a horizontal plane can be obtained. Based on the playback angle correction information corresponding to the target user, the first playback angle is corrected to obtain a corrected playback angle for the first audio. The playback angle correction information is the difference between the playback angle of the test audio and the perceived listening angle reported by the target user during the playback of each test audio. The first audio is then played based on the corrected playback angle. This application utilizes the perceived listening angle of the test audio reported by the user and the original playback angle of the test audio during testing to obtain playback angle correction information. This information is then used to correct the first playback angle of each first audio to be played, resulting in a corrected playback angle. The first audio is then played based on the corrected playback angle. This audio processing method provides a more accurate, personalized, and realistic sound effect spatial sense for each user, making it more suitable for the head and ear shape of the target user, and providing a more realistic virtual sound effect experience.

[0109] In another alternative example, the audio processing device includes:

[0110] The display module is used to display the audio test interface;

[0111] The receiving module is configured to receive at least two target inputs corresponding to the at least two test audios in the audio test interface during the sequential playback of the at least two test audios, wherein different test audios have different playback angles, each target input is the input of the target user when playing its corresponding test audio, and the target input is used to indicate the target user's listening angle.

[0112] The determining module is used to determine the playback angle correction information based on the playback angles of the at least two test audios and the perceived listening angles indicated by the at least two target inputs.

[0113] In another alternative example, the determining module also includes:

[0114] A construction unit is configured to generate an angle matrix based on the playback angles of the at least two test audios and the perceived angles indicated by the at least two target inputs;

[0115] A fitting unit is used to fit the angle matrix to obtain a playback angle correction equation, wherein the playback angle correction equation is the playback angle correction information.

[0116] In another optional example, the playback angle correction equation includes:

[0117] D = a*θ + b

[0118] Where D is the first playback angle, θ is the corrected playback angle, a is the first coefficient, and b is the second coefficient.

[0119] In another alternative example, the audio output unit 123 further includes:

[0120] The acquisition unit is used to acquire the frequency of the first audio, the pitch angle of the sound source of the first audio, and the distance between the sound source and the center point of the target user's head;

[0121] The generation unit is configured to generate a first transfer function from the sound source to the left ear of the target user and a second transfer function from the sound source to the right ear of the target user based on the corrected playback angle, the frequency of the first audio, the pitch angle of the sound source of the first audio, and the distance between the sound source and the center point of the target user's head.

[0122] A playback unit is configured to play the first audio based on the first transfer function and the second transfer function.

[0123] It should be understood that, in this embodiment, the input unit 124 may include a graphics processing unit (GPU) 1241 and a microphone 1242. The GPU 1241 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 126 may include a display panel 1261, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 127 includes at least one of a touch panel 1271 and other input devices 1272. The touch panel 1271 is also called a touch screen. The touch panel 1271 may include a touch detection device and a touch controller. Other input devices 1272 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0124] The memory 129 can be used to store software programs and various data. The memory 129 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 129 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 129 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0125] Processor 120 may include one or more processing units; optionally, processor 120 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 120.

[0126] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described audio processing method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0127] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0128] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described audio processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0129] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0130] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the audio processing method embodiment described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0131] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0132] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0133] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An audio processing method, characterized in that, include: Obtain the first audio to be played and the first playback angle of the first audio, wherein the first playback angle is the angle of the sound source playing the first audio on the horizontal plane relative to the target user in a virtual surround scene; Based on the playback angle correction equation corresponding to the target user, the first playback angle is corrected to obtain the corrected playback angle of the first audio. The playback angle correction equation is a linear equation obtained by fitting the difference between the playback angle of the test audio and the listening angle fed back by the target user during the playback of each test audio. The first audio is played based on the corrected playback angle.

2. The method according to claim 1, characterized in that, Before correcting the first playback angle based on the playback angle correction equation corresponding to the target user to obtain the corrected playback angle of the first audio, the method further includes: Display the audio test interface; During the sequential playback of at least two test audios, at least two target inputs corresponding to the at least two test audios are received in the audio test interface, wherein different test audios have different playback angles, each target input is the input of the target user when playing its corresponding test audio, and the target input is used to indicate the target user's listening angle. The playback angle correction equation is determined based on the playback angles of the at least two test audios and the perceived listening angles indicated by the at least two target inputs.

3. The method according to claim 2, characterized in that, The step of determining the playback angle correction equation based on the playback angles of the at least two test audios and the perceived listening angles indicated by the at least two target inputs includes: An angle matrix is ​​generated based on the playback angle of the at least two test audios and the perceived angle indicated by the at least two target inputs; By fitting the angle matrix, the playback angle correction equation is obtained.

4. The method according to claim 3, characterized in that, The playback angle correction equation includes: D = a*θ + b Where D is the first playback angle, θ is the corrected playback angle, a is the first coefficient, and b is the second coefficient.

5. The method according to claim 1, characterized in that, Playing the first audio based on the corrected playback angle includes: The frequency of the first audio, the pitch angle of the sound source of the first audio, and the distance between the sound source and the center point of the target user's head are obtained. Based on the corrected playback angle, the frequency of the first audio, the pitch angle of the first audio source, and the distance between the sound source and the center point of the target user's head, a first transfer function from the sound source to the left ear of the target user and a second transfer function from the sound source to the right ear of the target user are generated. The first audio is played based on the first transfer function and the second transfer function.

6. An audio processing apparatus, characterized in that, include: The acquisition module is used to acquire the first audio to be played and the first playback angle of the first audio. The first playback angle is the angle of the sound source playing the first audio on the horizontal plane relative to the target user in a virtual surround scene. The correction module is used to correct the first playback angle based on the playback angle correction equation corresponding to the target user, so as to obtain the corrected playback angle of the first audio. The playback angle correction equation is a linear equation obtained by fitting the difference between the playback angle of the test audio and the listening angle fed back by the target user during the playback of each test audio. The playback module is used to play the first audio based on the corrected playback angle.

7. The apparatus according to claim 6, characterized in that, The audio processing device further includes: The display module is used to display the audio test interface; The receiving module is configured to receive at least two target inputs corresponding to the at least two test audios in the audio test interface during the sequential playback of at least two test audios, wherein different test audios have different playback angles, each target input is the input of the target user when playing its corresponding test audio, and the target input is used to indicate the target user's listening angle. The determination module is used to determine the playback angle correction equation based on the playback angles of the at least two test audios and the perceived listening angles indicated by the at least two target inputs.

8. The apparatus according to claim 7, characterized in that, The determining module further includes: A construction unit is configured to generate an angle matrix based on the playback angles of the at least two test audios and the perceived angles indicated by the at least two target inputs; The fitting unit is used to fit the angle matrix to obtain the playback angle correction equation.

9. The apparatus according to claim 8, characterized in that, The playback angle correction equation includes: D = a*θ + b Where D is the first playback angle, θ is the corrected playback angle, a is the first coefficient, and b is the second coefficient.

10. The apparatus according to claim 6, characterized in that, The playback module also includes: The acquisition unit is used to acquire the frequency of the first audio, the pitch angle of the sound source of the first audio, and the distance between the sound source and the center point of the target user's head; The generation unit is configured to generate a first transfer function from the sound source to the left ear of the target user and a second transfer function from the sound source to the right ear of the target user based on the corrected playback angle, the frequency of the first audio, the pitch angle of the sound source of the first audio, and the distance between the sound source and the center point of the target user's head. A playback unit is configured to play the first audio based on the first transfer function and the second transfer function.

Images