Audio processing method and device, equipment and medium

Abstract

The invention provides an audio processing method and device, equipment and a medium, which are used for solving the technical problems that the realization cost cannot be ensured to be relatively low and the user requirements cannot be met in the aspects of sound field expansion, reverberation natural sense and listening sense in the prior art, and the method comprises the following steps: obtaining first audio data to be played; processing the first audio data to obtain early-stage reflected sound which is used for simulating a reflection effect when the first audio data is played during playing; generating late reverberation according to the early reflected sound, wherein the late reverberation is used for simulating a reverberation effect when the first audio data is played during playing; and obtaining second audio data for playing based on the early reflected sound and the late reverberation.

Description

Technical Field

[0001] The present invention relates to the technical field of audio signal processing, and in particular to an audio processing method, device, equipment and medium. Background Art

[0002] In related technologies, the car audio effects are often poor, with poor sound field expansion, natural reverberation and listening experience, which cannot meet user needs. Some vehicles will achieve this through convolution of real impulse responses, which not only requires huge calculations, but also requires additional digital signal processing to support, which is too costly. Summary of the invention

[0003] The present invention provides an audio processing method, device, equipment and medium, which are used to solve the technical problem that the related art cannot ensure low cost while meeting user needs in terms of sound field expansion, natural reverberation and hearing experience.

[0004] In a first aspect, an embodiment of the present invention provides an audio processing method, the method comprising:

[0005] Acquire first audio data to be played;

[0006] Processing the first audio data to obtain early reflection sound, where the early reflection sound is used to simulate the reflection effect of the first audio data during playback;

[0007] generating a late reverberation according to the early reflected sound, wherein the late reverberation is used to simulate the reverberation effect of the first audio data when it is played;

[0008] Based on the early reflected sound and the late reverberation, second audio data for playing is obtained.

[0009] In a possible implementation manner, in the method provided in the embodiment of the present invention, processing the first audio data to obtain the early reflection sound includes:

[0010] Utilizing the cloud server to calculate the first audio data to obtain mirror source audio data;

[0011] The features of the mirror source audio data are sparsely processed and then convolved to obtain the early reflection sound.

[0012] In a possible implementation manner, in the method provided in the embodiment of the present invention, obtaining early reflection sound by processing audio data includes:

[0013] Acquire reflected sound audio data, where the reflected sound audio data is the reflected sound of the collected first audio data;

[0014] The features of the reflected sound audio data are sparsely processed and then convolved to obtain the early reflected sound.

[0015] In a possible implementation manner, in the method provided in the embodiment of the present invention, generating late reverberation according to early reflected sound includes:

[0016] The early reflection sound is input into a pre-generated late reverberation model to obtain the late reverberation.

[0017] In a possible implementation manner, in the method provided by the embodiment of the present invention, after obtaining the first audio data to be played, the method further includes:

[0018] Performing music source amplitude adjustment on the first audio data to obtain direct sound, where the direct sound is the sound of the first audio data directly transmitted to the receiver without any reflection;

[0019] The early reflection sound is input into the pre-generated late reverberation model to obtain the late reverberation, including:

[0020] The early reflected sound and direct sound are input into the late reverberation model to obtain the late reverberation.

[0021] In a possible implementation manner, in the method provided in the embodiment of the present invention, obtaining second audio data for playback based on early reflection sound and late reverberation includes:

[0022] Second audio data is obtained based on the early reflected sound, the late reverberation and the direct sound.

[0023] In a possible implementation manner, in the method provided by the embodiment of the present invention, after obtaining the first audio data to be played, the method further includes:

[0024] Based on the user's settings or the pre-set desired strategy, obtain reverberation effect data;

[0025] After obtaining the second audio data based on the early reflected sound and the late reverberation, the method further includes:

[0026] The gain and delay of at least two speakers are determined according to the reverberation effect data, and the second audio data is played according to the gain and delay of each speaker.

[0027] In a second aspect, an embodiment of the present invention provides an audio processing device, including:

[0028] An acquisition unit, used for acquiring first audio data to be played;

[0029] A processing unit, used to process the first audio data to obtain early reflection sound, where the early reflection sound is used to simulate the reflection effect of the first audio data during playback;

[0030] A generating unit, used for generating a late reverberation according to the early reflected sound, wherein the late reverberation is used for simulating the reverberation effect of the first audio data when it is played;

[0031] The output unit is used to obtain second audio data for playing based on the early reflected sound and the late reverberation.

[0032] In a possible implementation manner, in the device provided by the embodiment of the present invention, the processing unit is used to:

[0033] Utilizing the cloud server to calculate the first audio data to obtain mirror source audio data;

[0034] The features of the mirror source audio data are sparsely processed and then convolved to obtain the early reflection sound.

[0035] In a possible implementation manner, in the device provided by the embodiment of the present invention, the processing unit is used to:

[0036] Acquire reflected sound audio data, where the reflected sound audio data is the reflected sound of the collected first audio data;

[0037] The features of the reflected sound audio data are sparsely processed and then convolved to obtain the early reflected sound.

[0038] In a possible implementation manner, in the device provided by the embodiment of the present invention, the generating unit is specifically used for:

[0039] The early reflection sound is input into a pre-generated late reverberation model to obtain the late reverberation.

[0040] In a possible implementation manner, in the device provided by the embodiment of the present invention, the acquiring unit is further used for:

[0041] Performing music source amplitude adjustment on the first audio data to obtain direct sound, where the direct sound is the sound of the first audio data directly transmitted to the receiver without any reflection;

[0042] The audio data is processed by adjusting the amplitude of the music source to obtain direct sound;

[0043] The generation unit is also used to:

[0044] The early reflected sound and direct sound are input into the late reverberation model to obtain the late reverberation.

[0045] In a possible implementation manner, in the device provided by the embodiment of the present invention, the output unit is specifically used for:

[0046] Second audio data is obtained based on the early reflected sound, the late reverberation and the direct sound.

[0047] In a possible implementation manner, in the device provided by the embodiment of the present invention, the acquiring unit is further used for:

[0048] Based on the user's settings or the pre-set desired strategy, obtain reverberation effect data;

[0049] The output unit is also used for:

[0050] The gain and delay of at least two speakers are determined according to the reverberation effect data, and the second audio data is played according to the gain and delay of each speaker.

[0051] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method provided in the first aspect of the embodiment of the present invention.

[0052] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implements the method provided in the first aspect of the embodiment of the present invention.

[0053] In the embodiment of the present invention, firstly, the first audio data to be played is obtained, and then the first audio data is processed to obtain early reflection sound, and then the late reverberation is generated according to the early reflection sound, and finally the second audio data for playing is obtained based on the early reflection sound and the late reverberation. Compared with the related art, the early reflection sound and the late reverberation are obtained by processing the first audio data, and the second audio data for final output is generated by using the early reflection sound and the late reverberation. Under the condition of low calculation amount, the reflection sound and the reverberation are simulated, thereby achieving the improvement of the sound field expansion, the natural sense of reverberation and the hearing sense, and meeting the listening needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flowchart of an audio processing method provided by an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of an implementation environment of an audio processing method provided by an embodiment of the present invention;

[0056] Figure 3 A schematic diagram of a specific flow chart of an audio processing method provided by an embodiment of the present invention;

[0057] Figure 4 A schematic diagram of the structure of an audio processing device provided by an embodiment of the present invention;

[0058] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] The following are explanations of some of the terms that appear in the text:

[0061] 1. In the embodiments of the present invention, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0062] 2. The term "early reflected sound" in the embodiment of the present invention refers to all reflected sounds that arrive after the direct sound and have a beneficial effect on the sound quality of the room. The time range is generally 50ms after the direct sound, and some people believe that it can be taken as 95ms. The ratio of early reflected sound energy to reverberant sound energy is called resolution.

[0063] 3. In the embodiments of the present invention, the term "direct sound", i.e. direct sound (DS), refers to the sound that is directly transmitted from the sound source to the receiver in a straight line without any reflection.

[0064] 4. The term "early reflected sound" in the embodiments of the present invention refers to all reflected sounds that arrive after the direct sound and have a positive effect on the sound quality of the room. In the present invention, it specifically refers to the first audio data used to simulate the early reflected sound.

[0065] 5. The term "late reverberation" in the embodiments of the present invention refers to the reflected waves that arrive at the microphone later due to multipath reflection of the audio signal from the sound source to the microphone when audio communication is carried out in a closed space. In the present invention, it specifically refers to the simulation of the reflection and reverberation of sound in space. By adjusting the parameters of the late reverberation, the reverberation characteristics of the sound in different environments can be simulated, thereby creating a rich sense of spatial atmosphere.

[0066] 6. The term "reverse sound energy ratio" in the embodiments of the present invention is a measure of the complexity and confusion of an audio signal. It is defined as the ratio of the sound energy propagating forward to the sound energy propagating backward in an audio signal. In an ideal situation without interference such as echo and noise, the reverse sound energy ratio should be close to 0 (all the sound energy propagates forward).

[0067] 7. The term "mirror source computing" in the embodiments of the present invention is a strategy used in network architecture and large-scale data processing. This strategy involves the collaboration between multiple servers, allowing users to request the same data or resources based on different physical locations.

[0068] In the related technologies, the car audio effects are often poor, and the sound field expansion, reverberation naturalness and listening experience are all poor, which cannot meet user needs. Some high-end vehicle models will be equipped with more complex audio systems, which are achieved through convolution of real impulse responses. Not only is the amount of calculation huge, but additional intelligent devices are also needed to realize digital signal processing, which is too costly.

[0069] The audio processing method, device, equipment and medium provided by the present invention are described in more detail below in conjunction with the accompanying drawings and embodiments.

[0070] The embodiment of the present invention provides an audio processing method, such as Figure 1 As shown, including:

[0071] Step S101, obtaining first audio data to be played.

[0072] During specific implementation, the first audio data to be played is obtained through the music source. At the same time, in this step, the expected strategy can also be obtained. The expected strategy can be set by the user or preset by the system to ultimately adjust the second audio data to be played.

[0073] Step S102: Process the first audio data to obtain early reflection sound.

[0074] In a specific implementation, the first audio data is calculated to obtain early reflected sound. Specifically, there are two ways to obtain the early reflected sound. One is to perform mirror source calculation on the first audio data to obtain mirror source audio data, and then perform convolution after thinning the features of the mirror source audio data to finally obtain the early reflected sound. Alternatively, the real reflected sound audio data can be directly obtained through a sensor, and then the features of the reflected sound audio data can be thinned and convolution is performed to finally obtain the early reflected sound. Specifically, when the features are sparse, a sparse filter can be used for convolution. A sparse filter is a method of sparsely inputting features to obtain a good representation of the data and to reduce the complexity of the model to some extent.

[0075] Step S103: generating late reverberation according to the early reflected sound.

[0076] In the specific implementation, the early reflected sound is input into the pre-trained late reverberation model to obtain the late reverberation. Specifically, the first audio data is firstly gain delayed, and then the early reflected sound is used as the input of the late reverberation model. While iteratively generating a higher density reverberation, the density of the front reverberation is multiplied. When in use, in order to restore the complex and real physical field, different channels are set to different reverberation times, so as to modify the feedback matrix parameters and obtain the reverberation output of different channels. When modifying the feedback matrix parameters, the reverberation time of the impulse response is generated by traversing different step length parameters through the Schroeder reverse integration method to obtain the desired reverberation effect.

[0077] Step S104: obtaining second audio data based on the early reflected sound and the late reverberation.

[0078] In a specific implementation, the generated early reflection sound is combined with the late reverberation to obtain the final audio output, which is played through the vehicle speakers, such as Figure 2 As shown, the vehicle computer part is responsible for processing the first audio data to be played from the music source, and finally transmits it to the vehicle speaker for playing.

[0079] If the desired strategy is obtained in S101, the gain and delay of different speakers are controlled by the desired strategy before the speakers play, thereby increasing the reverse sound energy ratio and obtaining the final output audio content.

[0080] In one example, if theater-mode audio output is to be implemented in a car computer, the expected strategy may be theater mode. According to this strategy, the playback gain and playback delay of multiple speakers in the front, middle, and rear of the car are set so that the reverberation effect at the user's location is similar to that in a theater, thereby making the sound heard by the user feel like in a theater.

[0081] like Figure 3 As shown, the specific process of the audio processing method provided by the embodiment of the present invention may include the following steps:

[0082] Step S301, obtaining first audio data to be played.

[0083] During specific implementation, the first audio data to be played is obtained through the music source. At the same time, in this step, the expected strategy can also be obtained. The expected strategy can be set by the user or preset by the system to ultimately adjust the second audio data to be played.

[0084] Step S302, adjusting the music source amplitude of the first audio data to obtain direct sound.

[0085] In a specific implementation, the audio data is subjected to music source amplitude processing to obtain direct sound, where the direct sound is the sound of the first audio data directly transmitted to the receiver without any reflection.

[0086] Step S303: Process the first audio data to obtain early reflection sound.

[0087] In a specific implementation, the first audio data is calculated to obtain early reflected sound. Specifically, there are two ways to obtain the early reflected sound. One is to perform mirror source calculation on the first audio data to obtain mirror source audio data, and then perform convolution after thinning the features of the mirror source audio data to finally obtain the early reflected sound. Alternatively, the real reflected sound audio data can be directly obtained through a sensor, and then the features of the reflected sound audio data can be thinned and convolution is performed to finally obtain the early reflected sound. Specifically, when the features are sparse, a sparse filter can be used for convolution. A sparse filter is a method of sparsely inputting features to obtain a good representation of the data and to reduce the complexity of the model to some extent.

[0088] Step S304: generating late reverberation according to the early reflected sound and the direct sound.

[0089] In specific implementation, the early reflected sound and the direct sound are input into a pre-trained late reverberation model to obtain the late reverberation. While iteratively generating a higher density reverberation, the density of the front reverberation is multiplied.

[0090] Step S305: obtaining second audio data based on the early reflected sound, the late reverberation and the direct sound.

[0091] In specific implementation, the generated early reflected sound, late reverberation and direct sound are combined to obtain the final audio output, and played through the vehicle speakers. If the desired strategy is obtained in S301, the gain and delay of different speakers are controlled by the desired strategy before the speakers play, thereby expanding the reverse sound energy ratio and obtaining the final output audio content.

[0092] By applying the audio processing method provided in this embodiment, the acoustic space characteristics of the real scene are continued in the reverberation by taking the real reflected sound as the input of the reverberation model. At the same time, without the need for a large amount of calculation, a complex reverberation strategy is provided in the car. With rich tuning freedom, multi-channel different parameter adjustment brings a more realistic sound field experience.

[0093] like Figure 4 As shown, based on the same inventive concept of the audio processing method, the present invention also provides an audio processing device, comprising:

[0094] An acquisition unit 401 is used to acquire first audio data to be played;

[0095] The processing unit 402 is used to process the first audio data to obtain early reflection sound, and the early reflection sound is used to simulate the reflection effect of the first audio data when playing;

[0096] A generating unit 403, configured to generate a late reverberation according to the early reflected sound, wherein the late reverberation is used to simulate a reverberation effect when the first audio data is played during playback;

[0097] The output unit 404 is used to obtain second audio data for playing based on the early reflection sound and the late reverberation.

[0098] In a possible implementation manner, in the device provided by the embodiment of the present invention, the processing unit 402 is used to:

[0099] Utilizing the cloud server to calculate the first audio data to obtain mirror source audio data;

[0100] The features of the mirror source audio data are sparsely processed and then convolved to obtain the early reflection sound.

[0101] In a possible implementation manner, in the device provided by the embodiment of the present invention, the processing unit 402 is used to:

[0102] Acquire reflected sound audio data, where the reflected sound audio data is the reflected sound of the collected first audio data;

[0103] The features of the reflected sound audio data are sparsely processed and then convolved to obtain the early reflected sound.

[0104] In a possible implementation manner, in the device provided by the embodiment of the present invention, the generating unit 403 is specifically used for:

[0105] The early reflection sound is input into a pre-generated late reverberation model to obtain the late reverberation.

[0106] In a possible implementation manner, in the device provided by the embodiment of the present invention, the acquiring unit 401 is further used for:

[0107] Performing music source amplitude adjustment on the first audio data to obtain direct sound, where the direct sound is the sound of the first audio data directly transmitted to the receiver without any reflection;

[0108] The audio data is processed by adjusting the amplitude of the music source to obtain direct sound;

[0109] Then the generating unit 403 is further used for:

[0110] The early reflected sound and direct sound are input into the late reverberation model to obtain the late reverberation.

[0111] In a possible implementation manner, in the device provided by the embodiment of the present invention, the output unit 404 is specifically used for:

[0112] Second audio data is obtained based on the early reflected sound, the late reverberation and the direct sound.

[0113] In a possible implementation manner, in the device provided by the embodiment of the present invention, the acquiring unit 401 is further used for:

[0114] Based on the user's settings or the pre-set desired strategy, obtain reverberation effect data;

[0115] Then the output unit 404 is further used for:

[0116] The gain and delay of at least two speakers are determined according to the reverberation effect data, and the second audio data is played according to the gain and delay of each speaker.

[0117] In addition, combined Figure 1-Figure 4 The audio processing method and apparatus described in the embodiments of the present invention may be implemented by an electronic device. Figure 5 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention is shown.

[0118] The following specific reference Figure 5 , which shows a structural schematic diagram of an electronic device 500 suitable for implementing the embodiments of the present disclosure. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0119] like Figure 5 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 to a random access memory (RAM) 503 to implement the voice control method of the embodiment described in the present disclosure. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0120] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0121] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the voice control method as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0122] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0123] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0124] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0125] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0126] Acquire first audio data to be played;

[0127] Processing the first audio data to obtain early reflection sound, where the early reflection sound is used to simulate the reflection effect of the first audio data during playback;

[0128] generating a late reverberation according to the early reflected sound, wherein the late reverberation is used to simulate the reverberation effect of the first audio data when it is played;

[0129] Based on the early reflected sound and the late reverberation, second audio data for playing is obtained.

[0130] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0131] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0132] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0133] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0134] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0135] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0136] In the embodiment of the present invention, firstly, the first audio data to be played is obtained, and then the first audio data is processed to obtain early reflection sound, and then the late reverberation is generated according to the early reflection sound, and finally the second audio data for playing is obtained based on the early reflection sound and the late reverberation. Compared with the related art, the early reflection sound and the late reverberation are obtained by processing the first audio data, and the second audio data for final output is generated by using the early reflection sound and the late reverberation. Under the condition of low calculation amount, the reflection sound and the reverberation are simulated, thereby achieving the improvement of the sound field expansion, the natural sense of reverberation and the hearing sense, and meeting the listening needs of users.

[0137] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0138] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0139] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0141] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0142] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An audio processing method, characterized in that: include: Acquire first audio data to be played; Processing the first audio data to obtain early reflection sound, wherein the early reflection sound is used to simulate the reflection effect of the first audio data when it is played; generating a late reverberation according to the early reflected sound, wherein the late reverberation is used to simulate a reverberation effect when the first audio data is played during playback; Based on the early reflected sound and the late reverberation, second audio data for playing is obtained.

2. The audio processing method according to claim 1, characterized in that: The processing of the first audio data to obtain early reflected sound includes: Utilizing a cloud server to calculate the first audio data to obtain mirror source audio data; The features of the image source audio data are thinned and then convolved to obtain the early reflected sound.

3. The audio processing method according to claim 1, characterized in that: The step of obtaining early reflected sound by processing the audio data comprises: Acquire reflected sound audio data, where the reflected sound audio data is the collected reflected sound of the first audio data; The features of the reflected sound audio data are sparsely processed and then convolved to obtain the early reflected sound.

4. The audio processing method according to claim 1, characterized in that: Generating late reverberation according to the early reflected sound comprises: The early reflected sound is input into a pre-generated late reverberation model to obtain the late reverberation.

5. The audio processing method according to claim 4, characterized in that: After obtaining the first audio data to be played, the method further includes: Performing music source amplitude adjustment on the first audio data to obtain direct sound, where the direct sound is the sound of the first audio data directly transmitted to the receiver without any reflection; Then the step of inputting the early reflected sound into the pre-generated late reverberation model to obtain the late reverberation includes: The early reflected sound and the direct sound are input into a late reverberation model to obtain the late reverberation.

6. The audio processing method according to claim 5, characterized in that: The obtaining of second audio data for playing based on the early reflected sound and the late reverberation includes: The second audio data is obtained based on the early reflected sound, the late reverberation and the direct sound.

7. The audio processing method according to any one of claims 1 to 6, characterized in that: After obtaining the first audio data to be played, the method further includes: Based on the user's settings or the pre-set desired strategy, obtain reverberation effect data; After obtaining the second audio data based on the early reflected sound and the late reverberation, the method further includes: The gain and delay of at least two speakers are determined according to the reverberation effect data, and the second audio data is played according to the gain and delay of each speaker.

8. An audio processing device, characterized in that: include: An acquisition unit, used for acquiring first audio data to be played; a processing unit, configured to process the first audio data to obtain early reflected sound, wherein the early reflected sound is used to simulate the reflection effect of the first audio data during playback; A generating unit, configured to generate a late reverberation according to the early reflected sound, wherein the late reverberation is used to simulate a reverberation effect of the first audio data when it is played; The output unit is used to obtain second audio data for playing based on the early reflected sound and the late reverberation.

9. An electronic device, characterized in that: include: At least one processor, at least one memory and computer program instructions stored in the memory, when the computer program instructions are executed by the processor, implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images