Audio information processing method, device, equipment and medium

By performing time-domain and frequency-domain analysis of audio data, extracting features and determining loading sound effects, the problem that smart speakers and other devices cannot automatically select sound effects is solved, and adaptive sound effects enhancement is achieved, suitable for all audio transmission methods, and users are enhanced Experience.

CN112309352BActive Publication Date: 2024-11-22DOUYIN VISION CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202010041956.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-01-15
Publication Date
2024-11-22
Estimated Expiration
2040-11-04

AI Technical Summary

Technical Problem

Playing devices such as smart speakers cannot automatically select appropriate sound effects based on the played audio content, and the Bluetooth connection method cannot transmit the audio type tag, resulting in the inability to determine specific sound effects based on the audio type.

Method used

By performing time-domain and frequency-domain analysis of the original audio data, audio features are extracted, and appropriate loading sound effects information is determined based on these features, linear superposition is performed to enhance the audio style, which is suitable for all audio transmission methods.

Benefits of technology

It realizes automatic selection of sound effects based on audio content, improves user experience, and is suitable for scenarios such as Bluetooth transmission, solving the problem that the existing technology cannot adaptively select sound effects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112309352B_ABST
    Figure CN112309352B_ABST
Patent Text Reader

Abstract

The embodiments of the present application disclose an audio information processing method, device, equipment and medium. A specific implementation of the method includes: performing time domain analysis and frequency domain analysis on the audio segments in the original audio data that are in a preset time period to obtain the audio features of the original audio data; determining the loaded sound effect information corresponding to the audio features, and the loaded sound effects represented by the loaded sound effect information are used to enhance the audio style determined based on the audio features; and superimposing the loaded sound effects on the basis of the original audio data. This embodiment can analyze the audio segments of the original audio data to obtain the audio features of the original audio data, and select appropriate loaded sound effect information according to the audio features, and superimpose it on the original audio data, thereby improving the user experience; and, since the loaded sound effects can be adaptively selected according to the identified audio features, it is suitable for all audio transmission methods such as Bluetooth transmission.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and specifically to an audio information processing method, apparatus, device and medium. Background Art

[0002] Currently, smart speakers and other playback devices can obtain specific sound effects corresponding to audio type tags based on preset audio type tags, but cannot automatically select appropriate sound effects based on the audio content being played. In addition, since Bluetooth connection cannot transmit audio type tags, the method of determining specific sound effects based on audio type tags is not applicable to Bluetooth connection playback scenarios. Summary of the invention

[0003] The embodiments of the present application provide an audio information processing method and device.

[0004] In a first aspect, an embodiment of the present application provides an audio information processing method, which includes: performing time domain analysis and frequency domain analysis on audio segments in a preset time period in the original audio data to obtain audio features of the original audio data; determining loaded sound effect information corresponding to the audio features, and the loaded sound effects represented by the loaded sound effect information are used to enhance the audio style determined based on the audio features; and superimposing the loaded sound effects on the original audio data.

[0005] In some embodiments, the above-mentioned determination of the loaded sound effect information corresponding to the audio feature includes: determining the audio type of the original audio data according to the audio feature; and determining the loaded sound effect information corresponding to the audio type according to the audio type.

[0006] In some embodiments, before performing time domain analysis and frequency domain analysis on the audio segments in the original audio data that are in a preset time period to obtain the audio features of the original audio data, the method further includes: performing time domain analysis and frequency domain analysis on the acquired sample audio data to obtain sample audio features of the sample audio data; acquiring loaded sound effect information for the sample audio data, and associating the sample audio features with the loaded sound effect information; and associating the sample audio features with the loaded sound effect information and storing them in a loaded sound effect set.

[0007] In some embodiments, the above-mentioned determination of the loaded sound effect information corresponding to the audio feature includes: determining whether the audio feature matches the sample audio feature in the loaded sound effect set; in response to determining that the audio feature matches the sample audio feature in the loaded sound effect set, determining the loaded sound effect information corresponding to the sample audio feature as the loaded sound effect information corresponding to the audio feature.

[0008] In some embodiments, the above-mentioned superimposing the loading sound effect on the basis of the original audio data includes: determining the loading duration corresponding to the loading sound effect information, the loading duration is used to characterize the time required to superimpose the loading sound effect; according to the loading duration, linearly superimposing the loading sound effect on the basis of the original audio data.

[0009] In the second aspect, an embodiment of the present application provides an audio information processing device, which includes: an audio analysis unit, configured to perform time domain analysis and frequency domain analysis on audio segments in a preset time period in the original audio data to obtain audio features of the original audio data; a sound effect determination unit, configured to determine loaded sound effect information corresponding to the audio features, and the loaded sound effects represented by the loaded sound effect information are used to enhance the audio style determined based on the audio features; and a sound effect superposition unit, configured to superimpose the loaded sound effects on the basis of the original audio data.

[0010] In a third aspect, an embodiment of the present application provides a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

[0011] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect.

[0012] The audio information processing method and device provided in the embodiment of the present application firstly performs time domain analysis and frequency domain analysis on the audio segments in the original audio data that are in a preset time period to obtain the audio features of the original audio data; then, the loaded sound effect information corresponding to the audio features is determined, and the loaded sound effect represented by the loaded sound effect information is used to enhance the audio style determined based on the audio features; finally, the loaded sound effect is superimposed on the original audio data. The present application can analyze the audio segments of the original audio data to obtain the audio features of the original audio data, and select appropriate loaded sound effect information according to the audio features, and superimpose it on the original audio data, thereby improving the user experience; and, since the loaded sound effect can be adaptively selected according to the identified audio features, it is suitable for all audio transmission methods such as Bluetooth transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0014] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present application may be applied;

[0015] Figure 2 is a flowchart of an embodiment of an audio information processing method according to the present application;

[0016] Figure 3 is a schematic diagram of an application scenario of the audio information processing method according to this embodiment;

[0017] Figure 4 is a flowchart of another embodiment of the audio information processing method according to the present application;

[0018] Figure 5 is a structural diagram of an embodiment of an audio information processing device according to the present application;

[0019] Figure 6 It is a structural diagram of a computer system suitable for implementing the embodiments of the present application. DETAILED DESCRIPTION

[0020] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present disclosure, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0021] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0022] Figure 1 An exemplary architecture 100 to which the audio information processing method and apparatus of the present application can be applied is shown.

[0023] like Figure 1 As shown, the system architecture 100 may include playback devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the playback devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0024] The playback devices 101, 102, 103 may be hardware devices or software that support network connection for data interaction and data processing. When the playback devices 101, 102, 103 are hardware, they may be various electronic devices that support functions such as information interaction, network connection, audio and video playback, including but not limited to smart speakers, Bluetooth speakers, smart phones, tablet computers, e-book readers, laptop computers, and desktop computers, etc. When the playback devices 101, 102, 103 are software, they may be installed in the electronic devices listed above. It may be implemented as multiple software or software modules, for example, for providing distributed services, or it may be implemented as a single software or software module. No specific limitation is made here.

[0025] The server 105 may be a server that provides various services, such as providing data analysis and processing, data transmission and other functions to the playback devices 101, 102, 103. The server may store or process various received data and feed back the processing results to the playback device.

[0026] It should be noted that the audio information processing method provided in the embodiments of the present disclosure can be executed by the playback devices 101, 102, 103, or by the server 105, or a part of the method can be executed by the playback devices 101, 102, 103, and another part can be executed by the server 105. Accordingly, the audio information processing device can be arranged in the playback devices 101, 102, 103, or in the server 105, or a part of the method can be arranged in the playback devices 101, 102, 103, and another part can be arranged in the server 105. No specific limitation is made here.

[0027] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server is software, it can be implemented as multiple software or software modules for providing distributed services, or it can be implemented as a single software or software module. No specific limitation is made here.

[0028] It should be understood that Figure 1 The number of playback devices and servers in the embodiment is only for illustration. Any number of playback devices and servers may be provided according to implementation requirements.

[0029] Continue to refer Figure 2 , shows a process 200 of an embodiment of an audio information processing method, comprising the following steps:

[0030] Step 201 , performing time domain analysis and frequency domain analysis on the audio segments in the preset time period in the original audio data to obtain the audio features of the original audio data.

[0031] In this embodiment, the execution subject (for example Figure 1 The playback device or server in the original audio data can perform time domain analysis and frequency domain analysis on the audio segments in the preset time period in the original audio data to obtain the audio features of the original audio data.

[0032] The original audio data may be pure audio data, such as voice data, music data, etc., or audio data in multimedia data. The preset time period may be any time period in the playback duration corresponding to the original audio data, that is, the audio segment may be any audio segment in the original audio data. For example, the preset time period may be a preset time period set at the beginning of the original audio, or a preset time period in the middle of the original audio.

[0033] The duration of the preset time period can be set according to the actual situation such as the analysis speed of the device analyzing the audio clip, the audio content, etc., and is not limited here.

[0034] In this embodiment, time domain analysis and frequency domain analysis are used to obtain the audio features of the original audio data. Among them, time domain analysis refers to analyzing the relationship between time and various physical quantities of the audio signal with the time axis as the horizontal axis. For example, filtering, amplification, statistical feature calculation, correlation analysis and other processing of the signal in the time domain are all time domain analysis of the audio signal. Frequency domain analysis refers to analyzing the relationship between frequency and various physical quantities of the audio information signal with the frequency axis as the horizontal axis. The frequency domain analysis of the audio clip can be performed through frequency domain analysis methods such as spectrum, energy spectrum, power spectrum, inverse spectrum, wavelet analysis, etc.

[0035] Audio features are used to characterize the audio type of the original audio data. The audio type can be roughly divided into speech audio and music audio, for example. Music audio can be further subdivided into pop music, classical music, rock music, country music, and other types.

[0036] Different types of audio have different audio features, and these audio features can be used to determine the audio type. For example, the audio features of typical speech audio are continuous words in the time domain, with intervals between adjacent words, and are concentrated in the mid-frequency band in the frequency domain; the audio features of pop music are often coherent with periodic beats in the time domain, and are mostly low-frequency and high-frequency in the frequency domain.

[0037] It should be noted that the execution subject of this step can be a playback device or a server. When the playback device has an audio analysis function, the execution subject of this step can be the playback device with an audio analysis function; otherwise, the execution subject of this step can be a server with an audio analysis function.

[0038] Step 202, determining the loaded sound effect information corresponding to the audio feature.

[0039] In this embodiment, the execution subject can determine the loaded sound effect information of the original audio data through the audio features, wherein the loaded sound effect represented by the loaded sound effect information is used to enhance the audio style determined based on the audio features. For example, the loaded sound effect information can be the loaded sound effect information indicating to reduce the volume of the low frequency and to increase the volume of the middle frequency to highlight the clarity of the voice audio; or the loaded sound effect information can be the loaded sound effect information indicating to increase the volume of some low frequencies and high frequencies to enhance the rhythm of pop music and the brightness of musical instruments.

[0040] In some optional implementations, the execution subject may first determine the audio type of the original audio data according to the audio features of the original audio data; and then determine the loaded sound effect information corresponding to the audio type according to the audio type. Specifically, the loaded sound effect information corresponding to various audio types may be pre-stored, and the audio type and the corresponding loaded sound effect information may be associated.

[0041] It should be noted that the execution subject of this step can be a playback device or a server. When the playback device stores the loaded sound effect information, the execution subject of this step can be the playback device storing the loaded sound effect information; otherwise, the execution subject of this step can be the server storing the loaded sound effect information.

[0042] Step 203, superimpose and load sound effects on the original audio data.

[0043] In this embodiment, the execution entity can collect, transform, filter, evaluate, enhance, compress, recognize and perform other synthesis processes on the original audio data and the determined loaded sound effect information, thereby superimposing the loaded sound effect information on the original audio data.

[0044] In some optional implementations, a DSP (Digital Signal Processing) system may be set in the execution subject, including the main components of a digital signal processor DSP, audio A / D (analog / digital) and D / A (digital / analog), RAM, ROM and peripheral processors. Through the DSP system, the execution subject superimposes and loads sound effects on the original audio data.

[0045] In order to avoid obvious sound changes caused by suddenly adding loading sound effects, which may cause auditory discomfort to users, loading sound effects can be added to the original audio data through delayed addition. For example, delayed addition may be performed by adding a portion of the loading sound effects of a preset value at intervals of a predetermined time until all loading sound effects are added.

[0046] In this embodiment, the execution subject can also perform the superposition of the loading sound effect according to the audio superposition law represented by the preset linear function. First, determine the loading duration corresponding to the loading sound effect information, and the loading duration is used to represent the time required for superimposing the loading sound effect. Generally speaking, the greater the change of the loading sound effect represented by the loading sound effect information, the greater the value of the loading duration. Then, according to the ratio of the loading sound effect information to the loading duration, determine the slope of the preset linear function, that is, the value represented by the superimposed loading sound effect information per unit time. On the basis of the original audio data, the loading sound effect is superimposed according to the audio superposition law represented by the slope of the preset linear function. For example, if the loading sound effect information is to increase the volume by 20dB, and the loading duration determined according to the loading sound effect information is 10S, then it can be determined that the slope of the preset linear function is 20dB / 10S=2dB / S, that is, based on the original audio data, the volume is increased by 2dB per second.

[0047] It should be noted that the execution subject of this step can be a playback device or a server. When the playback device has a sound effect superposition function, the execution subject of this step is the playback device with the sound effect superposition function; otherwise, the execution subject of this step is the server with the sound effect superposition function.

[0048] In this embodiment, the execution entity can analyze the audio fragments of the original audio data, obtain the audio features of the original audio data, and select appropriate loaded sound effect information based on the audio features, and superimpose it on the original audio data, thereby improving the user experience and the practicality of the playback device.

[0049] Figure 3 An application scenario of the audio information processing method of the present embodiment is schematically shown. User 301 is listening to a radio program through a playback device 302, and the radio program is playing pop music. The playback device 302 receives and caches pop music data sent by the server 303 through the network. The playback device 302 performs time domain analysis and frequency domain analysis on the audio segment at the beginning of the pop music data in the cache to obtain the audio features of the original audio data; then, the loaded sound effect information corresponding to the audio features is determined, and the loaded sound effect information is the loaded sound effect information indicating the enhancement of some low-frequency and high-frequency parts to enhance the rhythm of the pop music and the brightness of the musical instruments. Finally, the playback device 302 superimposes the loaded sound effect on the original audio data, and plays the pop music data superimposed with the loaded sound effect.

[0050] Continue to refer Figure 4 , shows a schematic process 400 of another embodiment of the audio information processing method according to the present application, comprising the following steps:

[0051] Step 401: Perform time domain analysis and frequency domain analysis on the acquired sample audio data to obtain sample audio features of the sample audio data.

[0052] In this embodiment, the sample audio data includes various sample audio data. For example, it includes sample audio data representing speech audio and sample audio data representing various types of music. For each audio type, only one typical sample audio data may be selected, or multiple different sample audio data may be selected.

[0053] In this embodiment, the sample audio data may be subjected to time domain analysis and frequency domain analysis in a manner similar to step 201 to obtain sample audio features of the sample audio data, which will not be described in detail herein.

[0054] Step 402, obtaining loaded sound effect information for sample audio data, and associating the sample audio features with the loaded sound effect information.

[0055] In this embodiment, the execution subject can obtain corresponding loading sound effect information for each sample audio data. The loading sound effect represented by the loading sound effect information is used to enhance the audio style determined based on the audio feature. In some optional implementations, the loading sound effect information can be the loading sound effect information obtained by professionals in sound effect processing after sound effect adjustment for the sample audio data.

[0056] In this embodiment, the execution subject needs to associate the sample audio feature with the loaded sound effect information so that the corresponding loaded sound effect information can be obtained through the sample audio feature.

[0057] Step 403: associate the sample audio features with the loaded sound effect information and store them in a loaded sound effect set.

[0058] In this embodiment, the loaded sound effect set stores sample audio features and loaded sound effect information of sample audio data of various audio types. Each audio type may correspond to only one sample audio data or to multiple different sample audio data.

[0059] In this embodiment, the loaded sound effect set may be stored in the playback device, or may be stored in a server connected to the playback device through a network.

[0060] Step 404: Perform time domain analysis and frequency domain analysis on the audio segments in the preset time period in the original audio data to obtain audio features of the original audio data.

[0061] In this embodiment, step 404 is performed in a manner similar to step 201 and will not be described in detail herein.

[0062] Step 405: Determine the loaded sound effect information corresponding to the audio feature.

[0063] In this embodiment, step 405 may be performed in a manner similar to step 202, and will not be described in detail herein.

[0064] In some optional implementations, the execution subject first determines whether the audio feature matches the sample audio feature in the loaded sound effect set. A preset threshold can be set, and the preset threshold is used to determine whether the audio feature matches the sample audio feature in the loaded sound effect set; and the matching degree between the audio feature and the sample audio feature in the loaded sound effect set is digitized to obtain a matching degree value. For example, each feature in the audio feature and the sample audio feature and the corresponding feature value can be digitized by encoding or the like; then, the digitized audio feature and each feature in the sample audio feature are compared item by item, and the number of features that are successfully matched is used as the matching degree value.

[0065] In response to determining that the matching degree value between the audio feature and the sample audio feature in the loaded sound effect set is greater than a preset threshold, the audio feature is considered to match the sample audio feature in the loaded sound effect set.

[0066] Then, in response to determining that the audio feature matches the sample audio feature in the loaded sound effect set, the loaded sound effect information corresponding to the sample audio feature is determined as the loaded sound effect information corresponding to the audio feature.

[0067] Step 406, superimpose and load sound effects on the original audio data.

[0068] In this embodiment, step 406 is performed in a manner similar to step 203 and will not be described in detail herein.

[0069] It can be seen from the embodiment including steps 401-406 that Figure 2 Compared with the corresponding embodiment, the process 400 of the audio information processing method in this embodiment specifically describes setting a loaded sound effect set, in which various sample audio features and corresponding loaded sound effect information are stored. The loaded sound effect of the original audio data is determined by matching the audio feature with the sample audio feature in the loaded sound effect set, thereby improving the efficiency of obtaining the loaded sound effect information.

[0070] Continue to refer Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an audio information processing device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0071] like Figure 5 As shown, the audio information processing device includes: an audio analysis unit 501, a sound effect determination unit 502 and a sound effect superposition unit 503.

[0072] The audio analysis unit 501 is configured to perform time domain analysis and frequency domain analysis on the audio segments in the preset time period in the original audio data to obtain the audio features of the original audio data; the sound effect determination unit 502 is configured to determine the loaded sound effect information corresponding to the audio features, and the loaded sound effect represented by the loaded sound effect information is used to enhance the audio style determined based on the audio features; the sound effect superposition unit 503 is configured to superimpose the loaded sound effect on the basis of the original audio data.

[0073] In some embodiments, the sound effect determination unit 502 is further configured to: determine the audio type of the original audio data according to the audio features; and determine the loaded sound effect information corresponding to the audio type according to the audio type.

[0074] In some embodiments, the above-mentioned device also includes: a sample audio analysis unit (not shown in the figure), configured to perform time domain analysis and frequency domain analysis on the acquired sample audio data to obtain sample audio features of the sample audio data; a sound effect acquisition unit (not shown in the figure), configured to obtain loaded sound effect information for the sample audio data, and associate the sample audio features with the loaded sound effect information; a sound effect storage unit (not shown in the figure), configured to associate the sample audio features with the loaded sound effect information and store them in a loaded sound effect set.

[0075] In some embodiments, the sound effect determination unit 502 is further configured to: determine whether the audio feature matches the sample audio feature in the loaded sound effect set; in response to determining that the audio feature matches the sample audio feature in the loaded sound effect set, determine the loaded sound effect information corresponding to the sample audio feature as the loaded sound effect information corresponding to the audio feature.

[0076] In some embodiments, the sound effect superposition unit 503 is further configured to: determine the loading duration corresponding to the loading sound effect information; and linearly superimpose the loading sound effect on the original audio data based on the loading duration.

[0077] Reference below Figure 6 , which shows a device suitable for implementing the embodiments of the present application (eg Figure 1 Schematic diagram of the structure of a computer system 600 of the devices 101, 102, 103, 105 shown. Figure 6 The device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0078] like Figure 6As shown, the computer system 600 includes a processor (e.g., CPU, central processing unit) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the system 600 are also stored in the RAM 603. The processor 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0079] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0080] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 609, and / or installed from a removable medium 611. When the computer program is executed by the processor 601, the above-mentioned functions defined in the method of the present application are executed.

[0081] It should be noted that the computer-readable medium of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0082] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the client computer, partially on the client computer, as a separate software package, partially on the client computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the client computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0083] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the device, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0084] According to one or more embodiments of the present disclosure, the present disclosure provides an audio information processing method, including: performing time domain analysis and frequency domain analysis on audio segments in a preset time period in original audio data to obtain audio features of the original audio data; determining loaded sound effect information corresponding to the audio features, wherein the loaded sound effects represented by the loaded sound effect information are used to enhance the audio style determined based on the audio features; and superimposing the loaded sound effects on the original audio data.

[0085] According to one or more embodiments of the present disclosure, in the audio information processing method provided by the present disclosure, determining the loaded sound effect information corresponding to the audio features includes: determining the audio type of the original audio data according to the audio features; and determining the loaded sound effect information corresponding to the audio type according to the audio type.

[0086] According to one or more embodiments of the present disclosure, in the audio information processing method provided by the present disclosure, before performing time domain analysis and frequency domain analysis on the audio segments in the original audio data that are in a preset time period to obtain the audio features of the original audio data, the method further includes: performing time domain analysis and frequency domain analysis on the acquired sample audio data to obtain sample audio features of the sample audio data; acquiring loaded sound effect information for the sample audio data, and associating the sample audio features with the loaded sound effect information; and associating the sample audio features with the loaded sound effect information and storing them in a loaded sound effect set.

[0087] According to one or more embodiments of the present disclosure, in the audio information processing method provided by the present disclosure, determining the loaded sound effect information corresponding to the audio feature includes: determining whether the audio feature matches the sample audio feature in the loaded sound effect set; in response to determining that the audio feature matches the sample audio feature in the loaded sound effect set, determining the loaded sound effect information corresponding to the sample audio feature as the loaded sound effect information corresponding to the audio feature.

[0088] According to one or more embodiments of the present disclosure, in the audio information processing method provided by the present disclosure, a loading sound effect is superimposed on the basis of the original audio data, including: determining a loading duration corresponding to the loading sound effect information, the loading duration is used to characterize the time required to superimpose the loading sound effect; according to the loading duration, linearly superimposing the loading sound effect on the basis of the original audio data.

[0089] According to one or more embodiments of the present disclosure, the present disclosure provides an audio information processing device, including: an audio analysis unit, configured to perform time domain analysis and frequency domain analysis on audio segments in a preset time period in original audio data to obtain audio features of the original audio data; a sound effect determination unit, configured to determine loaded sound effect information corresponding to the audio features, the loaded sound effects represented by the loaded sound effect information are used to enhance the audio style determined based on the audio features; and a sound effect superposition unit, configured to superimpose the loaded sound effects on the basis of the original audio data.

[0090] According to one or more embodiments of the present disclosure, in the audio information processing device provided by the present disclosure, the sound effect determination unit is further configured to: determine the audio type of the original audio data according to the audio features; and determine the loaded sound effect information corresponding to the audio type according to the audio type.

[0091] According to one or more embodiments of the present disclosure, the audio information processing device provided by the present disclosure further includes: a sample audio analysis unit, configured to perform time domain analysis and frequency domain analysis on the acquired sample audio data to obtain sample audio features of the sample audio data; a sound effect acquisition unit, configured to acquire loaded sound effect information for the sample audio data, and associate the sample audio features with the loaded sound effect information; and a sound effect storage unit, configured to associate the sample audio features with the loaded sound effect information and store them in a loaded sound effect set.

[0092] According to one or more embodiments of the present disclosure, in the audio information processing device provided by the present disclosure, the sound effect determination unit is further configured to: determine whether the audio feature matches the sample audio feature in the loaded sound effect set; in response to determining that the audio feature matches the sample audio feature in the loaded sound effect set, determine the loaded sound effect information corresponding to the sample audio feature as the loaded sound effect information corresponding to the audio feature.

[0093] According to one or more embodiments of the present disclosure, in the audio information processing device provided by the present disclosure, the sound effect superposition unit is further configured to: determine the loading duration corresponding to the loaded sound effect information, the loading duration is used to characterize the time required to superimpose the loaded sound effect; according to the loading duration, linearly superimpose the loaded sound effect on the basis of the original audio data.

[0094] The units involved in the embodiments described in the present application may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor comprising an audio analysis unit, a sound effect determination unit, and a sound effect superposition unit. Among them, the names of these units do not constitute a limitation on the unit itself in certain circumstances, for example, the audio analysis unit may also be described as a unit that "performs time domain analysis and frequency domain analysis on audio segments in a preset time period in the original audio data to obtain the audio features of the original audio data".

[0095] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the computer device: performs time domain analysis and frequency domain analysis on the audio segments in the original audio data that are in a preset time period to obtain the audio features of the original audio data; determines the loaded sound effect information corresponding to the audio features, and the loaded sound effect represented by the loaded sound effect information is used to enhance the audio style determined based on the audio features; and superimposes the loaded sound effect on the original audio data.

[0096] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the concept of the present disclosure. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the present application.

Claims

1. A method for processing audio information, the method include: Performing time domain analysis and frequency domain analysis on audio segments in a preset time period in the original audio data to obtain audio features of the original audio data; Numerize each feature in the audio feature and the sample audio features in the loaded sound effect set and the feature value corresponding thereto by encoding; For each feature of the digitized audio feature and the sample audio feature, a comparison is performed item by item, and the number of features that are successfully compared is used as the matching degree value; In response to determining that the audio feature matches the sample audio feature in the loaded sound effect set, the audio feature matches the sample audio feature; In response to determining that the audio feature matches the sample audio feature in the loaded sound effect set, determining the loaded sound effect information corresponding to the sample audio feature as the loaded sound effect information corresponding to the audio feature, and the loaded sound effect represented by the loaded sound effect information is used to enhance the audio style determined based on the audio feature; Determine a loading duration corresponding to the loading sound effect information, where the loading duration is used to represent the time required to superimpose the loading sound effect; Determining the slope of a preset linear function according to the ratio of the loading sound effect information to the loading duration; On the basis of the original audio data, the loaded sound effects are superimposed according to the audio superposition rule represented by the slope of the preset linear function.

2. The method according to claim 1, in, The determining the loaded sound effect information corresponding to the audio feature includes: Determining the audio type of the original audio data according to the audio feature; According to the audio type, loading sound effect information corresponding to the audio type is determined.

3. The method according to claim 1, in, Before performing time domain analysis and frequency domain analysis on the audio segments in the preset time period in the original audio data to obtain the audio features of the original audio data, the method further includes: Performing time domain analysis and frequency domain analysis on the acquired sample audio data to obtain sample audio features of the sample audio data; Acquire loaded sound effect information for the sample audio data, and associate the sample audio feature with the loaded sound effect information; The sample audio feature is associated with the loaded sound effect information and stored in a loaded sound effect set.

4. An audio information processing device, the device include: An audio analysis unit is configured to perform time domain analysis and frequency domain analysis on an audio segment in a preset time period in the original audio data to obtain audio features of the original audio data; A sound effect determination unit, configured to determine loaded sound effect information corresponding to the audio feature, wherein the loaded sound effect represented by the loaded sound effect information is used to enhance the audio style determined based on the audio feature; A sound effect superposition unit, configured to determine a loading duration corresponding to the loading sound effect information, wherein the loading duration is used to represent a time required to superimpose the loading sound effect; Determining the slope of a preset linear function according to the ratio of the loading sound effect information to the loading duration; On the basis of the original audio data, the loaded sound effects are superimposed according to the audio superposition rule represented by the slope of the preset linear function; The sound effect determination unit is specifically configured to digitize each feature of the audio feature and the sample audio features in the loaded sound effect set and the feature value corresponding thereto by encoding; For each feature of the digitized audio feature and the sample audio feature, a comparison is performed item by item, and the number of features that are successfully compared is used as the matching degree value; In response to determining that the audio feature matches the sample audio feature in the loaded sound effect set, the audio feature matches the sample audio feature; In response to determining that the audio feature matches the sample audio feature in the loaded sound effect set, the loaded sound effect information corresponding to the sample audio feature is determined as the loaded sound effect information corresponding to the audio feature.

5. A computer readable medium having a computer program stored thereon, in, When the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

6. An electronic device, include: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 3.

Citation Information

Patent Citations

  • Method, device and system for hiding audio stream error

    CN101155140A

  • Method and apparatus for auto-extracting wonderful segment of sports program

    CN101398826A

  • Device and method for analyzing audio data

    CN101685446A

  • Method and device for recognizing human voices in audio

    CN103489445A

  • Terminal volume adjustment processing method and terminal

    CN107360326A