Audio processing method and apparatus

Abstract

The present disclosure relates to the technical field of data processing. Disclosed are an audio processing method and apparatus, which are mainly intended to implement direct running in different clients and visualization in audio tracks for visual audio analysis. The main technical solution of the present disclosure is: acquiring an execution command; on the basis of the execution command, by using a preset instance method, acquiring, from a preset configuration file, configuration data corresponding to the execution command; on the basis of the execution command, acquiring audio data by using a preset acquisition rule; on the basis of the configuration data corresponding to the execution command, performing audio compression and format processing on the audio data by using a preset processing rule, so as to obtain target audio data; and on the basis of the target audio data, performing visualization processing by using a preset visualization rule, so as to obtain visual display corresponding to the target audio data, wherein the visual display comprises audio tracks and text annotations. The present disclosure is used for audio compression and visualization.

Description

Audio processing method and device

[0001] This disclosure claims priority to Chinese patent application No. 202311099735.X filed on August 29, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of data processing technology, and in particular to an audio processing method and device. Background Art

[0003] During work, the user end needs to automatically collect audio or introduce audio through plug-ins or SDK (English full name: Software Development Kit, Chinese full name: Software Development Kit) for audio compression and visualization.

[0004] Currently, the method for audio compression and visualization is to use FFmpeg method or instance method for audio encoding compression, and process it through Linux platform or server to visualize it in text form; wherein, the FFmpeg method refers to an open source computer program developed under the Linux platform that can be used to record, convert and transform digital audio; the instance method is an object method in JavaScript method, and audio encoding compression is performed by changing data parameters through the instance method.

[0005] However, the above processing methods are all processed using C / C++ and are all processed natively through Android / IOS. Therefore, they cannot be directly run on user terminals such as browsers, H5, and mini-programs; and they are all visualized in text form, not visualized in the audio track, and audio visualization analysis cannot be performed.

[0006] Summary of the Invention

[0007] In view of the above problems, the present disclosure provides an audio processing method and device, the main purpose of which is to achieve direct operation on different user terminals and visualize the audio into the audio track for audio visualization analysis.

[0008] To solve the above technical problems, the present disclosure proposes the following solutions:

[0009] In a first aspect, the present disclosure provides an audio processing method, the method comprising:

[0010] Get the execution command;

[0011] Based on the execution command, obtaining configuration data corresponding to the execution command from a preset configuration file using a preset instance method, wherein the preset configuration file at least includes an audio output mode, an audio encoding method, and an audio file format;

[0012] Based on the execution command, the audio data is acquired using a preset acquisition rule;

[0013] Based on the configuration data corresponding to the execution command, the audio data is compressed and formatted using a preset processing rule to obtain target audio data;

[0014] Based on the target audio data, visualization processing is performed using preset visualization rules to obtain a visualization display corresponding to the target audio data, where the visualization display includes an audio track and text annotations.

[0015] In a second aspect, the present disclosure provides an audio processing device, the device comprising:

[0016] A first acquiring unit, configured to acquire an execution command;

[0017] a second acquiring unit, configured to acquire, based on the execution command, configuration data corresponding to the execution command from a preset configuration file using a preset instance method, wherein the preset configuration file includes at least an audio output mode, an audio encoding method, and an audio file format;

[0018] A third acquiring unit, configured to acquire audio data based on the execution command using a preset acquiring rule;

[0019] a processing unit, configured to perform audio compression and format processing on the audio data using preset processing rules based on the configuration data corresponding to the execution command to obtain target audio data;

[0020] The visualization unit is used to perform visualization processing based on the target audio data using preset visualization rules to obtain a visualization display corresponding to the target audio data, wherein the visualization display includes an audio track and text annotations.

[0021] In order to achieve the above-mentioned purpose, according to the third aspect of the present disclosure, a storage medium is provided, which includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the audio processing method described in the first aspect.

[0022] In order to achieve the above-mentioned purpose, according to the fourth aspect of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, all or part of the steps for the audio processing device as described in the second aspect are implemented.

[0023] By means of the above technical solution, the audio processing method and device provided by the present disclosure is that due to the use of FFmpeg method or instance method for audio encoding compression, the audio data is processed and displayed visually in text form through the Linux platform or server, which has the problem that it cannot be directly run on the user end such as browser, H5, applet, etc.; and it is all displayed visually in text form, without visualization into the audio track, and audio visualization analysis cannot be performed. To this end, the present disclosure obtains an execution command; based on the execution command, obtains the configuration data corresponding to the execution command from a preset configuration file using a preset instance method, wherein the preset configuration file at least includes an audio output mode, an audio encoding method and an audio file format; based on the execution command, obtains audio data using a preset acquisition rule; based on the configuration data corresponding to the execution command, performs audio compression and format processing on the audio data using a preset processing rule to obtain target audio data; based on the target audio data, performs visualization processing using a preset visualization rule to obtain a visualization display corresponding to the target audio data, and the visualization display includes an audio track and text annotation. The present disclosure can lower the threshold for audio visualization and can visualize audio tracks for audio visualization analysis; reduce user and enterprise traffic and storage costs through audio compression; be compatible with different terminals including PC, H5, mini-programs, etc.; and select other customized configuration options to facilitate application in different scenarios.

[0024] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the specific implementation methods of the present disclosure are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present disclosure. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0026] FIG1 shows a flow chart of an audio processing method provided by an embodiment of the present disclosure;

[0027] FIG2 shows a flow chart of another audio processing method provided by an embodiment of the present disclosure;

[0028] FIG3 shows a block diagram of an audio processing device provided by an embodiment of the present disclosure;

[0029] FIG4 shows a block diagram of another audio processing device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0031] Explanation of terms:

[0032] FFmpeg: An open-source computer program for recording, converting, and streaming digital audio and video. Licensed under the LGPL or GPL, it provides a complete solution for recording, converting, and streaming audio and video.

[0033] Linux (GNU / Linux) is a free-to-use and freely distributable UNIX-like operating system. Its kernel was released by Linus Benedict Torvalds. It is a POSIX-based, multi-user, multi-tasking operating system with multi-threading and multi-CPU support. It supports 32-bit and 64-bit hardware and can run major Unix tools, applications, and network protocols.

[0034] WebAssembly: abbreviated as wasm, is a virtual instruction set architecture (virtual ISA). The overall architecture includes the core ISA definition, binary encoding, definition and execution of program semantics, and application programming interfaces (WebAssembly API) for different embedded environments (such as the Web).

[0035] H5: HTML5 is a language for describing web content. HTML5 is the next-generation standard for the internet, a language for building and presenting internet content. It is considered one of the core technologies of the internet.

[0036] PCM (Pulse Code Modulation) is a coding method used in digital communications. It samples analog signals such as voice and video at regular intervals to discretize them. The sampled values ​​are rounded off and quantized by layer unit. The sampled values ​​are then represented by a set of binary codes to represent the amplitude of the sampled pulses.

[0037] WAV: It is one of the most common sound file formats. It is a standard digital audio file developed by Microsoft specifically for Windows. The file can record various mono or stereo sound information and ensure that the sound is not distorted.

[0038] MP3 is an audio compression technology. Its full name is Moving Picture Experts Group Audio Layer III, or simply MP3. It is designed to significantly reduce the size of audio data. Using MPEG Audio Layer 3 technology, music can be compressed into smaller files at a compression ratio of 1:10 or even 1:12, while for most users, the playback sound quality is not noticeably reduced compared to the original uncompressed audio.

[0039] Ogg (OggVorbis), short for OGGVobis, is an audio compression format similar to music formats like MP3. Ogg is completely free, open, and patent-free. The OggVorbis file extension is ".ogg." The Ogg file format can be continuously improved in size and sound quality without affecting older encoders or players.

[0040] Canvas: The Canvas API is a new tag added in HTML5 for generating images in real time on web pages and manipulating image content. Basically, it is a bitmap that can be manipulated with JavaScript.

[0041] JavaScript, also known as "JS," is a lightweight, function-first, interpreted or just-in-time compiled programming language. While best known as a scripting language for web pages, it's also used in many non-browser environments. JavaScript is a multi-paradigm, dynamic scripting language based on prototype programming, supporting object-oriented, imperative, declarative, and functional programming paradigms.

[0042] ASR: Speech recognition technology, also known as automatic speech recognition (ASR), aims to convert the lexical content of human speech into computer-readable input, such as keystrokes, binary codes, or character sequences. This differs from speaker identification and speaker verification, which attempt to identify or verify the speaker of the speech rather than the lexical content.

[0043] Opus: A lossy audio coding format developed by the Xiph.Org Foundation and later standardized by the IETF (Internet Engineering Task Force). Its goal is to replace Speex and Vorbis with a single format for both audio and speech, suitable for low-latency, real-time audio transmission over the Internet. The standard format is defined in RFC 6716. The Opus format is open and free of patents or restrictions.

[0044] CDN stands for Content Delivery Network. Built on the existing Internet, CDN systems create an intelligent virtual network composed of distributed node servers. Based on comprehensive information such as network traffic, node connectivity and load, distance to the user, and response time, CDN systems can redirect user requests to the nearest service node in real time. This allows users to access their desired content locally, alleviate Internet congestion, and improve website response times.

[0045] Raw: Usually called "source code", it means to output the audio format on the CD through coaxial and optical fiber without any processing.

[0046] Currently, FFmpeg methods or example methods are used for audio coding and compression, which are processed on a Linux platform or server-side and displayed visually in text form. However, since the above-mentioned methods are processed in C / C++ and are all processed natively on Android / IOS, they cannot be directly run on user terminals such as browsers, H5, and applets. Moreover, they are all displayed visually in text form, without being visualized in the audio track, making it impossible to perform audio visual analysis. To address this problem, the inventors came up with the idea of ​​using a preset example method and a preset configuration file in conjunction with secondary compression audio processing to perform audio track combination with text annotation and then visual analysis.

[0047] To this end, an embodiment of the present disclosure provides an audio processing method that can be directly run on different user terminals and visualized into audio tracks for audio visualization analysis. The specific execution steps are shown in Figure 1, including:

[0048] 101. Get the execution command.

[0049] When the user terminal operates an audio processing project that needs to be completed, the user terminal needs to first call a preset instance method through a preset call address, and then select the audio processing project to be completed through this instance method, such as audio compression, audio visualization, etc., which is not specifically limited in this embodiment; when the audio processing project to be processed is selected, an execution command is generated.

[0050] 102. Based on the execution command, obtain configuration data corresponding to the execution command from the preset configuration file using a preset instance method.

[0051] The preset configuration file includes at least an audio output mode, an audio encoding method, and an audio file format. The audio output mode can be "Raw" or "PCM", the audio encoding method can be "Waveform Coding" or "Hybrid Coding", and the audio file format can be WAV, MP3, ogg, etc., which are not specifically limited in this embodiment.

[0052] The execution command can be obtained from step 101. By parsing the execution command, the configuration items required to complete the selected audio processing project can be obtained, such as: audio output mode, audio encoding method, etc., which are not specifically limited in this embodiment; according to the required configuration items, the corresponding configuration data is obtained from the preset configuration file for subsequent processing of the collected audio data.

[0053] 103. Based on the execution command, obtain audio data using a preset acquisition rule.

[0054] The execution command generated from step 101 needs to obtain authorization from the user end. Only when the user end agrees to collect audio data can the audio data be obtained. Therefore, a preset acquisition rule can be set. The preset acquisition rule can be used to determine whether the user end authorizes to obtain audio data. When the user end authorizes, audio data can be collected to complete the audio processing project.

[0055] 104. Based on the configuration data corresponding to the execution command, perform audio compression and format processing on the audio data using preset processing rules to obtain target audio data.

[0056] From step 103 , the configuration data corresponding to the execution command can be obtained. The configuration data includes data required to complete audio compression and format processing, such as sampling rate, audio file format, etc., which are not specifically limited in this embodiment.

[0057] Based on the configuration data, the audio data can be directly compressed using a WebAssembly encapsulated FFmpeg method to obtain target audio data. Alternatively, the audio data can be initially compressed based on a compression ratio and then secondary compressed using a WebAssembly encapsulated FFmpeg method to obtain target audio data. This embodiment does not specifically limit this. The target audio data is audio data that has undergone audio compression and format conversion and can be used for visualization operations.

[0058] 105. Based on the target audio data, perform visualization processing using preset visualization rules to obtain a visualization display corresponding to the target audio data.

[0059] The visual display includes an audio track and text annotations. The preset visualization rule may be to use Canvas and JavaScript to form an audio track visual display of the target audio data.

[0060] The target audio data is obtained from step 104 and visualized as a track using Canvas and JavaScript. Text annotations, such as keywords and explanations, are also performed based on the corresponding relationship between the audio and the text, although this embodiment does not specifically limit this. Furthermore, operations such as track analysis can be performed on the visualized display by highlighting words or dragging the track.

[0061] Based on the implementation of the embodiment of Figure 1 above, it can be seen that the present disclosure provides an audio processing method, which obtains an execution command; based on the execution command, obtains the configuration data corresponding to the execution command from a preset configuration file using a preset instance method, wherein the preset configuration file at least includes an audio output mode, an audio encoding method, and an audio file format; based on the execution command, obtains audio data using a preset acquisition rule; based on the configuration data corresponding to the execution command, performs audio compression and format processing on the audio data using a preset processing rule to obtain target audio data; based on the target audio data, performs visualization processing using a preset visualization rule to obtain a visualization display corresponding to the target audio data, wherein the visualization display includes an audio track and text annotations. The present disclosure can lower the threshold for audio visualization, and can be visualized into an audio track for audio visualization analysis; reduce user and enterprise traffic and storage costs through audio compression; be compatible with different terminals including PC, H5, applets, etc.; and select other customized configuration options to facilitate application in different scenarios.

[0062] Furthermore, as a refinement and extension of the embodiment shown in FIG1 , the embodiment of the present disclosure also provides another audio processing method, as shown in FIG2 , and the specific steps are as follows:

[0063] 201. Use the preset calling rules to call the preset instance method.

[0064] Developers will design the preset instance method before the audio processing project and save it in a preset storage address. The user end can call the preset instance method through the preset storage address, or upload it to the user end using a mobile storage device for use.

[0065] For example: download by setting a command or import via CDN, and call the preset instantiation method.

[0066] 202. Use the preset instance method to create a preset configuration file.

[0067] After calling the preset instance method in advance through step 201, a preset configuration file can be created through the preset instance method. The creation process is to set the configuration item and the data corresponding to the configuration item through the preset instance method. For example, the configuration item can select the audio file format, and then the corresponding data is set to "WAV". This is not specifically limited in this embodiment.

[0068] 203. Get the execution command.

[0069] This step is combined with the description of step 101 in the above method, and the same content is not repeated here.

[0070] 204. Based on the execution command, obtain configuration data corresponding to the execution command from the preset configuration file using a preset instance method.

[0071] This step is combined with the description of step 102 in the above method, and the same contents are not repeated here.

[0072] The execution command can be obtained from step 203, and then the execution command is parsed to obtain the configuration items required to complete the execution command, such as: audio output mode, audio encoding method and audio file format, etc., which are not specifically limited in this embodiment; based on the configuration items, the configuration data corresponding to the configuration items are obtained from the preset configuration file using a preset instance method to obtain the configuration data corresponding to the execution command, for example: the audio output mode is "PCM", the audio encoding method is "waveform encoding" and the audio file format is "MP3", etc.

[0073] 205. Based on the execution command, obtain audio data using a preset acquisition rule.

[0074] This step is combined with the description of step 103 in the above method, and the same contents are not repeated here.

[0075] The execution command can be obtained from step 203, and then based on the execution command, it is determined whether the user terminal is authorized to obtain the audio data; if so, the audio data is obtained; if not, the process returns to the previous step to execute the action.

[0076] 206. Based on the configuration data corresponding to the execution command, perform audio compression and format processing on the audio data using preset processing rules to obtain target audio data.

[0077] This step is combined with the description of step 104 in the above method, and the same contents are not repeated here.

[0078] The configuration data may include a collection sampling rate and an output sampling rate, which is not specifically limited in this embodiment; after obtaining the configuration data corresponding to the execution command from step 205, first, the ratio of the collection sampling rate to the output sampling rate is calculated to obtain a compression rate, that is, the ratio is the compression rate; based on the compression rate, the audio data is compressed using a preset compression rule to obtain primary compressed audio data; the primary compressed audio data set is secondary compressed (i.e., opus compression is performed) using the method of encapsulating FFmpeg using Webassembly to obtain target audio data, thereby further improving the compression capability.

[0079] Among them, based on the compression rate, the audio data is compressed using preset compression rules to obtain primary compressed audio data, including: identifying noise data and empty data in the audio data, and deleting them to obtain audio data to be compressed. This processing can improve the compression capability; compressing the audio data to be compressed according to the compression rate to obtain the primary compressed audio data.

[0080] For example:

[0081] The client collects audio and processes the sampling rate from 48000 / 44100Hz to 8000 / 16000Hz, then performs encoding compression to compress the PCM stream into an OPUS data stream, and supports converting the PCM stream into audio formats such as WAV, MP3, and OGG for audio streaming transmission.

[0082] 207. Based on the target audio data, perform visualization processing using preset visualization rules to obtain a visualization display corresponding to the target audio data.

[0083] This step is combined with the description of step 105 in the above method, and the same contents are not repeated here.

[0084] The target audio data can be obtained from step 206, and then the target audio data is identified using ASR to obtain the audio and text correspondence, which can be stored on the server side; based on the audio and text correspondence and the target audio data (which can be obtained from the server side), Canvas and JavaScript are used to form a track visualization display, that is, JavaScript controls Canvas to draw the track, and the track visualization display includes text, for example: keywords are placed in the track; based on the track visualization display, audio analysis and annotation are performed using preset movement commands. The preset movement commands can be word marking or dragging the track, that is, audio analysis and annotation are achieved by marking words and dragging the track.

[0085] Based on the implementation method of Figure 2 above, it can be seen that the present disclosure provides an audio processing method. The secondary compression processing method of the present disclosure can make the memory occupied by the audio after compression smaller, which can overcome the problem in the prior art that the data packet is too large and occupies too much memory on the user side, and the use of C / C++ for encapsulation leads to large packets and large memory consumption during C / C++ runtime. At the same time, audio collection and transmission can be more conveniently performed in different terminals; the present disclosure adopts a preset instance method and a preset configuration file method, which is suitable for use by different user terminals for audio compression, overcoming the problem in the prior art that it cannot be directly applied to the user terminal when used; the present disclosure can combine audio tracks with text for audio track visualization, solve the problem of visual content analysis of audio in enterprise data analysis, and expand the value mining method and efficiency of audio assets.

[0086] Furthermore, as an implementation of the method shown in FIG. 1 , the present disclosure also provides an audio processing device for implementing the method shown in FIG. This device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will not further detail the aforementioned method embodiment. However, it should be clear that the device in this embodiment is capable of implementing all of the aforementioned method embodiments. As shown in FIG. 3 , the device includes:

[0087] A first acquiring unit 31 is configured to acquire an execution command;

[0088] a second acquiring unit 32 configured to acquire, based on the execution command obtained from the first acquiring unit 31, configuration data corresponding to the execution command from a preset configuration file using a preset instance method, wherein the preset configuration file includes at least an audio output mode, an audio encoding method, and an audio file format;

[0089] A third acquiring unit 33 is configured to acquire audio data using a preset acquisition rule based on the execution command obtained from the first acquiring unit 31;

[0090] a processing unit 34 configured to perform audio compression and format processing on the audio data obtained from the third obtaining unit 33 using preset processing rules based on the configuration data corresponding to the execution command obtained from the second obtaining unit 32 to obtain target audio data;

[0091] The visualization unit 35 is configured to perform visualization processing on the target audio data obtained from the processing unit 34 using preset visualization rules to obtain a visualization display corresponding to the target audio data, wherein the visualization display includes an audio track and text annotations.

[0092] Furthermore, as an implementation of the method shown in FIG. 2 , the present disclosure also provides another audio processing device for implementing the method shown in FIG. 2 . This device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will not further detail the aforementioned method embodiment. However, it should be clear that the device in this embodiment can implement all of the aforementioned method embodiments. As shown in FIG. 4 , the device includes:

[0093] A first acquiring unit 31 is configured to acquire an execution command;

[0094] a second acquiring unit 32 configured to acquire, based on the execution command obtained from the first acquiring unit 31, configuration data corresponding to the execution command from the preset configuration file obtained by the creating unit 37 using a preset instance method, wherein the preset configuration file includes at least an audio output mode, an audio encoding method, and an audio file format;

[0095] A third acquiring unit 33 is configured to acquire audio data using a preset acquisition rule based on the execution command obtained from the first acquiring unit 31;

[0096] a processing unit 34 configured to perform audio compression and format processing on the audio data obtained from the third obtaining unit 33 using preset processing rules based on the configuration data corresponding to the execution command obtained from the second obtaining unit 32 to obtain target audio data;

[0097] A visualization unit 35 is configured to perform visualization processing on the target audio data obtained from the processing unit 34 using preset visualization rules to obtain a visualization corresponding to the target audio data, wherein the visualization includes an audio track and text annotations;

[0098] A calling unit 36 ​​is configured to call the preset instance method using a preset calling rule;

[0099] The creating unit 37 is configured to create the preset configuration file using the preset instance method obtained from the calling unit 36 ​​.

[0100] Furthermore, the configuration data includes at least a collection sampling rate and an output sampling rate; the processing unit 34 includes:

[0101] A calculation module 341 is configured to calculate a ratio of the collection sampling rate to the output sampling rate to obtain a compression rate;

[0102] A first compression module 342 is configured to compress the audio data using a preset compression rule based on the compression rate obtained from the calculation module 341 to obtain primary compressed audio data;

[0103] The second compression module 343 is configured to perform secondary compression on the primary compressed audio data set obtained from the first compression module 342 by using a method of encapsulating FFmpeg in Webassembly to obtain target audio data.

[0104] Furthermore, the first compression module 342 includes:

[0105] an identification submodule 3421 for identifying noise data and empty data in the audio data and deleting them to obtain audio data to be compressed;

[0106] The compression submodule 3422 is configured to compress the to-be-compressed audio data obtained from the recognition submodule 3421 according to the compression rate to obtain the primary compressed audio data.

[0107] Furthermore, the visualization unit 35 includes:

[0108] Recognition module 351, used for using ASR to recognize the target audio data and obtain the corresponding relationship between audio and text;

[0109] A visualization module 352 is configured to generate a track visualization using Canvas and JavaScript based on the audio-text correspondence obtained from the recognition module 351 and the target audio data, wherein the track visualization includes text;

[0110] The analysis module 353 is configured to perform audio analysis and annotation based on the audio track visualization display obtained from the visualization module 352 using preset movement commands.

[0111] Furthermore, the second acquiring unit 32 includes:

[0112] A parsing module 321 is configured to parse the execution command to obtain configuration items required to complete the execution command;

[0113] The acquisition module 322 is configured to obtain configuration data corresponding to the configuration item from a preset configuration file using a preset instance method based on the configuration item obtained from the parsing module 321, thereby obtaining configuration data corresponding to the execution command.

[0114] Furthermore, the third obtaining unit 33 includes:

[0115] A determination module 331 is configured to determine, based on the execution command, whether the user terminal is authorized to obtain the audio data;

[0116] an acquisition module 332 configured to acquire the audio data if the user end authorization obtained from the determination module 331 is obtained to acquire the audio data;

[0117] The return module 333 is used to return to the previous step to perform an action if the user terminal obtained from the judgment module 331 does not authorize the acquisition of the audio data.

[0118] Furthermore, an embodiment of the present disclosure also provides a processor, which is used to run a program, wherein the program executes the audio processing method described in Figures 1-2 above when running.

[0119] Furthermore, an embodiment of the present disclosure also provides a storage medium, which is used to store a computer program, wherein when the computer program is running, it controls the device where the storage medium is located to execute the audio processing method described in Figures 1-2 above.

[0120] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0121] It is understood that the relevant features of the above methods and devices can be referenced to each other. In addition, the terms "first" and "second" in the above embodiments are used to distinguish between the embodiments, and do not represent the advantages and disadvantages of the embodiments.

[0122] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0123] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems may also be used together with the teachings herein. Based on the above description, it is apparent that the structure required for constructing such systems is suitable. In addition, the present disclosure is not directed to any specific programming language. It should be understood that various programming languages ​​may be utilized to implement the present disclosure described herein, and the description of specific languages ​​above is provided for the purpose of disclosing the best mode of implementation of the present disclosure.

[0124] In addition, the memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0125] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0127] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0129] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0130] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0131] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0132] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0133] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The above are merely examples of the present disclosure and are not intended to limit the present disclosure. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure are intended to be included within the scope of the claims of the present disclosure.

Claims

1. An audio processing method, characterized in that: The method comprises: Get the execution command; Based on the execution command, obtaining configuration data corresponding to the execution command from a preset configuration file using a preset instance method, wherein the preset configuration file at least includes an audio output mode, an audio encoding method, and an audio file format; Based on the execution command, the audio data is acquired using a preset acquisition rule; Based on the configuration data corresponding to the execution command, the audio data is compressed and formatted using a preset processing rule to obtain target audio data; Based on the target audio data, visualization processing is performed using preset visualization rules to obtain a visualization display corresponding to the target audio data, wherein the visualization display includes an audio track and text annotations.

2. The method according to claim 1, characterized in that: Before obtaining the execution command, the method further includes: Calling the preset instance method using a preset calling rule; The preset configuration file is created using the preset instance method.

3. The method according to claim 1, characterized in that The configuration data at least includes a collection sampling rate and an output sampling rate; The step of performing audio compression and format processing on the audio data based on the configuration data corresponding to the execution command using a preset processing rule to obtain target audio data includes: Calculating the ratio of the collection sampling rate to the output sampling rate to obtain a compression rate; Based on the compression rate, compressing the audio data using a preset compression rule to obtain primary compressed audio data; The primary compressed audio data set is compressed twice by encapsulating FFmpeg using Webassembly to obtain target audio data.

4. The method according to claim 3, characterized in that The step of compressing the audio data based on the compression rate using a preset compression rule to obtain primary compressed audio data includes: Identify the noise data and empty data in the audio data and delete them. To the audio data to be compressed; The audio data to be compressed is compressed according to the compression rate to obtain the primary compressed audio data.

5. The method according to any one of claims 1 to 4, characterized in that: The step of performing visualization processing based on the target audio data using preset visualization rules to obtain a visualization display corresponding to the target audio data includes: Using ASR to identify the target audio data to obtain the corresponding relationship between the audio and the text; Based on the corresponding relationship between the audio and the text and the target audio data, using Canvas and JavaScript to form a visual display of the audio track, wherein the visual display of the audio track includes text; Based on the audio track visualization display, audio analysis and annotation are performed using preset movement commands.

6. The method according to claim 5, characterized in that The obtaining, based on the execution command, configuration data corresponding to the execution command from a preset configuration file using a preset instance method includes: Parsing the execution command to obtain configuration items required to complete the execution command; Based on the configuration item, the configuration data corresponding to the configuration item is obtained from a preset configuration file using a preset instance method, so as to obtain the configuration data corresponding to the execution command.

7. The method according to claim 5, characterized in that The step of acquiring audio data based on the execution command using a preset acquisition rule includes: Based on the execution command, determining whether the user terminal is authorized to obtain the audio data; If yes, obtaining the audio data; If not, return to the previous step to execute the action.

8. An audio processing device, characterized in that: include: A first acquisition unit, used to acquire an execution command; A second acquisition unit is used to acquire configuration data corresponding to the execution command from a preset configuration file using a preset instance method based on the execution command, wherein the preset configuration file at least includes an audio output mode, an audio encoding method, and an audio file format; A third acquisition unit, configured to acquire audio data based on the execution command using a preset acquisition rule; A processing unit is used to use a preset processing rule based on the configuration data corresponding to the execution command. Then, the audio data is compressed and formatted to obtain target audio data; A visualization unit is used to perform visualization processing based on the target audio data using preset visualization rules to obtain a visualization display corresponding to the target audio data, wherein the visualization display includes an audio track and text annotations.

9. A storage medium comprising a stored program, characterized in that: When the program is running, the device where the storage medium is located is controlled to execute the audio processing method according to any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the audio processing method according to any one of claims 1 to 7 is implemented.

Images