Audio processing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of audio processing, and discloses an audio processing method and device, equipment and a storage medium, and the method comprises the steps: carrying out the first audio processing of the current to-be-processed audio data through a first audio processing algorithm under the condition of receiving the current to-be-processed audio data, and obtaining a first audio processing result, the data volume in the first audio processing result is the first processing data volume; when the first processing data volume is not lower than the second processing data volume, performing second audio processing on the first audio processing result through a second audio processing algorithm to obtain a target processing result; and when the first processing data volume is lower than the second processing data volume, caching the first audio processing result, waiting for the first audio processing algorithm to output a new first audio processing result, complementing the cached first processing result and the new first audio processing result, and performing second audio processing on the complementing result through a second audio processing algorithm, and obtaining a target processing result. The delay is reduced.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to an audio processing method, apparatus, device and storage medium. Background Technology

[0002] Currently, to improve the listening experience, various audio devices commonly employ audio processing algorithms (such as active noise cancellation, echo cancellation, and speech separation) to process the audio data in each frame of audio captured. In practical applications, a complete audio processing flow often requires multiple algorithms to process data sequentially. However, because the amount of data processed by upstream and downstream algorithms differs each time (e.g., 5 versus 3), the real-time result produced by the upstream algorithm (such as active noise cancellation) cannot directly match the amount of input data required by the downstream algorithm (such as echo cancellation). For example, the active noise cancellation algorithm produces 5 data points, while the echo cancellation algorithm requires 3 data points to activate, hindering the smooth data transfer at this point.

[0003] To address this challenge, existing technologies commonly employ a "data caching" mechanism. This involves setting up a temporary buffer between two algorithm levels. The system first accumulates the audio data produced by the previous algorithm in the buffer until the accumulated data volume reaches or exceeds an integer multiple of the input data required by the next algorithm (e.g., finding the least common multiple of 5 and 3, 15). Then, the entire block of data is retrieved from the buffer and segmented into portions (e.g., three at a time) that conform to the format required by the next algorithm before inputting. While this caching mechanism ensures data integrity, its inherent processing flow introduces significant latency. Summary of the Invention

[0004] The main purpose of this application is to provide an audio processing method that aims to solve the technical problem of how to reduce audio processing latency.

[0005] To achieve the above objectives, this application proposes an audio processing method, which is applied to an audio processing device. The audio processing device includes a first audio processing algorithm and a second audio processing algorithm. The first audio processing algorithm corresponds to a first amount of data to be processed, and the second audio processing algorithm corresponds to a second amount of data to be processed. The processing priority of the first audio processing algorithm is higher than that of the second audio processing algorithm. The method includes: Upon receiving the current audio data to be processed, the current audio data to be processed is subjected to first audio processing through the first audio processing algorithm to obtain a first audio processing result, wherein the amount of data in the first audio processing result is the amount of the first processed data. When the first amount of processed data is not less than the second amount of processed data, the first audio processing result is subjected to second audio processing by the second audio processing algorithm to obtain the target processing result. When the first processing data volume is lower than the second processing data volume, the first audio processing result is cached, and the first audio processing algorithm outputs a new first audio processing result. The cached first processing result and the new first audio processing result are then filled in until the data volume in the filled result is not lower than the second processing data volume. Then, the second audio processing algorithm is used to perform second audio processing on the filled result to obtain the target processing result.

[0006] In one embodiment, the step of performing first audio processing on the current audio data to be processed using the first audio processing algorithm to obtain a first audio processing result includes: The audio data in the current audio data to be processed is sequentially segmented according to the first processing data volume; The obtained first segmentation results are sequentially input into the first audio processing algorithm for first audio processing to obtain the first audio processing result.

[0007] In one embodiment, the first audio processing algorithm includes at least two different first processing modes; The step of sequentially inputting the obtained first segmentation results into the first audio processing algorithm for first audio processing to obtain the first audio processing result includes: Obtain the first mode flag bit corresponding to each audio data in the current audio data to be processed, and determine the target first processing mode corresponding to each audio data in the current audio data to be processed based on the first mode flag bit; The obtained first segmentation result is sequentially input into the first audio processing algorithm, and the first audio processing algorithm performs first audio processing on the first segmentation result according to the target first processing mode to obtain the first audio processing result.

[0008] In one embodiment, the step of performing second audio processing on the first audio processing result using the second audio processing algorithm to obtain the target processing result when the first processed data volume is not less than the second processed data volume includes: When the first processed data volume is not less than the second processed data volume, the audio data in the first audio processing result is sequentially segmented according to the second processed data volume. The obtained second segmentation results are sequentially input into the second audio processing algorithm for second audio processing to obtain the target processing result.

[0009] In one embodiment, the second audio processing algorithm includes at least two different second processing modes; The step of sequentially inputting the obtained second segmentation results into the second audio processing algorithm for second audio processing to obtain the target processing result includes: Obtain the second mode flag bit corresponding to each audio data in the current audio data to be processed, and determine the target second processing mode corresponding to each audio data in the current audio data to be processed based on the second mode flag bit; The obtained second segmentation results are sequentially input into the second audio processing algorithm, and the second audio processing algorithm performs second audio processing on the second segmentation results according to the target second processing mode to obtain the target processing result.

[0010] In one embodiment, the step of sequentially segmenting each audio data in the first audio processing result according to the second processing data volume when the first processed data volume is not less than the second processed data volume includes: When the first amount of processed data is not less than the second amount of processed data, the current number of processing steps and the current processing remainder are determined based on the first amount of processed data and the second amount of processed data. The audio data in the first audio processing result is sequentially segmented according to the second processing data volume and the current processing count. After the step of sequentially inputting the obtained second segmentation results into the second audio processing algorithm for second audio processing to obtain the target processing result, the method further includes: The audio data corresponding to the current processing remainder in the first audio processing result is cached. When a new first audio processing result is obtained, the new first audio processing result and the audio data corresponding to the current processing remainder are padded together until the amount of data in the padded result is not less than the amount of data processed by the second processing. Then, the padded result is processed by the second audio processing algorithm to obtain a new target processing result.

[0011] In one embodiment, after the step of obtaining the target processing result, the method further includes: After the current audio data to be processed has been received, the total number of data receptions and the total number of processing operations of the second audio processing algorithm are determined based on the first amount of data to be processed and the second amount of data to be processed. The target processing result is checked for loss based on the total number of data receptions and the total number of processing operations to obtain the check result.

[0012] Furthermore, to achieve the above objectives, this application also proposes an audio processing apparatus, the apparatus comprising: The data receiving module is used to perform first audio processing on the current audio data to be processed by a first audio processing algorithm when receiving the current audio data to be processed, and obtain a first audio processing result, wherein the amount of data in the first audio processing result is the first processed data amount; The first processing module is used to perform second audio processing on the first audio processing result using a second audio processing algorithm when the first processing data volume is not less than the second processing data volume, so as to obtain the target processing result. The second processing module is used to cache the first audio processing result when the first processing data volume is lower than the second processing data volume, wait for the first audio processing algorithm to output a new first audio processing result, complete the cached first processing result and the new first audio processing result, until the data volume in the completed result is not lower than the second processing data volume, and then perform second audio processing on the completed result through the second audio processing algorithm to obtain the target processing result. Wherein, the first audio processing algorithm corresponds to the first amount of data to be processed, the second audio processing algorithm corresponds to the second amount of data to be processed, and the processing priority of the first audio processing algorithm is higher than that of the second audio processing algorithm.

[0013] In addition, to achieve the above objectives, this application also proposes an audio processing device, the device comprising: a memory, a processor, and an audio processing program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the audio processing method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium that is a computer-readable storage medium, on which an audio processing program is stored, which, when executed by a processor, implements the steps of the audio processing method described above.

[0015] This application proposes an audio processing method, apparatus, device, and storage medium. The method is applied to an audio processing device, which includes a first audio processing algorithm and a second audio processing algorithm. The first audio processing algorithm corresponds to a first processing data volume, and the second audio processing algorithm corresponds to a second processing data volume. The processing priority of the first audio processing algorithm is higher than that of the second audio processing algorithm. The method includes: upon receiving current audio data to be processed, performing first audio processing on the current audio data to be processed using the first audio processing algorithm to obtain a first audio processing result, wherein the data volume in the first audio processing result is the first processing data volume; when the first processing data volume is not less than the second processing data volume, performing second audio processing on the first audio processing result using the second audio processing algorithm to obtain a target processing result; when the first processing data volume is less than the second processing data volume, caching the first audio processing result, waiting for the first audio processing algorithm to output a new first audio processing result, and completing the cached first processing result and the new first audio processing result until the data volume in the completed result is not less than the second processing data volume, and then performing second audio processing on the completed result using the second audio processing algorithm to obtain the target processing result.

[0016] This application enables differentiated real-time processing paths by determining the relationship between the first and second processing data volumes in real time. Specifically, when the first processing data volume output by the first audio processing algorithm is not less than the second processing data volume, the data can immediately enter the second audio processing algorithm after processing by the first audio processing algorithm. When the first processing data volume output by the first audio processing algorithm is less than the second processing data volume, the cache can be padded after waiting for new first audio processing results until the data volume in the padded results is not less than the second processing data volume, at which point it is input into the second audio processing algorithm for processing. This avoids the step in traditional solutions where data must accumulate to a fixed multiple before processing, thereby fundamentally reducing system processing latency. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1This is a flowchart illustrating the data transmission process in the audio processing method proposed in this application. Figure 2 This is a flowchart of the first embodiment of the audio processing method proposed in this application; Figure 3 This is a flowchart of a second embodiment of the audio processing method proposed in this application; Figure 4 This is a flowchart of the third embodiment of the audio processing method proposed in this application; Figure 5 A diagram of an audio processing apparatus provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an audio processing device suitable for implementing the embodiments of this application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not intended to limit this application.

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0024] Understandably, to improve the listening experience, various audio devices currently employ audio processing algorithms (such as active noise reduction, echo cancellation, and speech separation) to process the audio data in each frame of the acquired audio. However, these algorithms are typically built upon specific mathematical models, thus having a strict "frame length dependency" requirement on the length of the input data. That is, different algorithms require a fixed but potentially inconsistent number of audio data points for each input. For example, an active noise reduction algorithm might require five consecutive audio data points for processing each time, while an echo cancellation algorithm might require three consecutive audio data points.

[0025] In practical applications, a complete audio processing workflow often requires multiple algorithms to process data sequentially. For example, a piece of raw audio data may first be processed by an active noise reduction algorithm, and its result is immediately sent to the next-level echo cancellation algorithm for further processing before finally outputting the desired high-quality audio. However, because the amount of data processed by the upstream and downstream algorithms differs each time (e.g., 5 versus 3), the amount of data produced by the upstream algorithm (such as active noise reduction) cannot directly match the amount of input data required by the downstream algorithm (such as echo cancellation). The active noise reduction algorithm produces 5 data points, while the echo cancellation algorithm requires 3 data points to start, resulting in a data transmission bottleneck.

[0026] To address this challenge, existing technologies generally employ a "data caching" mechanism. This involves setting up a temporary buffer between two levels of algorithms. The system first accumulates the audio data produced by the previous level algorithm in the buffer until the accumulated data reaches or exceeds an integer multiple of the input data required by the next level algorithm (for example, finding the least common multiple of 5 and 3, which is 15). Then, the entire block of data is retrieved from the buffer and segmented into a format that meets the requirements of the next level algorithm (e.g., 3 segments at a time) for input.

[0027] While this caching mechanism ensures data integrity, its inherent processing flow introduces significant latency. Specifically, the next-level audio processing algorithm must wait until the data accumulates in the buffer to an integer multiple of the previous level before it can begin processing, causing a pause in the processing flow, resulting in noticeable latency and a poor user experience.

[0028] Therefore, to address the technical problem of reducing audio processing latency, this embodiment proposes an audio processing method. The method includes: upon receiving current audio data to be processed, performing first audio processing on the current audio data to be processed using a first audio processing algorithm to obtain a first audio processing result, wherein the data volume within the first audio processing result is a first processing data volume; when the first processing data volume is not less than a second processing data volume, performing second audio processing on the first audio processing result using a second audio processing algorithm to obtain a target processing result; when the first processing data volume is less than the second processing data volume, caching the first audio processing result, waiting for the first audio processing algorithm to output a new first audio processing result, and completing the cached first processing result and the new first audio processing result until the data volume within the completed result is not less than the second processing data volume, and then performing second audio processing on the completed result using the second audio processing algorithm to obtain the target processing result.

[0029] This embodiment can determine the relationship between the first and second processing data volumes in real time, realizing differentiated real-time processing paths. Specifically, in this embodiment, when the first processing data volume output by the first audio processing algorithm is not less than the second processing data volume, the data can immediately enter the second audio processing algorithm after the first audio processing algorithm completes its processing. When the first processing data volume output by the first audio processing algorithm is less than the second processing data volume, the buffer can be padded after waiting for a new first audio processing result until the data volume in the padded result is not less than the second processing data volume, at which point it is input into the second audio processing algorithm for processing. This avoids the step in traditional solutions where data must accumulate to a fixed multiple before processing, thereby fundamentally reducing system processing latency.

[0030] For ease of understanding, the following is combined with Figures 1 to 6 The audio processing methods provided in the embodiments of this application, as well as the audio processing methods, apparatus, devices, and storage media provided in the following embodiments, will be described in detail.

[0031] It is understood that the executing entity of this embodiment can be a multifunctional machine device with audio processing capabilities, such as a mobile phone, a cloud server, or a device capable of performing the aforementioned functions. This embodiment uses an audio processing device (hereinafter referred to as the device) for description. The audio processing device includes a first audio processing algorithm and a second audio processing algorithm. The first audio processing algorithm corresponds to a first amount of data to be processed, and the second audio processing algorithm corresponds to a second amount of data to be processed. The processing priority of the first audio processing algorithm is higher than that of the second audio processing algorithm.

[0032] It should be noted that the aforementioned device may have its own application (APP) for receiving audio data to be processed, or it may be connected to an application (APP) that transmits the audio data to be processed. (See reference) Figure 1 , Figure 1 This is a flowchart illustrating the data transmission process in the audio processing method proposed in this application. Figure 1 As shown, this embodiment can use a server deployed in the cloud or a local device (such as a mobile phone) as the audio processing device to perform audio processing.

[0033] like Figure 1 As shown, in this embodiment, the local device can be equipped with an APP. When the APP is directly connected to an audio device (such as headphones or a microphone) via a Serial Port Profile (SPP) (for example, when extremely low latency is required to ensure real-time communication within the game), the local device is used as the audio processing device for audio processing (i.e., ...). Figure 1(Interacting with the local device), calling the algorithm function set in the local device, and processing the current audio to be processed using the algorithms in the algorithm function set, outputting the processed audio data (i.e., Figure 1 (output audio data in the file).

[0034] When an app does not use SPP to directly connect to a specific device, but instead uses the phone's own microphone or other non-SPP audio sources (such as recorded audio files), the cloud can be used as an audio processing device for audio processing (i.e., Figure 1 (Interacting with the cloud) Through the APP, users select the set of algorithm functions to process the audio to be processed, and process the audio according to the algorithms in the set, outputting the processed audio data (i.e., Figure 1 (output audio data in the file).

[0035] Furthermore, it should be noted that the aforementioned audio processing device can be a hardware entity that performs audio capture, processing, and output, such as a smartphone, smart headphones, audio processing chip, or cloud server. The aforementioned first audio processing algorithm can be an algorithm that performs preliminary processing on the raw audio, such as an active noise reduction algorithm or a speech separation algorithm. The aforementioned second audio processing algorithm can be an algorithm that relies on the output of the first audio processing algorithm for processing, such as an echo cancellation algorithm or a speech enhancement algorithm. The aforementioned first processing data volume can be the fixed input data unit length required for a single execution of the first audio processing algorithm, for example, processing 5 consecutive audio data samples each time. The aforementioned second processing data volume can be the fixed input data unit length required for a single execution of the second audio processing algorithm, for example, processing 3 consecutive audio data samples each time. The aforementioned processing priority can refer to the priority of the first audio processing algorithm over the second audio processing algorithm in the audio processing flow, that is, the output result of the first audio processing algorithm can only be used as the input of the second audio processing algorithm after the first audio processing algorithm has completed processing the current frame.

[0036] This application provides an audio processing method, referring to... Figure 2 , Figure 2 This is a flowchart of the first embodiment of the audio processing method proposed in this application.

[0037] like Figure 1 As shown, the method includes: Step S10: Upon receiving the current audio data to be processed, perform first audio processing on the current audio data to be processed using the first audio processing algorithm to obtain a first audio processing result. The amount of data in the first audio processing result is the amount of the first processed data.

[0038] It should also be noted that the aforementioned audio data to be processed can be audio data captured in real time by an audio acquisition device (such as a microphone) or read from a storage file. The aforementioned first audio processing result can be the audio data output by the first audio processing algorithm after processing the current audio data to be processed.

[0039] In its implementation, the aforementioned device acquires the current audio data stream to be processed via an audio acquisition device or reads it from a storage file. The device then segments this continuous data stream according to a first processing data volume corresponding to a first audio processing algorithm. For each data segment that reaches the first processing data volume, the device invokes its internally integrated first audio processing algorithm to perform first audio processing on that segment. After processing is complete, the device obtains an output data segment of the same length as the input data segment, which is the first audio processing result.

[0040] Step S20: When the first amount of processed data is not less than the second amount of processed data, the first audio processing result is subjected to second audio processing by the second audio processing algorithm to obtain the target processing result.

[0041] It should be noted that the aforementioned target processing result can be the final output audio data of the entire audio processing flow, that is, the audio data after continuous processing by the first and second audio processing algorithms. In specific implementation, after obtaining the first audio processing result produced by the first audio processing algorithm, the device can compare its data volume with the second processing data volume required by the second audio processing algorithm. When it is determined that the first processing data volume is not less than the second processing data volume, the device will call its internally integrated second audio processing algorithm. The device sequentially divides the first audio processing result according to the second processing data volume, forming several data blocks that meet the input requirements of the second audio processing algorithm. The device sequentially inputs these data blocks into the second audio processing algorithm for second audio processing. After processing each data block, the device sequentially combines the output data generated by each processing step to form the final target processing result.

[0042] For ease of understanding, the following example illustrates the concept, but does not impose specific limitations on this embodiment. Assume the device described above is a smart headset with call noise reduction functionality. Its first audio processing algorithm is an active noise reduction algorithm with a first processing data volume of 5; its second audio processing algorithm is an echo cancellation algorithm with a second processing data volume of 3. When the active noise reduction algorithm processes one frame of data and produces a first audio processing result containing 5 sampling points, the device determines that 5 is greater than 3, satisfying the immediate processing condition. The device then divides the result of the 5 sampling points into a data block of length 3, obtaining a first data block of 3 sampling points and the remaining 2 sampling points. The device immediately sends the first data block of 3 sampling points into the echo cancellation algorithm for processing, obtaining a portion of the target processing result and outputting it. Simultaneously, the device buffers the remaining 2 sampling points, waiting to concatenate them with the previous sampling point of the next frame's first audio processing result, until a total of 3 sampling points are obtained, before sending them into the echo cancellation algorithm for processing.

[0043] Step S30: When the first processed data volume is lower than the second processed data volume, cache the first audio processing result, wait for the first audio processing algorithm to output a new first audio processing result, and complete the cached first processing result and the new first audio processing result until the data volume in the completed result is not lower than the second processed data volume. Then, perform second audio processing on the completed result through the second audio processing algorithm to obtain the target processing result.

[0044] It should be noted that the aforementioned completion result can be audio data formed after completion operations, with the total data volume reaching or exceeding a specific requirement. In specific implementation, after obtaining the first audio processing result produced by the first audio processing algorithm, the device compares its data volume with the second processing data volume required by the second audio processing algorithm. When it is determined that the first processing data volume is lower than the second processing data volume, the device stores the first audio processing result in a designated buffer. The device then enters a waiting state until the first audio processing algorithm processes subsequent audio data and outputs a new first audio processing result. The device retrieves the previously cached first result from the buffer and concatenates it sequentially with this new result to form a completion result. The device immediately checks the total data volume within this completion result. If it is still lower than the second processing data volume, the device uses the entire completion result as new buffer content, continues to wait and obtain the next new first audio processing result, and repeats the concatenation process until the total data volume within the completion result reaches or exceeds the second processing data volume. Once the condition is met, the device immediately calls the second audio processing algorithm to perform second audio processing on the entire completion result. After the algorithm completes execution, the device obtains the target processing result for this process. Any remaining data that was not consumed during this process is stored in a cache as the starting data for the next completion operation.

[0045] For ease of understanding, the following example illustrates the concept, but does not impose specific limitations on this embodiment. Assume the device described above is an audio conferencing system. Its first audio processing algorithm is a noise suppression algorithm, requiring 3 sampling points per processing step (first processing data volume = 3); the second audio processing algorithm is a high-precision echo cancellation algorithm, requiring 5 sampling points per processing step (second processing data volume = 5). After the noise suppression algorithm processes the first frame of data and produces the first audio processing result of 3 sampling points (denoted as data block A1), the device determines that 3 < 5, and therefore caches A1. Next, the system collects and processes the next frame of data, and the noise suppression algorithm produces a new result of 3 sampling points (denoted as data block A2). The device concatenates the cached A1 with the new A2 to form the complete result A1 + A2, with a total data volume of 6. The device determines that 6 > 5, satisfying the start condition of the second audio processing algorithm. Subsequently, starting from the beginning of these 6 sampling points, the device extracts the first 5 sampling points to form a complete data block, and immediately sends it to the echo cancellation algorithm for processing to obtain a portion of the target processing result. After processing, the device retains the remaining 1 sampling point (i.e., the 6th sampling point) in the buffer. When the third noise suppression result A3 (3 sampling points) is generated, the device concatenates the buffered 1 sampling point with A3 to form 4 sampling points of data. This is still less than 5, so it continues to be buffered to wait for A4. After A4 arrives, it is concatenated, truncated, and processed again.

[0046] Furthermore, to ensure the reliability of the end-to-end audio processing flow, after the step of obtaining the target processing result, the following steps are also included: Step S40: After the current audio data to be processed has been received, determine the total number of data receptions and the total number of processing operations of the second audio processing algorithm based on the first amount of data to be processed and the second amount of data to be processed; Step S50: Perform a loss check on the target processing result based on the total number of data receptions and the total number of processing operations to obtain the check result.

[0047] It should be noted that the total number of data receptions mentioned above can refer to the total number of times the first audio processing algorithm outputs the first audio processing result during the entire process of processing the current audio data to be processed. The total number of processing iterations mentioned above can refer to the number of times the second audio processing algorithm is successfully triggered and executed during the entire process of processing the current audio data to be processed. The loss check mentioned above can refer to detecting whether there is data loss or abnormal operation during the data processing by comparing the consistency between the expected data stream and the actual data stream. The check result mentioned above can be the judgment conclusion output after the loss check process is completed, used to indicate whether the target processing result is complete and reliable.

[0048] In its implementation, after confirming that all audio data to be processed has been received, the device first determines the theoretically expected total number of frames that the first audio processing algorithm should output, i.e., the total number of data receptions for the second audio processing algorithm, based on the ratio of the total length of the entire audio data stream to the first processing data volume. Then, it calculates the total number of processing operations for the second audio processing algorithm based on the mathematical relationship (such as the least common multiple) between the total length of the entire audio data stream and the first and second processing data volumes. The device uses an internal counter to obtain the actual number of data receptions and the actual number of processing operations for the second audio processing algorithm, and compares the actual reception count with the theoretical total number of data receptions to verify whether any output frames from the first audio processing algorithm were lost during transmission. Simultaneously, the device compares the actual number of processing operations with the theoretical total number of processing operations to verify whether the second audio processing algorithm performed all the expected processing operations. Based on these two sets of comparison results, the device generates the final verification result. If both sets of comparisons are consistent, the verification result indicates that the data is complete; if either set of comparisons is inconsistent, the verification result indicates that data loss has occurred, and the approximate point of loss can be located based on the discrepancy.

[0049] For ease of understanding, the following example is used for illustration, but it does not impose specific limitations on this embodiment. Assume the device described above is a smart speaker. Its first audio processing algorithm is voice activity detection, with a first processing data volume of 3 sampling points; the second audio processing algorithm is speech recognition, with a second processing data volume of 5 sampling points. The device records a voice command with a length of 30 sampling points as the current audio data to be processed. After processing, the device begins verification. It first calculates the theoretical value: total number of data receptions = total length 30 / first processing data volume 3 = 10 times. This means the voice activity detection algorithm should output 10 frames of results. The total number of processing times is related to the connection efficiency of the data flow through the two-stage algorithm. Based on the least common multiple LCM(3,5) = 15, it can be deduced that for every 15 sampling points processed (one complete adaptation cycle), the second audio processing algorithm will be triggered 3 times. Therefore, for 30 sampling points, the theoretical total number of processing times = (30 / 15) * 3 = 6 times. The device then checks its internal status: it verifies that it did indeed receive 10 frames of data from the first audio processing algorithm (actual reception count = 10), and that the speech recognition algorithm was successfully invoked 6 times (actual processing count = 6). The two sets of data match perfectly, and the device therefore generates a verification result of "Data complete, no loss." Conversely, if the device finds that the actual processing count is only 5, while the theoretical value is 6, it indicates that approximately 5 sampling points of valid speech data were not successfully recognized. The verification result will be marked as "Data incomplete, loss occurred during processing," thus prompting the system to re-acquire data or report the error.

[0050] In this embodiment, during audio processing, a differentiated real-time processing path is achieved by determining the relationship between the first and second processing data volumes: when the first processing data volume is not less than the second processing data volume, the data can immediately enter the second audio processing algorithm after the first audio processing algorithm is completed. This avoids the step in traditional solutions where data must accumulate to a fixed multiple before processing, thereby fundamentally reducing system processing latency. When the first processing data volume is low, the method only requires caching the current result and performing minimal supplementation with the new result of the next frame, rather than waiting to cache a large amount of data, reducing the occupation of temporary cache memory and lowering hardware resource overhead.

[0051] Based on the first embodiment, in the second embodiment, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 , Figure 3 This is a flowchart of a second embodiment of the audio processing method proposed in this application. Further, to ensure that the current audio data to be processed can be processed completely and promptly, the step of performing first audio processing on the current audio data to be processed using the first audio processing algorithm to obtain a first audio processing result includes: Step S11: Divide the audio data in the current audio data to be processed into segments in sequence according to the first processing data volume; Step S12: The obtained first segmentation results are sequentially input into the first audio processing algorithm to perform first audio processing and obtain the first audio processing result.

[0052] It should be noted that the first segmentation result mentioned above can be an independent data block that meets the first processing data volume requirement after being segmented in sequence, such as a data segment consisting of 5 consecutive sampling points.

[0053] In its implementation, after receiving a continuous stream of audio data to be processed, the device first determines the length of each data block based on a preset first processing data volume. The device creates a data buffer to temporarily store the input audio data stream and sets a data pointer to identify the current processing position. When the accumulated audio data in the buffer reaches the first processing data volume, the device extracts data blocks starting from the beginning of the buffer, according to the size of the first processing data volume. Each extraction operation acquires a continuous first processing data volume of audio data units, forming an independent first segmentation result. The device immediately transmits this first segmentation result to the input interface of the first audio processing algorithm, triggering the algorithm to execute the corresponding first audio processing operation. During the operation of the first audio processing algorithm, the device continues to monitor the reception status of the input data stream and performs the next data block segmentation operation when the data volume in the buffer reaches the first processing data volume. After the algorithm completes processing, the device obtains the first audio processing result from the algorithm output interface; the data volume of this result is consistent with the first processing data volume. Simultaneously, the device updates the buffer state, removes processed data, and moves subsequent unprocessed data forward to reserve space for receiving new audio data. This process is repeated until all the currently input audio data to be processed has been segmented and processed.

[0054] For ease of understanding, the following example illustrates the concept, but does not impose specific limitations on this embodiment. Assume the device described above is a smart headset, and its first audio processing algorithm is an active noise reduction algorithm, with a first processing data volume set to 5 sampling points. When the headset microphone acquires a segment of audio data containing 30 sampling points, the device initiates the processing flow. The aforementioned audio data first flows into a data buffer created by the device. When the data volume in the buffer reaches the first processing data volume of 5, it is segmented. The first segmentation obtains sampling points 1 to 5, forming the first segmentation result, which is then sent to the active noise reduction algorithm for first audio processing. The algorithm outputs 5 noise-reduced sampling points, i.e., the first first audio processing result. Simultaneously, the device immediately performs a second segmentation, obtaining sampling points 6 to 10, forming the second first segmentation result, which is then sent to the algorithm to obtain the second first audio processing result. This process continues, segmenting 5 consecutive sampling points each time, processing sampling points 11 to 15, 16 to 20, 21 to 25, and 26 to 30 sequentially, generating a total of six first segmentation results, corresponding to six first audio processing results.

[0055] Furthermore, in order to ensure the synchronization of the processing strategy and the audio content and prevent algorithm state misalignment, the first audio processing algorithm includes at least two different first processing modes. The step of sequentially inputting the obtained first segmentation results into the first audio processing algorithm for first audio processing to obtain the first audio processing result includes: Step S121: Obtain the first mode flag bit corresponding to each audio data in the current audio data to be processed, and determine the target first processing mode corresponding to each audio data in the current audio data to be processed based on the first mode flag bit.

[0056] Step S122: The obtained first segmentation result is sequentially input into the first audio processing algorithm, and the first audio processing algorithm performs first audio processing on the first segmentation result according to the target first processing mode to obtain the first audio processing result.

[0057] It should be noted that the first processing mode mentioned above can be a different processing mode of the first audio processing algorithm on the audio. Since different audio can be set with different processing degrees for the same operation, this embodiment can set different processing modes for each audio processing algorithm, and different processing modes can correspond to different processing degrees.

[0058] It is understood that the aforementioned audio data can be unit data constituting the current audio data to be processed, i.e., a single byte of audio data. The aforementioned first mode flag can be a metadata tag embedded in the audio data, such as a 2-bit mode identifier contained in the header of each audio frame. The aforementioned target first processing mode can be a processing mode for processing the current audio data based on the first audio processing algorithm parsed from the flag, such as the conference mode noise reduction corresponding to the identifier "01".

[0059] In its implementation, when initiating the audio processing flow, the device first initializes multiple first processing modes of the first audio processing algorithm. Each first processing mode corresponds to a different set of algorithm parameters and processing logic. While receiving the audio data to be processed, the device simultaneously parses the first mode flag bit attached to each audio data unit. The device segments the input audio stream according to the first processing data volume, forming continuous first segmentation results. Before sending each first segmentation result into the first audio processing algorithm, the device reads the first mode flag bit corresponding to that result. Based on a preset mapping relationship, the flag bit value is converted into a specific target first processing mode number. Subsequently, the device inputs the first segmentation result into the algorithm with the completed mode configuration to execute the first audio processing. The algorithm performs calculations on the input data according to the currently activated target first processing mode, outputting a first audio processing result that conforms to the characteristics of that mode. The device records the correspondence between each output result and the processing mode and sends the result to the next level of processing. When processing continuous audio streams, the device dynamically monitors changes in the first mode flag. When the target first processing mode of two adjacent first segmentation results is different, the device inserts an algorithm reconfiguration stage between two processing cycles to ensure that each data block can be correctly processed according to its corresponding target first processing mode.

[0060] For ease of understanding, the following example illustrates the concept, but does not impose specific limitations on this embodiment. Assume the device described above is a high-end noise-canceling headphone, and its first audio processing algorithm is a multi-mode adaptive noise reduction algorithm, containing three first processing modes: Mode A (deep noise reduction, suitable for high-noise environments such as airplanes and subways), Mode B (normal noise reduction, suitable for medium-noise environments such as streets and offices), and Mode C (transparency mode, allowing some ambient sound to enter). The first processing data volume is 256 sampling points. When the device starts processing audio, the microphone array collects ambient sound and embeds a 2-bit first mode flag (00 represents Mode A, 01 represents Mode B, and 10 represents Mode C) at the beginning of each 256-sampling-point audio frame. After receiving the current audio data to be processed, which contains multiple audio frames, the device segments it according to 256 sampling points to obtain multiple first segmentation results. When processing the first first segmentation result, the device parses its flag bit as "00", thereby determining the target first processing mode as Mode A. The device is configured with a deep noise reduction algorithm, processes the data block, and outputs the first audio processing result after strong noise reduction. Immediately upon processing the second first segmentation result, the device detects that its flag has changed to "01", and thus reconfigures the algorithm to normal noise reduction parameters (Mode B), outputting a result with moderate noise reduction. When the user enters a scene requiring environmental awareness, the flag of subsequent data blocks changes to "10", and the device immediately switches the algorithm to transparency mode parameters (Mode C), outputting audio that preserves ambient sound.

[0061] Based on the first and second embodiments, in the third embodiment, the content that is the same as or similar to that in Embodiments 1 and 2 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 , Figure 4 This is a flowchart of a third embodiment of the audio processing method proposed in this application. Further, it ensures that the currently processed audio data can be processed completely and promptly. The step of performing second audio processing on the first audio processing result using the second audio processing algorithm to obtain the target processing result when the first processing data volume is not less than the second processing data volume includes: Step S21: When the first processed data volume is not less than the second processed data volume, the audio data in the first audio processing result is segmented in sequence according to the second processed data volume; Step S22: Input the obtained second segmentation results into the second audio processing algorithm in sequence to perform second audio processing and obtain the target processing result.

[0062] It should be noted that the aforementioned sequential segmentation refers to dividing a larger data block into multiple smaller data blocks according to the chronological order of the data. The aforementioned second segmentation result can be a data block that meets the input requirements of the second audio processing algorithm, obtained by segmenting the first audio processing result according to the second processing data volume.

[0063] In its implementation, after obtaining the first audio processing result output by the first audio processing algorithm, the device first compares the size of the first processed data volume with that of the second processed data volume. When it is confirmed that the first processed data volume is not less than the second processed data volume, the device initiates a data forwarding process. The device sequentially segments the audio data contained in the first audio processing result according to the specifications of the second processed data volume. The segmentation process starts from the beginning of the data and extracts consecutive data units in sequence, using the second processed data volume as the step size, forming several independent second segmentation results. Each second segmentation result contains a complete data unit that exactly meets the requirements of the second processed data volume. The device establishes a processing queue and arranges the obtained second segmentation results in the segmentation order. Subsequently, the device sequentially retrieves the second segmentation results from the queue and inputs them into the data interface of the second audio processing algorithm. For each input data block, the second audio processing algorithm performs the corresponding second audio processing operation. After the algorithm completes the processing, it outputs the corresponding processing result segment. The device collects all output segments and combines them into a complete target processing result according to the processing order.

[0064] Furthermore, the second audio processing algorithm includes at least two different second processing modes; The step of sequentially inputting the obtained second segmentation results into the second audio processing algorithm for second audio processing to obtain the target processing result includes: Step S221: Obtain the second mode flag bit corresponding to each audio data in the current audio data to be processed, and determine the target second processing mode corresponding to each audio data in the current audio data to be processed based on the second mode flag bit; Step S222: The obtained second segmentation result is sequentially input into the second audio processing algorithm, and the second audio processing algorithm performs second audio processing on the second segmentation result according to the target second processing mode to obtain the target processing result.

[0065] It should be noted that the first processing mode mentioned above can be a different processing mode of the first audio processing algorithm for the audio. Since different audio can be set with different processing levels for the same operation, this embodiment can set different processing modes for each audio processing algorithm. Different processing modes can correspond to different processing levels, such as echo suppression intensity levels used for different acoustic environments.

[0066] The aforementioned second mode flag can be a metadata tag embedded in the audio data, such as a 2-bit mode identifier contained in the header of each audio frame. The aforementioned target second processing mode can be the processing mode of the second audio processing algorithm for processing the current audio data, such as the conference mode echo cancellation corresponding to the identifier "01".

[0067] In its implementation, after completing the first audio processing and preparing the second segmentation result, the device initiates the mode configuration process for the second audio processing algorithm. The device first initializes multiple second processing modes of the second audio processing algorithm. While receiving the current audio data to be processed, the device simultaneously parses the second mode flag bit attached to each audio data unit. The device segments the first audio processing result according to the second processing data volume, forming continuous second segmentation results. Before sending each second segmentation result into the second audio processing algorithm, the device reads the second mode flag bit corresponding to that result. Based on a preset mapping relationship, the flag bit value is converted into a specific target second processing mode number. The second audio processing algorithm is then configured based on the target second processing mode number. Subsequently, the device inputs the second segmentation result into the algorithm with the completed mode configuration to execute the second audio processing. The algorithm operates on the input data according to the currently activated target second processing mode, outputting a target processing result that conforms to the characteristics of that mode. The device records the correspondence between each output result and the processing mode, and sends the result to the output process or the next processing stage. When processing continuous audio streams, the device dynamically monitors changes in the second mode flag. If the target second processing mode differs between two adjacent second segmentation results, the device inserts an algorithm reconfiguration phase between two processing cycles to ensure that each data block is correctly processed according to its corresponding target second processing mode. Furthermore, in another embodiment, the device can simultaneously establish a mode switching buffer mechanism to prevent audio quality degradation due to abrupt mode changes at processing boundaries.

[0068] For ease of understanding, the following example illustrates the concept, but does not impose specific limitations on this embodiment. Assume the device described above is an intelligent conference system, and its second audio processing algorithm is a multi-mode echo cancellation algorithm, containing three second processing modes: Mode X (strong cancellation mode, suitable for large conference rooms), Mode Y (standard cancellation mode, suitable for small conference rooms), and Mode Z (adaptive mode, dynamically adjusted according to the environment). The second processing data volume is 512 sampling points. When the device processes the first audio processing result from the first audio processing algorithm, it first segments the data according to the 512 sampling point specification, obtaining multiple second segmentation results. Each second segmentation result carries a corresponding second mode flag bit, which is determined during the audio acquisition stage and transmitted here along with the data processing flow. When processing the first second segmentation result, the device parses its flag bit as "10", thereby determining the target second processing mode as Mode X. The device configures the echo cancellation algorithm with strong cancellation parameters, then processes the data block and outputs the target processing result after deep echo cancellation. When processing the second second-segment result, the device detects that its flag bit has changed to "01", and therefore reconfigures the algorithm to standard cancellation parameters (mode Y), outputting a result with moderate echo cancellation. When the system detects a change in the ambient acoustic characteristics, the flag bit of the subsequent data block changes to "11", and the device immediately switches the algorithm to adaptive mode parameters (mode Z), outputting audio that is adjusted in real time according to the environment. During processing, the device pays special attention to the smooth transition during mode switching. When switching from mode X to mode Y, the device inserts a transition processing between the two processing frames to avoid audio distortion caused by abrupt changes in echo cancellation intensity.

[0069] Further, the step of sequentially segmenting each audio data in the first audio processing result according to the second processing data volume when the first processed data volume is not less than the second processed data volume includes: Step S211: When the first amount of processed data is not less than the second amount of processed data, determine the current number of processing steps and the current processing remainder based on the first amount of processed data and the second amount of processed data; Step S212: Divide the audio data in the first audio processing result into segments in sequence according to the second processing data volume and the current processing count.

[0070] It should be noted that the aforementioned current processing count can be the total number of processing iterations calculated based on the numerical relationship between the first and second processing data volumes, assuming the first processing data volume is not less than the second processing data volume. The aforementioned current processing remainder can be the number of data units remaining after the first audio processing result of the current frame has undergone a total of processing iterations.

[0071] In its implementation, after obtaining the first audio processing result, if the device determines that the first processed data volume is not less than the second processed data volume, it divides the first processed data volume by the second processed data volume, using the quotient as the current processing count and the remainder as the current processing remainder. Based on the current processing count, the device sequentially segments the audio data in the first audio processing result according to the second processed data volume. The segmentation process starts from the beginning of the data and continuously extracts data blocks for the current processing count, with each data block containing an audio data volume strictly equal to the second processed data volume. The device assigns a sequence number to each segmented data block and establishes a processing queue. Simultaneously, the device separately marks and caches the data unit corresponding to the current processing remainder.

[0072] Accordingly, after the step of sequentially inputting the obtained second segmentation results into the second audio processing algorithm for second audio processing to obtain the target processing result, the method further includes: Step S23: Cache the audio data corresponding to the current processing remainder in the first audio processing result, and when a new first audio processing result is obtained, complete the new first audio processing result with the audio data corresponding to the current processing remainder until the amount of data in the completed result is not less than the amount of the second processing data. Then, perform second audio processing on the completed result through the second audio processing algorithm to obtain a new target processing result.

[0073] It should be noted that the aforementioned new first audio processing result can be a new output data block generated by the first audio processing algorithm processing subsequent input audio data. The aforementioned completion can be the process of concatenating multiple data segments to form a complete data block. The aforementioned completion result can be a new data set that meets the processing requirements after data concatenation. The aforementioned data volume can be the number of basic data units contained in the data set. The aforementioned new target processing result can be the final output data obtained after data completion processing.

[0074] In its implementation, after completing the segmentation of the current frame data, the device stores the audio data corresponding to the current processing remainder in the first audio processing result into a designated buffer area. The device maintains the buffer state, waiting for the first audio processing algorithm to output a new first audio processing result. When a new first audio processing result is obtained, the device reads the previously saved remainder data from the buffer and concatenates it sequentially with the beginning part of the new first audio processing result to form a complete result. The device immediately checks the amount of data in the complete result; if it is still lower than the second processing data amount, it continues to wait for the next new first audio processing result and concatenates it again until the amount of data in the complete result reaches or exceeds the second processing data amount. When the condition is met, the device calls the second audio processing algorithm to perform second audio processing on the complete result to obtain a new target processing result. After processing, the device updates the buffer content, saving the remaining data that was not consumed in this processing as new remainder data.

[0075] This embodiment also provides a first embodiment of an audio processing device, please refer to... Figure 5 , Figure 5 This is a diagram of an audio processing apparatus provided in an embodiment of this application. The audio processing apparatus includes: The data receiving module 401 is used to perform first audio processing on the current audio data to be processed by a first audio processing algorithm when receiving the current audio data to be processed, and obtain a first audio processing result, wherein the amount of data in the first audio processing result is the first processing data amount; The first processing module 402 is used to perform second audio processing on the first audio processing result by a second audio processing algorithm to obtain the target processing result when the first processing data volume is not less than the second processing data volume. The second processing module 403 is used to cache the first audio processing result when the first processing data volume is lower than the second processing data volume, wait for the first audio processing algorithm to output a new first audio processing result, complete the cached first processing result and the new first audio processing result, until the data volume in the completed result is not lower than the second processing data volume, and then perform second audio processing on the completed result through the second audio processing algorithm to obtain the target processing result. Wherein, the first audio processing algorithm corresponds to the first amount of data to be processed, the second audio processing algorithm corresponds to the second amount of data to be processed, and the processing priority of the first audio processing algorithm is higher than that of the second audio processing algorithm. The data receiving module 401 is further configured to, after the current audio data to be processed has been received, determine the total number of data receptions and the total number of processing operations of the second audio processing algorithm based on the first processing data volume and the second processing data volume; and perform loss verification on the target processing result based on the total number of data receptions and the total number of processing operations to obtain a verification result.

[0076] Referring to the first embodiment of the audio processing device, this embodiment also proposes a second embodiment of the audio processing device. The contents that are the same as or similar to those in the first embodiment of the audio processing device can be referred to the above description, and will not be repeated hereafter.

[0077] The first processing module 402 is further configured to sequentially segment each audio data in the current audio data to be processed according to the first processing data volume, and input the obtained first segmentation result into the first audio processing algorithm to perform first audio processing to obtain the first audio processing result; The first processing module 402 is further configured to obtain the first mode flag bit corresponding to each audio data in the current audio data to be processed, and determine the target first processing mode corresponding to each audio data in the current audio data to be processed based on the first mode flag bit; sequentially input the obtained first segmentation result into the first audio processing algorithm, and perform first audio processing on the first segmentation result according to the target first processing mode through the first audio processing algorithm to obtain the first audio processing result.

[0078] Referring to the first embodiment and the second embodiment of the audio processing device, this embodiment also proposes a third embodiment of the audio processing device. The contents that are the same as or similar to those of the first embodiment and the second embodiment of the audio processing device can be referred to the above description, and will not be repeated hereafter.

[0079] The first processing module 402 is further configured to obtain the second mode flag bit corresponding to each audio data in the current audio data to be processed, and determine the target second processing mode corresponding to each audio data in the current audio data to be processed based on the second mode flag bit; sequentially input the obtained second segmentation result into the second audio processing algorithm, and perform second audio processing on the second segmentation result according to the target second processing mode through the second audio processing algorithm to obtain the target processing result; The first processing module 402 is further configured to obtain the second mode flag bit corresponding to each audio data in the current audio data to be processed, and determine the target second processing mode corresponding to each audio data in the current audio data to be processed based on the second mode flag bit; sequentially input the obtained second segmentation result into the second audio processing algorithm, and perform second audio processing on the second segmentation result according to the target second processing mode through the second audio processing algorithm to obtain the target processing result.

[0080] The first processing module 402 is further configured to determine the current number of processing steps and the current processing remainder based on the first processing data volume and the second processing data volume when the first processing data volume is not less than the second processing data volume; and to sequentially segment each audio data in the first audio processing result according to the second processing data volume and the current processing steps.

[0081] The first processing module 402 is further configured to cache the audio data corresponding to the current processing remainder in the first audio processing result, and when a new first audio processing result is obtained, to complete the new first audio processing result and the audio data corresponding to the current processing remainder until the amount of data in the completed result is not less than the amount of the second processing data, and then to perform second audio processing on the completed result through the second audio processing algorithm to obtain a new target processing result.

[0082] The audio processing apparatus provided in this embodiment, employing the audio processing method described in the above embodiments, can solve the technical problem of how to reduce audio processing latency. Compared with the prior art, the beneficial effects of the audio processing apparatus provided in this embodiment are the same as those of the audio processing method described in the above embodiments, and other technical features in the audio processing apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0083] This embodiment provides an audio processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the audio processing method in the first embodiment described above.

[0084] The following is for reference. Figure 6 , Figure 6This is a schematic diagram of the structure of an audio processing device suitable for implementing the embodiments of this application. The audio processing device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The audio processing device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0085] like Figure 6 As shown, the audio processing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the audio processing device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the audio processing device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows audio processing devices with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0086] Specifically, according to this embodiment, the process described above with reference to the flowchart can be implemented as a computer software program. For example, this embodiment includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the disclosed embodiments of this embodiment.

[0087] The audio processing device provided in this embodiment, employing the audio processing method described in the above embodiments, can solve the technical problem of how to reduce audio processing latency. Compared with the prior art, the beneficial effects of the audio processing device provided in this embodiment are the same as those of the audio processing method provided in the above embodiments, and other technical features of this audio processing device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0088] It should be understood that the various parts disclosed in this embodiment can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0089] The above description is merely a specific implementation of this embodiment, but the protection scope of this embodiment is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this embodiment should be included within the protection scope of this embodiment. Therefore, the protection scope of this embodiment should be determined by the protection scope of the claims.

[0090] This embodiment provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the audio processing method in the above embodiment.

[0091] The computer-readable storage medium provided in this embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0092] The aforementioned computer-readable storage medium may be included in an audio processing device or may exist independently without being assembled into an audio processing device.

[0093] The aforementioned computer-readable storage medium carries one or more programs that, when executed by an audio processing device, cause the audio processing device to perform audio processing.

[0094] Computer program code for performing the operations of this embodiment can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this embodiment. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0096] The modules described in this embodiment can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0097] The readable storage medium provided in this embodiment is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described audio processing method, thereby solving the technical problem of how to reduce audio processing latency. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this embodiment are the same as those of the audio processing method provided in the above embodiments, and will not be repeated here.

[0098] The above descriptions are only some embodiments and do not limit the patent scope of this embodiment. All equivalent structural transformations made based on the technical concept of this application and the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included within the patent protection scope of this application.

Claims

1. An audio processing method, characterized in that, The method is applied to an audio processing device, which includes a first audio processing algorithm and a second audio processing algorithm. The first audio processing algorithm corresponds to a first amount of data to be processed, and the second audio processing algorithm corresponds to a second amount of data to be processed. The processing priority of the first audio processing algorithm is higher than that of the second audio processing algorithm. The method includes: Upon receiving the current audio data to be processed, the current audio data to be processed is subjected to first audio processing through the first audio processing algorithm to obtain a first audio processing result, wherein the amount of data in the first audio processing result is the amount of the first processed data. When the first amount of processed data is not less than the second amount of processed data, the first audio processing result is subjected to second audio processing by the second audio processing algorithm to obtain the target processing result. When the first processing data volume is lower than the second processing data volume, the first audio processing result is cached, and the first audio processing algorithm outputs a new first audio processing result. The cached first processing result and the new first audio processing result are then filled in until the data volume in the filled result is not lower than the second processing data volume. Then, the second audio processing algorithm is used to perform second audio processing on the filled result to obtain the target processing result.

2. The method as described in claim 1, characterized in that, The step of performing first audio processing on the current audio data to be processed using the first audio processing algorithm to obtain a first audio processing result includes: The audio data in the current audio data to be processed is sequentially segmented according to the first processing data volume; The obtained first segmentation results are sequentially input into the first audio processing algorithm for first audio processing to obtain the first audio processing result.

3. The method as described in claim 2, characterized in that, The first audio processing algorithm includes at least two different first processing modes; The step of sequentially inputting the obtained first segmentation results into the first audio processing algorithm for first audio processing to obtain the first audio processing result includes: Obtain the first mode flag bit corresponding to each audio data in the current audio data to be processed, and determine the target first processing mode corresponding to each audio data in the current audio data to be processed based on the first mode flag bit; The obtained first segmentation result is sequentially input into the first audio processing algorithm, and the first audio processing algorithm performs first audio processing on the first segmentation result according to the target first processing mode to obtain the first audio processing result.

4. The method as described in claim 1, characterized in that, The step of performing second audio processing on the first audio processing result using the second audio processing algorithm to obtain the target processing result when the first processed data volume is not less than the second processed data volume includes: When the first processed data volume is not less than the second processed data volume, the audio data in the first audio processing result is sequentially segmented according to the second processed data volume. The obtained second segmentation results are sequentially input into the second audio processing algorithm for second audio processing to obtain the target processing result.

5. The method as described in claim 4, characterized in that, The second audio processing algorithm includes at least two different second processing modes; The step of sequentially inputting the obtained second segmentation results into the second audio processing algorithm for second audio processing to obtain the target processing result includes: Obtain the second mode flag bit corresponding to each audio data in the current audio data to be processed, and determine the target second processing mode corresponding to each audio data in the current audio data to be processed based on the second mode flag bit; The obtained second segmentation results are sequentially input into the second audio processing algorithm, and the second audio processing algorithm performs second audio processing on the second segmentation results according to the target second processing mode to obtain the target processing result.

6. The method as described in claim 4, characterized in that, The step of sequentially segmenting each audio data in the first audio processing result according to the second processing data volume when the first processed data volume is not less than the second processed data volume includes: When the first amount of processed data is not less than the second amount of processed data, the current number of processing steps and the current processing remainder are determined based on the first amount of processed data and the second amount of processed data. The audio data in the first audio processing result is sequentially segmented according to the second processing data volume and the current processing count. After the step of sequentially inputting the obtained second segmentation results into the second audio processing algorithm for second audio processing to obtain the target processing result, the method further includes: The audio data corresponding to the current processing remainder in the first audio processing result is cached. When a new first audio processing result is obtained, the new first audio processing result and the audio data corresponding to the current processing remainder are padded together until the amount of data in the padded result is not less than the amount of data processed by the second processing. Then, the padded result is subjected to second audio processing by the second audio processing algorithm to obtain a new target processing result.

7. The method as described in claim 1, characterized in that, After the step of obtaining the target processing result, the method further includes: After the current audio data to be processed has been received, the total number of data receptions and the total number of processing operations of the second audio processing algorithm are determined based on the first amount of data to be processed and the second amount of data to be processed. The target processing result is checked for loss based on the total number of data receptions and the total number of processing operations to obtain the check result.

8. An audio processing apparatus, characterized in that, The audio processing device includes: The data receiving module is used to perform first audio processing on the current audio data to be processed by a first audio processing algorithm when receiving the current audio data to be processed, and obtain a first audio processing result, wherein the amount of data in the first audio processing result is the first processed data amount; The first processing module is used to perform second audio processing on the first audio processing result using a second audio processing algorithm when the first processing data volume is not less than the second processing data volume, so as to obtain the target processing result. The second processing module is used to cache the first audio processing result when the first processing data volume is lower than the second processing data volume, wait for the first audio processing algorithm to output a new first audio processing result, complete the cached first processing result and the new first audio processing result, until the data volume in the completed result is not lower than the second processing data volume, and then perform second audio processing on the completed result through the second audio processing algorithm to obtain the target processing result. Wherein, the first audio processing algorithm corresponds to the first amount of data to be processed, the second audio processing algorithm corresponds to the second amount of data to be processed, and the processing priority of the first audio processing algorithm is higher than that of the second audio processing algorithm.

9. An audio processing device, characterized in that, The audio processing device includes: a memory, a processor, and an audio processing program stored in the memory and executable on the processor, wherein the audio processing program, when executed by the processor, implements the steps of the audio processing method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores an audio processing program, which, when executed by a processor, implements the steps of the audio processing method as described in any one of claims 1 to 7.

Images