Audio data processing method and device, electronic equipment and storage medium

Abstract

The invention discloses an audio data processing method and device, electronic equipment and a storage medium, and belongs to the technical field of data processing. The method comprises the following steps: determining the size of a data block based on an obtained audio parameter; according to the size of the data block, pre-distributing a memory space with a target size from a memory as a buffer area; writing the first audio data into the buffer area according to the size of the data block, or reading the second audio data from the buffer area according to the size of the data block; wherein the target size is an integral multiple of the size of the data block, and the buffer area is an annular buffer area.

Description

Technical Field

[0001] This application belongs to the technical field of data processing, and particularly relates to an audio data processing method, apparatus, electronic device, and storage medium Background Art

[0002] In the field of digital audio processing, the processing of real-time audio streams has always been a key challenge. Traditional audio processing systems have problems with low processing efficiency when processing audio data in fields such as gaming, live streaming, and professional audio production Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide an audio data processing method, apparatus, electronic device, storage medium, and program product, which can improve the efficiency of audio data processing

[0004] In a first aspect, the embodiments of this application provide an audio data processing method, which includes:

[0005] Determine the data block size based on the obtained audio parameters

[0006] Pre-allocate a memory space of a target size from the memory as a buffer according to the data block size

[0007] Write first audio data to the buffer according to the data block size, or read second audio data from the buffer according to the data block size

[0008] Wherein, the target size is an integer multiple of the data block size, and the buffer is a circular buffer

[0009] In a second aspect, the embodiments of this application provide an audio data processing apparatus, including:

[0010] A size determination module, configured to determine the data block size based on the obtained audio parameters

[0011] A memory allocation module, configured to pre-allocate a memory space of a target size from the memory as a buffer according to the data block size

[0012] A read / write module, configured to write first audio data to the buffer according to the data block size, or read second audio data from the buffer according to the data block size

[0013] Wherein, the target size is an integer multiple of the data block size, and the buffer is a circular buffer

[0014] In a third aspect, the embodiments of this application provide an electronic device, which includes a processor and a memory. The memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented

[0015] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instructions are stored, and when the program or instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run a program or instructions to implement the method described in the first aspect.

[0017] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0018] In the embodiment of the present application, based on the acquired audio parameters, the data block size is determined; according to the data block size, a memory space with a target size is pre-allocated from the memory as a buffer; the first audio data is written to the buffer according to the data block size, or the second audio data is read from the buffer according to the data block size; wherein, the target size is an integer multiple of the data block size, and the buffer is a circular buffer. In the embodiment of the present application, the data block size matching the size of the audio data to be processed is determined according to the audio parameters, and a memory space with a target size is pre-allocated according to the data block size as a buffer, so that the buffer size matches the data volume of the audio data to be processed, thereby avoiding the time overhead caused by subsequent dynamic memory allocation, reducing the processing delay, improving the real-time performance of data processing, and moreover, the buffer adopts a circular buffer, so that when reading and writing, there is no need to switch the buffer, further reducing the time overhead in the processing process. In addition, writing to or reading from the buffer according to the data block size can reduce the space waste caused by data fragmentation, improve the space utilization rate of the buffer, and thus improve the efficiency of audio data processing. Description of the Drawings

[0019] Figure 1 is a flowchart of an audio data processing method provided by some embodiments of the present application;

[0020] Figure 2 is a schematic diagram of an audio data processing device provided by some embodiments of the present application;

[0021] Figure 3 is a schematic diagram of an electronic device provided by some embodiments of the present application;

[0022] Figure 4 is a schematic structural diagram of an electronic device provided by some embodiments of the present application. Detailed Embodiments

[0023] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0024] The terms "first", "second", etc. in the description and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually of the same category, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and / or" in the description and claims means at least one of the connected objects, and the character " / " generally means that the related objects before and after are in an "or" relationship.

[0025] Next, in conjunction with the accompanying drawings, the audio data processing method, device, equipment, storage medium, and program product provided by the embodiments of the present application will be described in detail through specific embodiments and their application scenarios.

[0026] The audio data processing method provided by the present application can be applied to an image browsing scenario. Next, in conjunction with Figure 1 the audio data processing method provided by the embodiments of the present application will be described in detail. It should be noted that for the audio data processing method provided by the embodiments of the present application, the execution subject can be an audio data processing device. In the embodiments of the present application, taking the audio data processing device as the execution subject of the audio data processing method as an example, the audio data processing method provided by the embodiments of the present application will be described.

[0027] See Figure 1 , which is a schematic flowchart of the audio data processing method provided by some embodiments of the present application. As Figure 1 shown, the method includes steps 110-step 140, which will be specifically described below.

[0028] Step 110. Determine the data block size based on the acquired audio parameters.

[0029] Here, the audio parameters refer to the parameters related to the audio data to be processed in the audio processing algorithm used when performing audio data processing. Among them, the audio processing algorithm is an algorithm used for audio data processing, and the audio processing algorithm used in audio data processing can be preset according to the actual situation, and the audio parameters can be directly obtained from the set audio processing algorithm.

[0030] A data block is the basic processing unit of a buffer, and the data block size refers to the size of the data block in the buffer.

[0031] In some embodiments of the present application, based on the acquired audio parameters, a data block size matching the data volume of the audio data to be processed in audio data processing is determined. Based on this, when performing audio data processing, the data volume of the audio data to be processed can be determined based on the acquired audio parameters, and the data block size can be determined based on the data volume of the audio data. In this way, the data block size can be made to match the data volume of the audio data that needs to be processed.

[0032] In some embodiments of the present application, in order to reduce the overhead of data reorganization, the data block size is greater than or equal to the data volume of the audio data to be processed. In this way, one data block can store all the audio data to be processed in audio data processing. Subsequently, when performing audio data processing based on an audio processing algorithm, only one data block needs to be called from the buffer to obtain all the audio data to be processed, without the need for data reorganization, thereby reducing the data reorganization overhead in audio processing.

[0033] In some embodiments of the present application, the audio parameters include the sample rate, bit depth, channel count, and processing duration. The sample rate refers to the number of samples extracted from a continuous audio signal per second and assembled into a discrete signal, measured in Hertz (Hz). The bit depth indicates how many bits are used to represent the amplitude information of the audio at each sampling point. Common bit depths are 16 bits, 24 bits, etc. The channel count refers to the number of channels included in the audio signal, such as mono (1 channel), stereo (2 channels), etc. Based on this, the data volume of the audio data to be processed can be calculated according to the following formula:

[0034] Data volume of audio data to be processed = (Sample rate × Bit depth × Channel count × Processing duration) / 8

[0035] Since 1 byte is equal to 8 bits, dividing by 8 in the above formula can obtain the data volume of the audio data to be processed in bytes. Exemplarily, assuming the sample rate is 44100 Hz, the channel count is 2, the processing duration is 0.1 second, and the bit depth is 16 bits, then based on the above formula, the data volume of the audio data to be processed can be calculated as 17640 bytes.

[0036] Step 120. Pre-allocate a memory space with a target size from the memory as a buffer according to the data block size.

[0037] In some embodiments of the present application, the target size is an integer multiple of the data block size. The specific multiple can be preset by the user according to the actual situation, and this embodiment does not make specific limitations. Based on this, after the data block size is determined, the target size will also be determined, and when the audio parameters remain unchanged, the target size will not change. Therefore, during the process of audio data processing, the target size can be regarded as a fixed value. Thus, through step 120, a memory space with a fixed size can be pre-allocated from the memory as a buffer, that is, this buffer is a static buffer with a fixed size.

[0038] In this way, by pre-allocating a memory space with a fixed size as a buffer, the time overhead caused by subsequent dynamic allocation can be avoided.

[0039] Step 130. Write the first audio data into the buffer according to the data block size, or read the second audio data from the buffer according to the data block size.

[0040] In some embodiments of the present application, the buffer includes a circular buffer with a write pointer and a read pointer. The write pointer is a marker used to indicate the position where data is written in the buffer, and the read pointer is a marker used to indicate the position where data is read from the buffer. When the buffer is initialized, both pointers are set to point to the starting position of the buffer, that is, the head of the buffer. The audio data processing device writes to or reads from the buffer based on the write pointer and the read pointer. In this way, the buffer can be directly operated without the need to create additional temporary storage locations, reducing intermediate links. In the audio data processing device, new audio data can be quickly written into the buffer based on the write pointer, avoiding performance losses caused by complex function calls and data conversions. When accessing the buffer using the read pointer, there is no need to copy the data to other temporary storage locations either. For example, when a certain data element in the buffer needs to be modified, it can be directly modified in place in the buffer through the pointer, without first copying the data out, modifying it, and then copying it back, thereby reducing data copying and improving efficiency.

[0041] In some embodiments of the present application, the buffer provides a block-aligned write interface. Among them, block alignment refers to organizing and operating data according to the data block size. The write interface is a set of specifications and methods for writing data to the buffer, which allows an external data source to write data into the buffer. The write interface stipulates how to safely and efficiently place external data into the buffer, including many details such as the starting position of the data, the writing order, the data format, and the handling method in special cases such as when the buffer is full. Based on this, when an external data source needs to write first audio data to the buffer, the write interface is called. In response to the call of the write interface of the buffer, the audio data processing device aligns the write size to the data block size based on the write interface, that is, splits the first audio data into at least one data block according to the data block size, where the size of the last data block is the actual data block size. Then, based on the position pointed to by the write pointer, each split data block is sequentially written into the buffer. In this way, it can be ensured that the starting position and length of the audio data written into the buffer are integer multiples of the set data block size, reducing the space waste caused by data fragmentation and improving the space utilization rate of the buffer. Thereby improving the efficiency of audio data processing.

[0042] In some embodiments of the present application, the buffer provides a block-aligned read interface. The read interface of the buffer is a set of specifications and methods for extracting data from the buffer. It defines how to start from a specific position in the buffer, in what way (such as sequential reading, random reading) and in what quantity to obtain data, and stipulates the form and meaning of the return value, and also includes the handling method in special cases such as when the buffer is empty. Based on this, when a reading device needs to access the audio data stored in the buffer, the read interface of the buffer can be scheduled. In response to the call of the read interface of the buffer, the audio data processing device returns the readable memory address and the length of the memory address in the buffer through the read interface. By returning the readable memory address and length, it can be ensured that the reading device correctly obtains the data and avoids reading data beyond the actual storage range of the buffer. The reading device determines the second audio data to be read according to the readable memory address and the length of the memory address returned by the buffer, and sends a first read request for instructing the reading of the second audio data to the audio data processing device through the read interface. In response to the first read request, when the data volume of the second audio data is less than the data block size or an integer multiple of the data block size, the audio data processing device reads the complete second audio data from the buffer based on the read pointer and returns it to the user. Among them, the reading device includes but is not limited to devices such as mixers that consume audio data in the buffer.

[0043] Through the above reading method, when the data volume of the second audio data is an integer multiple of the data block size, the complete second audio data with a data volume that is an integer multiple of the data block size is directly read from the buffer, so that the efficiency of data processing can be guaranteed.

[0044] In some embodiments of the present application, when the reading interface is called, the audio data processing device can determine the readable memory address and the length of the memory address in the buffer according to the read pointer position and the write pointer position. Among them, the read pointer position is used to indicate the starting point of the readable memory address, and the write pointer position is used to indicate the ending point of the readable memory address. Based on this, the memory address between the read pointer position and the write pointer position is the readable memory address, and the length of the memory address between the read pointer position and the write pointer position is the length of the readable memory address.

[0045] In some embodiments of the present application, when reading data in the buffer, after each data reading, the processed memory space is released by advancing the read pointer.

[0046] In some embodiments of the present application, atomic operations with block alignment are performed on the buffer, which can reduce thread competition in a multi-threaded scenario. Because the buffer transfers data in units of data blocks, if the data is block-aligned, the data involved in the atomic operation is more likely to be completely located in one data block. In a multi-threaded access scenario, assuming that multiple threads access different block-aligned atomic variables, since these variables are neatly arranged in memory, the data blocks where each variable is located are not easily accessed and modified by multiple threads at the same time. This reduces thread competition caused by cache line sharing. Therefore, block-aligned atomic operations, through reasonable memory layout, reduce competition generated by sharing cache lines and other situations when multiple threads access and modify atomic variables. Compared with the lock mechanism adopted by the traditional buffer, the real-time performance and stability of data processing are improved.

[0047] In some embodiments of the present application, the buffer adopts a circular buffer that supports wrap-around processing. Based on this, when writing the first audio data to the buffer based on the write pointer, before writing the next data block to be written to the buffer each time, first determine the size of the memory space between the write pointer and the end of the buffer to determine whether the memory space between the write pointer and the end of the buffer is sufficient to write the next data block to be written. When the memory space between the write pointer and the end of the buffer is sufficient to write the next data block to be written, directly write the next data block to be written based on the position of the write pointer. When the memory space between the write pointer and the end of the buffer is not sufficient to write the next data block to be written, perform wrap-around processing and wrap back to the beginning position of the buffer to continue data writing, thereby achieving circular writing. Similarly, when reading the second audio data from the buffer based on the read pointer, when all the audio data to be read has not been read, when the read pointer reaches the end of the buffer, wrap the read pointer back to the beginning position of the buffer to continue reading the audio data.

[0048] Through wrap-around processing, it is possible to recycle the same memory space during the audio data processing. For example, when audio data is written to the buffer from the beginning position of the buffer and reaches the end of the buffer, it will continue to be written from the beginning position of the buffer, which avoids frequent allocation and release of memory and improves the memory utilization efficiency. Moreover, in a traditional linear buffer, as data is frequently written and deleted, memory fragmentation may occur. However, the buffer wrap-around processing can effectively reduce the generation of memory fragmentation by recycling the memory space, because it does not need to allocate new memory blocks as frequently as a linear buffer to meet the data storage requirements, thus maintaining the integrity of the memory space. In addition, in real-time audio stream processing, the buffer wrap-around processing also plays a key role. During the audio stream processing, audio data is continuously generated and needs to be processed, and the circular buffer can ensure that the audio data can be continuously stored and read in the buffer without interruption due to the buffer boundary limitations. Even if the data "wraps around" in the buffer, its continuity can still be guaranteed, providing stable data support for real-time processing. Moreover, in scenarios where multiple devices or multiple modules work together, the buffer wrap-around processing also helps to maintain data continuity and synchronization. Due to the wrap-around characteristics of the buffer, the data transmission and processing can be carried out at a relatively stable pace, which is conducive to achieving synchronous operations at the writing end and the reading end.

[0049] In some embodiments of the present application, to prevent data loss, before writing new data to the buffer each time through the write pointer, it is also possible to first check whether the buffer is full. The condition for determining whether the buffer is full can be: the write pointer is in front of the read pointer, and the memory space between the write pointer and the read pointer is less than the data block size. When the memory space between the write pointer and the read pointer is less than the data block size, it means that the next position of the write pointer is the position of the read pointer, indicating that there is no space in the buffer to write new data. When it is determined that the buffer is not full, new data is written to the buffer. In this way, the written data can be prevented from overwriting the unread data, thereby avoiding data loss.

[0050] In some embodiments of the present application, if the buffer is full when writing data to the buffer, it is possible to choose to discard the new data to be written according to specific application requirements, or wait until there is enough space in the buffer to write, or other processing strategies can also be adopted. This embodiment does not make specific limitations on this.

[0051] The audio data processing method provided by the embodiments of the present application determines the data block size based on the obtained audio parameters; pre-allocates a memory space with a target size from the memory as the buffer according to the data block size; writes the first audio data to the buffer according to the data block size, or reads the second audio data from the buffer according to the data block size; wherein, the target size is an integer multiple of the data block size, and the buffer is a circular buffer. In the embodiments of the present application, the data block size matching the size of the audio data to be processed is determined according to the audio parameters, and a memory space with a target size is pre-allocated from the memory as the buffer according to the data block size, so that the buffer size matches the data volume of the audio data to be processed, thereby avoiding the time overhead caused by subsequent dynamic memory allocation, reducing the processing delay, and improving the real-time performance of data processing. Moreover, since the buffer uses a circular buffer, there is no need to switch the buffer during reading and writing, further reducing the time overhead in the processing process. In addition, writing to or reading from the buffer according to the data block size can reduce the space waste caused by data fragmentation and improve the space utilization rate of the buffer, thereby improving the efficiency of audio data processing.

[0052] In some embodiments, the audio processing device supports a callback mechanism, allowing external code to define specific audio processing algorithms. Based on this, in step 110 above, the data block size can be determined based on the audio parameters in the audio processing callback function.

[0053] Here, the audio processing callback function is a user-defined function for audio processing.

[0054] Through the callback mechanism, users are allowed to flexibly customize the audio processing callback function according to specific application scenarios and requirements. Different users can process different audio data by passing in different audio processing callback functions, thus adapting to various audio processing scenarios and improving the flexibility and customization of audio data processing. And determining the data block size based on the audio processing callback function enables the flexible design of the data block size, thereby supporting the audio processing of non-standard sampling rates.

[0055] In some embodiments, in step 130 above, when reading the second audio data from the buffer according to the data block size, it further includes:

[0056] In response to the first read request for the second audio data, when the data volume of the second audio data is greater than the data block size and not an integer multiple of the data block size, based on the read pointer of the buffer, the third audio data is read from the buffer, where the data volume of the third audio data is an integer multiple of the data block size.

[0057] In some embodiments of the present application, when the data volume of the second audio data is greater than the data block size and not an integer multiple of the data block size, calculate the quotient of the data volume of the second audio data divided by the data block size, and determine the integer in the quotient as the first multiple. The multiple between the data volume of the third audio data and the data block size is less than or equal to the first multiple.

[0058] In the above manner, when reading the buffer, preferentially returning data with a complete data block size can ensure the efficiency of data processing.

[0059] In some embodiments of the present application, the audio data processing device can support non-block-aligned reading requirements. Based on this, after reading the third audio data, since the third audio data is less than the second audio data, the reading device can send a second read request to the audio data processing device to indicate reading the fourth audio data. The data volume of the fourth audio data is the difference between the data volume of the second audio data and the data volume of the third audio data. The audio data processing device responds to the second read request and reads the fourth audio data from the buffer based on the read pointer of the buffer.

[0060] In the above manner, the reading device can obtain the complete second audio data.

[0061] In some embodiments, in a mixing scenario, the audio processing device can be a mixing device. In this case, after reading the second audio data from the buffer according to the data block size, the following steps can further be performed:

[0062] Convert the data type of the second audio data read from the buffer to a preset data type to obtain audio sampling data;

[0063] Perform mixing processing on the audio sampling data.

[0064] In some embodiments of the present application, the preset data type may be the storage format of audio data pre-specified by the mixing device. The preset data type includes but is not limited to Float32 (32-bit floating point), Int16 (16-bit integer), and Int24 (24-bit integer). The second audio data read from the buffer is usually byte data, which is inconsistent with the preset data type. Therefore, before performing the mixing process, first convert the data type of the read second audio data to the preset data type to obtain audio sampling data. Then, perform mixing processing on the audio sampling data based on a preset mixing algorithm. Among them, the mixing algorithm can be a user-defined mixing algorithm.

[0065] In the above manner, in a mixing scenario, audio data can be read from the buffer based on a block alignment method, and then the read audio data can be mixed, so that the audio output can be made more stable, which is particularly suitable for professional audio processing scenarios and can effectively solve the audio stuttering problem.

[0066] In some embodiments, the audio data processing device supports buffer reset operations, allowing the buffer to be emptied and restarted when needed. Based on this, after the above step 120, the following steps can also be performed:

[0067] In response to the first operation, clear the data stored in the buffer, and update the write pointer and read pointer of the buffer to the start position of the buffer.

[0068] In some embodiments of the present application, the first operation is an operation for resetting the buffer. Exemplarily, the above first operation includes but is not limited to: the user's click input on the buffer reset control in the audio data processing device through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible inputs, which can be specifically determined according to actual usage requirements, and the embodiments of the present invention are not limited. The specific gesture in the embodiments of the present application can be any one of a click gesture, a swipe gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double press gesture, and a double click gesture; the click input in the embodiments of the present application can be a single click input, a double click input, or a click input of any number of times, and can also be a long press input or a short press input.

[0069] Through buffer reset, the same memory area can be reused as a buffer between different data processing stages or different data batches. This avoids frequent memory allocation and deallocation, reducing the memory management overhead. For buffers in some hardware devices, the reset function enables these hardware resources to be fully utilized in various application scenarios, reducing the cost of hardware upgrades because there is no need to excessively increase hardware resources to handle more data.

[0070] In the audio data processing method provided by the embodiments of the present application, the execution subject may be an audio data processing device. In the embodiments of the present application, taking the audio data processing device as an example to execute the audio data processing method, the audio data processing device provided by the embodiments of the present application is described.

[0071] See Figure 2 , which is a schematic diagram of the audio data processing device provided by the embodiments of the present application. As Figure 2 shown, the device 200 includes the following modules:

[0072] The size determination module 210 is configured to determine the data block size based on the acquired audio parameters;

[0073] The memory allocation module 220 is configured to pre-allocate a memory space with a target size from the memory as a buffer according to the data block size;

[0074] The reading and writing module 230 is configured to write first audio data to the buffer according to the data block size, or read second audio data from the buffer according to the data block size;

[0075] Among them, the target size is an integer multiple of the data block size, and the buffer is a circular buffer.

[0076] The audio data processing device provided by the embodiment of the present application determines the data block size based on the acquired audio parameters; according to the data block size, pre-allocates a memory space of a target size from the memory as a buffer; writes the first audio data to the buffer according to the data block size, or reads the second audio data from the buffer according to the data block size; wherein, the target size is an integer multiple of the data block size, and the buffer is a circular buffer. In the embodiment of the present application, the data block size matching the size of the audio data to be processed is determined according to the audio parameters, and a memory space of a target size is pre-allocated as a buffer according to the data block size, so that the buffer size matches the data volume of the audio data to be processed, thereby avoiding the time overhead caused by subsequent dynamic memory allocation, reducing the processing delay, improving the real-time performance of data processing, and moreover, the buffer adopts a circular buffer, so that when reading and writing, there is no need to switch the buffer, further reducing the time overhead in the processing process. In addition, writing to or reading from the buffer according to the data block size can reduce the space waste caused by data fragmentation and improve the space utilization rate of the buffer. Thereby improving the efficiency of audio data processing.

[0077] In some embodiments, the size determination module 210 is specifically configured to:

[0078] Determine the data block size based on the audio parameters in the audio processing callback function.

[0079] In some embodiments, the reading and writing module 230 is specifically configured to:

[0080] In response to a first read request for the second audio data, when the data volume of the second audio data is greater than the data block size and not an integer multiple of the data block size, read the third audio data from the memory address based on the read pointer of the buffer, wherein the data volume of the third audio data is an integer multiple of the data block size.

[0081] In some embodiments, the buffer is integrated in the mixing device, and the device 200 further includes: a mixing module, configured to:

[0082] After reading the second audio data from the buffer according to the data block size, convert the data type of the second audio data read from the buffer into a preset data type to obtain audio sampling data;

[0083] Perform mixing processing on the audio sampling data.

[0084] In some embodiments, the device 200 further includes: a reset module, configured to:

[0085] After allocating a memory space of a specified size from the memory as a buffer, in response to a first operation, clear the data stored in the buffer and update the write pointer and read pointer of the buffer to the start position of the buffer.

[0086] The audio data processing device in the embodiments of the present application may be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than terminals. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a handheld computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It may also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0087] The audio data processing device in the embodiments of the present application may be a device with an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems. The embodiments of the present application do not make specific limitations.

[0088] The audio data processing device provided in the embodiments of the present application can implement Figure 1 each process implemented by the method embodiments. To avoid repetition, it will not be elaborated here.

[0089] Optionally, as Figure 3 shown, the embodiments of the present application further provide an electronic device 300, including a processor 301 and a memory 302. A program or instruction that can run on the processor 301 is stored on the memory 302. When the program or instruction is executed by the processor 301, it implements each step of the above-mentioned audio data processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0090] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0091] Figure 4 Schematic diagram of the hardware structure of an electronic device for implementing the embodiments of the present application.

[0092] The electronic device 400 includes, but is not limited to, components such as a radio frequency unit 401, a network module 402, an audio output unit 403, an input unit 404, a sensor 405, a display unit 406, a user input unit 407, an interface unit 408, a memory 409, and a processor 410.

[0093] Those skilled in the art can understand that the electronic device 400 may further include a power supply (such as a battery) for supplying power to each component. The power supply can be logically connected to the processor 410 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 4 The structure of the electronic device shown does not limit the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0094] The processor 410 is configured to determine the data block size based on the acquired audio parameters;

[0095] The processor 410 is further configured to pre-allocate a memory space with a target size from the memory as a buffer according to the data block size;

[0096] The processor 410 is further configured to write first audio data to the buffer according to the data block size, or read second audio data from the buffer according to the data block size;

[0097] Wherein, the target size is an integer multiple of the data block size, and the buffer is a circular buffer.

[0098] The electronic device provided by the embodiment of the present application determines the data block size based on the acquired audio parameters; pre-allocates a memory space with a target size from the memory as a buffer according to the data block size; writes first audio data to the buffer according to the data block size, or reads second audio data from the buffer according to the data block size; wherein, the target size is an integer multiple of the data block size, and the buffer is a circular buffer. In the embodiment of the present application, the data block size matching the size of the audio data to be processed is determined according to the audio parameters, and a memory space with a target size is pre-allocated from the memory as a buffer according to the data block size, so that the size of the buffer matches the data volume of the audio data to be processed, thereby avoiding the time overhead caused by subsequent dynamic memory allocation, reducing the processing delay, and improving the real-time performance of data processing. Moreover, since the buffer uses a circular buffer, there is no need to switch the buffer during reading and writing, further reducing the time overhead during the processing. In addition, writing to or reading from the buffer according to the data block size can reduce the space waste caused by data fragmentation and improve the space utilization rate of the buffer. Thereby improving the efficiency of audio data processing.

[0099] In some embodiments, the processor 410 is specifically configured to:

[0100] Determine the data block size based on the audio parameters in the audio processing callback function.

[0101] In some embodiments, the processor 410 is specifically configured to:

[0102] In response to a first read request for second audio data, when the data volume of the second audio data is greater than the data block size and not an integer multiple of the data block size, based on the read pointer of the buffer, read third audio data from the memory address, where the data volume of the third audio data is an integer multiple of the data block size.

[0103] In some embodiments, the buffer is integrated in the mixing device, and the processor 410 is further configured to:

[0104] After reading the second audio data from the buffer according to the data block size, convert the data type of the second audio data read from the buffer to a preset data type to obtain audio sampling data;

[0105] Perform mixing processing on the audio sampling data.

[0106] In some embodiments, the processor 410 is further configured to:

[0107] After allocating a memory space with a specified size from the memory as a buffer, in response to a first operation, clear the data stored in the buffer and update the write pointer and read pointer of the buffer to the start position of the buffer.

[0108] It should be understood that in the embodiments of the present application, the input unit 404 may include a Graphics Processing Unit (GPU) 4041 and a microphone 4042. The graphics processor 4041 processes the image data of static pictures or videos obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 406 may include a display panel 4061, and the display panel 4061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 407 includes at least one of a touch panel 4071 and other input devices 4072. The touch panel 4071 is also called a touch screen. The touch panel 4071 may include two parts: a touch detection device and a touch controller. The other input devices 4072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, which will not be elaborated here.

[0109] The memory 409 can be used to store software programs and various data. The memory 409 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area can store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 409 can include volatile memory or non-volatile memory, or the memory 409 can include both volatile and non-volatile memory. Among them, the non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 409 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0110] The processor 410 may include one or more processing units; optionally, the processor 410 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 410 either.

[0111] The embodiments of the present application also provide a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the above embodiments of the audio data processing method and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0112] Among them, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs.

[0113] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above audio data processing method embodiment, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0114] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip.

[0115] The embodiments of the present application provide a computer program product. The program product is stored in a storage medium and is executed by at least one processor to implement each process of the above audio data processing method embodiment, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0116] It should be noted that in this article, the term "including", "comprising", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article, or device. Without more limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device including that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, the features described with reference to certain examples may be combined in other examples.

[0117] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0118] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection scope of the present application.

Claims

1. An audio data processing method, characterized in that: include: Determine the data block size based on the acquired audio parameters; According to the size of the data block, pre-allocate a memory space of a target size from the memory as a buffer; Writing the first audio data into the buffer according to the data block size, or reading the second audio data from the buffer according to the data block size; The target size is an integer multiple of the data block size, and the buffer is a ring buffer.

2. The method according to claim 1, characterized in that The step of determining the data block size based on the acquired audio parameters includes: Determine the data block size based on the audio parameters in the audio processing callback function.

3. The method according to claim 1, characterized in that The step of reading the second audio data from the buffer according to the data block size comprises: In response to a first read request for second audio data, when the data amount of the second audio data is greater than the data block size and is not an integer multiple of the data block size, third audio data is read from the buffer based on the read pointer of the buffer, wherein the data amount of the third audio data is an integer multiple of the data block size.

4. The method according to claim 1, characterized in that: The buffer is integrated in the audio mixing device. After reading the second audio data from the buffer according to the data block size, the method further includes: Converting the data type of the second audio data read from the buffer into a preset data type to obtain audio sampling data; Perform mixing processing on the audio sampling data.

5. The method according to any one of claims 1 to 4, characterized in that: After allocating a memory space of a specified size from the memory as a buffer, the method further comprises: In response to the first operation, the data stored in the buffer is cleared, and the write pointer and the read pointer of the buffer are updated to the beginning position of the buffer.

6. An audio data processing device, characterized in that: include: A size determination module, used for determining a data block size based on the acquired audio parameters; A memory allocation module, used for pre-allocating a target size memory space from the memory as a buffer according to the size of the data block; A read / write module, configured to write the first audio data into the buffer according to the data block size, or read the second audio data from the buffer according to the data block size; The target size is an integer multiple of the data block size, and the buffer is a ring buffer.

7. The device according to claim 6, characterized in that The size determination module is specifically used for: Determine the data block size based on the audio parameters in the audio processing callback function.

8. The device according to claim 6, characterized in that The read-write module is specifically used for: In response to a first read request for second audio data, when the data amount of the second audio data is greater than the data block size and is not an integer multiple of the data block size, third audio data is read from the buffer based on the read pointer of the buffer, wherein the data amount of the third audio data is an integer multiple of the data block size.

9. The device according to claim 6, characterized in that The buffer is integrated in a mixing device, and the apparatus further comprises a mixing module, which is used to: After reading the second audio data from the buffer according to the data block size, converting the data type of the second audio data read from the buffer into a preset data type to obtain audio sampling data; Perform mixing processing on the audio sampling data.

10. The device according to any one of claims 6 to 9, characterized in that: The device further comprises: a reset module, configured to: After allocating a memory space of a specified size from the memory as a buffer, in response to a first operation, the data stored in the buffer is cleared, and the write pointer and the read pointer of the buffer are updated to the beginning position of the buffer.

11. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the audio data processing method according to any one of claims 1 to 5 are implemented.

12. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the audio data processing method according to any one of claims 1 to 5 are implemented.

Images