Synchronous recording and broadcasting method of low-delay microphone based on Android system and related equipment

Abstract

The invention relates to the technical field of audio processing, and provides a synchronous recording and broadcasting method of a low-delay microphone based on an Android system and related equipment. Acquiring an original audio stream through a microphone and transmitting the original audio stream to the memory buffer area to generate a standby audio stream; performing quantization processing and weighted fusion processing on the acquired network state parameters and the equipment performance parameters to generate a comprehensive evaluation result, and dynamically adjusting a preset audio buffer area; creating a recording thread and a playing thread in the Android system, capturing a standby audio stream in the recording thread, and writing the standby audio stream into an audio buffer area; reading the standby audio stream from the audio buffer area through the playing thread; and carrying out sound mixing processing on the standby audio stream and the audio to be played to generate a sound-mixed audio and playing the sound-mixed audio. The recording and playing threads are created to respectively process the writing and reading of the audio so as to realize parallel operation, and finally, the audio is effectively integrated through audio mixing playing, so that low-delay microphone synchronous recording and playing are realized, and the audio interaction experience is improved.

Description

Synchronous recording and broadcasting method and related equipment based on low-latency microphones of Android system Technical Field

[0001] This application relates to the field of audio processing technology, and more specifically, to a synchronous recording and broadcasting method, apparatus, computer-readable medium, and electronic device based on a low-latency microphone of the Android system. Background Technology

[0002] In today's social entertainment landscape, live streaming and real-time interactive audio social networking are booming. Whether it's the instant communication between streamers and viewers in game live streams or the smooth multi-person chat in voice chat rooms, the real-time performance of audio is crucial. Meanwhile, in applications such as intelligent voice assistants and intelligent conferencing systems, low-latency microphone-based synchronous recording and playback methods are essential. Furthermore, for professional audio creators, such as podcasters and audiobook recorders, Android devices have become a commonly used creation tool due to their widespread availability and convenience. However, the latency issues inherent in traditional recording and playback methods significantly impact the broadcaster's control of rhythm, the quality of sound recording, and the playback effect. Summary of the Invention

[0003] This application provides a synchronous recording and playback method and related equipment based on a low-latency microphone of the Android system, which can at least to some extent solve the problem of high latency in traditional recording and playback methods.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to one aspect of this application, a synchronous recording and playback method based on a low-latency microphone in an Android system is provided, comprising: acquiring an original audio stream through a microphone, transmitting the original audio stream to a memory buffer to generate a backup audio stream; performing quantization and weighted fusion processing on acquired network state parameters and device performance parameters to generate a comprehensive evaluation result, and dynamically adjusting a preset audio buffer based on the comprehensive evaluation result; creating a recording thread and a playback thread in the Android system, capturing the backup audio stream through the recording thread and writing it into the audio buffer; reading the backup audio stream from the audio buffer through the playback thread; performing mixing processing on the backup audio stream and the audio to be played obtained from the playback buffer corresponding to the playback thread based on the mixing weight determined by the sampling value of the backup audio stream to generate mixed audio, and playing the mixed audio.

[0006] In this application, based on the aforementioned scheme, the step of acquiring the original audio stream through a microphone and transmitting the original audio stream to a memory buffer to generate a backup audio stream includes: acquiring the original audio stream through a USB microphone; sending a data transfer request to the DMA controller; and the DMA controller transmitting the original audio stream from the USB microphone to a previously allocated memory buffer according to pre-configured memory address information to generate a backup audio stream.

[0007] In this application, based on the aforementioned scheme, the step of quantifying and weightedly fusing the acquired network status parameters and device performance parameters to generate a comprehensive evaluation result, and then dynamically adjusting the preset audio buffer based on the comprehensive evaluation result, includes: obtaining network status parameters from network bandwidth through a preset network monitoring submodule, obtaining device performance parameters from system call interfaces through a device performance monitoring submodule, and statistically analyzing historical buffer usage efficiency based on a preset data structure; quantifying the network status parameters, the device performance parameters, and the historical buffer usage efficiency, and then weightedly fusing them based on preset weights to generate a comprehensive evaluation result; and dynamically adjusting the audio buffer according to the comprehensive evaluation result and a preset mapping relationship.

[0008] In this application, based on the aforementioned scheme, the step of creating a recording thread and a playback thread in the Android system, capturing the backup audio stream through the recording thread, and writing it into the audio buffer includes: creating and initializing a recording thread and a playback thread in the Android system based on multi-threaded programming technology; listening to whether the microphone has captured new audio data through the recording thread, and if so, reading the backup audio stream from the memory buffer through the driver interface; and writing the backup audio stream into the audio buffer based on a synchronization mechanism.

[0009] In this application, based on the aforementioned scheme, the step of reading the spare audio stream from the audio buffer through the playback thread includes: monitoring the occupancy information of the audio buffer through the playback thread, dynamically adjusting the speed and quantity of data reading based on the occupancy information, and reading the spare audio stream from the audio buffer based on the speed and quantity of data reading.

[0010] In this application, based on the aforementioned scheme, the mixing weight determined based on the sampled values ​​of the backup audio stream, the mixing processing of the backup audio stream and the audio to be played obtained from the playback buffer corresponding to the playback thread to generate mixed audio, and the playback of the mixed audio, includes: obtaining the audio to be played from the playback buffer corresponding to the playback thread; wherein, the audio to be played includes the mixed audio from the previous moment or silence data; analyzing the energy distribution of the backup audio based on the sampled values ​​of the backup audio stream, and determining the mixing weight according to the energy distribution; monitoring the time delay between recording and playback through a timing module, and determining a compensation strategy according to the time delay; mixing the backup audio stream and the audio to be played through the playback thread based on the mixing weight and the compensation strategy to generate mixed audio; playing the mixed audio, and saving the mixed audio to the playback buffer.

[0011] In this application, based on the aforementioned scheme, playing the mixed audio includes: converting the mixed audio into a sound signal through the playback thread; and playing the sound signal through a speaker.

[0012] According to one aspect of this application, a synchronous recording and playback device based on a low-latency microphone of an Android system is provided, comprising: an acquisition module, configured to acquire an original audio stream through a microphone, transmit the original audio stream to a memory buffer, and generate a backup audio stream; a buffer module, configured to quantize and weightedly fuse the acquired network status parameters and device performance parameters to generate a comprehensive evaluation result, and dynamically adjust a preset audio buffer based on the comprehensive evaluation result; a writing module, configured to create a recording thread and a playback thread in the Android system, capture the backup audio stream through the recording thread, and write it into the audio buffer; a reading module, configured to read the backup audio stream from the audio buffer through the playback thread; and a mixing module, configured to mix the backup audio stream and the audio to be played obtained from the playback buffer corresponding to the playback thread based on the mixing weight determined by the sampling value of the backup audio stream, generate mixed audio, and play the mixed audio.

[0013] According to one aspect of this application, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the synchronous recording and broadcasting method for a low-latency microphone based on the Android system as described in the above embodiments.

[0014] According to one aspect of this application, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the synchronous recording and broadcasting method of a low-latency microphone based on the Android system as described in the above embodiments.

[0015] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the synchronous recording and playback method for a low-latency microphone based on the Android system provided in the various optional implementations described above.

[0016] The main differences and advantages of this technical solution compared to existing technologies are as follows: Firstly, existing technologies introduce significant latency in processing USB microphone audio through buffering mechanisms and mixing operations. Users experience a considerable delay after speaking before hearing their own voice feedback, severely impacting the user experience. This technical solution optimizes the audio data stream architecture by introducing DMA technology to directly transfer the raw audio data captured by the USB microphone to a memory buffer, avoiding the time consumption caused by multiple data copies. Simultaneously, the audio buffer size is dynamically adjusted based on real-time network conditions and device performance, flexibly changing the buffer capacity to minimize buffering time while ensuring no audio data loss. Furthermore, in the parallel recording and playback thread design, independent threads handle recording and playback tasks separately, reducing the overall processing time and effectively lowering audio latency, thus improving real-time performance—something difficult to achieve with existing technologies.

[0017] Secondly, existing technologies exhibit significant performance fluctuations in audio processing under varying network environments and device performance conditions. For instance, sudden narrowing of network bandwidth or increased CPU usage can lead to audio stuttering and interruptions. This technical solution quantifies and weights network status parameters and device performance parameters to generate a comprehensive evaluation result, and dynamically adjusts the audio buffer based on this result. This adaptive adjustment mechanism can rationally allocate system resources according to real-time changing environmental factors, ensuring relatively stable audio processing performance under various complex environments. In contrast, existing technologies lack this intelligent dynamic adjustment capability and cannot effectively cope with complex and ever-changing environments, while this solution better adapts to various real-world scenarios and guarantees audio quality.

[0018] Thirdly, existing technologies suffer from significant time differences between recording and playback due to various factors, and the mixing process is not intelligent enough, resulting in poor audio synchronization and affecting the overall listening experience. This technical solution, in the mixing stage, not only analyzes the frequency components and energy distribution of the backup audio to determine mixing weights, but also monitors the time delay between recording and playback through a timing module and formulates compensation strategies. This comprehensive mixing approach, considering both audio characteristics and time delay, can more accurately achieve intelligent mixing and delay compensation, ensuring audio synchronization. Existing technologies often overlook these details, leading to poor audio synchronization. This solution has a significant advantage in this regard, providing a superior audio experience.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0021] Figure 1 schematically illustrates a flowchart of a synchronous recording and broadcasting method based on a low-latency microphone of the Android system in one embodiment of this application.

[0022] Figure 2 schematically illustrates a flowchart of generating and playing mixed audio in one embodiment of this application.

[0023] Figure 3 schematically illustrates a synchronous recording and broadcasting device based on a low-latency microphone of the Android system in one embodiment of this application.

[0024] Figure 4 shows a schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application. Detailed Implementation

[0025] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0026] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, systems, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0027] It should be noted that the data acquisition or information collection in this embodiment is performed after authorization by the user or the object of collection, and its process and purpose strictly follow the relevant regulations.

[0028] The block diagrams shown in the attached figures are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, or in one or more hardware modules composed of smart chips, smart integrated circuits, or application-specific integrated circuits (ASICs), or in different network and / or processor devices and / or microcontroller devices.

[0029] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0030] The following details the implementation of the technical solution of this application: Figure 1 shows a flowchart of a synchronous recording and broadcasting method based on a low-latency microphone of an Android system according to an embodiment of this application. Referring to Figure 1, the synchronous recording and broadcasting method based on a low-latency microphone of an Android system includes at least steps S110 to S150, which are described in detail below: S110, acquiring the original audio stream through the microphone, transmitting the original audio stream to a memory buffer, and generating a backup audio stream.

[0031] In this embodiment, when the low-latency microphone synchronous recording and playback function based on the Android system is activated, the microphone is first activated to start capturing sound signals, thereby obtaining the raw audio stream. According to a pre-set audio data transmission mechanism, a transmission command is sent to the Direct Memory Access (DMA) controller. The DMA controller accurately transmits the raw audio stream obtained from the microphone to a designated memory buffer according to a pre-planned memory address path. The audio stream in the memory buffer then becomes a backup audio stream for subsequent processing.

[0032] In one embodiment of this application, acquiring the original audio stream via a microphone and transmitting the original audio stream to a memory buffer to generate a backup audio stream includes: acquiring the original audio stream via a USB microphone; sending a data transfer request to a DMA controller; and the DMA controller transmitting the original audio stream from the USB microphone to a previously allocated memory buffer according to pre-configured memory address information to generate a backup audio stream.

[0033] After the audio device starts up, the audio acquisition application is initialized, a corresponding memory buffer is allocated for the USB microphone, and the Direct Memory Access (DMA) controller is configured. When the USB microphone begins capturing raw audio data, traditionally, this data is first passed to the AudioRecord module in the system audio framework for buffering. However, in this solution, due to the introduction of DMA technology, the raw audio data captured by the USB microphone does not need to go through the AudioRecord buffering process.

[0034] After capturing audio data, the USB microphone, acting as a peripheral device, sends a data transfer request to the DMA controller. Upon receiving the request, the DMA controller takes over control of the system bus. According to the pre-configured memory address information, it directly transfers the raw audio data from the USB microphone to the previously allocated memory buffer. This process completely bypasses the CPU's conventional intervention in data transfer, avoiding multiple copy operations in traditional software processing. For example, data might previously require multiple steps, such as copying from the microphone driver to the system kernel buffer and then from the kernel buffer to the user space buffer. Now, DMA directly completes the transfer from the microphone to the target memory buffer, significantly shortening the path and time for data to reach a usable state.

[0035] While audio data is being transferred to the memory buffer via DMA, the audio data stream manager monitors the status of the memory buffer in real time. Once new audio data is detected arriving via DMA, its storage location and related information are immediately retrieved. Internally, the audio data stream manager maintains a complex data structure to record metadata such as the arrival time and length of the audio data, and categorizes and manages this data according to subsequent playback and recording needs.

[0036] On one hand, for the audio data that needs to be played, the audio data stream manager quickly reads the data from the memory buffer according to the real-time needs of the playback thread and passes it to the playback thread. Optionally, during the transmission process, the manager reasonably adjusts the data transmission rate and batch size according to the current system load and resource status to ensure that the playback thread can stably and smoothly obtain audio data for playback.

[0037] On the other hand, for the audio data that needs to be recorded, the audio data stream manager also accurately extracts it from the memory buffer and writes the data into the storage device in an orderly manner according to the recording file format requirements. Throughout the process, the manager interacts with the storage subsystem to optimize the data write operation and avoid data loss or increased latency due to performance fluctuations of the storage device.

[0038] Through the coordinated processing of the audio data stream manager, the entire process of audio data from capture to playback and recording becomes more efficient and orderly, reducing the time consumption caused by multiple intermediate processing steps and mixing operations in traditional solutions. This enables the system to respond to the processing needs of audio data more quickly, thereby achieving low-latency recording and playback functionality.

[0039] The above process avoids the buffering time of AudioTrack and AudioRecord, as well as the mixing time of AudioFlinger, found in traditional solutions. Simultaneously, DMA technology reduces data copying time, significantly lowering the latency from audio data capture to playback. In actual testing, the latency can reach below 150ms, achieving a level of delay imperceptible to the human ear. Reducing intermediate processing steps frees up CPU resources, enabling more efficient processing of audio data and other tasks. The design of the audio data stream manager makes data reception, storage, and distribution more orderly and efficient, further improving the overall system performance. Primarily based on software architecture improvements and the application of DMA technology, no hardware modifications are required, thus ensuring good compatibility and applicability to computer systems and USB microphone devices with different configurations.

[0040] The above process, by acquiring the raw audio stream through a USB microphone, ensures a clear and stable source of audio input. Based on a pre-defined audio data architecture, a data transfer request is sent to the DMA controller. The DMA controller then directly transfers the raw audio stream from the USB microphone to the pre-allocated memory buffer according to the pre-configured memory address information. This process bypasses the multiple intermediate copying steps that may be involved in traditional data transfer, significantly reducing data transfer latency and additional processing overhead. This allows for the rapid generation of a backup audio stream, providing an efficient data foundation for subsequent audio processing.

[0041] S120: The obtained network status parameters and device performance parameters are quantified and weighted and fused to generate a comprehensive evaluation result. Based on the comprehensive evaluation result, the preset audio buffer is dynamically adjusted.

[0042] In this embodiment, the network monitoring module continuously monitors the network status and collects relevant status information such as network bandwidth, while the device performance monitoring module also obtains device performance parameters from the system level. Then, according to predetermined rules, these different types and forms of network status parameters and device performance parameters are transformed into a unified and measurable standard form. Next, the quantified parameters are fused according to preset weight ratios, and a comprehensive evaluation result is generated after considering various factors. Finally, based on this comprehensive evaluation result, the pre-set audio buffer size is flexibly adjusted through a preset data interface.

[0043] In one embodiment of this application, the acquired network status parameters and device performance parameters are quantified and weighted fused to generate a comprehensive evaluation result. Based on the comprehensive evaluation result, a preset audio buffer is dynamically adjusted. This includes: obtaining network status parameters from network bandwidth through a preset network monitoring submodule, obtaining device performance parameters from a system call interface through a device performance monitoring submodule, and statistically analyzing historical buffer usage efficiency based on a preset data structure; quantifying the network status parameters, the device performance parameters, and the historical buffer usage efficiency, and weighting and fusing them based on preset weights to generate a comprehensive evaluation result; and dynamically adjusting the audio buffer according to the comprehensive evaluation result and a preset mapping relationship.

[0044] In this embodiment, the network monitoring submodule senses changes in network bandwidth in real time at the network stack interface, collects relevant data such as network transmission rate at preset time intervals, and performs preliminary processing on this raw data as network status parameters. For example, it records bandwidth fluctuations over a period of time, removes some instantaneous abnormal peaks or valleys, and obtains relatively stable data that reflects the current actual network condition.

[0045] Meanwhile, the device performance monitoring submodule obtains CPU utilization information through the system call interface provided by the operating system. It continuously tracks CPU usage at different times, analyzes the current computing resource load status of the device, and uses this as a device performance parameter. These network status parameters and device performance parameters are passed to subsequent processing stages, providing a basis for adjusting the buffer size.

[0046] Because network bandwidth and CPU utilization have different numerical ranges and units, they need to be normalized for easier subsequent comprehensive calculations. For network bandwidth data, historical data over a period of time is analyzed to identify the maximum and minimum values. Then, the currently perceived network bandwidth value is mapped to a range of zero to one according to a certain ratio. In this way, regardless of how the actual network bandwidth value changes, after normalization, it can be converted into a relative value between zero and one, more intuitively reflecting the relative level of the current network bandwidth within the historical fluctuation range. Similarly, CPU utilization is itself a percentage value; directly mapping it to the range of zero to one gives it the same unit range as other normalized data, facilitating subsequent comprehensive calculations.

[0047] This embodiment pre-defines a dedicated data structure for recording the historical buffer usage efficiency. Each time the buffer performs data storage and retrieval operations, relevant efficiency metrics, such as buffer fill speed, read speed, and data loss rate, are recorded and written into this data structure. When calculating the current buffer adjustment strategy, this historical information is considered, and the concept of time decay is introduced. Historical data more recent to the current time has a greater impact on the current decision, while older data has a relatively smaller impact. By weighted averaging and other methods, a comprehensive value reflecting the historical buffer usage efficiency is integrated, providing an important reference for current buffer size adjustments.

[0048] The normalized network state parameters, device performance parameters, and integrated historical buffer usage efficiency are combined with weight coefficients pre-trained using a machine learning algorithm for comprehensive calculation. The machine learning algorithm, through learning and analyzing a large amount of real-world data, determines the importance of different factors in buffer size adjustment, which is reflected in the form of weight coefficients. These data from different sources are then fused according to their respective weights to obtain a comprehensive evaluation result. This result reflects the direction and magnitude of the buffer adjustment under the combined effects of current network conditions, device performance, and historical usage.

[0049] Based on the comprehensive evaluation results, the audio buffer is dynamically adjusted. If the evaluation results show that the current network condition is good, the device performance is sufficient, and the historical buffer usage efficiency is high, the buffer size can be appropriately reduced to reduce the audio data storage time and thus reduce latency. Conversely, if the evaluation results show that the network is unstable, the device load is high, or the historical buffer usage efficiency is low, the buffer size will be appropriately increased to ensure that there is enough space to store audio data, avoid data loss and playback stuttering, and ensure audio quality.

[0050] For example, based on the calculated comprehensive evaluation result B(t) value, it is converted into an actual buffer size adjustment amount according to a preset mapping relationship. For instance, if B(t) is between 0 and 0.3, the buffer size is appropriately reduced; if it is between 0.3 and 0.7, the buffer size remains unchanged; and if it is between 0.7 and 1, the buffer size is appropriately increased.

[0051] By calling the relevant interfaces provided by the audio processing framework, the memory allocation of the buffer is dynamically adjusted to ensure that the storage and retrieval of audio data can adapt to the new buffer size. The entire adjustment process is real-time and dynamic, and can respond promptly to the constantly changing external environment, ensuring that the audio recording and playback system is always in optimal working condition.

[0052] The above process involves obtaining network status parameters from network bandwidth through a pre-defined network monitoring submodule, while the device performance monitoring submodule obtains device performance parameters from system call interfaces. Based on a pre-defined data structure, historical buffer usage efficiency is statistically analyzed. Quantifying network status parameters, device performance parameters, and historical buffer usage efficiency unifies these parameters of different natures onto a comparable and calculable scale. Then, based on pre-defined weights, a weighted fusion is performed, comprehensively considering factors such as real-time changes in network bandwidth, fluctuations in device performance, and historical buffer usage. The resulting comprehensive evaluation result more comprehensively and accurately reflects the current system operating status. Based on this comprehensive evaluation result and a pre-defined mapping relationship, the audio buffer is dynamically adjusted. By dynamically adjusting the buffer size in real time according to network and device conditions, audio data backlog caused by an excessively large buffer is avoided, effectively reducing audio latency from recording to playback. While reducing latency, the actual conditions of the network and devices are comprehensively considered to ensure that appropriate buffer space is provided for audio data under various conditions, avoiding data loss or playback stuttering due to an insufficiently small buffer, thus guaranteeing audio quality. It can automatically adjust the audio buffer size according to real-time changing environmental factors, and minimize the latency caused by buffering while ensuring that audio data is not lost, so as to adapt to different network and device environments, making the system more adaptable and stable, and able to adapt to different network environments and device performance states.

[0053] S130, a recording thread and a playback thread are created in the Android system. The recording thread captures the backup audio stream and writes it to the audio buffer.

[0054] In this embodiment, under the Android system environment, independent recording and playback threads are created using multi-threaded programming techniques, and their initialization settings are configured. After the recording thread enters the working state, it listens for the generation of new backup audio streams. Once a new backup audio stream is detected, the recording thread accurately reads the backup audio stream data through the interaction interface with the hardware driver, and then writes the read backup audio stream into the audio buffer in an orderly and stable manner according to the pre-set synchronization mechanism, preparing for subsequent audio processing and playback.

[0055] In one embodiment of this application, a recording thread and a playback thread are created in the Android system. The recording thread captures the backup audio stream and writes it into the audio buffer. This includes: creating and initializing the recording thread and playback thread in the Android system based on multi-threaded programming technology; listening to whether the microphone captures new audio data through the recording thread; if so, reading the backup audio stream from the memory buffer through the driver interface; and writing the backup audio stream into the audio buffer based on a synchronization mechanism.

[0056] When launching an audio processing application in the Android system, two independent threads are created according to design requirements: RecordThread (recording thread) and PlayThread (playback thread). This is implemented using multithreaded programming techniques, which allow multiple threads to execute multiple tasks in parallel within the same timeframe. Specifically, in the Android system, new threads can be created by inheriting the Thread class or implementing the Runnable interface. For example, a new class can be created that inherits from the Thread class, and the startup method can be overridden before creating and starting a thread instance.

[0057] After creating the recording and playback threads, each thread will perform its own initialization tasks, such as allocating memory resources and setting thread priorities. During initialization, the recording thread will establish a connection with the USB microphone to prepare for receiving audio data; the playback thread will establish a connection with the audio output device (such as a speaker) to prepare for playing audio.

[0058] After the recording thread starts running, it continuously monitors the data output of the USB microphone. When new audio data is detected by the USB microphone, the recording thread reads this audio data through the corresponding driver interface as a backup audio stream. The read audio data is then stored in a dynamically adjusted audio buffer.

[0059] Optionally, to ensure the correctness and stability of data storage, the recording thread employs a synchronization mechanism, such as a mutex lock, when writing data to the buffer. When the recording thread needs to write data, it first acquires the mutex lock to ensure that only it can access the buffer and perform write operations at the same time. After writing is complete, it releases the mutex lock to prevent other threads from interfering with the data in the buffer at this time.

[0060] S140, the backup audio stream is read from the audio buffer via the playback thread.

[0061] In this embodiment, after the playback thread starts, it continuously monitors the state changes of the audio buffer and obtains its occupancy information in real time. Based on this occupancy information, the playback thread can perceive the storage status of the buffer and dynamically adjust its data reading strategy, such as speeding up or slowing down the reading speed, or increasing or decreasing the amount of data read each time. Subsequently, according to the adjusted strategy, the playback thread accurately extracts an appropriate amount of spare audio stream from the audio buffer for subsequent mixing and other processing operations.

[0062] In one embodiment of this application, reading the spare audio stream from the audio buffer via the playback thread includes: monitoring the occupancy information of the audio buffer via the playback thread, dynamically adjusting the speed and quantity of data reading based on the occupancy information, and reading the spare audio stream from the audio buffer based on the speed and quantity of data reading.

[0063] While the recording thread captures the backup audio stream, the playback thread is also running and continuously reading audio data from the audio buffer. Before reading data, the playback thread also needs to acquire the corresponding synchronization lock to ensure that the buffer data is not modified by the recording thread or other threads during the reading process.

[0064] Optionally, to ensure smooth playback and meet the real-time requirements of audio playback, the playback thread dynamically adjusts the speed and amount of data read based on the buffer's fill level, reading an appropriate amount of spare audio stream from the audio buffer at regular intervals. For example, if there is a lot of data in the buffer, the reading speed can be increased appropriately; if there is little data, the reading speed can be slowed down to avoid the buffer becoming empty due to excessively rapid data reading, thus preventing playback interruption.

[0065] Optionally, to ensure data synchronization between the recording and playback threads and to avoid race conditions, synchronization mechanisms such as semaphores or condition variables are used. Taking condition variables as an example, when the buffer is full, the recording thread may be blocked, waiting for a condition variable signal. When the playback thread reads a certain amount of data from the buffer, making the buffer have enough space, the playback thread sends a condition variable signal to notify the recording thread that it can continue writing data to the buffer. Similarly, when the buffer is empty, the playback thread is blocked, waiting for a signal sent by the recording thread after writing data. Through this synchronization mechanism, the two threads can work in coordination, ensuring the real-time nature and continuity of data, and avoiding problems such as data corruption or program crashes caused by data races.

[0066] In the Android system, the above process involves creating and initializing a recording thread and a playback thread using multithreaded programming techniques. This allows recording and playback tasks to execute in parallel without interference. The recording thread monitors the microphone for new audio data. Once captured, it reads the audio data as a backup audio stream through the driver interface and writes it to the audio buffer using a synchronization mechanism. This synchronization mechanism ensures the order and accuracy of data writing, avoiding data conflicts and chaos. The playback thread monitors the occupancy information of the audio buffer and dynamically adjusts the speed and amount of data read based on this information, reading an appropriate amount of backup audio stream from the audio buffer. This dynamic adjustment reading strategy can reasonably acquire data based on the real-time state of the buffer, preventing buffer overflow or reading empty data, and ensuring a stable supply of audio data.

[0067] S150, based on the mixing weights determined by the sample values ​​of the backup audio stream, the backup audio stream and the audio to be played obtained from the playback buffer corresponding to the playback thread are mixed to generate mixed audio, and the mixed audio is played.

[0068] In this embodiment, after completing the operations related to reading the backup audio stream from the audio buffer, the playback thread retrieves the audio to be played from the corresponding playback buffer. This audio may be previously mixed audio or data in a muted state. Next, the characteristics of the backup audio are analyzed in detail to determine appropriate mixing weights, while simultaneously monitoring the time difference between recording and playback and developing compensation methods. Subsequently, based on these settings, the playback thread skillfully mixes the backup audio stream and the audio to be played to generate a mixed audio, which is finally played through the speaker.

[0069] As shown in Figure 2, in one embodiment of this application, based on the mixing weight determined by the sampling value of the backup audio stream, the backup audio stream and the audio to be played obtained from the playback buffer corresponding to the playback thread are mixed to generate mixed audio, and the mixed audio is played, including: S210, obtaining the audio to be played from the playback buffer corresponding to the playback thread; wherein, the audio to be played includes the mixed audio from the previous moment or silence data; S220, analyzing the energy distribution of the backup audio based on the sampling value of the backup audio stream, and determining the mixing weight according to the energy distribution; S230, monitoring the time delay between recording and playback through a timing module, and determining a compensation strategy according to the time delay; S240, based on the mixing weight and the compensation strategy, mixing the backup audio stream and the audio to be played through the playback thread to generate mixed audio; S250, playing the mixed audio and saving the mixed audio to the playback buffer.

[0070] During its operation, the playback thread needs to acquire two key data sources. First, it retrieves the latest backup audio stream, denoted as x, from the audio data stream captured by the recording thread. This data is captured in real-time from the USB microphone and processed in the previous steps, representing the sound information currently being recorded. Second, the playback thread also needs to acquire the audio to be played, y. Specifically, y can be the mixed audio from the previous moment or silent data. For example, if there is no mixed audio currently available, or if the audio to be played is silent at the initial moment of playback, it is stored in a specific memory buffer, waiting for the playback thread to read and process it.

[0071] Specifically, the playback thread maintains its own playback buffer to store audio to be played. At the beginning of each processing cycle, the current audio to be played is read from the playback buffer. Based on the mixing weights determined by the sample values ​​of the backup audio stream, the backup audio stream and the audio to be played obtained from the playback buffer corresponding to the playback thread are mixed to generate mixed audio, which is then played.

[0072] After acquiring the backup audio stream and the audio to be played, the playback thread initiates the audio content analysis submodule, employing audio feature extraction algorithms to perform a detailed analysis of the backup audio stream. For example, it analyzes the frequency components of the backup audio to determine whether it is vocals, instrumental sounds, or other environmental sounds; it also analyzes the energy distribution of the backup audio to determine changes in sound intensity. Based on these analysis results, the mixing weights are dynamically adjusted. If the analysis reveals that vocals constitute a large proportion and are relatively clear in the current backup audio stream, the weight of the backup audio stream in the mixing is increased, making the vocals more prominent in the final mixed audio. Conversely, if the background music in the audio to be played is more important, the mixing weight of the audio to be played is appropriately increased to ensure the quality of the music. This dynamic adjustment mechanism ensures that the mixing can be optimized according to the actual situation of the audio, improving the overall audio quality.

[0073] Furthermore, a high-precision timing module continuously monitors the time delay between recording and playback, recording it as... This timing tool can be a high-precision clock interface provided by the system to accurately record the time difference between the time the audio is generated and the time the playback thread prepares to process the corresponding playback moment, thereby obtaining the time delay and ensuring the accuracy of the delay measurement.

[0074] After obtaining the time delay, the playback thread operates according to a compensation strategy determined through extensive experimentation. During the experiments, the audio synchronization effect is tested under different delay scenarios, and the optimal compensation method is found by adjusting relevant parameters. In actual processing, if a certain time delay is detected between recording and playback, a corresponding compensation strategy is determined to adjust the audio to be played according to the magnitude of the time delay. For example, when the delay is large, the volume of the audio to be played is appropriately reduced or attenuated to a certain extent to reduce the dissonance caused by the delay; when the delay is small, the adjustment will be relatively smaller. This compensation processing is based on a deep understanding of the relationship between delay and audio effect, using flexible adjustments to improve audio synchronization performance.

[0075] After weight adjustment and delay compensation, the playback thread mixes the backup audio stream and the adjusted audio to be played according to appropriate ratios. This superimposes the two different sound signals according to a certain intensity ratio to form a new audio signal, which serves as the mixed audio. The mixed audio is obtained using a specific formula. as follows: in, The mixing weights are dynamically adjusted based on the audio content. This dynamic adjustment is achieved by analyzing the characteristics of the audio data, such as frequency distribution and energy level, to determine appropriate weight values. x and y represent the backup audio stream and the audio to be played, respectively. Indicates a time delay; This represents the delay compensation coefficient, determined experimentally by simulating different delay scenarios during the experiment. Observe the audio synchronization effect after mixing, and finally choose a value that can make the audio synchronization reach the best state; Used to determine the time delay between recording and playback Adjust the attenuation of the playback sample y.

[0076] Specifically, the energy distribution of the backup audio stream is analyzed based on the sampled values ​​of the backup audio stream, and the mixing weights are determined according to the energy distribution. This includes: extracting the sampled values ​​of the backup audio stream at preset times, summing the squares of the sampled values ​​at each time to generate short-time energy; calculating the quotient between the short-time energy and a preset maximum energy reference value; and linearly processing the quotient through a preset adjustment factor to generate mixing weights.

[0077] In this embodiment, during the determination of mixing weights, features are extracted from the backup audio stream x captured by the recording thread. These audio features may include short-time energy, zero-crossing rate, or Mel-frequency cepstral coefficients. Taking short-time energy as an example, the sampled value of the backup audio stream at time m is obtained. Short-time energy reflects the energy intensity distribution change of the audio signal. Assuming that the weights are mainly adjusted based on short-time energy, an energy threshold T is set. When the short-time energy of the recorded audio x is greater than the threshold T, it is considered that the current part is the human voice portion, and the weight of the backup audio stream is increased. Accordingly, the weight of the audio y to be played in the playback thread is reduced. Specifically, mixing weights are determined based on the audio features of the backup audio stream x. for: in, This represents the sample value of the backup audio stream at time m, where N represents the frame length and n represents the frame identifier in the backup audio stream. This represents the short-time energy, or energy distribution, of the nth frame of audio. This indicates the preset maximum energy reference value. This represents the adjustment factor, ranging from 0 to 1, used to control the degree of influence of energy on the weights. When the weight is large, the weight is greatly affected by the energy of the current frame; when When the weights are smaller, the weights are relatively more stable. for The complement of.

[0078] After the mixing calculation is complete, the playback thread passes the mixed audio to the audio output device. Upon receiving the mixed audio, the output device converts it into a corresponding sound signal and plays it through speakers, etc. Simultaneously, the playback thread saves the current mixed audio into the playback buffer as the audio to be played at the next time point, enabling subsequent mixing processing and creating a smooth audio mixing effect and a continuous, stable processing flow.

[0079] It should be noted that this solution includes a memory buffer, an audio buffer, and a playback buffer. The memory buffer primarily serves to quickly and temporarily store the raw audio stream acquired from the USB microphone, providing a temporary storage location for the audio data. This ensures that data is not lost during subsequent processing and guarantees data continuity and integrity; it is the first point of contact for audio data entering the subsequent processing flow. The audio buffer coordinates the transmission and storage of audio data between the recording and playback threads. The recording thread writes the processed backup audio stream to the audio buffer, while the playback thread reads data from this buffer for subsequent mixing and other operations. It balances the differences in recording and playback speeds, ensuring a stable flow of audio data between different threads. The playback buffer primarily stores the audio data to be played, including the mixed audio and possibly the mixed audio from the previous moment or mute data. It provides a stable data source for the playback thread, ensuring that the speaker can continuously and stably play audio, avoiding audio interruptions or stuttering due to untimely data supply.

[0080] The system rapidly receives and temporarily stores the raw audio stream using a memory buffer, ensuring data integrity and preventing data loss, thus laying the foundation for subsequent processing. The audio buffer coordinates data interaction between the recording and playback threads, balancing their speed differences and ensuring orderly data flow, avoiding data chaos or waiting caused by speed mismatches. The playback buffer provides stable data for playback, guaranteeing continuous and smooth audio playback and preventing interruptions or stuttering. These three components flow data sequentially, forming a complete processing chain, with each stage working closely together from raw data capture to final playback. This design not only guarantees high efficiency and stability in audio processing but also effectively reduces audio latency and improves real-time performance. Furthermore, each buffer performs its specific function, resulting in a clear system structure that facilitates maintenance and optimization, ultimately providing users with a superior and smoother audio experience.

[0081] Through the above execution process, the audio to be played is obtained from the playback buffer corresponding to the playback thread. The frequency components and energy distribution of the backup audio are analyzed, and the mixing weights related to timbre and acoustic cavity are dynamically determined based on this information. This allows the mixing process to better match the actual audio situation and ensures the quality of the mixed audio. The timing module monitors the time delay between recording and playback, and determines a compensation strategy based on the delay. This effectively compensates for the small time difference that occurs during recording and playback, ensuring audio synchronization. Based on the mixing weights and compensation strategy, the playback thread mixes the backup audio stream and the audio to be played to generate mixed audio. Finally, the mixed audio is converted into a sound signal and played through the speaker. At the same time, the mixed audio is saved to the playback buffer to provide data support for subsequent audio processing and playback. Intelligent mixing and delay compensation can effectively improve audio quality and synchronization, bringing users a smoother and more natural recording and playback experience, and achieving low-latency synchronous recording and playback effects.

[0082] This application's technical solution ensures timely data supply by rapidly transmitting the raw audio stream acquired by the microphone to a memory buffer to generate a backup audio stream. It dynamically adjusts the audio buffer based on network and device conditions, balancing latency and stability. Recording and playback threads are created to handle audio writing and reading separately, achieving parallel operations. Finally, the mixing and playback effectively integrates the audio, achieving low-latency synchronous microphone recording and playback on the Android system, thus enhancing the audio interaction experience.

[0083] The following describes embodiments of the synchronous recording and broadcasting device based on a low-latency microphone using an Android system, which can be used to execute the synchronous recording and broadcasting method based on a low-latency microphone using an Android system described in the above embodiments of this application. It is understood that the synchronous recording and broadcasting device based on a low-latency microphone using an Android system can be a computer program (including program code) running on a computer device. For example, in industrial applications, the synchronous recording and broadcasting device based on a low-latency microphone using an Android system can install industrial application software or audio-based industrial control management software to achieve real-time industrial control based on synchronous recording and broadcasting; the synchronous recording and broadcasting device based on a low-latency microphone using an Android system can be used to execute the corresponding steps in the methods provided in the embodiments of this application. For details not disclosed in the embodiments of the synchronous recording and broadcasting device based on a low-latency microphone using an Android system of this application, please refer to the embodiments of the synchronous recording and broadcasting method based on a low-latency microphone using an Android system described above.

[0084] Optionally, the technical solution of this application can be adapted to applications based on metaverse technology. For example, edge nodes can be deployed around voice acquisition devices such as microphones, and the synchronous recording and broadcasting method of low-latency microphones based on the Android system in this embodiment can be implemented through edge computing or edge-cloud collaboration to achieve synchronous voice data and edge data transmission. Furthermore, through the synchronous recording and broadcasting method of low-latency microphones based on the Android system in this embodiment, voice input, transmission, and reception can be synchronized to achieve voice interaction and human-computer dialogue.

[0085] Figure 3 shows a block diagram of a synchronous recording and broadcasting device based on a low-latency microphone of the Android system according to an embodiment of this application.

[0086] Referring to Figure 3, a synchronous recording and playback device based on a low-latency microphone of an Android system according to an embodiment of this application includes: an acquisition module 310, used to acquire an original audio stream through a microphone, transmit the original audio stream to a memory buffer, and generate a backup audio stream; a buffer module 320, used to quantize and weightedly fuse the acquired network status parameters and device performance parameters to generate a comprehensive evaluation result, and dynamically adjust a preset audio buffer based on the comprehensive evaluation result; a writing module 330, used to create a recording thread and a playback thread in the Android system, capture the backup audio stream through the recording thread, and write it into the audio buffer; a reading module 340, used to read the backup audio stream from the audio buffer through the playback thread; and a mixing module 350, used to mix the backup audio stream and the audio to be played obtained from the playback buffer corresponding to the playback thread based on the mixing weight determined by the sampling value of the backup audio stream, generate mixed audio, and play the mixed audio.

[0087] In this application, based on the aforementioned scheme, the step of acquiring the original audio stream through a microphone and transmitting the original audio stream to a memory buffer to generate a backup audio stream includes: acquiring the original audio stream through a USB microphone; sending a data transfer request to the DMA controller; and the DMA controller transmitting the original audio stream from the USB microphone to a previously allocated memory buffer according to pre-configured memory address information to generate a backup audio stream.

[0088] In this application, based on the aforementioned scheme, the step of quantifying and weightedly fusing the acquired network status parameters and device performance parameters to generate a comprehensive evaluation result, and then dynamically adjusting the preset audio buffer based on the comprehensive evaluation result, includes: obtaining network status parameters from network bandwidth through a preset network monitoring submodule, obtaining device performance parameters from system call interfaces through a device performance monitoring submodule, and statistically analyzing historical buffer usage efficiency based on a preset data structure; quantifying the network status parameters, the device performance parameters, and the historical buffer usage efficiency, and then weightedly fusing them based on preset weights to generate a comprehensive evaluation result; and dynamically adjusting the audio buffer according to the comprehensive evaluation result and a preset mapping relationship.

[0089] In this application, based on the aforementioned scheme, the step of creating a recording thread and a playback thread in the Android system, capturing the backup audio stream through the recording thread, and writing it into the audio buffer includes: creating and initializing a recording thread and a playback thread in the Android system based on multi-threaded programming technology; listening to whether the microphone has captured new audio data through the recording thread, and if so, reading the backup audio stream from the memory buffer through the driver interface; and writing the backup audio stream into the audio buffer based on a synchronization mechanism.

[0090] In this application, based on the aforementioned scheme, the step of reading the spare audio stream from the audio buffer through the playback thread includes: monitoring the occupancy information of the audio buffer through the playback thread, dynamically adjusting the speed and quantity of data reading based on the occupancy information, and reading the spare audio stream from the audio buffer based on the speed and quantity of data reading.

[0091] In this application, based on the aforementioned scheme, the mixing weight determined based on the sampled values ​​of the backup audio stream, the mixing processing of the backup audio stream and the audio to be played obtained from the playback buffer corresponding to the playback thread to generate mixed audio, and the playback of the mixed audio, includes: obtaining the audio to be played from the playback buffer corresponding to the playback thread; wherein, the audio to be played includes the mixed audio from the previous moment or silence data; analyzing the energy distribution of the backup audio based on the sampled values ​​of the backup audio stream, and determining the mixing weight according to the energy distribution; monitoring the time delay between recording and playback through a timing module, and determining a compensation strategy according to the time delay; mixing the backup audio stream and the audio to be played through the playback thread based on the mixing weight and the compensation strategy to generate mixed audio; playing the mixed audio, and saving the mixed audio to the playback buffer.

[0092] In this application, based on the aforementioned scheme, playing the mixed audio includes: converting the mixed audio into a sound signal through the playback thread; and playing the sound signal through a speaker.

[0093] This application's technical solution ensures timely data supply by rapidly transmitting the raw audio stream acquired by the microphone to a memory buffer to generate a backup audio stream. It dynamically adjusts the audio buffer based on network and device conditions, balancing latency and stability. Recording and playback threads are created to handle audio writing and reading separately, achieving parallel operations. Finally, the mixing and playback effectively integrates the audio, achieving low-latency synchronous microphone recording and playback on the Android system, thus enhancing the audio interaction experience.

[0094] Figure 4 shows a schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application.

[0095] It should be noted that the computer system of the electronic device in this embodiment is only an example and should not impose any limitations on the function and scope of use of the embodiments of this application.

[0096] In this embodiment, the computer system includes a central processing unit 401, which can perform various appropriate actions and processes based on programs stored in read-only memory 402 or programs loaded from storage section 408 into random access memory 403, such as executing the synchronous recording and broadcasting method of low-latency microphone based on the Android system described in the above embodiment. The random access memory 403 also stores various programs and data required for system operation, thereby realizing big data storage and big data management. The central processing unit 401, read-only memory 402, and random access memory 403 are interconnected via bus 404. Input / output interface 405 is also connected to bus 404.

[0097] The following components are connected to the input / output interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 410 as needed so that computer programs read from it can be installed into the storage section 408 as needed.

[0098] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit 401, it performs various functions defined in the system of this application.

[0099] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0100] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0101] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations described above.

[0102] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the synchronous recording and broadcasting method of the low-latency microphone based on the Android system described in the above embodiments.

[0103] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0104] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A synchronous recording and broadcasting method based on a low-latency microphone of the Android system, characterized in that, include: The original audio stream is acquired through a microphone and transmitted to a memory buffer to generate a backup audio stream; The acquired network status parameters and device performance parameters are quantified and weighted and fused to generate a comprehensive evaluation result. Based on the comprehensive evaluation result, the preset audio buffer is dynamically adjusted. In the Android system, a recording thread and a playback thread are created. The recording thread captures the backup audio stream and writes it to the audio buffer. The playback thread reads the backup audio stream from the audio buffer; based on the mixing weights determined by the sample values ​​of the backup audio stream, it performs mixing processing on the backup audio stream and the audio to be played obtained from the playback buffer corresponding to the playback thread to generate mixed audio, and plays the mixed audio.

2. The synchronous recording and broadcasting method based on a low-latency microphone of the Android system according to claim 1, characterized in that, The process of acquiring a raw audio stream via a microphone and transferring the raw audio stream to a memory buffer to generate a backup audio stream includes: acquiring a raw audio stream via a USB microphone; sending a data transfer request to a DMA controller; and the DMA controller transferring the raw audio stream from the USB microphone to a previously allocated memory buffer according to pre-configured memory address information to generate a backup audio stream.

3. The synchronous recording and broadcasting method based on a low-latency microphone of the Android system according to claim 1, characterized in that, The acquired network status parameters and device performance parameters are quantified and weighted to generate a comprehensive evaluation result. Based on the comprehensive evaluation result, a preset audio buffer is dynamically adjusted. This includes: obtaining network status parameters from network bandwidth through a preset network monitoring submodule, obtaining device performance parameters from system call interfaces through a device performance monitoring submodule, and statistically analyzing historical buffer usage efficiency based on a preset data structure; quantifying the network status parameters, device performance parameters, and historical buffer usage efficiency, and weighting and fusing them based on preset weights to generate a comprehensive evaluation result; and dynamically adjusting the audio buffer according to the comprehensive evaluation result and a preset mapping relationship.

4. The synchronous recording and broadcasting method based on a low-latency microphone of the Android system according to claim 1, characterized in that, Creating a recording thread and a playback thread in the Android system, capturing the backup audio stream through the recording thread, and writing it into the audio buffer includes: creating and initializing the recording thread and the playback thread based on multi-threaded programming technology in the Android system; listening to whether the microphone has captured new audio data through the recording thread, and if so, reading the backup audio stream from the memory buffer through the driver interface; and writing the backup audio stream into the audio buffer based on a synchronization mechanism.

5. The synchronous recording and broadcasting method based on a low-latency microphone of the Android system according to claim 1, characterized in that, Reading the backup audio stream from the audio buffer via the playback thread includes: monitoring the occupancy information of the audio buffer via the playback thread, dynamically adjusting the speed and quantity of data reading based on the occupancy information, and reading the backup audio stream from the audio buffer based on the speed and quantity of data reading.

6. The synchronous recording and broadcasting method based on a low-latency microphone of the Android system according to claim 1, characterized in that, Based on the mixing weights determined by the sampled values ​​of the backup audio stream, the backup audio stream and the audio to be played obtained from the playback buffer corresponding to the playback thread are mixed to generate mixed audio, and the mixed audio is played. This includes: obtaining the audio to be played from the playback buffer corresponding to the playback thread; wherein the audio to be played includes the mixed audio from the previous moment or silence data; analyzing the energy distribution of the backup audio based on the sampled values ​​of the backup audio stream, and determining the mixing weights according to the energy distribution; monitoring the time delay between recording and playback through a timing module, and determining a compensation strategy according to the time delay; based on the mixing weights and the compensation strategy, mixing the backup audio stream and the audio to be played through the playback thread to generate mixed audio; playing the mixed audio, and saving the mixed audio to the playback buffer.

7. The synchronous recording and broadcasting method based on a low-latency microphone of the Android system according to claim 6, characterized in that, Analyzing the energy distribution of the backup audio stream based on the sampled values, and determining the mixing weights based on the energy distribution, includes: extracting the sampled values ​​of the backup audio stream at preset times, and summing the squares of the sampled values ​​at each time to generate short-time energy; calculating the quotient between the short-time energy and a preset maximum energy reference value; and linearly processing the quotient using a preset adjustment factor to generate mixing weights.

8. A synchronous recording and broadcasting device based on a low-latency microphone of the Android system, characterized in that, include: The acquisition module is used to acquire the original audio stream through the microphone, transmit the original audio stream to the memory buffer, and generate a backup audio stream. The buffer module is used to quantify and weightedly fuse the acquired network status parameters and device performance parameters to generate a comprehensive evaluation result, and dynamically adjust the preset audio buffer based on the comprehensive evaluation result. The writing module is used to create a recording thread and a playback thread in the Android system, capture the backup audio stream through the recording thread, and write it into the audio buffer; A reading module is used to read the spare audio stream from the audio buffer via the playback thread; The mixing module is used to perform mixing processing on the backup audio stream and the audio to be played obtained from the playback buffer corresponding to the playback thread based on the mixing weight determined by the sampling value of the backup audio stream, generate mixed audio, and play the mixed audio.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the synchronous recording and broadcasting method of the low-latency microphone based on the Android system as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the synchronous recording and broadcasting method of a low-latency microphone based on the Android system as described in any one of claims 1 to 7.

Images