Audio synchronous acquisition and transmission processing system

Abstract

The embodiment of the invention provides an audio synchronous acquisition and transmission processing system. The system comprises an image acquisition module, an audio acquisition module, an image processing module, an audio processing module, a storage module, a signal processing and transmission module, a power management module and a power supply module, wherein the image acquisition module and the audio acquisition module keep synchronous through a unified clock source in the acquisition process, the image processing module and the audio processing module coordinate a processing time sequence through the synchronous controller based on a timestamp mechanism, and the signal processing and transmission module performs resynchronization verification on a composite data stream before transmission so as to ensure audio and video synchronization. The synchronization of acquisition starting points is ensured through a unified clock source, the time sequence unification of acquisition and processing links is realized in combination with a timestamp coordination mechanism of a synchronous controller, and good cross-platform compatibility and energy efficiency performance are achieved.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of communication technology, and in particular to an audio synchronous acquisition and transmission processing system. Background Technology

[0002] Existing audio and video acquisition and transmission systems have inherent defects in synchronization. Since image and audio signals are processed through independent channels, timing deviations are prone to occur. Traditional systems use simple timestamp alignment methods, which are difficult to eliminate the cumulative delay between the acquisition module and the processing module. In complex application scenarios, the compatibility of existing systems is limited, and they cannot adapt to different operating systems and communication software environments. The lack of coordination between power management strategies and data processing flows affects the continuous working performance of the system. These factors together lead to insufficient audio and video synchronization accuracy and poor transmission stability.

[0003] Therefore, a better solution is urgently needed. Summary of the Invention

[0004] In view of this, the embodiments of this specification provide an audio synchronous acquisition and transmission processing system to solve the technical defects existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, an audio synchronization acquisition and transmission processing system is provided, comprising: The image acquisition module is used to acquire optical images and convert them into digital image signals; The audio acquisition module is used to collect ambient sound and convert it into digital audio signals; The image processing module is used to perform noise reduction, sharpening, and color correction on digital image signals; The audio processing module is used for sampling, encoding, and compressing digital audio signals; The storage module is used to temporarily store the acquired digital image signals and digital audio signals, as well as the processed data; The signal processing and transmission module is used to package the processed image and audio data into a composite data stream and transmit it to a computer device via a USB interface. The power management module is used to provide power to all modules of the system. The image acquisition module and audio acquisition module maintain synchronization through a unified clock source during the acquisition process. The image processing module and audio processing module coordinate the processing timing through a synchronization controller based on a timestamp mechanism. The signal processing and transmission module performs resynchronization verification on the composite data stream before transmission to ensure audio and video synchronization.

[0006] In one possible implementation, the image acquisition module supports multiple image formats, including MJEPG, H.264, and YUV, and the resolution is adjustable to meet the needs of different application scenarios.

[0007] In one possible implementation, the audio acquisition module uses a microphone array to collect ambient sound. The microphone array is integrated with the image acquisition module and the synchronization of the acquisition start time is ensured by a unified clock source.

[0008] In one possible implementation, the image processing module performs noise reduction to eliminate random noise introduced by the image sensor, sharpening to enhance edge details, and color correction to adjust white balance and gamma values ​​to optimize image output quality.

[0009] In one possible implementation, the audio processing module uses a configurable sampling rate to adapt to different audio quality requirements, the encoding process converts the audio signal into PCM or AAC format, and the compression process reduces the amount of data through lossless or lossy algorithms.

[0010] In one possible implementation, the storage module uses non-volatile memory that supports high-speed read and write operations. It is used to buffer data streams and store configuration parameters and synchronization logs, facilitating subsequent diagnosis and optimization.

[0011] In one possible implementation, the signal processing and transmission module supports USB 3.0 or later transmission modes and has a dynamic bandwidth adjustment mechanism to adapt to the performance of different USB ports.

[0012] In one possible implementation, the power management module employs a multi-stage power design including a voltage regulator circuit and a power consumption control unit, dynamically adjusting the power supply strategy according to the system load, entering a low-power mode when idle and switching to full-power mode during data acquisition and transmission.

[0013] In one possible implementation, the resynchronization check step includes calculating the audio / video synchronization error index E, which is calculated as follows:

[0014] in, Let represent the synchronization error of the i-th audio / video pair, calculated as = - , It is the output timestamp of the i-th image frame. It is the output timestamp of the i-th audio block, which is generated from the synchronization controller; The window size is obtained from the configuration parameters in the storage module; The average synchronization error is expressed as follows: .

[0015] In one possible implementation, the resynchronization verification step includes calculating the synchronization adjustment amount A, which is calculated as follows:

[0016] in, Let represent the synchronization error of the i-th audio / video pair, calculated as = - , It is the output timestamp of the i-th image frame. It is the output timestamp of the i-th audio block, which is generated from the synchronization controller; This represents the average processing latency difference, calculated as follows: ; The processing delay for the i-th image frame is represented by , and is calculated as . , It is the acquisition timestamp of the i-th image frame, generated from the image acquisition module; The processing delay of the i-th audio block is represented by , calculated as . , It is the acquisition timestamp of the i-th audio block, generated from the audio acquisition module; the acquisition timestamp is based on a unified clock source; The window size is obtained from the configuration parameters in the storage module.

[0017] This specification provides an audio synchronization acquisition and transmission processing system, comprising: an image acquisition module for acquiring optical images and converting them into digital image signals; an audio acquisition module for acquiring ambient sounds and converting them into digital audio signals; an image processing module for performing noise reduction, sharpening, and color correction on the digital image signals; an audio processing module for sampling, encoding, and compressing the digital audio signals; a storage module for temporarily storing the acquired digital image signals, digital audio signals, and processed data; a signal processing and transmission module for packaging the processed image and audio data into a composite data stream and transmitting it to a computer device via a USB interface; and a power management module for providing power to all modules of the system. The image acquisition module and audio acquisition module maintain synchronization during acquisition using a unified clock source. The image processing module and audio processing module coordinate processing timing based on a timestamp mechanism using a synchronization controller. The signal processing and transmission module performs a re-synchronization check on the composite data stream before transmission to ensure audio and video synchronization. A unified clock source ensures synchronization of the acquisition start point, and the timing of acquisition and processing is unified by the timestamp coordination mechanism of the synchronization controller. The use of composite data stream packaging and resynchronization verification steps effectively eliminates accumulated errors during transmission. The modular design enhances system adaptability and supports multiple image formats and audio encoding standards. The combination of dynamic power management strategy and processing flow optimization ensures stable operation of the system under different load conditions. Overall, high-precision audio and video synchronization transmission is achieved, while also possessing good cross-platform compatibility and energy efficiency. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of an audio synchronous acquisition and transmission processing system provided in one embodiment of this specification. Detailed Implementation

[0019] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0020] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0021] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0022] This specification provides an audio synchronization acquisition and transmission processing system, which will be described in detail in the following embodiments.

[0023] See Figure 1 , Figure 1This diagram illustrates a system schematic of an audio synchronization acquisition and transmission processing system according to an embodiment of this specification. Specifically, it includes an image acquisition module for acquiring optical images and converting them into digital image signals; an audio acquisition module for acquiring ambient sound and converting it into digital audio signals; an image processing module for performing noise reduction, sharpening, and color correction on the digital image signals; an audio processing module for sampling, encoding, and compressing the digital audio signals; a storage module for temporarily storing the acquired digital image signals, digital audio signals, and processed data; a signal processing and transmission module for packaging the processed image and audio data into a composite data stream and transmitting it to a computer device via a USB interface; and a power management module for providing power to all modules of the system. The image acquisition module and audio acquisition module maintain synchronization during acquisition using a unified clock source. The image processing module and audio processing module coordinate processing timing based on a timestamp mechanism using a synchronization controller. The signal processing and transmission module performs a re-synchronization check on the composite data stream before transmission to ensure audio and video synchronization.

[0024] In practical applications, an image acquisition module can refer to a hardware or software component that acquires optical images and converts them into digital image signals. An audio acquisition module can refer to a hardware or software component that acquires ambient sound and converts it into digital audio signals. An image processing module can refer to a component that performs noise reduction, sharpening, and color correction on digital image signals. An audio processing module can refer to a component that samples, encodes, and compresses digital audio signals. A storage module can refer to a memory that temporarily stores the acquired digital image signals, digital audio signals, and processed data. A signal processing and transmission module can refer to a component that packages the processed image and audio data into a composite data stream and transmits it to a computer device via a USB interface. A power management module can refer to a power management component that provides power to all modules of the system. A unified clock source refers to a clock source that provides a unified time reference to ensure synchronized image and audio acquisition. A synchronization controller refers to a controller that coordinates the processing timing of the image and audio processing modules based on a timestamp mechanism. A timestamp mechanism refers to a mechanism that assigns time tags to image frames and audio blocks for synchronized processing. A composite data stream refers to a composite data stream containing packaged image and audio data for easy transmission. A USB interface refers to a Universal Serial Bus interface used for data transmission, allowing the composite data stream to be transmitted to computer devices. Resynchronization verification refers to the process of performing synchronization verification on the composite data stream before transmission to ensure audio and video synchronization.

[0025] As a concrete example: Upon system startup, the power management module provides 5V DC power. The image acquisition module acquires optical images at 1920x1080 resolution using a CMOS sensor, converting them into YUV format digital image signals. Simultaneously, the audio acquisition module acquires ambient sound using a microphone array at a 48kHz sampling rate, converting it into PCM format digital audio signals. Both modules use a unified clock source to ensure consistent acquisition start times. The acquired data is temporarily stored in the DDR4 memory of the storage module. The image processing module performs noise reduction on the digital image signals using Gaussian filtering, sharpening using the Laplacian operator, and color correction, adjusting the white balance to 6500K. The audio processing module performs sample-and-hold, encoding, and compression using the MP3 algorithm on the digital audio signals. The synchronization controller assigns millisecond-level time stamps to each frame of image and audio block based on a timestamp mechanism, coordinating processing timing. The processed data is packaged into a composite data stream by the signal processing and transmission module, embedding synchronization information, and transmitted to the running Windows system via a USB 3.0 interface. The computer with a capacity of 10 performs a resynchronization check before transmission and calculates the synchronization adjustment amount to ensure that the audio and video latency is less than 10 milliseconds. Throughout the process, the power management module dynamically adjusts the power consumption according to the load and switches to a 1.8V low-power mode when idle.

[0026] This application achieves precise timing coordination between the acquisition and processing stages through a unified clock source and synchronization controller, avoiding audio and video misalignment; the modular design supports multiple formats and resolutions, enhancing system compatibility and flexibility; the resynchronization verification step ensures high-quality synchronization of transmitted data, improving user experience; optimized power management extends equipment lifespan and adapts to different application scenarios; the overall system achieves efficient and stable audio and video acquisition and transmission.

[0027] In one possible implementation, the image acquisition module supports multiple image formats, including MJEPG, H.264, and YUV, and the resolution is adjustable to meet the needs of different application scenarios.

[0028] In practical applications, MJEPG can refer to a motion picture compression format based on JPEG, used to reduce image data size for easier storage and transmission. H.264 can refer to an efficient video coding standard that provides a high compression ratio while maintaining video quality. YUV can refer to a color space model that separates luminance and chrominance information to optimize image processing. Resolution can refer to the dimensionality of pixels in an image and can be adjusted to suit different display or storage needs.

[0029] As a concrete example: the image acquisition module uses a CMOS sensor to acquire optical images at adjustable resolutions, such as 640x480 for low-bandwidth scenarios or 1920x1080 for high-definition recording, and converts the raw data into YUV color format; subsequently, depending on the application requirements, the image processing module encodes the YUV data into MJPEG format for still image compression or H.264 format for dynamic video streaming, dynamically switching the resolution to 720p to adapt to video conferencing scenarios or 4K for professional recording; the processed image data is packaged by the signal processing and transmission module and output to the computer via USB interface. The entire process is synchronized with other modules through a synchronous controller.

[0030] This application enhances the system's flexibility and adaptability by supporting multiple image formats and adjustable resolutions, enabling it to meet the needs of different application scenarios such as video conferencing or high-definition recording. Multi-format compatibility improves data exchange efficiency and reduces storage and transmission overhead. The resolution adjustment function optimizes resource utilization, ensuring image quality is maintained in various environments. Overall, it improves the system's usability and user experience.

[0031] In one possible implementation, the audio acquisition module uses a microphone array to collect ambient sound. The microphone array is integrated with the image acquisition module and the synchronization of the acquisition start time is ensured by a unified clock source.

[0032] In practical applications, a microphone array refers to a system composed of multiple microphone units used to acquire ambient sound through beamforming technology. Ambient sound refers to sound signals in the surrounding environment, including speech and background noise, which can be captured by the audio acquisition module and converted into digital audio signals. Synchronization at the start of acquisition refers to the consistency of the start times of image and audio acquisition, which can be achieved through a unified clock source to avoid timing deviations.

[0033] As a concrete example: the audio acquisition module uses an array of four microphones to acquire ambient sounds, such as human voices and background noise in a conference room, at a sampling rate of 48kHz, and enhances directional acquisition through beamforming technology; the microphone array is integrated with the CMOS sensor hardware of the image acquisition module, and the two generate a synchronous trigger signal at the start of each acquisition through a unified clock source, ensuring that the error at the start of acquisition is less than 1 millisecond; the acquired ambient sounds are converted into PCM format digital audio signals, which are temporarily stored in the storage module along with the image data, and the timing is subsequently coordinated by the synchronization controller.

[0034] This application achieves high-precision synchronization at the start of acquisition by integrating a microphone array with an image acquisition module and using a unified clock source, effectively reducing initial audio and video deviations. Beamforming technology improves the quality of ambient sound acquisition and enhances the system's adaptability in noisy environments. Overall, it improves the synchronization and reliability of audio and video data, meeting the needs of real-time communication and recording applications.

[0035] In one possible implementation, the image processing module performs noise reduction to eliminate random noise introduced by the image sensor, sharpening to enhance edge details, and color correction to adjust white balance and gamma values ​​to optimize image output quality.

[0036] In practical applications, an image sensor refers to a semiconductor device that converts optical images into electrical signals to capture raw image data. Random noise refers to random interference signals generated by the image sensor during signal conversion, which can be eliminated through noise reduction processing to improve image clarity. Edge details refer to the high-frequency information of object contours and textures in an image, which can be enhanced through sharpening processing to improve visual clarity. White balance refers to the process of adjusting the color temperature of an image to present true colors, used to correct color deviations and ensure color accuracy. Gamma value refers to a parameter describing the nonlinear relationship between image brightness and signal voltage, which can be adjusted to optimize image contrast and brightness performance.

[0037] As a concrete example: The image processing module receives YUV format data from the CMOS image sensor. First, it uses a median filtering algorithm to reduce random noise, lowering the signal-to-noise ratio to below 30dB. Then, it uses the Laplacian operator for sharpening to enhance the detail of the contours of people and the edges of text in the image. Finally, it performs color correction, adjusting the white balance to the standard color temperature of 5500K and setting the gamma value to 2.2, so that the output image can be accurately reproduced in the sRGB color space. The processed image and synchronized audio data are then transmitted to the computer after being coordinated by a synchronization controller.

[0038] This application effectively improves image quality through a systematic image processing workflow, significantly reduces noise interference introduced by the sensor through noise reduction processing, enhances the image detail representation through sharpening processing, ensures true and natural color reproduction through color correction, and optimizes the overall image output quality while ensuring synchronization and coordination with audio data.

[0039] In one possible implementation, the audio processing module uses a configurable sampling rate to adapt to different audio quality requirements, the encoding process converts the audio signal into PCM or AAC format, and the compression process reduces the amount of data through lossless or lossy algorithms.

[0040] In practical applications, an audio processing module can refer to a component that samples, encodes, and compresses digital audio signals to execute the audio signal processing flow. Sampling processing refers to the process of discretizing audio signals, converting continuous audio signals into digital sequences. The sampling rate refers to the number of audio samples collected per second, used to determine the frequency response and sound quality of the audio signal. Audio quality requirements refer to specific requirements for audio output quality, such as fidelity or bandwidth, to adapt to different application scenarios. Encoding processing refers to the process of converting audio data into a specific format to compress or standardize audio signals. An audio signal can refer to an electrical or digital signal representing sound that can be processed and analyzed. PCM format refers to Pulse Code Modulation format, used for lossless storage of audio data. AAC format refers to Advanced Audio Coding Format, providing efficient lossy compression. Compression processing refers to the process of reducing the amount of audio data to save storage space and transmission bandwidth. Lossless algorithms refer to algorithms that can completely recover the original data after compression, used to maintain audio quality. Lossy algorithms refer to algorithms that lose some data after compression, significantly reducing the amount of data. Data volume can refer to the size or volume of audio data, which can affect storage and transmission efficiency.

[0041] As a concrete example: The audio processing module receives digital audio signals from the audio acquisition module, first performs sampling processing, using a configurable sampling rate such as 44.1kHz for high-fidelity music or 8kHz for voice communication to adapt to different audio quality requirements; then it performs encoding processing, converting the sampled audio signal into PCM format for original data preservation or AAC format for streaming media transmission; finally, it performs compression processing, reducing the data size to 60% of the original size using lossless algorithms such as FLAC or reducing the data size to 10% of the original size using lossy algorithms such as MP3. The processed audio data and image data are then transmitted to the computer device after being coordinated by a synchronization controller.

[0042] This application flexibly adapts to different audio quality requirements and improves system applicability through configurable sampling rates and support for multiple encoding formats; compression processing effectively reduces data volume and optimizes storage and transmission efficiency; overall, it enhances the flexibility and performance of audio processing and ensures synchronous coordination with image data.

[0043] In one possible implementation, the storage module uses non-volatile memory that supports high-speed read and write operations. It is used to buffer data streams and store configuration parameters and synchronization logs, facilitating subsequent diagnosis and optimization.

[0044] In practical applications, non-volatile memory refers to storage media that retains data even after power failure, used for persistent storage of critical system data. High-speed read / write refers to rapid data access capabilities, improving data transmission efficiency. Buffered data streams refer to the process of temporarily storing flowing data to balance differences in data processing speeds. Configuration parameters refer to the settings required for system operation, used to adjust the working status of various modules. Synchronization logs refer to log files that record timing synchronization information, assisting in troubleshooting system problems. Post-diagnosis refers to the later analysis process of the system's operating status, used to identify potential faults. Optimization refers to the process of improving system performance, which can improve overall work efficiency.

[0045] As a concrete example: the storage module uses NVMe solid-state drives as non-volatile memory, achieving a high-speed read and write speed of 3.5GB per second, continuously buffering 1080p video streams from the image acquisition module and 48kHz audio data streams from the audio acquisition module; it also stores configuration parameters such as resolution settings and sampling rate parameters, as well as timestamp synchronization logs generated by the synchronization controller; when the system detects audio and video synchronization deviations, it performs subsequent diagnosis by reading the stored synchronization logs, analyzes the latency generation points, and adjusts the parameter configuration of the synchronization controller accordingly to optimize the system.

[0046] This application ensures the reliability and access efficiency of data storage by using high-performance non-volatile memory, guarantees smooth data flow processing by high-speed read and write characteristics, and provides complete data support for system diagnosis and optimization by storing configuration parameters and synchronous logs, thereby enhancing the overall system stability and maintainability.

[0047] In one possible implementation, the signal processing and transmission module supports USB 3.0 or later transmission modes and has a dynamic bandwidth adjustment mechanism to adapt to the performance of different USB ports.

[0048] In practical applications, USB 3.0 refers to the third generation standard of the Universal Serial Bus, used to support high-speed data transmission at rates of 5 gigabits per second and above. Transmission mode refers to the protocol used for data transmission, defining how data packets are sent and received to adapt to different needs. Dynamic bandwidth adjustment mechanism refers to a system that automatically adjusts bandwidth allocation based on real-time conditions, optimizing data transmission efficiency and avoiding congestion. USB port performance refers to the data transmission capability of the USB interface, affecting connection stability and speed consistency.

[0049] As a concrete example: the signal processing and transmission module detects the type of USB port connected to the computer. If it identifies a USB 3.0 port, it enables ultra-high-speed transmission mode and sends composite data streams at a rate of 5Gbps. If a USB 2.0 port is detected, it limits the transmission rate to 480Mbps through a dynamic bandwidth adjustment mechanism and switches to bulk transmission mode to ensure data integrity. Throughout the process, it monitors USB port performance indicators such as signal strength and bit error rate in real time, dynamically adjusts packet size and buffering strategies, and achieves a stable connection with the computer device.

[0050] This application effectively adapts to the performance of different USB ports by supporting high-speed USB transmission mode and dynamic bandwidth adjustment mechanism, thereby improving the reliability and efficiency of data transmission; enhancing system compatibility and ensuring stable operation in various hardware environments; and optimizing the overall transmission quality of audio and video streams, thus improving the user experience.

[0051] In one possible implementation, the power management module employs a multi-stage power design including a voltage regulator circuit and a power consumption control unit, dynamically adjusting the power supply strategy according to the system load, entering a low-power mode when idle and switching to full-power mode during data acquisition and transmission.

[0052] In practical applications, multi-stage power supply design refers to a power supply architecture that includes multiple voltage conversion stages to provide stable and adjustable power output. A voltage regulator circuit refers to an electronic circuit that maintains a constant output voltage, eliminating the impact of input voltage fluctuations. A power consumption control unit refers to a functional module that manages power consumption, monitoring and adjusting the energy consumption of each module in real time. System load refers to the combined workload of all modules in the current system, used as a basis for adjusting power supply strategies. Dynamic power supply strategy adjustment refers to a method of changing power supply configuration based on real-time conditions, optimizing the balance between energy efficiency and performance. Low-power mode refers to an energy-saving state that reduces operating frequency and voltage to reduce energy consumption during standby. Full-power mode refers to a full-load state where all modules operate at rated parameters, ensuring the performance requirements of data processing and transmission.

[0053] As a concrete example: the power management module adopts a multi-stage power supply design that includes a DC-DC converter and an LDO regulator. When the system is idle, the power consumption control unit reduces the core voltage to 1.2V and the clock frequency to 100MHz, entering a low-power mode, at which point the total power consumption does not exceed 0.5W. When the image acquisition module starts acquiring 1080p video and the audio acquisition module works synchronously, the system load reaches its peak, and the power consumption control unit immediately switches to full-power mode, increasing the core voltage to 3.3V and the clock frequency to 500MHz, and ensuring that the voltage fluctuation does not exceed ±2% through the voltage regulation circuit. Throughout the process, the power supply strategy is dynamically adjusted to complete the mode switching in milliseconds according to the real-time load changes, ensuring that the system maintains the best energy efficiency performance throughout the video conference.

[0054] This application achieves high-efficiency management of the system under different working conditions through multi-level power supply design and intelligent power consumption control, significantly improving the device's battery life; dynamic power supply strategy ensures an optimized balance between performance and power consumption, enhancing system adaptability; voltage regulation circuit ensures power quality and improves system operational stability, achieving overall energy-saving and efficient power utilization.

[0055] In one possible implementation, the resynchronization check step includes calculating the audio / video synchronization error index E, which is calculated as follows:

[0056] in, Let represent the synchronization error of the i-th audio / video pair, calculated as = - , It is the output timestamp of the i-th image frame. It is the output timestamp of the i-th audio block, which is generated from the synchronization controller; The window size is obtained from the configuration parameters in the storage module; The average synchronization error is expressed as follows: .

[0057] In practical applications, the resynchronization check step refers to the process of synchronizing and checking the composite data stream before transmission to ensure audio and video synchronization. The audio / video synchronization error index E is a comprehensive indicator that quantifies audio and video synchronization deviation and can evaluate overall synchronization performance. Synchronization error refers to the time difference of the i-th audio / video pair, reflecting the synchronization status of a single data pair. The output timestamp refers to the output time marker of the i-th image frame, marking the completion time of image processing. The output timestamp refers to the output time marker of the i-th audio block, marking the completion time of audio processing. The synchronization controller refers to a controller that coordinates processing timing based on a timestamp mechanism and can generate output timestamps. The window size refers to the number of samples used to calculate the synchronization error, controlling the scope of statistical evaluation. The storage module refers to a component that stores system configuration parameters, providing the window size. The average synchronization error refers to the average value of the synchronization errors, representing the overall deviation level.

[0058] As a concrete example: In the resynchronization verification step, the synchronization controller reads the configuration parameter of window size N=100 from the storage module and obtains the output timestamps of the most recent 100 image frames and audio blocks. and ,For example For 150ms The time interval is 148ms, and the synchronization error is calculated to be 2ms. Then, the average synchronization error is calculated, assuming an average synchronization error of 1.8ms. Next, the audio and video synchronization error index E is calculated. If E exceeds the threshold of 5ms, the adjustment mechanism is triggered. Throughout the process, resynchronization verification ensures that the audio and video synchronization error of the composite data stream is controlled within the allowable range before transmission through the USB interface.

[0059] This application achieves precise quantification of synchronization performance by calculating the audio and video synchronization error index, thereby improving the synchronization accuracy and stability of audio and video data. The resynchronization verification step, combined with multi-parameter evaluation, effectively reduces latency and misalignment, enhancing the reliability of the system in real-time transmission. Overall, it optimizes the user experience and ensures the consistency and adaptability of the audio and video acquisition and transmission system in various application scenarios.

[0060] In one possible implementation, the resynchronization verification step includes calculating the synchronization adjustment amount A, which is calculated as follows:

[0061] in, Let represent the synchronization error of the i-th audio / video pair, calculated as = - , It is the output timestamp of the i-th image frame. It is the output timestamp of the i-th audio block, which is generated from the synchronization controller; This represents the average processing latency difference, calculated as follows: ; The processing delay for the i-th image frame is represented by , and is calculated as . , It is the acquisition timestamp of the i-th image frame, generated from the image acquisition module; The processing delay of the i-th audio block is represented by , calculated as . , It is the acquisition timestamp of the i-th audio block, generated from the audio acquisition module; the acquisition timestamp is based on a unified clock source; The window size is obtained from the configuration parameters in the storage module.

[0062] In practical applications, synchronization adjustment can refer to adjustment parameters used to correct synchronization deviations, guiding synchronization control operations. Average processing delay difference can refer to the average difference between the delays of the image and audio processing paths, used to identify system processing asymmetry. Processing delay can refer to the time interval from acquisition to completion of processing of the i-th image frame, reflecting image processing efficiency. Processing delay can refer to the time interval from acquisition to completion of processing of the i-th audio block, reflecting audio processing efficiency. Acquisition timestamp can refer to the moment when acquisition of the i-th image frame begins, used to mark the start point of image acquisition. Acquisition timestamp can refer to the moment when acquisition of the i-th audio block begins, marking the start point of audio acquisition. A unified clock source can refer to a clock module that provides a global time reference, ensuring time consistency across modules.

[0063] As a concrete example: In the synchronization verification step, the system obtains the configuration parameter of window size N=60 from the storage module. The synchronization controller provides the output timestamps of the most recent 60 image frames and audio blocks, for example, image frame output timestamp = 200ms, audio block output timestamp = 197ms, and the synchronization error is calculated to be 3ms. Simultaneously, it obtains the acquisition timestamp = 180ms from the image acquisition module, calculating the image processing delay as 20ms, and the acquisition timestamp = 179ms from the audio acquisition module, calculating the audio processing delay as 18ms. Then, it calculates the average processing delay difference, assuming... =2.5ms; Finally, the synchronization adjustment amount A is calculated. When the value of A exceeds the predetermined threshold, the system automatically adjusts the parameter settings of the synchronization controller to achieve real-time correction.

[0064] This application achieves accurate evaluation and dynamic correction of the system's synchronization state by calculating synchronization adjustment amounts, effectively improving the synchronization accuracy of audio and video data; it comprehensively considers multi-dimensional parameters such as processing delay differences and synchronization errors to enhance the system's adaptive capabilities; and it optimizes the synchronization control strategy to ensure stable synchronization performance under various workloads, thereby improving the overall system reliability and user experience.

[0065] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0066] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0067] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. An audio synchronous acquisition and transmission processing system, characterized in that, include: The image acquisition module is used to acquire optical images and convert them into digital image signals; The audio acquisition module is used to collect ambient sound and convert it into digital audio signals; The image processing module is used to perform noise reduction, sharpening, and color correction on the digital image signal; An audio processing module is used to sample, encode, and compress the digital audio signal; A storage module is used to temporarily store the acquired digital image signals and digital audio signals, as well as the processed data; The signal processing and transmission module is used to package the processed image and audio data into a composite data stream and transmit it to a computer device via a USB interface. The power management module is used to provide power to all modules of the system. The image acquisition module and the audio acquisition module maintain synchronization through a unified clock source during the acquisition process. The image processing module and the audio processing module coordinate the processing timing through a synchronization controller based on a timestamp mechanism. The signal processing and transmission module performs a resynchronization check on the composite data stream before transmission to ensure audio and video synchronization.

2. The audio synchronous acquisition and transmission processing system according to claim 1, characterized in that, The image acquisition module supports multiple image formats, including MJPEG, H.264, and YUV, and its resolution is adjustable to meet the needs of different application scenarios.

3. The audio synchronous acquisition and transmission processing system according to claim 1, characterized in that, The audio acquisition module uses a microphone array to collect ambient sound. The microphone array is integrated with the image acquisition module and the synchronization of the acquisition start time is ensured by the unified clock source.

4. The audio synchronous acquisition and transmission processing system according to claim 1, characterized in that, The image processing module's noise reduction process eliminates random noise introduced by the image sensor, sharpening process enhances edge details, and color correction process adjusts white balance and gamma value to optimize image output quality.

5. The audio synchronous acquisition and transmission processing system according to claim 1, characterized in that, The audio processing module employs a configurable sampling rate to adapt to different audio quality requirements, the encoding process converts the audio signal into PCM or AAC format, and the compression process reduces the amount of data through lossless or lossy algorithms.

6. The audio synchronous acquisition and transmission processing system according to claim 1, characterized in that, The storage module uses non-volatile memory, supports high-speed read and write, and is used to buffer data streams and store configuration parameters and synchronization logs, which facilitates subsequent diagnosis and optimization.

7. The audio synchronous acquisition and transmission processing system according to claim 1, characterized in that, The signal processing and transmission module supports USB 3.0 or higher transmission modes and has a dynamic bandwidth adjustment mechanism to adapt to the performance of different USB ports.

8. The audio synchronous acquisition and transmission processing system according to claim 1, characterized in that, The power management module adopts a multi-level power design, including a voltage regulator circuit and a power consumption control unit. It dynamically adjusts the power supply strategy according to the system load, entering a low-power mode when idle and switching to full-power mode when collecting and transmitting data.

9. The audio synchronous acquisition and transmission processing system according to claim 1, characterized in that, The resynchronization verification step includes calculating the audio / video synchronization error index E, which is calculated as follows: in, Let represent the synchronization error of the i-th audio / video pair, calculated as = - , It is the output timestamp of the i-th image frame. It is the output timestamp of the i-th audio block, which is generated from the synchronization controller; The window size is obtained from the configuration parameters of the storage module; The average synchronization error is expressed as follows: .

10. The audio synchronization acquisition and transmission processing system according to claim 1, characterized in that, The resynchronization verification step includes calculating the synchronization adjustment amount A, which is calculated as follows: in, Let represent the synchronization error of the i-th audio / video pair, calculated as = - , It is the output timestamp of the i-th image frame. It is the output timestamp of the i-th audio block, which is generated from the synchronization controller; This represents the average processing latency difference, calculated as follows: ; The processing delay for the i-th image frame is represented by , and is calculated as . The It is the acquisition timestamp of the i-th image frame, generated from the image acquisition module; The processing delay of the i-th audio block is represented by , calculated as . The It is the acquisition timestamp of the i-th audio block, generated from the audio acquisition module; the acquisition timestamp is based on the unified clock source; The window size is obtained from the configuration parameters in the storage module.

Images