Audio test data processing method, system and equipment based on zero-copy NIO and dynamic sliding window

Abstract

The invention discloses an audio test data processing method, system and device based on zero-copy NIO and a dynamic sliding window, and the method comprises the following steps: S1, building a memory mapping file channel, and directly mapping an audio file to a process address space; s2, constructing a double-buffer processing pipeline, and realizing asynchronous decoupling of a collection thread and a calculation thread by adopting a producer-consumer mode; and S3, a dynamic sliding window mechanism is adopted to maintain a real-time audio data interval, and the window size is dynamically adjusted according to a television CPU load. According to the technical scheme of the invention, the problems of high memory occupation, high detection delay and low cross-application data reading efficiency existing in audio testing of a television system in a multi-application concurrent scene are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent television audio detection, and particularly relates to an audio test data processing method, system and device based on zero-copy NIO and dynamic sliding window. BACKGROUND

[0002] The current television audio test field generally adopts traditional file IO to read PCM audio data in sequence, performs quantization value calculation through full-quantity cached historical data, and performs quality determination by using a fixed sampling window. This method causes linear growth of memory consumption with test time length due to full-quantity data caching, causes high detection delay due to blocking IO reading and writing, and causes limited sampling rate due to the fixed window, which cannot adapt to the high sampling requirement of a television multi-microphone array. SUMMARY

[0003] The present application aims to provide an audio test data processing method, system and device based on zero-copy NIO and dynamic sliding window, so as to solve the problems of high memory occupation, high detection delay and low cross-application data reading efficiency of television system audio test in a multi-application concurrent scene.

[0004] In order to achieve the above-mentioned purpose, the present application adopts the following technical scheme:

[0005] An audio test data processing method based on zero-copy NIO and dynamic sliding window comprises the following steps:

[0006] S1. Establishing a memory mapping file channel to directly map an audio file to a process address space;

[0007] S2. Constructing a double-buffering processing pipeline to realize asynchronous decoupling of a collection thread and a calculation thread by using a producer-consumer mode;

[0008] S3. Using a dynamic sliding window mechanism to maintain a real-time audio data interval, and dynamically adjusting the window size according to the television CPU load.

[0009] In some embodiments, the step S1 specifically comprises: step 11, locating an audio file in a storage directory, obtaining a file descriptor and a complete path thereof; step 12, dividing the audio file into data blocks of a fixed size according to a television system audio sampling rate; and step 13, establishing a memory mapping view to generate an object that can directly access file data.

[0010] In some embodiments, the step S2 specifically comprises: the double-buffer asynchronous processing pipeline comprises that the producer thread writes the original audio data to the buffer A in a non-blocking manner; the consumer thread reads the data from the buffer B to perform RMS calculation and decibel conversion; the read-write pointer is automatically switched when the buffer A is full, and the buffer size is adaptively adjusted according to the sampling rate of the television audio interface.

[0011] In some embodiments, the step S3 specifically comprises:

[0012] A dynamic sliding window mechanism is adopted to maintain a real-time audio data interval, and the window size is dynamically adjusted according to the television CPU load: the window size is reduced when the CPU utilization exceeds a preset threshold, and the window size is increased when garbage collection is not triggered in a continuous processing period.

[0013] In some embodiments, the step S3 further comprises a multi-level validity verification step:

[0014] The valid speech segment is identified by a short-time zero-crossing rate or an RMS value, and the mute data is filtered out; the valid reading times of each channel are counted, and the minimum number is taken as the overall test validity criterion.

[0015] The embodiment also provides an audio test system, comprising: a cross-application data access module for breaking through the Android sandbox limit to access audio streams of a specified directory; a zero-copy processing engine comprising a memory mapping manager and a direct buffer allocator; and a dynamic analysis module integrating a sliding window controller and a television chip adapter.

[0016] In some embodiments, the cross-application data access module specifically comprises: monitoring a.pcm file change event under an audio storage directory; obtaining a file descriptor to establish a shared memory channel with a television system audio service.

[0017] In some embodiments, the zero-copy processing engine comprises a memory mapping fault recovery unit that automatically degrades to a segmented mapping mode when the television memory is insufficient; and a byte sequence converter that adapts to the PCM data storage format of different television chip architectures.

[0018] In some embodiments, the dynamic analysis module further comprises a television microphone frequency response curve calibration database that stores the microphone frequency response characteristics of each brand of television.

[0019] The embodiment also provides a smart television device integrating any one of the audio test systems, and automatically generates a diagnostic report containing an SN code, an abnormal timestamp and a frequency band analysis result when an abnormal microphone is detected; and displays a three-dimensional frequency spectrum graph through a television OSD interface.

[0020] The audio test data processing method, system and device based on zero-copy NIO and dynamic sliding window disclosed in the application can bring beneficial effects, which include but are not limited to:

[0021] The data copy is eliminated through zero-copy memory mapping, the memory is allocated on demand in combination with dynamic sliding window, and the invalid resources are released through multi-level verification, so that the memory occupation is greatly reduced, the problem of high memory occupation is solved, the thread-level parallelism is realized through double-buffering pipeline, the invalid calculation is reduced through noise filtering, and the problem of high detection delay is solved; the memory mapping file channel enables cross-application data access to bypass the traditional I / O path, the reading efficiency is greatly improved, and the problem of low cross-application data reading efficiency is solved. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A low memory occupation and cross-application efficient reading scheme flowchart.

[0023] Figure 2 A low-delay real-time processing scheme flowchart.

[0024] Figure 3 A memory-sensitive dynamic window adjustment algorithm schematic diagram.

[0025] Figure 4 A delay-sensitive dynamic window adjustment algorithm schematic diagram. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the application more clear, the application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and do not limit the application.

[0027] On the contrary, the application covers any alternative, modification, equivalent method and scheme made on the essence and scope of the application defined by the claims. Further, in order to make the public better understand the application, some specific details are described in detail in the following detailed description of the application. The application can also be completely understood without the description of these details by those skilled in the art.

[0028] As Figure 1 shown, an audio test data processing method based on zero-copy NIO and dynamic sliding window includes the following steps:

[0029] Step 1, establish a memory mapping file channel, directly map an audio file to a process address space, bypass traditional disk I / O and kernel buffer copy, solve the problem of low cross-application data reading efficiency; the calculation thread directly operates file data, avoids creating a user state data copy, and solves the problem of high memory occupation;

[0030] Step 2, build a double-buffering processing pipeline, use the producer-consumer mode to decouple the acquisition thread and the calculation thread, both can handle data asynchronously, reduce thread blocking waiting time, solve the problem of high detection delay;

[0031] Step 3, implement dynamic sliding window control, dynamically adjust the window size according to the real-time performance of the system, reduce the window size when CPU utilization is too high, solve the problem of high memory occupation;

[0032] Step 4, perform multi-level validity verification, including: filtering out invalid data generated by environmental noise, avoiding invalid data occupying computing resources, reducing delay caused by redundant processing, solving the problem of high detection delay; count the number of valid reads of each channel, terminate the calculation task of invalid channels in advance, release memory resources, use the minimum valid read number of each channel as the overall test validity criterion, solve the problem of high memory occupation.

[0033] Further, the step 1 specifically comprises: step 11, locating the audio file in the storage directory, obtaining its file descriptor and complete path; step 12, dividing the audio file into fixed-size data blocks according to the television system audio sampling rate; step 13, establishing a memory mapping view to generate an object that can directly access file data.

[0034] Among them, the memory mapping file technology specifically includes FileChannel.map() mapping the audio file under the specified directory to the memory address space, and ShortBuffer view directly accessing 16-bit PCM sample data to avoid heap memory copy.

[0035] Further, the double-buffering asynchronous processing pipeline includes: the producer thread writes raw audio data to buffer A in a non-blocking manner; the consumer thread reads data from buffer B for RMS calculation and decibel conversion; when buffer A is full, the read-write pointer is automatically switched, and the buffer size is adaptively adjusted according to the sampling rate of the television audio interface.

[0036] Further, the step 3 specifically comprises: step 31, initializing the window size;

[0037] Step 32, real-time monitoring of system performance indicators, when CPU utilization exceeds a certain value, gradually reduce the window size;

[0038] Step 33, when it is detected that no GC is triggered for 3 consecutive processing periods, increase the window size by a certain amplitude until the initial value.

[0039] Among them, the initial size of the sliding window is 1 second of data volume of the television audio sampling rate, and the adjustment strategy of the dynamic sliding window can be:

[0040] W_{\text{new}}=\left(\frac{\text{FreeMemory}}{\text{TotalMemory}}\right)\times W_{\text{initial}}

[0041] and W_{\text{new}} = \max(40\text{ms}, W_{\text{current}} \times \phi) \quad (\phi=0.618).

[0042] Further, the step 4 specifically includes:

[0043] Step 41, identify valid speech segments, filter out invalid data generated by environmental noise;

[0044] Step 42, count the number of valid readings of each channel, and take the minimum number as the overall test effectiveness criterion.

[0045] The embodiment also provides an audio test system, comprising: a cross-application data access module for breaking through the Android sandbox restriction to access the audio stream of a specified directory; a zero-copy processing engine including a memory mapping manager and a direct buffer allocator; a dynamic analysis module integrating a sliding window controller and a TV chip adapter.

[0046] In some embodiments, the cross-application data access module specifically implements the following steps: monitoring the change event of the.pcm file under the audio storage directory; obtaining the file descriptor to establish a shared memory channel with the TV system audio service.

[0047] In some embodiments, the zero-copy processing engine includes a memory mapping fault recovery unit that automatically degrades to a segmented mapping mode when the TV memory is insufficient; a byte sequence converter that adapts the PCM data storage format of different TV chip architectures.

[0048] In some embodiments, the dynamic analysis module further includes a TV microphone frequency response curve calibration database that stores the microphone frequency response characteristics of each brand of TV.

[0049] The embodiment also provides a smart TV device integrating any of the audio test systems, and when detecting microphone abnormalities, automatically generates a diagnostic report containing the SN code, abnormal timestamp and frequency band analysis result; and displays a three-dimensional frequency spectrum graph through the TV OSD interface.

[0050] In order to make the purpose, technical scheme and advantages of the present application more clear, the following will be described in detail in combination with Embodiment 1 (low memory and high efficiency reading scheme) and Embodiment 2 (low delay optimization scheme). The specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0051] Embodiment 1: Solve the problems of high memory occupation and low cross-application reading efficiency by zero-copy memory mapping and memory-sensitive dynamic window control.

[0052] As shown in Figure 1 , the embodiment includes the following steps:

[0053] Step 1, establish a memory channel, which is specifically implemented as:

[0054] Step 11, call FileInputStream.getChannel() to obtain the file channel of the PCM audio file to be tested under the storage directory, and obtain the physical storage path of the file through FileDescriptor to ensure cross-process access;

[0055] Step 12, divide the file into fixed-size data blocks according to the television microphone sampling rate, and assign independent logical offset addresses to each data block to facilitate on-demand loading;

[0056] Step 13, create MappedByteBuffer using FileChannel.map(MapMode.READ_ONLY, 0, fileSize), and the calculation thread directly accesses the PCM data under the storage directory through buffer.get() to avoid user-mode memory copying;

[0057] Step 2, as shown in Figure 3 , implement memory-driven dynamic sliding window control, which is specifically implemented as:

[0058] Step 21, initialize the window size to 200ms of audio data length, and define the minimum threshold of the window to 40ms to ensure the retention of critical data;

[0059] Step 22, monitor Runtime.getRuntime().freeMemory() in real time, and when the system free memory is less than 30% of the total memory, the new window size is calculated according to the formula: new window size = current free memory / total memory x initial window linear reduction, and when the free memory rises to more than 50%, the window is expanded by 10ms / period;

[0060] Step 3, implement memory release type multi-level validity verification, which is specifically implemented as:

[0061] Step 31, calculate the RMS value of each channel 200ms window, if the relative decibel value of RMS of a channel is <-60dBFS for 5 times in succession, it is determined as invalid silence section;

[0062] Step 32, for the channel marked as invalid, call MappedByteBuffer.force() to release the physical mapping, and the channel data block no longer occupies the user space;

[0063] Embodiment 2: Reduce end-to-end processing delay by double buffering pipeline and delay sensitive dynamic window control, solve the problem of high detection delay.

[0064] As shown in Figure 2 , the embodiment includes the following steps:

[0065] Step 1, build a double buffering processing pipeline, which is specifically implemented as:

[0066] Step 11, create two 256KB ring buffers, each buffer containing 100ms of audio data, implement lock-free pointer switching through AtomicInteger to ensure thread safety;

[0067] Step 12, write audio data to the currently active buffer, trigger compareAndSwap to switch to Buffer B when it is full, and the calculation thread reads data from the inactive buffer, waits for the next switch after processing is completed;

[0068] Step 13, if the processing time of the calculation thread is >20ms, immediately switch the buffer and enable 8bit reduced precision calculation, and the delay after degradation is forced to be within 50ms;

[0069] Step 2, as shown in Figure 4 , implement delay-driven dynamic sliding window control, which is specifically implemented as:

[0070] Step 21, initialize the window size to 100ms, and define the minimum window size to 40ms;

[0071] Step 22, when the delay exceeds the limit, for example when the processing delay is >20ms, the window is quickly reduced according to the Fibonacci sequence: window size = max(40ms, current value x 0.618);

[0072] Step 23, gradually recover at low delay, after 3 consecutive delays <15ms, the window is expanded by 5ms / period step by step.

[0073] Step 3, implement low-delay multi-level validity verification, which is specifically implemented as:

[0074] Step 31, a short-time zero-crossing rate of the 50ms window is calculated in parallel by using a SIMD instruction set, and the short-time zero-crossing rate is used to identify an effective speech segment;

[0075] Step 32, if the zero-crossing rate is less than a threshold value, the segment is marked as a mute segment and verification is skipped, and a proportion of effective frames in a non-mute segment is dynamically counted.

[0076] The above merely describes preferred embodiments of the present application and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing audio test data based on zero-copy NIO and dynamic sliding window, characterized in that: The following steps are involved: S1 establishes a memory-mapped file channel and directly maps the audio file to the process address space; S2 builds a double-buffered processing pipeline and adopts a producer-consumer model to achieve asynchronous decoupling of the acquisition thread and the calculation thread; S3 uses a dynamic sliding window mechanism to maintain the real-time audio data interval, and the window size is dynamically adjusted according to the TV CPU load.

2. The audio test data processing method based on zero-copy NIO and dynamic sliding window according to claim 1, characterized in that: The step S1 specifically includes: step 11, locating the audio file in the storage directory and obtaining its file descriptor and complete path; step 12, dividing the audio file into fixed-size data blocks according to the audio sampling rate of the television system; step 13, establishing a memory mapping view and generating an object that can directly access the file data.

3. The audio test data processing method based on zero-copy NIO and dynamic sliding window according to claim 1, characterized in that: The step S2 specifically includes: the double-buffered asynchronous processing pipeline includes: the producer thread writes the original audio data to buffer A in a non-blocking manner; the consumer thread reads data from buffer B to perform RMS calculation and decibel conversion; when buffer A is full, the read and write pointers are automatically switched, and the buffer size is adaptively adjusted according to the sampling rate of the TV audio interface.

4. The audio test data processing method based on zero-copy NIO and dynamic sliding window according to claim 1, characterized in that: The step S3 specifically includes: A dynamic sliding window mechanism is used to maintain the real-time audio data interval, and the window size is dynamically adjusted according to the TV CPU load: the window is reduced when the CPU utilization exceeds the preset threshold, and the window is increased when continuous processing cycles do not trigger garbage collection.

5. The audio test data processing method based on zero-copy NIO and dynamic sliding window according to claim 1, characterized in that: It also includes multi-level validity verification steps: Valid speech segments are identified through short-time zero-crossing rate or RMS value, and silent data is filtered out. The number of valid readings for each channel is counted, and the minimum number is used as the overall test effectiveness criterion.

6. An audio testing system based on the method according to any one of claims 1 to 5, characterized in that: include: Cross-application data access module, used to break through the Android sandbox restrictions and access the audio stream of the specified directory; Zero-copy processing engine, including memory map manager and direct buffer allocator; Dynamic analysis module, integrated sliding window controller and TV chip adapter.

7. The system according to claim 6, characterized in that The cross-application data access module is specifically implemented by monitoring .pcm file change events in the audio storage directory; obtaining file descriptors, and establishing a shared memory channel with the television system audio service.

8. The system according to claim 6, wherein: The zero-copy processing engine includes a memory mapping fault recovery unit, which automatically degrades to a segmented mapping mode when the TV memory is insufficient; and a byte order converter, which adapts to the PCM data storage format of different TV chip architectures.

9. The system according to claim 6, wherein: The dynamic analysis module also includes a television microphone frequency response curve calibration database, which stores the microphone frequency response characteristics of various brands of televisions.

10. A smart TV device, characterized in that: The audio test system according to any one of claims 6 to 9 is integrated, and when a microphone abnormality is detected, a diagnostic report including an SN code, an abnormality timestamp and a frequency band analysis result is automatically generated; and a three-dimensional spectrum diagram is displayed through a TV OSD interface.

Images