Intelligent audio mute segment detection and skip playing method, system and device and medium

Abstract

The invention discloses an intelligent audio mute segment detection and skip playing method, system and device and a medium, a mute segment detection algorithm adopted by the method can dynamically adjust a threshold according to audio content characteristics, high-precision mute segment identification is realized in combination with an overlapped sliding window, the false detection rate and the omission rate are reduced, and the detection efficiency is improved. The robustness and universality of the algorithm are improved; according to the method, a lightweight real-time detection algorithm is adopted, efficient operation can be achieved in the JavaScript execution environment of the Web browser, good real-time performance and user experience are guaranteed, and meanwhile the CPU occupancy rate and memory consumption are controlled; the method supports streaming processing, mute detection and skip control can be carried out in real time in the audio playing process, dynamic interaction operation of a user is supported, and the whole audio file does not need to be pre-processed; besides, according to the method, an intelligent caching strategy and a resource management mechanism are adopted, so that repeated calculation is avoided, the overall operation efficiency of the system is improved, and occupation of system resources is reduced.

Description

Technical Field

[0001] This application belongs to the field of audio technology, specifically relating to an intelligent audio silence detection and skip playback method, system, device, and medium. Background Technology

[0002] In digital audio processing, audio signals are typically digitized at sampling rates (e.g., 44.1kHz, 48kHz) to form discrete digital signal sequences. Audio signal analysis primarily includes two methods: time-domain analysis and frequency-domain analysis. Time-domain analysis directly analyzes the amplitude variations of the audio signal, with commonly used metrics including root mean square (RMS) value and peak value. Frequency-domain analysis, on the other hand, uses a Fast Fourier Transform (FFT) to convert the signal to the frequency domain for analysis. Simultaneously, the Web Audio API technology is utilized; it is a high-level JavaScript API provided by modern browsers for processing and synthesizing audio in a web environment. This API provides core interfaces such as AudioContext, AudioBuffer, and AudioNode, supporting functions such as audio loading, decoding, playback, and analysis. Through the Web Audio API, real-time audio processing, audio visualization, and sound effects processing can be achieved. The existing silence detection technology solutions are mainly divided into: (1) The existing audio players generally adopt silence detection technology based on fixed threshold. The implementation principle is: a fixed volume threshold (such as -40dB or 0.01 normalization value) is set in advance, and the audio is sampled and analyzed according to a fixed time interval (usually 10-100 milliseconds). The average energy or RMS value of the audio signal in each time window is calculated. The calculated energy value is compared with the preset threshold. If it is lower than the threshold, it is judged as silent. Only when multiple consecutive time windows are judged as silent is it marked as a silent segment. When a silent segment is detected during playback, it directly jumps to the next non-silent position. This technology has the following technical drawbacks: It uses a fixed global threshold, which cannot be dynamically adjusted based on the audio content. For audio with a large dynamic range (such as classical music), a fixed low threshold may cause soft passages to be misjudged as silent. For audio with a small dynamic range (such as voice calls), a fixed high threshold may not be able to effectively detect true silent segments. These misjudgments and omissions lead to insufficient detection accuracy, directly affecting the user's playback experience. Furthermore, using a fixed-size detection window makes it impossible to accurately locate the start and end boundaries of silent segments. When the detection interval is large (e.g., 100 milliseconds), it may miss brief audio content; when the detection interval is small, it increases the computational burden and affects performance. This coarse timing precision may cause useful audio content to be truncated when skipping playback. Finally, the fixed-parameter detection scheme cannot adaptively adjust to the characteristics of different types of audio. Different types of audio, such as voice, music, and podcasts, have different silence characteristics. Fixed detection parameters cannot cover all scenarios, limiting the system's versatility and practicality.(2) The existing technology of speech activity detection (VAD) is also proposed, which is mainly used in speech recognition and communication systems. Its implementation principle is as follows: extract the short-time energy, zero cross rate, spectral centroid and other features of audio, establish a statistical model of speech and non-language (including silence), use likelihood ratio detection or machine learning classifier to make speech activity judgment, reduce false detection through smoothing filtering and continuity constraints, and update the background noise model according to the detection results. The technology has the following technical defects: VAD technology requires the extraction of multi-dimensional features and complex statistical modeling and classification calculations. Its computational complexity far exceeds the simple silence detection requirements. In the Web browser environment, the excessive computational load will lead to performance problems such as page lag and increased battery consumption, which seriously affects the user experience. The design goal of VAD technology is to accurately distinguish between speech and non-speech, rather than silence detection specifically optimized for playback experience. Its performance on non-speech audio such as music is often unsatisfactory, and it cannot effectively handle deliberate silence segments in audio production (such as music interludes, dramatic effects, etc.). VAD technology usually requires complex model training and parameter tuning, which makes it difficult to integrate with existing Web audio playback systems in a lightweight manner. Its implementation cost and maintenance cost are both high, making it unsuitable for widespread application in ordinary audio playback scenarios. (3) The existing professional audio editing software has also proposed a silence segment deletion function. Its implementation principle is: to perform a complete scan and analysis of the entire audio file, and the user manually sets parameters such as silence threshold and minimum silence length, marks the position of all silence segments that meet the conditions, and provides batch editing functions for deleting, shortening or replacing silence segments. The user can preview the processing results and adjust them manually. This technology has the following technical shortcomings: It adopts an offline batch processing mode, requiring users to wait for the complete file analysis and processing, which cannot meet the real-time needs of online playback and streaming media scenarios. Users cannot dynamically enable or disable the mute / skip function during playback. It requires users to manually configure multiple technical parameters (thresholds, duration, etc.), placing excessive technical demands on ordinary users. Furthermore, each processing step requires user intervention, preventing intelligent automatic optimization and resulting in a poor user experience. The complete file loading and processing consumes significant memory and computing resources, potentially leading to memory overflow or page crashes in a web browser environment. Its design architecture cannot adapt to the sandbox limitations and resource constraints of the web environment. It only provides audio editing functions, lacking deep integration with the player control system, making real-time detection and dynamic skipping during playback impossible, as well as providing real-time feedback on playback status and user interaction. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this application proposes an intelligent audio silence detection and skip playback method, system, device, and medium.

[0004] This application is achieved through the following technical solution:

[0005] A method for intelligent audio silence detection and skipping playback includes:

[0006] The input audio resource is checked for cache. If the cached data exists, it is loaded directly, skipping the network request, decoding process and silence detection process. If the cached data does not exist, a network request is initiated and audio decoding is performed, and the result is cached.

[0007] Perform player initialization and determine whether silent skip is enabled. If so, use a silent detection algorithm to detect silent segments. Specifically, this includes: traversing the entire audio data according to the overlapping sliding window, calculating the RMS value of the audio signal in each overlapping sliding window, comparing the calculated RMS value with an adaptive threshold, performing silent segment identification and continuity verification, and caching the identified silent segments.

[0008] Real-time playback monitoring is performed. An interval search algorithm is used to detect whether the current playback position is in a silent segment. If it is, a time jump is executed to jump to a non-silent segment for playback. Otherwise, the standard playback mode is used for normal playback.

[0009] Determine if playback has ended. If so, playback is complete; otherwise, return to the real-time playback monitoring step to continue playback monitoring.

[0010] In some implementations, the use of a silence detection algorithm for silence segment detection includes:

[0011] Determine the parameters of the overlapping sliding window, including: window size, number of sampling points per window, and sliding step size, with the sliding step size being smaller than the window size;

[0012] The entire audio data is traversed using overlapping sliding windows, and the RMS value of each overlapping sliding window is calculated.

[0013] Accumulate the RMS values ​​of all overlapping sliding windows and calculate the average value to obtain the average RMS value;

[0014] The adaptive threshold is calculated based on the average RMS value; wherein the adaptive threshold is calculated by multiplying the average RMS value by a preset coefficient and taking the maximum value of the product and the preset threshold as the adaptive threshold.

[0015] The RMS value is compared with the adaptive threshold window by window to identify silent segments and verify the continuity of the identified silent segments. Valid silent segments are determined and their information is cached.

[0016] In some implementations, the process of identifying silent segments and verifying the continuity of the identified silent segments includes:

[0017] Initialization state, including initializing the start time of the silence segment and the continuous silence window;

[0018] The RMS value is compared with the adaptive threshold window by window. When the RMS value is less than the adaptive threshold, the start time is recorded and counted continuously. When the RMS value is greater than or equal to the adaptive threshold, the continuous duration is checked to see if it meets the minimum duration requirement. If it does, it is determined to be a valid silent segment.

[0019] In some implementations, the method of using an interval search algorithm to detect whether the current playback position is in a silent segment includes:

[0020] For each silent segment in the silent segment array, if the current playback time is greater than or equal to the start time of the silent segment and the current playback time is less than or equal to the end time of the silent segment, then the end time of the silent segment is updated to the original end time of the silent segment plus the buffer time; otherwise, continue to check the next silent segment.

[0021] If none of the silent segments match, the current playback position will be maintained and playback will proceed normally in standard playback mode.

[0022] In some implementations, the method further includes:

[0023] The time-domain waveform of the audio signal is displayed in real time, and silence segments are marked on the time-domain waveform.

[0024] Real-time display of statistics on silent segments;

[0025] Provides a user interface.

[0026] In some implementations, the method further includes:

[0027] A hierarchical caching architecture is adopted to cache audio binary data, silence segment detection results, and configuration parameters respectively;

[0028] A timestamp-based cache expiration mechanism is adopted;

[0029] In response to the memory limitations of the web environment, the following optimization strategies are implemented in real time: audio files exceeding the threshold size are processed using streaming, infrequently used cached items are treated with weak references, and memory cleanup is proactively triggered at appropriate times.

[0030] Secondly, this application proposes an intelligent audio silence detection and skip playback system for implementing any of the above-described playback methods, comprising:

[0031] The audio processing module is used to receive audio resources transmitted from the audio input interface, process them to obtain digital audio signals, and transmit them to the silence detection module, playback control module, and buffer management module.

[0032] The audio detection module is used to receive digital audio signals from the audio processing module, and when mute skipping is enabled, it uses a mute detection algorithm to identify mute segments and transmits the detection results to the playback control module and the buffer management module in the form of mute segment data structure.

[0033] The playback control module is used to receive audio signals and silence segment information, execute playback control logic, including time jump and continuous playback, and finally transmit the audio signal to the audio output interface and output it to the audio output device for playback.

[0034] The cache management module is used to establish a bidirectional data connection with the audio processing module and the silence detection module, and is responsible for the intelligent caching and management of audio data and silence segment detection results;

[0035] In addition, a user interaction module is used to interact with the silence detection module and the playback control module through event information, and to transmit user configuration parameters and control commands.

[0036] In some embodiments, the audio detection module includes:

[0037] A sliding window analyzer is used to determine the parameters of an overlapping sliding window, including: window size, number of sampling points per window, and sliding step size, with the sliding step size being smaller than the window size;

[0038] An RMS volume calculator is used to traverse the entire audio data according to the overlapping sliding windows and calculate the RMS value of each overlapping sliding window.

[0039] An adaptive threshold adjuster is used to accumulate the RMS values ​​of all overlapping sliding windows and calculate the average value to obtain the average RMS value, and then calculate the adaptive threshold based on the average RMS value.

[0040] A silence segment identifier is used to compare the RMS value with the adaptive threshold window by window to identify silence segments.

[0041] In addition, a continuity verifier is used to verify the minimum duration of the silent segments identified by the silent segment recognizer to determine the valid silent segments.

[0042] Thirdly, this application proposes an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the above-described playback methods.

[0043] Fourthly, this application proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described playback methods.

[0044] This application proposes an intelligent audio silence detection and skip playback method. Addressing the insufficient accuracy of fixed-parameter (fixed threshold, fixed window, etc.) detection schemes, this method employs a silence detection algorithm that dynamically adjusts the threshold based on audio content characteristics. It also combines overlapping sliding windows (where each calculation window partially overlaps with adjacent windows) to achieve high-precision silence identification, while reducing false positive and false negative rates and improving the algorithm's robustness and versatility. Furthermore, addressing the excessive computational complexity of existing VAD (Voice over Detection) schemes, this method uses a lightweight real-time detection algorithm that can run efficiently in a web browser's JavaScript execution environment, ensuring good real-time performance and user experience while controlling CPU usage and memory consumption. In terms of resource consumption, this method addresses the issues of high latency and poor flexibility in existing offline processing solutions. It supports streaming processing, enabling real-time silence detection and skip control during audio playback, and supports dynamic user interaction without pre-processing the entire audio file. To address the low resource utilization efficiency of existing solutions, this method employs intelligent caching strategies and resource management mechanisms to avoid redundant calculations, improving overall system efficiency and reducing resource consumption. Furthermore, to address the issues of complex user interaction and low automation in existing solutions, this method utilizes an intelligent parameter adaptation mechanism and an intuitive user interface, allowing ordinary users to easily use the silence skip function while providing personalized configuration options to meet the needs of different users.

[0045] Accordingly, the intelligent audio silence detection and skip playback system, device and medium proposed in this application also have the same technical effects as described above. Attached Figure Description

[0046] The accompanying drawings, which are included to provide a further understanding of the embodiments of this application and form part of this application, do not constitute a limitation on the embodiments of this application. In the drawings:

[0047] Figure 1 This is a flowchart of the playback method proposed in an embodiment of this application;

[0048] Figure 2 This is a block diagram illustrating the principle of the playback system proposed in an embodiment of this application;

[0049] Figure 3 A schematic diagram of the user interface provided for the user interaction module in an embodiment of this application;

[0050] Figure 4 This is a schematic diagram of the electronic device proposed in the embodiments of this application;

[0051] Figure 5 This is a schematic diagram of a computer-readable storage medium proposed in an embodiment of this application;

[0052] Figure reference numerals and corresponding component names:

[0053] 200-Playback system, 201-Audio input interface, 202-Audio processing module, 203-Mute detection module, 204-Playback control module, 205-Audio output interface, 206-Buffer management module, 207-User interaction module, 208-Audio output device, 300-User interface, 301-Audio waveform display area, 302-Mute segment marking area, 303-Playback control area, 304-Mute skip switch, 305-Detection information display area, 306-Parameter configuration area, 307-Playback progress bar, 308-Volume controller, 400-Electronic device, 410-Memory, 420-Processor, 411-Computer program A, 500-Computer-readable storage medium, 511-Computer program B. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this application are only for explaining this application and are not intended to limit this application.

[0055] Existing audio processing technologies are mainly divided into the following three categories: (1) silence detection technology based on fixed thresholds, which has the following technical defects: insufficient detection accuracy, high false detection rate and false negative rate; rough time accuracy and inaccurate boundary positioning; lack of adaptive ability and poor versatility; (2) audio processing technology based on voice activity detection, which has the following technical defects: excessive computational complexity and insufficient real-time performance; insufficient targeting and mismatch of optimization targets; high integration complexity and high development cost; (3) the silence segment deletion function in audio editing software, which has the following technical defects: the processing mode is not applicable and cannot meet the needs of online playback; user operation is complicated and the degree of automation is low; large resource consumption and poor adaptability to the Web environment; single function and lack of playback control integration. In view of the above technical defects, this application proposes an intelligent audio silence segment detection and skip playback method, such as Figure 1 As shown, the playback method includes:

[0056] Step 1: Check the cache of the input audio resource. If the cached data exists, load the cached data directly, skipping the network request, decoding process and silence detection process. If the cached data does not exist, initiate a network request and perform audio decoding processing, and cache the processing result.

[0057] Step 2: Initialize the player and determine whether to enable silent skip. If so, use a silent detection algorithm to detect silent segments. Specifically, this includes: traversing the entire audio data according to the overlapping sliding window, calculating the RMS value of the audio signal in each overlapping sliding window, comparing the calculated RMS value with the adaptive threshold, identifying silent segments and verifying continuity, and caching the identified silent segments.

[0058] Step 3: Perform real-time playback monitoring. Use the interval search algorithm to detect whether the current playback position is in a silent segment. If so, perform a time jump to jump to a non-silent segment for playback. Otherwise, use the standard playback mode for normal playback.

[0059] Step 4: Determine if playback has ended. If yes, playback is complete; otherwise, return to step 3 to continue playback monitoring.

[0060] Furthermore, in step 1 of this application embodiment, to address the problem of resource waste caused by existing repeated loading, a smart caching mechanism based on parameter hashing is adopted, including:

[0061] Cache key-value generation algorithm: Generate a unique cache identifier (cache index) based on audio request parameters. Specifically, this includes: combining API type, file identifier, audio quality parameters, etc. into a string; performing hash calculation on the combined string to generate a cache key-value; where the cache key-value format is: ${api}_${parameter hash value}.

[0062] Cache data structure design: The cache item contains the following data fields: blob field: stores audio binary data; timestamp field: records the cache creation timestamp, used for expiration judgment; silenceSegments field: stores the silence segment detection results to avoid duplicate calculations.

[0063] Intelligent caching strategy: On the first request, check the cache; if it does not exist, initiate a network request and perform audio decoding, then cache the result. For subsequent requests, load data directly from the cache (audio data and silence detection results), skipping the network request, decoding process, and silence detection process. Cache expires automatically after a preset time (e.g., 30 minutes) to balance memory usage and performance.

[0064] Compared to existing technologies, this intelligent caching mechanism reduces network requests by 85%, avoids redundant calculations, and improves audio loading speed by 300%.

[0065] Specifically, the compressed audio decoding and mono (floating-point PCM) waveform extraction process includes: creating an audio processing context using the AudioContext interface of the Web Audio API, calling the decodeAudioData method to decode the audio Blob into an AudioBuffer object; extracting the sampled data of the first channel to obtain the time-domain signal in PCM floating-point time-domain sample sequence (Float32Array) format.

[0066] Furthermore, the silence detection algorithm used in step 2 of this application embodiment specifically includes:

[0067] The overlapping sliding window parameters are determined, and the entire audio data is traversed according to the overlapping sliding window to calculate the RMS value of each window. The overlapping sliding window parameters include: window size; number of sampling points per window, which is equal to the product of the sampling rate and the window size, rounded down; and sliding step size, which is an integer greater than or equal to 1 and cannot exceed the window size. The sliding step size must be divisible by or reasonably handle boundaries to ensure that the step size cycle covers all data. In this embodiment, the preferred sliding step size is equal to the number of sampling points per window divided by 4 and rounded down (i.e., the sliding window step size is 1 / 4 of the window size, achieving 75% overlap analysis. A higher overlap rate results in more accurate boundary detection, but increases computational load. This embodiment prefers a 75% overlap rate to achieve a balance between performance and effect). The RMS value is calculated by traversing all sampling points within the window, summing their squares, dividing by the number of sampling points, and taking the square root. The overlapping sliding window improves boundary detection accuracy by 75% compared to existing technologies.

[0068] Accumulate the RMS values ​​of all windows and calculate the average: The average RMS value is equal to the cumulative RMS value of all windows divided by the number of windows; this average RMS value reflects the overall volume level of the audio.

[0069] The adaptive threshold is calculated based on the average RMS value. The adaptive threshold is calculated by multiplying the average RMS value by a coefficient and taking the maximum value of the product and a preset threshold as the adaptive threshold. This ensures that the threshold is neither lower than the sensitivity (preset threshold) nor does it have to adapt to the audio characteristics. The coefficient is preferably 0.1, which has been determined through a large number of tests and performs optimally on different types of audio. The embodiments of this application adopt an adaptive threshold algorithm based on global audio features, which solves the problem that a fixed threshold cannot adapt to different audio dynamic ranges.

[0070] The process involves comparing the RMS value with an adaptive threshold window by window to identify silent segments and verify their continuity. Valid silent segments are then identified and cached. The specific steps include: initializing the state, initializing the start time of the silent segment, and the continuous silent window; and the detection process: comparing the RMS value with the adaptive threshold window window by window. When the RMS value is less than the adaptive threshold, the start time is recorded and counted continuously. When the RMS value is greater than or equal to the adaptive threshold, the continuous duration is checked to see if it meets the minimum requirement (i.e., whether the continuous duration is greater than or equal to the preset minimum duration). If it does, the segment is determined to be a valid silent segment.

[0071] Furthermore, step 3 of this embodiment employs a real-time playback control and intelligent skip algorithm, which specifically includes:

[0072] The playback time monitoring mechanism uses the Web Audio API's timeupdate event to monitor playback in real time. Specifically, it includes: event triggering frequency, which is triggered once every preset period (e.g., 100 seconds) to ensure timely response; a callback function that receives the current playback time parameter currentTime; and updating the playback progress displayed on the user interface.

[0073] The silence segment position detection uses an interval search algorithm to check whether the current playback position is within a silence segment. The algorithm logic is as follows: For each silence segment in the silence segment array, if the current playback time is greater than or equal to the start time of the silence segment and less than or equal to the end time of the silence segment, then the end time of the silence segment is updated to the end time of the silence segment plus a small buffer time (e.g., 0.1 seconds). This small buffer is used to avoid boundary issues. Otherwise, the next silence segment is checked. If none of the silence segments match, the current position is maintained and normal playback is performed using the standard playback mode (i.e., no silence segment exists).

[0074] When a silent segment is detected during playback, a time jump is executed: the `setTime` method of the `WaveSurfer` instance is called to set the new playback position; the `play` method is called to ensure playback continues after the jump; and a short buffer time is added to avoid boundary oscillation issues. The response time latency of this jump mechanism is less than 200 milliseconds, and the user will not perceive a noticeable playback interruption.

[0075] Furthermore, the playback method proposed in this application embodiment also includes:

[0076] The visualization and user interaction of the silence segment are implemented as follows:

[0077] Mark silent segments on the audio waveform graph. For example, silent segments can be marked with semi-transparent red areas. The marking method is as follows: call the clearRegions method of the regions plugin to clear the existing markings; add silent segment regions one by one and create visual markings using the addRegion method; set the region properties to red semi-transparent (rgba(255,0,0,0.2)) and disable dragging and adjustment.

[0078] Real-time display of silent segment statistics, including: the total number of detected silent segments, the sum of the durations of all silent segments, and a detailed list showing the start time, end time, and duration of the first few (e.g., 5) silent segments;

[0079] The user interface provides an intuitive user control interface, including: a silent skip switch, which can enable or disable the silent skip function at any time; parameter adjustment, which supports dynamic configuration of thresholds and minimum duration; and real-time feedback, which displays the current detection status and the number of skips.

[0080] Furthermore, the playback method proposed in this application embodiment also includes:

[0081] Intelligent cache management and performance optimization specifically include:

[0082] A hierarchical caching architecture is adopted. Preferably, a three-layer caching structure can be used. The L1 cache is used to cache audio binary data and is directly stored in the Map structure; the L2 cache is used to cache silence segment detection results to avoid duplicate calculations; and the L3 cache is used to cache configuration parameters to improve the personalized experience.

[0083] A timestamp-based cache expiration mechanism is adopted, including: a default expiration time, which can be set to 30 minutes to balance memory usage and performance; lazy cleanup, which checks for expired items and deletes them automatically when accessed; and proactive cleanup, which scans and cleans up expired items periodically.

[0084] To address memory limitations in the web environment, the following optimization strategies are implemented: large files are chunked, and audio files exceeding a threshold size (e.g., 50MB) are streamed; weak references are used for infrequently used cached items; and garbage collection is used in conjunction with proactive memory cleanup at appropriate times.

[0085] This intelligent caching management and performance optimization mechanism reduces memory usage by 71.7% compared to existing technologies, while improving response speed by 300%.

[0086] Example 1: To verify the technical effectiveness of the playback method proposed in this application embodiment on different audio types, this application embodiment selected classical music with a dynamic range of 40-70dB for testing. Classical music has a large dynamic range, and existing fixed threshold schemes are prone to misjudging weak segments as silence. Using the adaptive threshold mechanism proposed in this application embodiment, the global average RMS value of the audio is first calculated to be 0.05, and the adaptive threshold is set to max(0.01, 0.05×0.1) = 0.01. Through overlay window analysis, the true silent segments between musical movements are accurately identified, avoiding misjudgment of weak segments. Compared to the existing technology's detection accuracy of 72.3%, this application embodiment achieves a detection accuracy of 94.1% in classical music scenarios, an improvement of 30.1 percentage points.

[0087] Example 2: A podcast was selected for testing. Podcasts typically contain natural pauses between sentences and silent segments between chapters. The detection algorithm was required to preserve the natural rhythm of the speech while skipping meaningless, long periods of silence. This embodiment sets the minimum silence duration to 1 second to avoid misjudging natural speech pauses as skippable silent segments. Furthermore, a continuity verification mechanism ensures that only silences lasting longer than 1 second are marked as skippable segments. In audio content scenarios, detection accuracy improved from 79.5% to 96.2%, and user feedback indicated a smoother and more natural playback experience.

[0088] Example 3: To verify the performance advantages of the embodiments of this application, performance tests were conducted using the Chrome browser in a test environment with an Intel i7-10700K processor and 16GB of memory. The specific test process included:

[0089] Use browser developer tools to monitor CPU usage during JavaScript execution and test CPU utilization.

[0090] Monitor memory allocation and garbage collection using the Performance API, and test memory usage.

[0091] Record the time delay from audio loading to the start of playback to test response time.

[0092] The verification results showed that CPU utilization decreased from 15.2% in the existing solution to 4.7%, a reduction of 69.1%; memory usage decreased from 45.3MB to 12.8MB, a reduction of 71.7%; and the initial playback delay was shortened from 8.4 seconds to 1.2 seconds, a reduction of 85.7%.

[0093] Compared to existing technologies that use fixed thresholds and fixed windows for silence detection, this application's embodiment employs an adaptive threshold-based sliding window RMS analysis algorithm for silence detection. It combines an adaptive threshold based on global audio features with a 75% overlap sliding window RMS analysis algorithm, along with a continuity verification mechanism to reduce false positives or false negatives, significantly improving detection accuracy. Compared to existing offline batch processing methods, this application's embodiment uses a layered caching design and a real-time streaming processing architecture that detects while playing, solving the problem of high processing latency. The initial playback latency is reduced by up to 85.7%, improving resource utilization efficiency and significantly reducing memory usage. This application's embodiment also employs a real-time jump control algorithm, using a fast response mechanism within 200 milliseconds to ensure smooth and continuous playback, solving the problem of poor real-time performance in existing solutions.

[0094] This application also proposes an intelligent audio silence detection and skip playback system to implement the above playback method, such as... Figure 2 As shown, the playback system 200 includes:

[0095] The audio processing module 202 is used to receive audio resources (audio files / data streams) transmitted by the audio input interface 201, process them to obtain digital audio signals and transmit them to the silence detection module 203, the playback control module 204 and the buffer management module 206.

[0096] The audio detection module 203 is used to receive digital audio signals from the audio processing module 202 and is configured to: when mute skipping is enabled, use a mute detection algorithm to identify mute segments and transmit the detection results to the playback control module 204 and the buffer management module 206 in the form of mute segment data structure.

[0097] The playback control module 204 is used to receive audio signals and silence segment information, execute playback control logic, including time jump and continuous playback, and finally transmit the audio signal to the audio output interface 205 and output it to the audio output device 208 for playback.

[0098] The cache management module 206 is used to establish a bidirectional data connection with the audio processing module 202 and the silence detection module 203, and is responsible for the intelligent caching and management of audio data and silence segment detection results.

[0099] In addition, the user interaction module 207 is used to interact with the silence detection module 203 and the playback control module 204 through event information, and to transmit user configuration parameters and control commands.

[0100] Furthermore, the audio detection module 203 in this embodiment specifically includes:

[0101] A sliding window analyzer is used to determine the parameters of overlapping sliding windows;

[0102] The RMS volume calculator is used to traverse the entire audio data by overlapping sliding windows and calculate the RMS value of each window.

[0103] An adaptive threshold adjuster is used to accumulate the RMS values ​​of all windows and calculate the average value to obtain the average RMS value. The adaptive threshold is then calculated based on this average RMS value.

[0104] A silence segment identifier is used to compare RMS values ​​with an adaptive threshold window by window to identify silence segments.

[0105] In addition, a continuity verifier is used to verify the minimum duration of the silent segments identified by the silent segment recognizer to determine the valid silent segments.

[0106] It should be noted that the specific implementation of each unit of the silence detection module 203 is as described in the above method, and will not be repeated here.

[0107] Furthermore, the playback control module 204 in this embodiment specifically includes:

[0108] Playback Status Manager is used to monitor playback in real time via events;

[0109] The silent skip actuator is used to activate the time jump controller to perform a time jump when the playback position is detected to be in a silent segment.

[0110] And a waveform visualizer for visualizing audio waveforms.

[0111] It should be noted that the algorithm or playback mechanism used by the playback control module in this application embodiment is as described in the above method, and will not be repeated here.

[0112] Furthermore, the user interaction module 207 in this embodiment of the application implements functions such as user operation (parameter configuration, operation instructions), system response (processing and response after receiving user operation), interface update (updating the user interface display according to the system processing result), playback control (executing specific playback control operations), and status feedback (providing feedback on operation results and system status to the user) through the user interface 300. When the user activates the mute skip function, the system automatically skips the marked mute segment during playback, achieving a continuous and efficient audio playback experience. Figure 3 As shown, the user interface 300 mainly includes the following key components:

[0113] The audio waveform display area 301 displays the time-domain waveform of the audio signal in real time.

[0114] The silent segment is marked in area 302 and displayed as a red semi-transparent overlay on the waveform.

[0115] Playback control area 303 includes basic control buttons such as play, pause, and stop;

[0116] Mute Skip Switch 304: A toggle switch for users to enable / disable the mute skip function.

[0117] The detection information display area 305 displays statistical information such as the number of detected silent segments and the total duration;

[0118] The parameter configuration area 306 allows users to adjust parameters such as the detection threshold and minimum silence duration.

[0119] The playback progress bar (307) displays the current playback position and supports dragging to jump to the next position.

[0120] In addition, there is a volume controller 308, a volume adjustment slider, and a mute button.

[0121] Based on the same technical concept described above, this application also proposes an electronic device, such as... Figure 4 As shown, the electronic device 400 includes: a memory 410, a processor 420, and a computer program A411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program A411, it performs the following steps:

[0122] The input audio resource is checked for cache. If the cached data exists, it is loaded directly, skipping the network request and decoding process. If the cached data does not exist, a network request is initiated and audio decoding is performed, and the result is cached.

[0123] Perform player initialization and determine whether silent skip is enabled. If so, use a silent detection algorithm to detect silent segments. Specifically, this includes: traversing the entire audio data according to the overlapping sliding window, calculating the RMS value of the audio signal in each overlapping sliding window, comparing the calculated RMS value with an adaptive threshold, performing silent segment identification and continuity verification, and caching the identified silent segments.

[0124] Real-time playback monitoring is performed. An interval search algorithm is used to detect whether the current playback position is in a silent segment. If it is, a time jump is executed to jump to a non-silent segment for playback. Otherwise, the standard playback mode is used for normal playback.

[0125] Determine if playback has ended. If so, playback is complete; otherwise, return to the real-time playback monitoring step to continue playback monitoring.

[0126] Optionally, when the processor 420 executes the computer program A411, it can implement any of the embodiments in the corresponding examples of the playback method described above.

[0127] It should be noted that the electronic device proposed in this application embodiment is a device used to implement the above-described playback method. Therefore, based on the above-described playback method proposed in this application embodiment, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this application embodiment. Therefore, how the electronic device specifically implements the above-described playback method will not be described in detail here. Any electronic device used by those skilled in the art to implement the above-described playback method falls within the scope of protection of this application.

[0128] Based on the same technical concept described above, embodiments of this application also propose a computer-readable storage medium, such as... Figure 5 As shown, the computer-readable storage medium 500 stores a computer program B511, which, when executed by a processor, performs the following steps:

[0129] The input audio resource is checked for cache. If the cached data exists, it is loaded directly, skipping the network request and decoding process. If the cached data does not exist, a network request is initiated and audio decoding is performed, and the result is cached.

[0130] Perform player initialization and determine whether silent skip is enabled. If so, use a silent detection algorithm to detect silent segments. Specifically, this includes: traversing the entire audio data according to the overlapping sliding window, calculating the RMS value of the audio signal in each overlapping sliding window, comparing the calculated RMS value with an adaptive threshold, performing silent segment identification and continuity verification, and caching the identified silent segments.

[0131] Real-time playback monitoring is performed. An interval search algorithm is used to detect whether the current playback position is in a silent segment. If it is, a time jump is executed to jump to a non-silent segment for playback. Otherwise, the standard playback mode is used for normal playback.

[0132] Determine if playback has ended. If so, playback is complete; otherwise, return to the real-time playback monitoring step to continue playback monitoring.

[0133] Optionally, when the computer program B511 is executed by the processor, it can implement any of the embodiments corresponding to the above playback method.

[0134] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0135] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0137] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0138] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0139] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An intelligent audio silent section detection and skip play method, characterized in that, Comprise: Cache check on input audio resource, if cache data exists, directly load cache data, skip network request, decoding process and silence segment detection process, if cache data does not exist, initiate network request and carry out audio decoding processing, cache processing result; Carry out player initialization, and judge whether to enable silence skipping, if yes, adopt silence detection algorithm to carry out silence segment detection, specifically comprising: traverse entire audio data according to overlapping sliding window, calculate RMS value of audio signal in each overlapping sliding window, compare calculated RMS value with adaptive threshold, carry out silence segment identification and continuity verification and cache identified silence segment; Carry out real-time playing monitoring, utilize interval search algorithm to detect whether current playing position is in silence segment, if yes, execute time jump, jump to non-silence segment position to carry out playing, otherwise, adopt standard playing mode to normally play; Judge whether playing is ended, if yes, playing is completed, otherwise, return to carry out real-time playing monitoring step to continue playing monitoring.

2. The method of claim 1, wherein the method further comprises: The silence detection algorithm comprises: Determine overlapping sliding window parameters, including: window size, sampling point number per window and sliding step, and sliding step should be less than window size; Traverse entire audio data according to overlapping sliding window, calculate RMS value of each overlapping sliding window; Accumulate RMS value of all overlapping sliding windows and calculate average value, obtain average RMS value; Calculate adaptive threshold according to average RMS value, wherein calculation method of adaptive threshold is: calculate product of average RMS value and preset coefficient, and take maximum value of product and preset threshold as adaptive threshold; Compare RMS value with adaptive threshold window by window, carry out silence segment identification and continuity verification on identified silence segment, determine effective silence segment and cache effective silence segment information.

3. The method of claim 2, wherein the method further comprises: The silence segment identification and continuity verification on identified silence segment comprises: Initialization state, including initialization of silence segment start time and continuous silence window; Compare RMS value with adaptive threshold window by window, when RMS value is less than adaptive threshold, record start time and continuously count, when RMS value is greater than or equal to adaptive threshold, check whether continuous duration meets minimum duration requirement, if yes, determine as effective silence segment.

4. The method of claim 1-3, wherein, The interval search algorithm comprises: For each silence segment in silence segment array, if current playing time is greater than or equal to silence segment start time and current playing time is less than or equal to silence segment end time, update silence segment end time as time after original silence segment end time plus buffer time; Otherwise, continue to check next silence segment; If all silence segments do not match, keep current playing position to normally play in standard playing mode.

5. The method of claim 1-3, wherein, Further comprise: Real-time display time domain waveform graph of audio signal, and mark silence segment on time domain waveform graph; Real-time display silence segment statistical information; Provide user interface.

6. The method of claim 1-3, wherein, Further comprise: The layered cache architecture is adopted to cache audio binary data, silence segment detection result and configuration parameters respectively; The timestamp-based cache expiration mechanism is adopted; And the following optimization strategies are adopted in real time to limit the memory of the Web environment: streaming processing is adopted for audio files exceeding the threshold size, weak reference is used for infrequently used cache items, and memory cleaning is actively triggered at appropriate times.

7. An intelligent audio silent section detection and skip play system, characterized in that, The playback method of any one of claims 1-6 is implemented, comprising: An audio processing module configured to receive audio resources transmitted by an audio input interface and process the audio resources to obtain digital audio signals and transmit the digital audio signals to a silence detection module, a playback control module and a cache management module; An audio detection module configured to receive the digital audio signals from the audio processing module, and when silence skipping is enabled, perform silence segment identification using a silence detection algorithm and transmit the detection result in the form of a silence segment data structure to the playback control module and the cache management module; A playback control module configured to receive audio signals and silence segment information, execute playback control logic including time skipping and continuous playback, and finally transmit the audio signals to an audio output interface and output the audio signals to an audio output device through the audio output interface for playback; A cache management module configured to establish a bidirectional data connection with the audio processing module and the silence detection module, and responsible for intelligent caching and management of audio data and silence segment detection results; And a user interaction module configured to interact with the silence detection module and the playback control module through event information, transmit user configuration parameters and control instructions.

8. The intelligent audio silent section detection and skip play system of claim 7, wherein, The audio detection module comprises: A sliding window analyzer configured to determine overlapping sliding window parameters, including window size, number of sampling points per window and sliding step, and the sliding step should be smaller than the window size; An RMS volume calculator configured to traverse the entire audio data according to the overlapping sliding window, and calculate the RMS value of each overlapping sliding window; An adaptive threshold adjuster configured to accumulate the RMS values of all overlapping sliding windows and calculate the average value to obtain the average RMS value, and calculate the adaptive threshold value according to the average RMS value; A silence segment identifier configured to compare the RMS value with the adaptive threshold value window by window to identify silence segments; And a continuity verifier configured to perform minimum length verification on the silence segments identified by the silence segment identifier to determine valid silence segments. 9.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor executes the computer program to implement the playback method of any one of claims 1-6.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the playback method of any one of claims 1-6.

Images