Conference window conversation system based on voice sensor

Abstract

The invention discloses a meeting window call system based on a voice sensor, and relates to the technical field of communication. Comprising a full-space voice acquisition module, a dynamic confidence evaluation module, a phase coupling enhancement module, a multi-dimensional speech mode recognition module, a dynamic gain adjustment and tone quality enhancement module and a full-link voice integrity verification module, an array type voice sensing unit is deployed based on audio reflection characteristics, a multi-band full-space acquisition matrix is constructed, voice signals of two parties meeting are acquired, and a basic audio data stream with a high signal-to-noise ratio is output. According to the invention, through array sensing acquisition, dynamic confidence evaluation, phase coupling and harmonic enhancement, multi-dimensional speech recognition and dynamic gain adjustment, accurate recognition and fidelity enhancement of low-volume speech are realized, and through combination of full-link integrity verification and mistaken killing backtracking, weak signals are ensured not to be missed and distorted, so that the accuracy of speech recognition is improved. And the privacy, the security and the information integrity of the meeting call are improved.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a meeting window communication system based on a voice sensor. Background Art

[0002] A voice sensor-based meeting window communication system is designed for use in isolated locations such as meeting rooms, banks, and service windows. By integrating a highly sensitive voice sensor, a digital signal processing module, and a voice amplifier, it enables clear voice communication between people on both sides of soundproof glass. The system uses a voice sensor to accurately capture the speaker's voice signal. It also incorporates noise suppression, echo cancellation, and voice enhancement algorithms to effectively filter out ambient noise and glass reflections, enhancing the clarity and confidentiality of voice calls. The system supports full-duplex and half-duplex call modes, avoiding voice overlap and crosstalk, ensuring natural and smooth communication between the two parties. It is particularly suitable for window-based communication scenarios where security, privacy, and call quality are paramount.

[0003] The existing technology has the following deficiencies: Existing meeting window call systems lack the ability to recognize special voice states. Specifically, when one of the two parties in a meeting is in a depressed mood, psychologically depressed, or uses low-energy voice expressions such as whispering for privacy reasons, the resulting voice signal has characteristics such as low volume, slow speech speed, and small frequency variation. These signals are easily misidentified by the call system as background noise and automatically blocked or ignored. Because these voice signals are not effectively collected, amplified, and forwarded in the audio transmission link, the speech content is completely unperceivable at the receiving end, leading to information loss, semantic interruption, or misunderstanding during the call. This problem can lead to the omission of important information, the failure of communication goals, and even the inaccurate assessment of key factors such as the parties' psychological state and behavioral tendencies in highly sensitive conversations, such as legal visits, psychological counseling, and family comfort. This poses a high risk.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the present invention is to provide a meeting window call system based on voice sensors, which realizes accurate recognition and fidelity enhancement of low-volume voice through array sensor acquisition, dynamic confidence assessment, phase coupling and harmonic enhancement, multi-dimensional speech recognition and dynamic gain adjustment. Combined with full-link integrity verification and false positive backtracking, it ensures that weak signals are not missed or distorted, and improves the privacy, security and information integrity of meeting calls to solve the problems in the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a voice sensor-based meeting window communication system, comprising a full-space voice acquisition module, a dynamic confidence assessment module, a phase coupling enhancement module, a multi-dimensional speech pattern recognition module, a dynamic gain adjustment and sound quality enhancement module, and a full-link voice integrity verification module: The full-space voice acquisition module deploys an array of voice sensor units within the meeting window space based on audio reflection characteristics, constructs a multi-band full-space acquisition matrix, collects the voice signals of both parties in the meeting, and outputs a basic audio data stream with a high signal-to-noise ratio; The dynamic confidence assessment module performs dynamic confidence assessment on the basic audio data stream, extracts energy distribution and spectrum stability features, and generates a set of feature weights to distinguish speech from background noise; The phase coupling enhancement module constructs a phase coupling enhancement channel based on a set of feature weights. It enhances the structural coherence and energy density of the audio signal through phase consistency mapping and harmonic synchronous amplification. A multi-dimensional speech pattern recognition module performs multi-dimensional speech pattern recognition based on the enhanced audio signal, identifying human speech attributes based on pitch continuity, rhythm changes, and pause patterns; Dynamic gain adjustment and sound quality enhancement module, which dynamically adjusts the gain mapping curve and spectrum channel parameters of the audio signal transmission chain based on the recognized human voice attributes to implement sound quality fidelity enhancement; The full-link voice integrity verification module performs full-link voice integrity verification on the enhanced audio signal. Through attribute feature comparison and false positive backtracking verification, it achieves complete restoration and accurate reception of low-volume voice signals.

[0007] Preferably, the basic audio data stream acquisition steps are as follows: Based on the material, location, and acoustic reflection characteristics of the walls, ceiling, floor, and soundproof glass in the meeting window space, perform sound field simulation and modeling to determine the sound reflection path, attenuation coefficient, and standing wave distribution; Based on the sound field simulation results, highly sensitive capacitive or piezoelectric voice sensors are selected, arranged in a mixed pattern of equal and non-equal spacing, and sensors with tilted or directional pickup are set up to build a full-space acquisition matrix covering multiple frequency bands. The signal of each voice sensor is pre-amplified and digitized through the preamplifier circuit and analog-to-digital conversion unit, and then enters the multi-channel signal synchronization platform based on timing synchronization to achieve phase consistency and time synchronization; The digitized signal is divided into frequency bands and weighted based on a multi-band filter bank, and background noise is reduced by weighted averaging and spectral subtraction. Dynamic amplitude normalization processing with adaptive gain control is then performed to output a basic audio data stream with a high signal-to-noise ratio.

[0008] Preferably, the steps for generating the feature weight set are as follows: Based on the basic audio data stream, the energy distribution characteristics, spectrum stability characteristics, spectrum envelope similarity, spectrum entropy and main frequency component persistence of each frame signal are extracted to form a multi-dimensional feature vector of the time series; The multi-dimensional feature vector sequence is input into a deep learning model composed of a convolutional neural network and a bidirectional long short-term memory network in series, and the confidence score of each frame signal is output; Smoothing and thresholding of the scoring sequence to identify valid speech areas where the scores of consecutive frames are higher than the threshold; Based on the confidence score and each feature, a feature weight set is generated for signal enhancement and recognition processing.

[0009] Preferably, the specific steps of enhancing the structural coherence and energy density of the audio signal by using phase consistency mapping and harmonic synchronous amplification based on the feature weight set are as follows: Based on the energy distribution characteristics and spectrum stability characteristics of the feature weight set, a phase tracking model of the signal is established. The instantaneous phase information is extracted through Hilbert transform, the phase trajectory is constructed, and the target frequency with continuous phase and stable amplitude is selected. Perform phase consistency mapping on the target frequency and adjacent frequencies, perform phase compensation based on phase similarity, correct phase offset, and maintain phase consistency of frequency components; Apply dynamic gain control to the main frequency and its multiple harmonic components, dynamically adjust the gain according to the characteristic weight set, enhance the energy density and maintain the natural harmonic energy distribution ratio; Perform structural coherence review in the time-frequency domain, dynamically optimize based on phase spectrum difference, spectrum envelope change rate and waveform smoothness indicators, and output enhanced audio signals with high energy density and strong structural coherence.

[0010] Preferably, based on the enhanced audio signal, the specific steps of identifying whether the signal has human speech attributes through multi-dimensional analysis of pitch continuity, rhythm changes and pause patterns are as follows: The enhanced audio signal is subjected to short-time Fourier transform to extract the fundamental frequency and harmonics, form a pitch contour curve, and calculate the fundamental frequency smoothness and stability; The energy envelope curve and the fundamental frequency change rate are used to jointly analyze the rhythm density, beat interval and accent position, and the periodic and non-periodic rhythm components are identified by combining wavelet transform and rhythm synchronization function. Long short-term memory networks are used to predict the continuation and pause probabilities of speech segments based on time series features, distinguishing between continuous pronunciation and natural pauses, and forming pause-continuation patterns. The features of pitch continuity, rhythm changes and pause patterns are input into the support vector machine classification model, and the human speech attribute confidence score of each signal segment is output to determine the speech attributes of the audio signal.

[0011] Preferably, the specific steps of dynamically adjusting the gain mapping curve and spectrum channel parameters of the audio signal transmission chain to implement sound quality fidelity enhancement are as follows: Based on the recognized human speech attributes, a dynamic gain mapping curve is constructed, the gain boost coefficient is set according to the main frequency and harmonic distribution, and gain suppression is set for the non-speech frequency band; Dynamically adjust the spectrum channel parameters according to the speech attribute weight, adjust the center frequency, bandwidth and quality factor of the filter, enhance the voice band and suppress the noise band; The dynamic gain mapping curve and spectrum channel parameters are mapped to each time frame and frequency unit through digital signal processing algorithms, and phase compensation and spectrum reconstruction are applied to perform frame-by-frame and frequency band enhancement of the audio signal. Real-time sound quality evaluation is performed on the enhanced signal. Based on the signal-to-noise ratio, spectral smoothness, dynamic range and harmonic distortion rate indicators, the gain mapping curve and spectral channel parameters are dynamically optimized through a feedback mechanism to ensure the enhanced sound quality.

[0012] Preferably, the specific steps for achieving complete restoration and accurate reception of low-volume voice signals through attribute feature comparison and false positive backtracking verification are as follows: Extract five attribute features of the audio signal after adjusting the gain mapping curve and spectrum channel parameters: energy distribution, spectrum stability, pitch continuity, rhythm density and pause pattern; The five attribute features of the enhanced and unenhanced signals are compared using cosine similarity, dynamic time warping, Euclidean distance, rhythm density comparison, and edit distance algorithms to form a global speech similarity score. For signal segments with scores below the consistency threshold, we trace back to the original signal, apply different gain and spectrum parameter configurations for enhancement, and select the optimal configuration through Bayesian optimization to replace the original signal segment. The five types of features are extracted again from the corrected signal and the signal-to-noise ratio, harmonic distortion rate and dynamic range are calculated. The integrity and sound quality indicators are reviewed, and the final enhanced signal is output after meeting the standards.

[0013] In the above technical solution, the technical effects and advantages provided by the present invention are: The present invention improves the spatial and frequency coverage of signal acquisition through an array sensor layout, and realizes the intelligent distinction between human voice and background noise through dynamic confidence evaluation and feature weight construction. Phase coupling and harmonic enhancement effectively restore the energy density and structural coherence of low-volume speech. Furthermore, through multi-dimensional speech pattern recognition and dynamic gain adjustment, the human voice attributes are precisely enhanced to avoid synchronous noise amplification, and under the protection of full-link integrity verification and false positive backtracking mechanism, weak signal information is not missed or distorted. The system is particularly suitable for application scenarios with high dialogue sensitivity and strict information integrity requirements, such as legal visits, psychological counseling, and family comfort. It significantly improves the comprehensive capabilities and application value of the call system in terms of privacy, security, and complete transmission of voice information. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction to the drawings required for use in the embodiments will be given below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0015] Figure 1 The module diagram of a voice sensor-based meeting window communication system of the present invention is shown in FIG. DETAILED DESCRIPTION

[0016] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0017] The present invention provides Figure 1 The voice sensor-based meeting window communication system shown in the figure includes a full-space voice acquisition module, a dynamic confidence assessment module, a phase coupling enhancement module, a multi-dimensional speech pattern recognition module, a dynamic gain adjustment and sound quality enhancement module, and a full-link voice integrity verification module: The full-space voice acquisition module deploys an array of voice sensor units within the meeting window space based on audio reflection characteristics, constructing a full-space acquisition matrix covering multiple frequency bands. This module collects full-band voice signals generated by both parties and outputs a basic audio data stream with a high signal-to-noise ratio. The following steps are specifically included for deploying an array of voice sensor units in the meeting window space to achieve high signal-to-noise ratio acquisition of full-band voice signals: Based on the physical layout of the meeting window, sound field simulation and modeling are performed according to the material, location, and acoustic reflection characteristics of the walls, ceiling, floor, and soundproof glass within the meeting space to determine the sound reflection path, attenuation coefficient, and standing wave distribution within the space. Based on the sound field simulation results, highly sensitive capacitive or piezoelectric voice sensors are selected and arranged in a targeted array according to the identified sound reflection hotspots, areas prone to low-frequency standing wave accumulation, and areas prone to high-frequency reflection dissipation. The voice sensors in the array are arranged in a mixed manner of equal and non-equal spacing to ensure optimal collection angles and coverage for sound waves in multiple frequency bands from low to high frequencies. At the same time, inclined or directional sound pickup sensors are set in the front, side, and diagonal directions of the window to enhance the direct sound wave collection effect of users on both sides of the window and reduce the interference of spatial multipath reflections on the original signal.

[0018] To improve the quality and integrity of voice signals, the picked-up signal from each voice sensor is pre-amplified and digitized through a preamplifier circuit and an analog-to-digital conversion unit. The preamplifier circuit used features low noise, wide bandwidth, and high dynamic range, ensuring effective energy boosting of weak voice signals during the pickup phase while suppressing sensor noise and environmental electromagnetic interference. The sampling rate of the analog-to-digital conversion unit is set to no less than 96kHz, with a quantization accuracy of 24 bits to ensure spectrum restoration and detail preservation in subsequent digital signal processing. After digitization, each sensor signal enters a multi-channel signal synchronization platform based on timing synchronization. This platform applies precise timestamps to the signals of all channels to ensure phase consistency and time synchronization of the multi-channel signals in subsequent signal fusion and enhancement processing.

[0019] For the digitized audio signals obtained from multiple sensors, the signals are divided and weighted based on a preset multi-band filter group. The filter group divides the audio signals into three main frequency bands: low frequency, medium frequency, and high frequency, and sets adaptive filtering parameters based on the noise characteristics of each frequency band. In this process, a method based on weighted averaging and spectral subtraction is used to effectively reduce the background noise, wind noise, and electromagnetic noise in the signal, while improving the signal-to-noise ratio of the voice signal while maintaining the integrity of the spectrum. During the weighted averaging process, different weights are applied to the signals of sensors at different positions based on the aforementioned sound field simulation data to enhance the amplitude of the forward voice signals from people on both sides of the window, while weakening the energy of secondary or multiple reflected sound waves from the spatial reflection path to avoid signal blurring or ghosting effects caused by reflected sound waves.

[0020] After completing multi-band filtering and weighted fusion, dynamic amplitude normalization processing based on adaptive gain control is performed to uniformly adjust the signals of different channels and frequency bands to the optimal dynamic range, preventing information loss or signal distortion caused by uneven amplitude or too narrow dynamic range in subsequent signal processing. The normalized signal undergoes real-time signal-to-noise ratio evaluation and data integrity verification to form the final high-signal-to-noise ratio basic audio data stream. This data stream has the characteristics of balanced spectral distribution, moderate amplitude dynamics, and consistent phase timing, and can provide a high-quality original audio foundation for subsequent in-depth processing such as dynamic confidence assessment, phase coupling enhancement, and multi-dimensional speech recognition.

[0021] This step aims to lay a high-quality, complete signal foundation for the subsequent intelligent recognition, enhancement, and fidelity transmission of audio signals. This is especially important for achieving accurate and comprehensive audio information collection for the special low-volume, low-frequency speech conditions found in meeting window spaces. Meeting window spaces typically contain structures such as soundproof glass, hard walls, and ceilings. These structures can cause multipath reflections, sound wave attenuation, and standing wave effects, resulting in weak signal energy transmitted directly from the sound source to the receiver, which is easily masked by background noise. By deploying array-type voice sensor units, combined with audio reflection characteristics and spatial layout, a multi-band, full-space acquisition matrix can be constructed within the space, enabling simultaneous capture of sound from multiple directions and frequency dimensions, avoiding sound source omissions or signal distortion caused by single-point sensor acquisition. This acquisition matrix can evenly pick up direct sound and some valuable reflected sound, while simultaneously improving the spatial resolution and signal-to-noise ratio of the signal through sound source localization and phase alignment of the array structure. The resulting basic audio data stream boasts high fidelity, high integrity, and a high signal-to-noise ratio. It not only fully preserves the speaker's original voice characteristics, but also effectively isolates and attenuates ambient noise, glass reflections, and spatial reverberation. This signal foundation is the prerequisite for subsequent processing steps such as dynamic confidence assessment, phase coupling enhancement, and multi-dimensional speech pattern recognition. It ensures accurate transmission and restoration without missing information or distorting sound quality in complex communication scenarios, including low-energy speech, whispers, and emotional distress.

[0022] A dynamic confidence assessment module performs dynamic speech signal confidence assessment on the basic audio data stream, extracts energy distribution characteristics and spectrum stability characteristics in the audio signal, and generates a set of feature weights for distinguishing speech signals from background noise; Performing dynamic speech signal confidence assessment on the basic audio data stream specifically includes the following steps: Based on the collected high signal-to-noise ratio basic audio data stream, a multi-level signal analysis model is constructed for the time domain and frequency domain characteristics of the audio signal. This model obtains the spectral distribution and energy density of each frame of audio by performing a short-time Fourier transform on the audio signal. At the same time, a sliding window is used in the time domain to calculate the energy envelope curve of the signal to capture the energy change trend of the audio signal. To ensure the precision of the analysis, the window length of the short-time Fourier transform is set to 32 milliseconds and the frame shift is set to 10 milliseconds, thereby ensuring both spectral resolution and time resolution. Through this step, it is possible to fully capture the frequency components, energy intensity and transient changes of the signal in each time segment.

[0023] The energy envelope curve refers to the description of the energy change trend of the audio signal in the time series. It is a time-varying curve formed by integrating or averaging the amplitude or power of the audio signal in each short time segment. Its essence is to fit the amplitude change of the original audio signal with a smooth curve, reflecting the strength fluctuations of the audio signal in different time periods. The energy envelope curve is used in the present invention to track and quantify the dynamic changes of energy in the basic audio data stream in real time, especially for low-volume, whispering, depressed speech and other low-energy speech. Although the overall volume is low, its fluctuations in time still have certain speech characteristics, which are different from the randomness and stable low-amplitude changes of background noise. By analyzing the energy envelope curve, the speech rhythm, pronunciation fluctuations and other laws in the signal can be effectively identified, providing a time domain signal strength basis for subsequent dynamic confidence assessment, and combined with the spectral characteristics, improving the ability to distinguish low-energy speech from noise. This has a key discriminant value in preventing weak signals from being misjudged as noise and being blocked.

[0024] Based on the above signal analysis model, the energy distribution characteristics and spectral stability characteristics of the audio signal in each time window are further extracted. The energy distribution characteristics are described by calculating the energy proportion of each frequency sub-band, the center frequency drift amplitude and the energy change gradient, thereby reflecting the strength and stability of the signal. The spectral stability characteristics quantify the persistence and structural coherence of the signal in the frequency domain by calculating the similarity of the spectral envelope, spectral entropy and persistence of the main frequency component between adjacent frames. In particular, for low-volume, slow-speaking speech signals, their spectral changes are relatively stable but have the harmonic characteristics of human voice. The above feature extraction can effectively distinguish them from the broadband and random spectral characteristics presented in background noise.

[0025] The similarity of the spectral envelopes, spectral entropy, and persistence of the dominant frequency components between adjacent frames can be used to quantify the persistence and structural coherence of audio signals in the frequency domain from multiple dimensions. Specifically, the audio signal is first divided into short time frames of equal length. The spectral envelope curve of each frame is obtained through Fourier transform. The cosine similarity or correlation coefficient of the spectral envelopes of two adjacent frames is then calculated. A high similarity indicates temporal continuity of the spectral morphology, reflecting the stability of the signal's frequency components. Next, the spectral entropy of each frame is calculated. Spectral entropy reflects the degree of disorder in the frequency energy distribution. Low spectral entropy indicates that energy is concentrated at a specific frequency, indicating a well-structured signal, while noise typically has a high spectral entropy and irregular energy distribution. Finally, the dominant frequency of each frame is detected, i.e., the frequency component with the most concentrated energy. Its occurrence and persistence across multiple consecutive frames are counted. If the dominant frequency component persists across multiple frames with minimal fluctuation, it indicates that the signal has stable harmonics or fundamental frequency, reflecting structural coherence. Through the joint evaluation of these three, the stability and regularity of the signal in the frequency domain can be accurately quantified, thereby distinguishing ordered signals such as human voice from disordered signals such as background noise.

[0026] Signal discrimination is performed based on the extracted energy distribution features and spectral stability features using a dynamic confidence assessment model based on machine learning. This model uses a structure that combines a convolutional neural network and a recurrent neural network. The former is used to capture spatial features in the spectrum, and the latter is used to learn the time series characteristics of the signal. The model input is the sequence of feature vectors extracted in the first two steps, and the output is the confidence score of each frame of the signal. The higher the score, the more likely the frame signal is a human speech signal. To improve the model's ability to recognize low-volume and whispered speech, the model introduces a large number of low-signal-to-noise ratio, low-energy speech samples during the training phase, and uses comparative learning to improve the model's sensitivity to subtle speech features. This step ensures that the speech attribute confidence of each signal segment can be accurately assessed in real time in a dynamically changing noise environment.

[0027] The structure combining convolutional neural networks and recurrent neural networks refers to a deep model architecture formed by sequentially cascading convolutional neural networks and recurrent neural networks. The convolutional neural network first performs local perception and spatial pattern extraction on the input feature vector sequence, while the recurrent neural network performs temporal dependency analysis and dynamic feature modeling on the sequence data after convolution extraction. In this invention, the convolutional neural network uses multiple layers of convolution kernels to extract local information such as the spectral characteristics, energy distribution, and spectral envelope of each frame of the signal. This effectively extracts the local frequency patterns and structural features implicit in the speech signal, thereby enhancing the model's ability to distinguish speech from noise at the spectral level. The recurrent neural network, particularly the long short-term memory (LSTM) or bidirectional LSTM, learns the dynamic changes and dependencies between previous and subsequent frames of the signal along the time series dimension based on the feature sequence output by the convolutional layer, capturing the coherence and rhythmicity of the speech signal along the time axis, as well as the random fluctuations of the noise signal. Through this combination, the model performs in-depth signal discrimination in both spatial and temporal dimensions. Finally, the output of the recursive neural network is mapped to a confidence score for each frame of the signal through a fully connected layer and an activation function. The higher the value, the more likely the frame signal is human speech, achieving accurate signal discrimination and dynamic scoring.

[0028] By inputting the feature vector sequence extracted in the first two steps, the confidence score of each frame signal is output. The specific steps include: The energy distribution characteristics, spectrum stability characteristics, spectrum envelope similarity, spectrum entropy and main frequency component persistence corresponding to each frame of audio signal are combined into a multidimensional feature vector in a unified format, and a feature vector sequence is formed in the order of time series.

[0029] The feature vector sequence is input into a pre-trained deep learning model, which consists of a convolutional neural network layer and a bidirectional long short-term memory network layer (BiLSTM) in series. The convolutional layer is used to extract the local spatial features and spectral change patterns in the feature sequence, and the BiLSTM layer captures the temporal dependencies and contextual information in the feature sequence, improving the model's ability to recognize speech continuity and structure.

[0030] The output of BiLSTM is mapped into a one-dimensional score sequence corresponding to the number of time frames through a fully connected layer, and the score is normalized to between 0 and 1 through a sigmoid activation function, indicating the probability that each frame signal is human speech.

[0031] The scoring sequence is smoothed and thresholded, and signal segments with scores higher than the set threshold for multiple consecutive frames are determined to be valid speech areas, achieving accurate quantification of the confidence of each frame signal.

[0032] Based on the scoring results of the dynamic confidence assessment model, combined with the energy distribution characteristics and spectral stability characteristics of each time segment, a feature weight set is generated for subsequent signal enhancement and recognition processing. This feature weight set is a multidimensional weight matrix that contains weighting coefficients for the energy of each frequency band, enhancement factors for spectral coherence, and normalized confidence scores. It serves as an important parameter basis for guiding subsequent phase coupling enhancement and multidimensional speech pattern recognition. Through this feature weight set, differentiated enhancement efforts can be applied to signals in different frequency bands and time slices in the subsequent signal enhancement process, achieving targeted optimization of low-volume speech while suppressing interference from background noise.

[0033] This step aims to accurately identify valid speech segments and background noise by dynamically assessing the confidence of the underlying audio data stream. This provides a scientific and objective basis for subsequent speech signal enhancement, recognition, and fidelity transmission. In a meeting window call scenario, multiple complex noise sources exist, such as glass reflections, low-frequency ambient noise, and non-speech movements by both parties. Furthermore, the inherently weak signal strength of speech in unusual speech states, such as low volume, whispering, and suppressed emotions, can easily be misclassified as noise and filtered out. Therefore, relying solely on simple energy detection or traditional noise thresholding is difficult to achieve accurate signal discrimination. This step first extracts the audio signal's energy distribution features, such as energy density and energy gradient in each frequency band, to quantify the signal's strength and dynamics. Spectral stability features, such as spectral envelope similarity, spectral entropy, and persistence of the main frequency component, are then used to reflect the signal's structural coherence and stability in the frequency domain. These features are dynamically evaluated by a deep learning model to generate a confidence score for each time frame, which is then used to form a set of feature weights to guide signal enhancement and recognition. This feature weight set can not only dynamically adapt to changes in environmental noise and signal strength, but also provide refined parameter guidance for subsequent steps such as phase coupling enhancement and multi-dimensional speech pattern recognition, ensuring that low-volume and special voice state signals are fully preserved and accurately identified throughout the entire link, thereby improving the overall perception ability and information restoration of the call system.

[0034] The phase coupling enhancement module constructs a phase coupling enhancement channel for low-volume speech signals based on the energy distribution and spectral stability characteristics corresponding to the feature weight set. It applies phase consistency mapping and harmonic synchronous amplification to enhance the structural coherence and energy density of the audio signal. Based on the energy distribution characteristics and spectral stability characteristics corresponding to the feature weight set, constructing a phase coupling enhancement channel for low-volume speech signals specifically includes the following steps: Based on the feature weight set generated in the previous step, the energy distribution characteristics and spectral stability characteristics are used as input to establish a phase tracking model for the signal. The model continuously tracks the phase of the main frequency component and its harmonic components of each frame of the audio signal, extracts the instantaneous phase information using the Hilbert transform, and constructs a phase trajectory on the time series. By matching with the spectral stability factor in the feature weight set, the frequency components with continuous phase changes and smooth amplitude changes are screened out as the target frequencies for subsequent enhancement. For the screened target frequencies, a phase differential mapping table of their adjacent frequency components is further established to ensure that the phase relationship between adjacent frequencies is effectively preserved during the subsequent phase coupling enhancement process, preventing spectral distortion or phase distortion after signal enhancement.

[0035] The Hilbert transform is a mathematical transformation that can convert a real signal into a complex analytical signal. By introducing the orthogonal components of the signal, the instantaneous phase and instantaneous amplitude of the signal at each time point can be calculated. In this step, the Hilbert transform is used to extract the phase of the frequency components of the low-volume speech signal. By obtaining the instantaneous phase information of the signal, the phase change trajectory of the signal on the time axis is dynamically depicted to assist in determining whether the frequency components of the signal have continuity and harmonic characteristics. The specific steps include: Apply Hilbert transform to the enhanced audio signal to obtain the complex analytical signal corresponding to the signal, the real part of which is the original signal and the imaginary part is the orthogonal signal with a 90-degree phase shift.

[0036] The instantaneous phase at each time point is calculated by analyzing the signal. The instantaneous phase is obtained by the inverse tangent function of the real and imaginary parts of the complex signal, accurately describing the phase angle of the signal at that moment.

[0037] The instantaneous phase values ​​at consecutive time points are expanded and the 2π phase jump is eliminated, and a continuous phase sequence is formed through the phase unwrapping operation.

[0038] A phase trajectory is constructed based on this continuous phase sequence. The phase consistency and coherence of each main frequency and its harmonics in the signal are evaluated based on the speed of phase change of different frequency components. If the phase trajectory is smooth and continuous, it indicates that the frequency component belongs to a stable human speech signal; otherwise, it is judged to be background noise or interference frequency. This process ensures dynamic tracking and structural enhancement of the speech signal at the phase level.

[0039] After completing the phase tracking of the target frequency and establishing the mapping table, perform phase consistency mapping processing. The specific method is to synchronously adjust the phase of the target frequency and its adjacent frequencies, calculate the phase similarity of the adjacent frequency components, identify the frequency points with abnormal phase offset, and correct them through the phase compensation algorithm to keep the phases of all target frequencies and their harmonic components highly consistent. This step eliminates the phase disorder caused by noise interference, signal attenuation or recording errors in multi-band signals through synchronous phase adjustment, thereby improving the overall structural coherence of the signal in the frequency domain. The signal after phase consistency mapping presents a clearer human voice harmonic structure in the spectral distribution, forming a phase spectrum that is highly consistent with the characteristics of the original speech signal.

[0040] Based on the audio signal that has completed phase consistency mapping, harmonic synchronous amplification processing is implemented. Specifically, dynamic gain control is applied to the identified main frequency and its harmonic components. The gain value is dynamically adjusted according to the energy distribution weight and the spectrum stability weight in the feature weight set to ensure that the enhanced amplitude matches the energy characteristics of the signal itself. In order to avoid the imbalance of sound quality caused by excessive enhancement of a single frequency, a cross-frequency energy balance mechanism is introduced to ensure that the amplitude ratio of the enhanced main frequency and its harmonic components is consistent with the harmonic energy distribution of natural human voice. In this way, the naturalness and clarity of the voice signal can be maintained while improving the overall energy density of the signal. In particular, when processing low-volume and whisper-like signals, their audibility and transmission fidelity can be significantly enhanced.

[0041] For the audio signal after harmonic synchronous amplification, a structural consistency review in the time-frequency domain is performed to ensure the continuity and smoothness of its phase spectrum, amplitude spectrum, and time-domain waveform after signal enhancement. The specific method is to dynamically adjust the gain curve and phase compensation amount by calculating the phase spectrum difference, spectrum envelope change rate, and waveform smoothness index of the signal before and after enhancement to avoid introducing new phase distortion or spectral artifacts during the enhancement process. After passing the review, the final enhanced audio signal is output. This signal has the characteristics of high energy density, strong structural coherence, and natural spectral distribution. It can provide a high-quality signal foundation for subsequent multi-dimensional speech pattern recognition and dynamic gain adjustment, ensuring the clear transmission and restoration of low-volume speech in complex noise environments.

[0042] The purpose of this step is to address the problem that low-volume voice signals are easily misjudged as background noise by the call system due to their weak energy, small frequency variation amplitude, and unstable signal structure. By deeply enhancing the phase and spectral structure of the signal, the structural coherence and energy density of the audio signal are comprehensively improved, and the weak voice signal can be accurately restored and robustly enhanced. The harmonic distribution of low-volume voice signals in the frequency domain is not obvious enough, the amplitude of the main frequency and harmonic components is low, and the phase information on the time axis is easily affected by noise and becomes chaotic, resulting in reduced signal integrity and recognition. Based on the energy distribution characteristics and spectral stability characteristics in the feature weight set, the important frequency components and corresponding timing characteristics in the signal can be first screened out, and then the phases of the various frequency components of the signal can be synchronized and aligned through phase consistency mapping to eliminate phase confusion caused by noise or energy attenuation and ensure the coherence and continuity between frequency components. Subsequently, combined with harmonic synchronous amplification technology, the harmonic components of the human voice are dynamically amplified according to the natural energy ratio between the main frequency and its multiples. This not only increases the overall energy density but also restores the harmonic structure of the voice signal, making weak voice signals more recognizable and natural in the spectrum. Through this series of processing, the sound quality and integrity of low-volume voice signals are significantly improved when traversing noisy environments and signal transmission links. This solves the technical pain point of traditional communication systems' inability to capture and restore weak voice signals, making it particularly suitable for communication scenarios such as meeting windows where information accuracy and privacy are extremely important.

[0043] A multi-dimensional speech pattern recognition module performs multi-dimensional speech pattern recognition on the audio signal enhanced by the phase-coupled enhancement channel, identifying the corresponding human speech attributes in the audio signal based on pitch continuity, rhythm changes, and pause patterns; The multi-dimensional speech pattern recognition is performed based on the audio signal enhanced by the phase coupling enhancement channel, specifically comprising the following steps: Pitch feature extraction is performed on the audio signal processed by the phase-coupled enhancement channel. This is primarily done by converting the audio signal into spectral data through a short-time Fourier transform. The fundamental frequency and its harmonic distribution are then extracted within each time frame to form a pitch contour curve. To improve the accuracy of pitch extraction for low-volume speech, a weighted autocorrelation method is combined with a time-frequency domain fusion algorithm to ensure the accuracy of fundamental frequency detection while also taking into account the continuity and details of pitch changes. Furthermore, the smoothness and stability of the fundamental frequency within consecutive time frames are calculated to determine the coherence of the pitch. If the pitch curve shows a continuous and gentle change trend on the time axis, and the harmonic components are distributed proportionally to the fundamental frequency, the signal is preliminarily determined to possess the pitch properties of human speech.

[0044] Based on the extracted pitch features, the rhythm variation pattern is further analyzed. By dynamically detecting the energy envelope curve and fundamental frequency change rate of the audio signal in the time series, the rhythm density, beat interval, and stress position of each frame signal are calculated to form a complete rhythm feature sequence. In order to distinguish natural speech from background noise or mechanical interference, an analysis method combining wavelet transform and rhythm synchronization function is used to identify the periodic and non-periodic components in the rhythm. The rhythmic changes of the periodic components often correspond to the speaking speed and stress pattern of human language, while the non-periodic components are often associated with noise or non-speech signals. In this way, the rhythmic contour of the speech signal can be effectively captured, providing a dynamic feature basis for further determination of speech attributes.

[0045] The energy envelope curve refers to a smooth curve that describes the energy intensity change trend of an audio signal in a time series. By performing envelope fitting on the short-time energy or amplitude of the signal, it can reflect the loudness fluctuations and dynamic changes of the sound. The fundamental frequency change rate refers to the speed at which the fundamental frequency changes between adjacent time frames, indicating the amplitude and rate of change of the pitch over time, and can depict the changing rhythm of the pitch in the speech. In the present invention, the two are used together to reveal the rhythm and stress position of the speech signal, and then extract the rhythm features. The specific steps are: The short-time energy of each frame of audio signal is calculated, and envelope fitting is applied to the energy values ​​of all frames to form a smooth energy envelope curve. The peak of the curve corresponds to the accent or strong pronunciation part in the speech, and the valley corresponds to the pause or weak sound.

[0046] By extracting the fundamental frequency of each frame and calculating the difference between the fundamental frequencies of adjacent frames, the fundamental frequency change rate is obtained, which reflects the speed of the change in pitch.

[0047] The energy envelope curve and the fundamental frequency change rate are jointly analyzed to detect the local maximum values ​​in the envelope curve and the corresponding fundamental frequency fluctuations. These points are marked as potential accent positions, and the beat interval is further calculated by counting the time intervals between peaks.

[0048] Based on the distribution of stress positions and beat intervals, a rhythm density index is formed, and the number of stresses per unit time is counted. Finally, a complete rhythm feature sequence including rhythm density, beat intervals and stress positions is generated, providing accurate data support for subsequent speech attribute recognition and rhythm pattern analysis.

[0049] The analysis method combining the wavelet transform with a rhythm synchronization function uses the wavelet transform to perform a multi-scale decomposition of the audio signal, extracting rhythmic features at different time scales. The rhythm synchronization function then measures the synchronization of the decomposed signal energy with rhythmic variations, thereby distinguishing periodic components (such as the prosody and rhythm of natural speech) from non-periodic components (such as irregular fluctuations such as background noise and mechanical interference). The wavelet transform unfolds the audio signal in the time-frequency domain, capturing not only transient changes but also rhythmic features across different frequency bands, thus avoiding the limitations of the traditional Fourier transform, which cannot account for both time-frequency locality. The rhythm synchronization function, often a tempogram or autocorrelation function, quantifies the degree of synchronization between the audio signal and different rhythmic templates at various time points.

[0050] The specific steps are: Perform continuous wavelet transform on the audio signal to obtain the energy distribution at different time scales, especially focusing on the frequency bands related to speech rhythm, such as the modulation frequency in the range of 1Hz to 10Hz, to capture the rhythmic pulses related to speech prosody.

[0051] The energy envelope curve of the wavelet coefficients is used to calculate the rhythm synchronization function at each scale. A common method is to perform an autocorrelation function on the energy sequence at each scale to extract the periodicity of the sequence. The peak of the periodicity represents the basic period of the rhythm.

[0052] Combining the synchronization results of multiple scales, a time-rhythm-frequency synchronization matrix is ​​constructed. The periodic rhythm components of the signal are identified by detecting the strong synchronization areas in the synchronization matrix, and the parts with low synchronization and no significant period are marked as non-periodic components.

[0053] By separating periodic and non-periodic components, we can effectively distinguish natural speech rhythm from background noise and mechanical interference, ensuring that the true rhythm pattern in low-volume speech signals can be accurately extracted and recognized.

[0054] Based on the rhythmic and tonal features, the recognition of pause patterns is implemented. The pause pattern reflects the structure of pauses and continuous pronunciations in the speech stream. By calculating the time interval between the local minimum and maximum values ​​in the energy envelope curve and combining whether the fundamental frequency changes intermittently, the natural pause positions in the speech are identified. Especially for low-volume, whispering and other signals, the pauses are not obvious, so a time series prediction model based on a long short-term memory network is introduced in this step to predict the continuation probability and pause probability of continuous speech segments, ensuring that the pause relationship can be accurately distinguished even in weak speech. By quantifying the pause duration, frequency and its correlation with rhythm and tone, a complete description of the pause pattern is formed, thereby further confirming the natural speech properties of the signal.

[0055] The time series prediction model based on the long short-term memory network refers to a deep learning method that uses the long short-term memory network (LSTM) to model and predict the time series characteristics of audio signals. LSTM is a special recursive neural network that excels at capturing long-term and short-term dependencies in time series data and can effectively retain and forget key information in time series. In the present invention, the role of this model is to learn the time series characteristics of the audio signal, such as energy changes, fundamental frequency change rate, and spectral stability, for low-volume speech signals after signal enhancement. It predicts the continuation probability (i.e., the possibility of subsequent speech) and pause probability (i.e., the possibility of subsequent pauses or speech interruptions) of continuous speech segments in future time steps, thereby accurately determining the pause-continuation relationship in the signal, especially when the speech signal is weak and the pause is not obvious, to avoid misjudgment.

[0056] The specific steps are: The enhanced audio signal is divided into time windows of fixed length, and features such as energy envelope, fundamental frequency, and spectrum stability in each window are extracted to form a time series feature matrix.

[0057] The feature sequence is input into a pre-trained LSTM model. The model captures the dynamic changes and trends of the speech signal in the time dimension through its memory unit and forget gate, input gate, and output gate mechanism, and outputs the continuation probability and pause probability corresponding to each time point.

[0058] By setting a dynamic threshold, if the continuation probability at a certain time point is greater than the pause probability and continues for more than a set number of frames, the segment is judged to be in a continuous speech state; conversely, if the pause probability is higher than the continuation probability and lasts for several frames, it is marked as a pause position.

[0059] The continuous speech and pause marker sequences output by the model are cross-validated with the energy envelope and fundamental frequency changes to ensure the accuracy and robustness of the markers. This allows the continuous speech and pauses to be clearly distinguished even in weak and ambiguous speech, fully restoring the rhythm and language structure of the speech.

[0060] The recognition results of the three dimensions of pitch continuity, rhythm changes and pause patterns are integrated, and a decision fusion mechanism is used to determine whether the audio signal has human speech attributes. The specific method is: the feature indicators of the three dimensions are normalized and input into the trained support vector machine classification model. Combined with the multi-dimensional weight parameters of the model, a confidence score of the human speech attribute is output for each signal segment. Signal segments with scores higher than the set threshold are identified as having human speech attributes, and those with scores lower than the threshold are determined to be non-speech or noise signals. By executing this series of steps, not only can human speech be accurately identified in low-volume and weak speech conditions, but the accuracy and robustness of speech attribute recognition can also be guaranteed in complex noise backgrounds, providing a solid recognition foundation for subsequent dynamic gain adjustment and sound quality fidelity enhancement.

[0061] A trained support vector machine (SVM) classification model is a classification decision model trained using a supervised learning method using a support vector machine (SVM) algorithm based on a large number of labeled human speech and non-speech signal samples. The core principle of this model is to construct an optimal hyperplane in a high-dimensional feature space to classify the input audio signal into two categories: "human speech attribute signals" and "non-speech or noise signals." In this invention, the SVM model uses a normalized vector of multidimensional features such as pitch continuity, rhythm changes, and pause patterns as input. By learning the boundary relationship between these features and signal attributes during the training phase, the model can output a confidence score for each audio clip in practical applications, reflecting the probability that the signal belongs to human speech. SVMs have the advantage of discriminating small samples and nonlinear features, making them particularly suitable for processing low-volume signals with unclear speech features but a certain regularity. Therefore, they can effectively identify speech clips in complex noise backgrounds, providing a reliable decision basis for speech attribute determination and subsequent dynamic gain adjustment.

[0062] This step aims to comprehensively identify whether complex speech signals, such as low-volume, weak, or whispered speech signals, processed by the phase-coupled enhancement channel, possess the essential attributes of human speech through multi-dimensional speech pattern recognition, ensuring accurate distinction between speech and non-speech signals in complex, low-signal-to-noise ratio environments. Because low-volume speech has sparse energy distribution and weak structural coherence across the spectrum, traditional methods based on energy thresholds or simple spectral discrimination often misclassify it as noise or invalid signals, resulting in information loss. By introducing multi-dimensional recognition of pitch continuity, rhythm changes, and pause patterns in this step, the natural properties of speech can be fully restored from different speech feature levels. Pitch continuity analysis identifies the stability of the fundamental frequency and harmonics, rhythm changes reflect the unique stress and speech rate patterns of the language, and pause patterns reveal the natural pauses and pronunciation continuity in the speech stream. The comprehensive examination of these dimensions avoids the limitations of single feature recognition. Especially in low-volume scenarios, faint pronunciations still retain the pitch, rhythm, and speech flow structure of human voice. This step, through deep mining and pattern matching, can judge the human voice attributes in the signal with high confidence, providing accurate recognition basis for subsequent dynamic gain adjustment and sound quality fidelity enhancement, ensuring that no valuable voice information is missed in sensitive dialogue scenarios such as meetings, visits, and comforting, while improving the intelligence and reliability of the call system.

[0063] The dynamic gain adjustment and sound quality enhancement module dynamically adjusts the gain mapping curve and spectrum channel parameters of the audio signal transmission chain based on the recognized human voice attributes, and implements parameter mapping and sound quality fidelity enhancement on the audio signal; Dynamically adjusting the gain mapping curve and spectrum channel parameters of the audio signal transmission chain according to the recognized human speech attributes, and performing parameter mapping and sound quality fidelity enhancement on the audio signal specifically includes the following steps: Based on the human speech attribute results identified in the previous step, a comprehensive analysis is conducted on the audio signal's pitch characteristics, rhythm density, pause pattern, and the distribution characteristics of the signal's energy distribution in the spectrum to construct a dynamic gain mapping curve that matches the human speech attributes. The design of this gain mapping curve is based on the signal's energy share in different frequency bands and the overall dynamic range of the signal. It adopts a multi-band nonlinear gain strategy: for the main frequency and its harmonic frequency bands where human speech energy is concentrated, a higher gain boost coefficient is set to enhance the auditory clarity of the speech components; for low-frequency and high-frequency bands unrelated to speech, a gain suppression curve is set to avoid amplifying background noise or meaningless frequency components. The dynamic adjustment process of the gain curve introduces an adaptive threshold, dynamically adjusting the gain ratio based on the confidence level of the speech attribute identified for each signal segment, ensuring that while the speech signal is enhanced, the sound quality is not unbalanced due to the contamination of noise components.

[0064] A dynamic gain mapping curve dynamically adjusts the gain (volume amplification factor) at various frequencies or time periods based on the human speech attributes identified in the audio signal, as well as the signal's energy distribution and spectral characteristics across different frequency bands. Unlike fixed gain, this curve can selectively enhance the speech signal's frequency components by assigning different amplification factors based on the actual attributes of each audio segment. In this invention, the dynamic gain mapping curve targets the characteristics of low-volume, weak speech signals. Without blindly amplifying the entire signal, it focuses on boosting the energy of the primary frequency and harmonic components associated with human speech, while suppressing noise-related frequency bands to avoid deteriorating sound quality due to the simultaneous amplification of noise. The need for a dynamic gain mapping curve arises because low-volume speech signals often lack energy and partially overlap with background noise spectra. Using fixed or linear gain amplification can easily amplify both noise and distortion, ultimately reducing speech clarity and fidelity. Through a dynamic gain curve adapted to speech properties, targeted enhancement of the effective components of speech can be achieved, so that the enhanced signal reaches the optimal level in terms of energy density, natural sound quality, and auditory clarity, while ensuring the integrity and accuracy of information transmission during the call.

[0065] After the dynamic gain mapping curve is formulated, the spectral channel parameters of the audio signal are finely adjusted. This adjustment process is based on the dynamic configuration of the filter bank. According to the speech attribute weights of different frequency bands in the signal, the center frequency, bandwidth and Q value (quality factor) of the filter are adjusted in real time to achieve fine control of the speech signal spectrum. Specifically, for frequency bands in the spectrum that are highly correlated with the recognized human speech attributes, the bandwidth of the filter is appropriately widened to retain more natural speech details, while the Q value of the filter is increased to enhance frequency resolution and prevent interference from adjacent noise frequencies. For frequency bands that are unrelated to speech attributes or have a large number of noise components, the bandwidth is narrowed and the Q value is lowered to effectively suppress the noise frequency. This step ensures the dual goals of speech signal integrity and noise suppression in the frequency domain.

[0066] The dynamic gain mapping curve and the adjusted spectral channel parameters are synchronously applied to the audio signal processing chain to perform a parameter mapping operation. This operation uses a digital signal processing algorithm to map the gain and spectral parameters to each time frame and frequency unit of the audio signal, achieving precise enhancement of the signal frame by frame and frequency band by frequency band. To ensure the smoothness and naturalness of the mapping, a dynamic smoothing factor and a transition window function are used to buffer parameter changes to avoid signal distortion or spectral tearing caused by sudden changes in gain or filter parameters. At the same time, to maintain the fidelity of the sound quality, a spectral reconstruction and phase compensation mechanism is introduced during the mapping process to ensure that the phase consistency and temporal coherence of the original speech are not destroyed during the enhancement process.

[0067] Digital signal processing algorithms analyze, transform, and enhance discrete digital audio signals using mathematical methods and computational models. Common examples include the Fast Fourier Transform (FFT), Short-Time Fourier Transform (STFT), adaptive filtering, dynamic range compression, and spectral subtraction. In this step, the digital signal processing algorithm precisely maps and applies the dynamic gain mapping curve and spectral channel parameters derived from human speech attribute recognition to each time frame and corresponding frequency unit of the audio signal, achieving targeted signal enhancement and noise suppression.

[0068] The specific steps are: The short-time Fourier transform (STFT) is used to divide the time domain audio signal into frames and convert it into a time-frequency domain representation to obtain the amplitude and phase information of each time frame in different frequency units.

[0069] Based on the gain coefficient of each frequency in the dynamic gain mapping curve and the spectral channel parameters, such as the bandwidth and center frequency of the filter, the amplitude of each frequency unit in each time frame is multiplied or suppressed. At the same time, the bandwidth of the frequency distribution is adjusted according to the spectral parameters to enhance the frequency band related to speech characteristics and suppress the noise frequency band.

[0070] The phase compensation algorithm ensures that the phase information of the frequency unit remains consistent with the original signal after the amplitude adjustment, avoiding phase distortion caused by enhancement.

[0071] The inverse short-time Fourier transform (ISTFT) is applied to restore the adjusted time-frequency signal back to the time domain, resulting in an enhanced audio signal. This allows for precise enhancement and fidelity restoration on a frame-by-frame and frequency-band basis. This process not only improves the energy density and clarity of human speech components, but also suppresses background noise and invalid frequencies, improving overall sound quality and the call experience.

[0072] Real-time sound quality evaluation and adaptive feedback optimization are performed on the audio signal after parameter mapping and sound quality enhancement. By calculating indicators such as the signal-to-noise ratio, spectral smoothness, dynamic range, and harmonic distortion rate of the signal, the enhanced sound quality effect is dynamically evaluated. If the evaluation indicators do not meet the preset sound quality standards, the gain mapping curve and spectral channel parameter settings are automatically adjusted through the feedback mechanism to achieve iterative optimization. This step ensures that the optimal sound quality fidelity enhancement effect can be achieved in different environments and with different voice attributes, ensuring that low-volume, whispered, and emotionally expressed voice signals are transmitted clearly, naturally, and completely in the call system.

[0073] This step dynamically adjusts the gain mapping curves and spectral channel parameters in the audio signal transmission chain based on the identified human speech attributes, including pitch, rhythm, pause patterns, and spectral energy distribution. This provides refined signal enhancement and sound quality preservation, ensuring that low-volume, weak speech signals are reproduced clearly, naturally, and completely during calls. Because low-volume or whisper-like speech signals have low energy density and overlap with the ambient noise spectrum, traditional fixed gain or simple filtering methods can easily lead to simultaneous amplification of the entire frequency band, enhancing noise and distortion, and ultimately reducing speech perceptibility and clarity. By dynamically adjusting the gain mapping curve, a gain corresponding to the strength of the speech attribute is applied to each frequency band of the audio signal, resulting in higher amplification of speech-related frequency bands while suppressing gain in frequency bands associated with noise or non-speech attributes, avoiding meaningless enhancement. Furthermore, dynamic adjustment of spectral channel parameters, such as the filter's center frequency, bandwidth, and quality factor, optimizes the frequency channel configuration based on the identified speech characteristics, further eliminating interfering frequencies. This integrated mechanism of parameter mapping and audio quality enhancement not only improves signal energy density and speech clarity, but also preserves the natural timbre and spectral structure of speech, avoiding the distortion and discomfort often introduced by traditional enhancement methods. Ultimately, this process ensures accurate capture and high-quality restoration of voice information in meeting window call scenarios, even during low-energy speech, emotional expressions, or private whispers, effectively ensuring smooth and secure communication.

[0074] The full-link voice integrity verification module performs full-link voice integrity verification on the audio signal after adjustment using the gain mapping curve and spectrum channel parameters. By comparing the attribute characteristics of the audio signals before and after enhancement and conducting false positive back-testing verification, it ensures the complete restoration and accurate reception of low-volume voice signals. Perform full-link speech integrity verification on the audio signal after gain mapping curve and spectrum channel parameter adjustment. Compare the attribute characteristics of the audio signal before and after enhancement and verify false positives back-testing to ensure the complete restoration and accurate reception of low-volume speech signals. The specific steps are as follows: After the audio signal is processed using a dynamic gain mapping curve and spectral channel parameter adjustments, a series of detailed multidimensional attribute features are extracted from the enhanced audio signal. Specifically, these features include: energy distribution in the frequency domain, which uses a piecewise Fourier transform to divide the full-band signal from 0-8kHz into 500Hz subbands, for a total of 16 subbands, and calculates the average energy value for each subband; spectral stability, which calculates the cosine similarity between the spectral envelope of each frame and the envelopes of adjacent frames to form a similarity sequence for consecutive frames; pitch continuity, which uses a fundamental frequency extraction algorithm such as the YIN algorithm to extract the fundamental frequency over time and calculates its smoothness through first-order differences; rhythm density, which counts the number of peaks in the energy envelope curve per second, reflecting the rhythmic frequency of articulation per unit time; and pause-continuity pattern, which records the duration of continuous speech and silence segments and forms a time series of pauses and continuous articulations. Each of these features is fully recorded as a vector or sequence to ensure the rigor and integrity of subsequent comparative analysis.

[0075] The aforementioned feature sets of the audio signals before and after enhancement are rigorously compared one-to-one. Energy distribution features are compared using cosine similarity for each frequency subband, calculating the percentage similarity between the energy vectors of each subband before and after enhancement. Spectral stability is assessed using the Dynamic Time Warping (DTW) algorithm, which compares the spectral envelope similarity sequences before and after enhancement and calculates the minimum cumulative distance between the two sequences. For pitch continuity, the Euclidean distance is used to measure the difference between the fundamental frequency curves before and after enhancement, while also calculating the rate of change in smoothness. Rhythm density is assessed by directly comparing the change in the number of peaks per unit time, while pause-continuity patterns are assessed using the edit distance algorithm to quantify the similarity between pause and utterance time series. Finally, the differences between the five feature types are quantified into standardized scores and weighted to form a global speech similarity score. If the score falls below the 90% consistency threshold, the risk of false positives or feature loss is considered, and the false positive retrospective verification phase is initiated.

[0076] Dynamic Time Warping (DTW) is an algorithm used to measure the similarity between two time series of different lengths or asynchronous time series. Its core is to find a nonlinear alignment path through dynamic programming, so that the two sequences are optimally aligned in the time dimension, thereby calculating the minimum cumulative distance between them. In the present invention, DTW is used to align and compare the spectral envelope similarity sequences of the audio signal before and after enhancement point by point. This addresses the situation where the enhancement process may cause slight stretching, compression, or offset of the signal timing, amplitude, or spectral profile. It ensures that even if the signal undergoes slight changes on the time axis during the enhancement process, its spectral morphology consistency can still be accurately assessed. The specific steps are as follows: First, the spectral envelope similarity sequence extracted from the audio signal before enhancement is recorded as the reference sequence A, and the similarity sequence after enhancement is recorded as the target sequence B. Second, the DTW algorithm is used in a two-dimensional distance matrix, using the spectral envelope value of each time frame as the coordinate point. The square of the amplitude difference between each corresponding frame in A and B is calculated and filled into the matrix to form the local distance. Third, DTW recursively searches for a path in the matrix with the minimum cumulative cost from the starting point to the end point. This path allows for nonlinear stretching or compression of the time axis between sequences, minimizing the overall path cost (i.e., the sum of the squares of the amplitude differences). Finally, this minimum cumulative distance is used as a measure of the difference in spectral envelope similarity before and after enhancement. A smaller distance indicates that the enhancement process has better preserved spectral stability. Conversely, a smaller distance indicates that the signal structure may have been distorted or degraded due to enhancement, requiring further adjustment of the enhancement parameters.

[0077] The edit distance algorithm, also known as the Levenshtein distance, is an algorithm used to measure the difference between two sequences. It quantifies the difference by calculating the minimum number of editing operations required to transform one sequence into the other. These operations typically include insertions, deletions, and substitutions. In this invention, the edit distance algorithm accurately quantifies the similarity between the pause and utterance time series of the audio signal before and after enhancement, determining whether the original pause and continuous utterance structure has been mistakenly deleted, expanded, or altered during the enhancement process. Specifically, the audio signal before and after enhancement is first converted into a binary sequence, using "1" to represent an utterance segment and "0" to represent a pause segment, while recording the duration of each segment. The edit distance algorithm is then applied to compare the two sequences bit by bit, calculating the minimum number of insertions, deletions, or substitutions required to transform the original sequence into the enhanced one. For example, if a pause is expected after a continuous utterance in the original sequence, but it disappears after enhancement, an insertion or substitution operation is required. A lower total number of editing operations indicates greater consistency in the pause-and-connection pattern before and after enhancement. A higher number indicates that the enhancement process may have caused deviations in the rhythm and pause structure of the speech flow. This quantitative metric effectively assesses the fidelity of the speech rhythm structure during the enhancement process, ensuring that the continuity and pause relationships of low-volume speech are not disrupted, maintaining the natural fluency of speech.

[0078] The false positive retrospective verification process is initiated. For audio segments detected with insufficient similarity scores, the original unenhanced signal segments are traced back and gain mapping and spectrum channel adjustments with different parameter combinations are reapplied. Specifically, multiple sets of gain curve adjustment schemes are set, such as adjusting the gain of the main frequency and harmonic frequency bands from the original +6dB to +4dB or +8dB, and adjusting the suppression of the non-speech band from -10dB to -15dB or -5dB. At the same time, the center frequency of the spectrum channel filter is fine-tuned by ±100Hz and the bandwidth is expanded from 200Hz to 300Hz or contracted to 100Hz. The audio signal enhanced by each parameter combination then extracts the above five types of features and compares them with the original signal again. The parameter configuration with the highest similarity score is automatically selected through the Bayesian optimization algorithm to ensure the optimal balance between sound quality improvement and feature fidelity. After completion, the optimal enhancement result after the false positive retrospective is used to replace the original signal segment.

[0079] A full-link integrity review is performed on the retrospectively verified and corrected enhanced audio signal. This review not only re-extracts the five features of complete energy distribution, spectral stability, pitch continuity, rhythm density, and pause / continuity pattern to form a final similarity score, but also calculates sound quality indicators such as the signal-to-noise ratio (SNR), harmonic distortion (THD), and dynamic range (DR) across the entire frequency band. Specifically, the SNR is quantified in dB, the THD is controlled below 1%, and the dynamic range is maintained at at least 60 dB. If all indicators meet the set dual thresholds for integrity and sound quality, the enhanced link processing of the audio signal is confirmed to have met the standards, achieving complete restoration and high-fidelity reception of low-volume speech, which is then used as the final call output signal.

[0080] This step performs a full-link speech integrity check on the audio signal after adjustment using the gain mapping curve and spectral channel parameters. This check assesses whether the signal enhancement process has damaged or omitted the original properties and structure of the low-volume speech signal, ensuring that all human speech components are fully restored and accurately received. Low-volume speech signals are inherently fragile in terms of energy, spectrum, and rhythm. Without targeted integrity monitoring during the enhancement process, operations such as filtering, gain adjustment, and spectral compression can easily lead to the inadvertent deletion of valuable speech information (i.e., "false negatives") or the introduction of distortion and information loss. This is particularly true for whispers and suppressed speech, which are more likely to be mistakenly processed as background noise and erased. Therefore, this step extracts multi-dimensional attribute features of the audio signal before and after enhancement, such as energy distribution, spectral stability, pitch continuity, rhythm density, and pause patterns. It then uses algorithms such as dynamic time warping and edit distance to accurately compare these features and identify differences in various attributes between the enhanced and original signals. A false positive backtracking verification mechanism is also introduced. For signal segments with insufficient similarity, the system traces back to the original signal, readjusts the gain and spectrum adjustment parameters, and, after trying multiple solutions, selects the best enhancement that best matches the original properties. This full-link verification and backtracking ensures that low-volume, weak speech is enhanced while improving signal clarity and energy density while preserving the original pronunciation rhythm, tonal characteristics, and pause-to-dial relationships. This ensures that every voice message transmitted during a call is complete, accurate, and authentic, significantly improving the quality of voice interaction and information restoration in meeting window call scenarios.

[0081] The voice sensor-based meeting window call system proposed in this invention can systematically and fully address the problem of existing call systems' insufficient recognition of special voice states such as low volume, low frequency changes, and low speech speeds, achieving a full-process closed-loop processing of high-sensitivity acquisition, precise recognition, intelligent enhancement, and fidelity restoration of weak voice signals. This solution improves the spatial and frequency coverage of signal acquisition through an array sensor layout. Dynamic confidence assessment and feature weight construction achieve intelligent differentiation between human voice and background noise. Phase coupling and harmonic enhancement effectively restore the energy density and structural coherence of low-volume speech. Furthermore, through multi-dimensional speech pattern recognition and dynamic gain adjustment, human voice attributes are precisely enhanced to avoid synchronous noise amplification. Under the protection of full-link integrity verification and false positive backtracking mechanisms, weak signal information is not missed or distorted. The system is particularly suitable for application scenarios with high conversation sensitivity and strict information integrity requirements, such as legal visits, psychological counseling, and family comfort. It significantly improves the comprehensive capabilities and application value of the call system in terms of privacy, security, and complete transmission of voice information.

[0082] The above description is merely illustrative of certain exemplary embodiments of the present invention. It goes without saying that those skilled in the art will be able to modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims.

[0083] It should be noted that, in this document, if there are relational terms such as first and second, etc., they are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0084] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0085] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0086] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0087] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0088] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0089] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0090] The above description is merely illustrative of certain exemplary embodiments of the present invention. It goes without saying that those skilled in the art will be able to modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims.

Claims

1. A meeting window communication system based on voice sensor, characterized in that: It includes a full-space speech acquisition module, a dynamic confidence assessment module, a phase coupling enhancement module, a multi-dimensional speech pattern recognition module, a dynamic gain adjustment and sound quality enhancement module, and a full-link speech integrity verification module: The full-space voice acquisition module deploys an array of voice sensor units within the meeting window space based on audio reflection characteristics, constructs a multi-band full-space acquisition matrix, collects the voice signals of both parties in the meeting, and outputs a basic audio data stream with a high signal-to-noise ratio; The dynamic confidence assessment module performs dynamic confidence assessment on the basic audio data stream, extracts energy distribution and spectrum stability features, and generates a set of feature weights to distinguish speech from background noise; The phase coupling enhancement module constructs a phase coupling enhancement channel based on a set of feature weights. It enhances the structural coherence and energy density of the audio signal through phase consistency mapping and harmonic synchronous amplification. A multi-dimensional speech pattern recognition module performs multi-dimensional speech pattern recognition based on the enhanced audio signal, identifying human speech attributes based on pitch continuity, rhythm changes, and pause patterns; Dynamic gain adjustment and sound quality enhancement module, which dynamically adjusts the gain mapping curve and spectrum channel parameters of the audio signal transmission chain based on the recognized human voice attributes to implement sound quality fidelity enhancement; The full-link voice integrity verification module performs full-link voice integrity verification on the enhanced audio signal. Through attribute feature comparison and false positive backtracking verification, it achieves complete restoration and accurate reception of low-volume voice signals.

2. A voice sensor-based meeting window communication system according to claim 1, characterized in that: The steps to obtain the basic audio data stream are as follows: Based on the material, location, and acoustic reflection characteristics of the walls, ceiling, floor, and soundproof glass in the meeting window space, perform sound field simulation and modeling to determine the sound reflection path, attenuation coefficient, and standing wave distribution; Based on the sound field simulation results, highly sensitive capacitive or piezoelectric voice sensors are selected, arranged in a mixed pattern of equal and non-equal spacing, and sensors with tilted or directional pickup are set up to build a full-space acquisition matrix covering multiple frequency bands. The signal of each voice sensor is pre-amplified and digitized through the preamplifier circuit and analog-to-digital conversion unit, and then enters the multi-channel signal synchronization platform based on timing synchronization to achieve phase consistency and time synchronization; The digitized signal is divided into frequency bands and weighted based on a multi-band filter bank, and background noise is reduced by weighted averaging and spectral subtraction. Dynamic amplitude normalization processing with adaptive gain control is then performed to output a basic audio data stream with a high signal-to-noise ratio.

3. The voice sensor-based meeting window communication system according to claim 1, characterized in that: The steps for generating feature weight sets are as follows: Based on the basic audio data stream, the energy distribution characteristics, spectrum stability characteristics, spectrum envelope similarity, spectrum entropy and main frequency component persistence of each frame signal are extracted to form a multi-dimensional feature vector of the time series; The multi-dimensional feature vector sequence is input into a deep learning model composed of a convolutional neural network and a bidirectional long short-term memory network in series, and the confidence score of each frame signal is output; Smoothing and thresholding of the scoring sequence to identify valid speech areas where the scores of consecutive frames are higher than the threshold; Based on the confidence score and each feature, a feature weight set is generated for signal enhancement and recognition processing.

4. The voice sensor-based meeting window communication system according to claim 1, characterized in that: The specific steps for enhancing the structural coherence and energy density of audio signals using phase consistency mapping and harmonic synchronization amplification based on a feature weight set are as follows: Based on the energy distribution characteristics and spectrum stability characteristics of the feature weight set, a phase tracking model of the signal is established. The instantaneous phase information is extracted through Hilbert transform, the phase trajectory is constructed, and the target frequency with continuous phase and stable amplitude is selected. Perform phase consistency mapping on the target frequency and adjacent frequencies, perform phase compensation based on phase similarity, correct phase offset, and maintain phase consistency of frequency components; Apply dynamic gain control to the main frequency and its multiple harmonic components, dynamically adjust the gain according to the characteristic weight set, enhance the energy density and maintain the natural harmonic energy distribution ratio; Perform structural coherence review in the time-frequency domain, dynamically optimize based on phase spectrum difference, spectrum envelope change rate and waveform smoothness indicators, and output enhanced audio signals with high energy density and strong structural coherence.

5. The voice sensor-based meeting window communication system according to claim 1, characterized in that: Based on the enhanced audio signal, the specific steps for identifying whether the signal has human speech attributes are as follows through multi-dimensional analysis of pitch continuity, rhythm changes, and pause patterns: The enhanced audio signal is subjected to short-time Fourier transform to extract the fundamental frequency and harmonics, form a pitch contour curve, and calculate the fundamental frequency smoothness and stability; The energy envelope curve and the fundamental frequency change rate are used to jointly analyze the rhythm density, beat interval and accent position, and the periodic and non-periodic rhythm components are identified by combining wavelet transform and rhythm synchronization function. Long short-term memory networks are used to predict the continuation and pause probabilities of speech segments based on time series features, distinguishing between continuous pronunciation and natural pauses, and forming pause-continuation patterns. The features of pitch continuity, rhythm changes and pause patterns are input into the support vector machine classification model, and the human speech attribute confidence score of each signal segment is output to determine the speech attributes of the audio signal.

6. The voice sensor-based meeting window communication system according to claim 1, characterized in that: The specific steps for dynamically adjusting the gain mapping curve and spectral channel parameters of the audio signal transmission chain to enhance sound quality are as follows: Based on the recognized human speech attributes, a dynamic gain mapping curve is constructed, the gain boost coefficient is set according to the main frequency and harmonic distribution, and gain suppression is set for the non-speech frequency band; Dynamically adjust the spectrum channel parameters according to the speech attribute weight, adjust the center frequency, bandwidth and quality factor of the filter, enhance the voice band and suppress the noise band; The dynamic gain mapping curve and spectrum channel parameters are mapped to each time frame and frequency unit through digital signal processing algorithms, and phase compensation and spectrum reconstruction are applied to perform frame-by-frame and frequency band enhancement of the audio signal. Real-time sound quality evaluation is performed on the enhanced signal. Based on the signal-to-noise ratio, spectral smoothness, dynamic range and harmonic distortion rate indicators, the gain mapping curve and spectral channel parameters are dynamically optimized through a feedback mechanism to ensure the enhanced sound quality.

7. The voice sensor-based meeting window communication system according to claim 1, characterized in that: The specific steps for achieving complete restoration and accurate reception of low-volume speech signals through attribute feature comparison and false positive backtesting are as follows: Extract five attribute features of the audio signal after adjusting the gain mapping curve and spectrum channel parameters: energy distribution, spectrum stability, pitch continuity, rhythm density and pause pattern; The five attribute features of the enhanced and unenhanced signals are compared using cosine similarity, dynamic time warping, Euclidean distance, rhythm density comparison, and edit distance algorithms to form a global speech similarity score. For signal segments with scores below the consistency threshold, we trace back to the original signal, apply different gain and spectrum parameter configurations for enhancement, and select the optimal configuration through Bayesian optimization to replace the original signal segment. The five types of features are extracted again from the corrected signal and the signal-to-noise ratio, harmonic distortion rate and dynamic range are calculated. The integrity and sound quality indicators are reviewed, and the final enhanced signal is output after meeting the standards.

Images