Voice sensing system for assisting rail transit hearing impairment

Abstract

The invention discloses a voice sensing system for assisting rail transit hearing impairment. The voice sensing system comprises an audio acquisition module, an audio signal processing module, a power amplifier module and an output module. Specifically, the audio acquisition module is used for receiving an input audio; the audio signal processing module is used for performing signal enhancement processing on the input audio to obtain an enhanced audio signal; the power amplifier module is used for amplifying the enhanced audio signal to obtain an amplified audio signal; and the output module is used for outputting the amplified audio signal. Thus, clearer voice information is provided for the hearing impaired person through audio signal processing, the hearing experience of the hearing impaired person in the rail transit environment is effectively improved, and the ability of the hearing impaired person to acquire useful sound information is enhanced.

Description

Technical Field

[0001] The present application relates to the field of intelligent assistance, and more specifically, to a voice sensing system for assisting the hearing-impaired in rail transit. Background Art

[0002] With the accelerated development of urbanization, rail transit has become one of the indispensable public transportation tools in modern cities, greatly facilitating people's daily travel. However, for the hearing-impaired group, it is often difficult to effectively obtain sound information in the rail transit environment (such as station announcements, emergency broadcasts, etc.), which not only affects their travel experience, but also may bring safety hazards. Therefore, how to provide effective auxiliary hearing services for the hearing-impaired has become one of the current problems that need to be solved urgently.

[0003] Traditional hearing aids mainly rely on physical amplification to improve the audibility of sound. However, in rail transit environments, background noise interference is particularly serious, such as vehicle operation noise, passenger conversation noise, etc., making it difficult for traditional hearing aids to achieve the ideal hearing aid effect.

[0004] Therefore, an optimized voice sensing system for assisting the hearing-impaired in rail transit is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a voice sensing system for assisting the hearing-impaired in rail transit, which decomposes the input audio by adopting CEEMDAN technology, and uses the signal frequency domain analysis technology based on deep learning to perform frequency domain analysis on each IMF component after decomposition, so as to extract the frequency domain feature representation of each IMF component, and then filter out the noise components in the audio signal by performing feature filtering based on semantic contribution and full-frequency domain feature aggregation on each IMF component, and capture key semantic information to achieve enhanced processing of the audio signal. Finally, the enhanced audio signal is amplified and output, thereby providing clearer voice information for the hearing-impaired. In this way, the auditory experience of the hearing-impaired in the rail transit environment can be effectively improved, and their ability to obtain useful sound information can be enhanced.

[0006] According to one aspect of the present application, there is provided a voice sensing system for assisting hearing impairment in rail transit, which includes: an audio acquisition module, an audio signal processing module, a power amplifier module and an output module;

[0007] The audio acquisition module is used to receive input audio;

[0008] The audio signal processing module is used to perform signal enhancement processing on the input audio to obtain an enhanced audio signal;

[0009] The power amplifier module is used to amplify the enhanced audio signal to obtain an amplified audio signal;

[0010] The output module is used to output the amplified audio signal;

[0011] Among them, the audio signal processing module includes: a signal decomposition unit, used to perform CEEMDAN decomposition on the input audio to obtain a set of audio signal IMF components; a frequency domain feature extraction unit, used to respectively extract the frequency domain features of each audio signal IMF component in the set of audio signal IMF components to obtain a set of audio signal IMF component frequency domain feature vectors; a feature filtering unit, used to perform feature filtering based on semantic contribution on the set of audio signal IMF component frequency domain feature vectors to obtain a set of filtered audio signal IMF component frequency domain feature vectors; a signal restoration unit, used to perform signal restoration based on the set of filtered audio signal IMF component frequency domain feature vectors to obtain the enhanced audio signal.

[0012] Compared with the prior art, the speech sensing system for assisting the hearing-impaired in rail transit provided by the present application decomposes the input audio by using CEEMDAN technology, and uses the signal frequency domain analysis technology based on deep learning to perform frequency domain analysis on each IMF component after decomposition, so as to extract the frequency domain feature representation of each IMF component, and then filter out the noise components in the audio signal by performing feature filtering based on semantic contribution and full-frequency domain feature aggregation on each IMF component, and capture key semantic information to achieve enhanced processing of the audio signal. Finally, the enhanced audio signal is amplified and output, thereby providing clearer speech information for the hearing-impaired. In this way, the auditory experience of the hearing-impaired in the rail transit environment can be effectively improved, and their ability to obtain useful sound information can be enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0014] Figure 1 is a block diagram of a voice sensing system for assisting hearing impairment in rail transit according to an embodiment of the present application;

[0015] Figure 2 A schematic diagram of data flow of a voice sensing system for assisting hearing impairment in rail transit according to an embodiment of the present application;

[0016] Figure 3 It is a flow chart of the working principle of MCU according to an embodiment of the present application;

[0017] Figure 4 The present invention is a block diagram of an audio signal processing module in a voice sensing system for assisting the hearing-impaired in rail transit according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0019] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0020] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0021] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0022] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0023] Traditional hearing aids mainly rely on physical amplification to improve the audibility of sound. However, in the rail transit environment, the interference of background noise is particularly serious, such as vehicle operation noise, passenger conversation noise, etc., making it difficult for traditional hearing aids to achieve the ideal hearing aid effect. Therefore, an optimized voice sensing system for rail transit hearing impairment assistance is expected.

[0024] In the technical solution of the present application, a speech sensing system for assisting the hearing-impaired in rail transit is proposed. Figure 1 The block diagram of the speech sensing system for assisting the hearing-impaired in rail transit according to an embodiment of the present application. Figure 2 FIG. 1 is a data flow diagram of a voice sensing system for assisting hearing-impaired people in rail transit according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to an embodiment of the present application, a voice sensing system 300 for assisting the hearing-impaired in rail transit includes: an audio acquisition module 310 for receiving input audio; an audio signal processing module 320 for performing signal enhancement processing on the input audio to obtain an enhanced audio signal; a power amplifier module 330 for amplifying the enhanced audio signal to obtain an amplified audio signal; and an output module 340 for outputting the amplified audio signal.

[0025] Those skilled in the art should know that according to the principles of electromagnetism, when a current passes through a long straight wire, magnetic lines of force will be generated around it. According to the right-hand rule, the magnetic lines of force are concentric circles in the same plane and are perpendicular to the wire. The magnetic lines of force are centered on the wire, from inside to outside, from dense to sparse, and the magnetic field changes from strong to weak. When the current in a closed loop changes, the magnetic field generated by the current also changes accordingly. After the surrounding coil generates an induced magnetic field in the compartment space, passengers wearing hearing aids in the compartment can use the induction coil inside the hearing aid to cut the magnetic lines of force in the magnetic field to generate current by switching to the hearing aid "T gear" ("T" represents the magnetoelectric induction input mode), thereby picking up the electromagnetic signal of a specific frequency around and converting it into an electrical signal - the basic principle based on this process is the law of electromagnetic induction, which is expressed as: the magnitude of the induced electromotive force in any closed circuit is equal to the rate of change of the magnetic flux passing through this circuit. The process of electric field transforming magnetic field and magnetic field transforming electric field is the basic principle of the induction coil system. Therefore, the signal transmission and reception process of the audio induction loop system is that the host filters, amplifies and outputs the received audio signal to a long straight wire, and an alternating magnetic field that changes with the audio can be formed in the car; at this time, the coil of the hearing aid in the magnetic field induces an audio current, which is properly amplified and converted into a voice signal, so that the hearing-impaired can receive clearer voice content.

[0026] In particular, the audio acquisition module 310 is used to receive input audio. Accordingly, after receiving the input audio, the audio acquisition module 310 transmits the input audio to the audio signal processing module 320 for signal processing.

[0027] Accordingly, in a feasible implementation, after receiving the input audio, the audio acquisition module 310 performs A / D conversion on it, converts the analog quantity into a digital quantity, and then inputs it to the audio signal processing module 320 for further signal enhancement processing. It is worth mentioning that the audio acquisition module is mainly responsible for receiving up to two external audio inputs, and filtering the noise in the external audio through an isolation transformer, and then outputting it to the digital voice processing module.

[0028] In one example, after the audio acquisition module 320 receives the input audio, it gives it to the MCU. The MCU converts the analog quantity into a digital quantity through A / D conversion, and performs FFT transformation for spectrum analysis, and processes and digitally filters the frequency band with frequency response problems in the audio signal. The signal processed by the MCU is output to the hardware filter for signal filtering and smoothing, and then output to the external loop after power amplification (that is, after power amplification processing by the power amplifier module 330).

[0029] like Figure 3 As shown in the figure, MCU is mainly composed of two parts: digital voice processing and controller; digital voice processing is mainly responsible for analog quantity analysis, A / D conversion, digital quantity analysis, FFT transformation, spectrum analysis, gain amplification, anti-aliasing filtering, etc. of the audio transmitted by the voice acquisition module. STM32F407 is used for voice processing, with a main frequency of up to 168MHZ, providing 3 ADCs, and a maximum configuration of 12-bit resolution. DMA mode is used for continuous high-speed AD conversion to convert analog quantity into data quantity, and then the floating-point operation unit FPU is used for FFT operation to realize the conversion from time domain to frequency domain; the controller mainly feeds back to the MCU based on the control signal collected by DI and the gain control of equipment debugging; the MCU outputs the signals of digital voice processing and controller to the power amplifier module, among which DI acquisition is mainly responsible for collecting the external switch quantity input corresponding to the two-way audio signal, and at the same time provides the collected signal to the controller in the MCU; in addition, if there is only one audio input and output, the corresponding control switch can be internally short-circuited.

[0030] In particular, since audio signals, especially audio signals from real environments, often contain complex nonlinear components and transient changes, this characteristic makes it difficult for traditional signal processing methods to effectively extract their useful information.

[0031] Therefore, in a preferred embodiment of the present application, the audio signal processing module 320 uses another method to perform signal enhancement processing to obtain an enhanced audio signal. Specifically, Figure 4As shown, the audio signal processing module 320 includes: a signal decomposition unit 321, used to perform CEEMDAN decomposition on the input audio to obtain a set of audio signal IMF components; a frequency domain feature extraction unit 322, used to respectively extract the frequency domain features of each audio signal IMF component in the set of audio signal IMF components to obtain a set of audio signal IMF component frequency domain feature vectors; a feature filtering unit 323, used to perform feature filtering based on semantic contribution on the set of audio signal IMF component frequency domain feature vectors to obtain a set of filtered audio signal IMF component frequency domain feature vectors; a signal restoration unit 324, used to perform signal restoration based on the set of filtered audio signal IMF component frequency domain feature vectors to obtain the enhanced audio signal.

[0032] Specifically, the signal decomposition unit 321 is used to perform CEEMDAN decomposition on the input audio to obtain a set of IMF components of the audio signal. It should be understood that since audio signals, especially audio signals from actual environments, often contain complex nonlinear components and transient changes, this characteristic makes it difficult for traditional signal processing methods to effectively extract useful information. CEEMDAN is an adaptive decomposition method for nonlinear and non-stationary signals, which can effectively process audio signals containing multiple frequency components and complex noise. During the decomposition process, CEEMDAN can decompose the signal into a series of IMF components with different frequency scales by adding white noise for multiple iterations and applying the EMD (Empirical Mode Decomposition) algorithm, thereby separating noise and useful signals in different IMF components. For example, high-frequency IMF components usually contain more transient changes and noise, while low-frequency IMF components may carry more stable and useful information. In this way, it is helpful to process each IMF component separately, thereby effectively suppressing noise interference while retaining important signal features.

[0033] Specifically, the frequency domain feature extraction unit 322 is used to respectively extract the frequency domain features of each audio signal IMF component in the set of audio signal IMF components to obtain a set of audio signal IMF component frequency domain feature vectors. In a specific example of the present application, each audio signal IMF component in the set of audio signal IMF components is input into an audio signal component frequency domain feature extractor based on a 1D-CNN model to obtain a set of audio signal IMF component frequency domain feature vectors. That is, in order to further extract key feature information from each audio signal IMF component, the present application adopts a 1D-CNN model as an audio signal component frequency domain feature extractor to perform frequency domain feature extraction on each audio signal IMF component in the set of audio signal IMF components. By stacking multiple convolutional layers, activation layers (such as ReLU) and pooling layers (such as maximum pooling), the 1D-CNN model can make full use of the local correlation of audio signals on the time axis, and gradually extract the deep features of audio signals at different frequency scales through layer-by-layer convolution operations, so as to mine the frequency domain characteristics of each audio signal IMF component, thereby obtaining a set of frequency domain feature vectors of the audio signal IMF components, providing strong data support for subsequent feature filtering.

[0034] Specifically, the feature filtering unit 323 is used to perform feature filtering on the set of frequency domain feature vectors of the IMF component of the audio signal based on semantic contribution to obtain a set of filtered frequency domain feature vectors of the IMF component of the audio signal. In order to remove redundant information and noise components in the audio signal to achieve enhanced processing of the audio signal, the present application further performs feature filtering on the set of frequency domain feature vectors of the IMF component of the audio signal. It is worth mentioning that in order to construct a more refined feature filtering mechanism, the present application introduces a feature filtering method based on semantic contribution, which can intelligently evaluate and filter out the most critical feature information for the overall audio signal in the frequency domain feature vectors of each IMF component by introducing the concept of pseudo-anchor center and combining with the semantic contribution query mechanism. In a specific example of the present application, first, cluster analysis is performed on the set of frequency domain feature vectors of the audio signal IMF component to obtain a representation vector of the frequency domain feature cluster center of the audio signal IMF component as a pseudo anchor center; by clustering analysis of the set of frequency domain feature vectors of the audio signal IMF component to capture the overall intrinsic structure and distribution of the data, the representation vector of the frequency domain feature cluster center of the audio signal IMF component is extracted as a pseudo anchor center, thereby providing a reference for subsequent feature evaluation. Specifically, the positional mean vector of the set of frequency domain feature vectors of the audio signal IMF component is calculated as the representation vector of the frequency domain feature cluster center of the audio signal IMF component. Then, based on the semantic contribution of each audio signal IMF component frequency domain feature vector in the set of frequency domain feature vectors of the audio signal IMF component relative to the pseudo anchor center, feature filtering is performed on the set of frequency domain feature vectors of the audio signal IMF component to obtain the filtered set of frequency domain feature vectors of the audio signal IMF component. Specifically, the semantic contribution representation matrix of each audio signal IMF component frequency domain feature vector in the set of the audio signal IMF component frequency domain feature vectors relative to the pseudo anchor center is calculated to obtain a set of audio signal IMF component frequency domain feature semantic contribution representation matrices; and based on the set of audio signal IMF component frequency domain feature semantic contribution representation matrices, the set of audio signal IMF component frequency domain feature vectors is feature modulated to obtain the set of filtered audio signal IMF component frequency domain feature vectors. Here, by calculating the semantic contribution of each audio signal IMF component frequency domain feature vector relative to the pseudo anchor center, the correlation and relative importance of each audio signal IMF component frequency domain feature vector and the central trend of the audio signal are quantitatively represented to generate a sequence of semantic contribution representation matrices. Then, by performing random inactivation masking processing on each semantic contribution representation matrix, the focus on key features is strengthened, the generalization ability of the model is enhanced and the number of parameters is reduced.Finally, the semantic contribution representation matrices after masking are used as modulation matrices, and the original IMF component frequency domain feature vectors of the audio signal are weighted to filter out unimportant or noise features, thereby obtaining a set of filtered audio signal IMF component frequency domain feature vectors.

[0035] Among them, the process of calculating the semantic contribution representation matrix of each audio signal IMF component frequency domain feature vector in the set of the audio signal IMF component frequency domain feature vector relative to the pseudo anchor center to obtain a set of audio signal IMF component frequency domain feature semantic contribution representation matrices includes: multiplying the audio signal IMF component frequency domain feature vector by the transposed vector of the pseudo anchor center and then dividing it by the binary norm of the pseudo anchor center to obtain the audio signal IMF component frequency domain feature semantic contribution representation matrix.

[0036] More specifically, based on the set of frequency domain feature semantic contribution representation matrices of the audio signal IMF component frequency domain features, the process of performing feature modulation on the set of frequency domain feature vectors of the audio signal IMF component to obtain the set of frequency domain feature vectors of the filtered audio signal IMF component frequency domain features includes: performing contribution masked sparse processing based on a random deactivation mechanism on each audio signal IMF component frequency domain feature semantic contribution representation matrix in the set of frequency domain feature semantic contribution representation matrices of the audio signal IMF component frequency domain features to obtain a set of masked sparse audio signal IMF component frequency domain feature semantic contribution representation matrices; using each masked sparse audio signal IMF component frequency domain feature semantic contribution representation matrix in the set of masked sparse audio signal IMF component frequency domain feature semantic contribution representation matrices as a modulation matrix, and calculating the matrix product between it and each audio signal IMF component frequency domain feature vector to obtain the set of frequency domain feature vectors of the filtered audio signal IMF component frequency domain features.

[0037] In summary, in the above embodiment, the set of frequency domain feature vectors of the audio signal IMF component is subjected to feature filtering based on semantic contribution to obtain a set of filtered frequency domain feature vectors of the audio signal IMF component, including: processing the set of frequency domain feature vectors of the audio signal IMF component using the following feature filtering formula to obtain the set of filtered frequency domain feature vectors of the audio signal IMF component, wherein the feature filtering formula is:

[0038] X={x1,x2,...,x m}

[0039]

[0040]

[0041] {x i'}={WF i *x i}

[0042] Wherein, X represents the set of frequency domain feature vectors of the IMF component of the audio signal, x1, x2, x i and x m are the first, second, i-th and m-th audio signal IMF component frequency domain feature vectors in the set of the audio signal IMF component frequency domain feature vectors, respectively, the value of m is the number of the audio signal IMF component frequency domain feature vectors, x c represents the pseudo anchor center, ||·||2 represents the bi-norm of the eigenvector, (·) T represents the transpose of the vector, softmax is the normalized exponential function, dropout[·] represents random inactivation processing, WF i represents the semantic contribution matrix of the frequency domain feature of the IMF component of the i-th masked sparse audio signal, * represents matrix multiplication, x i ' represents the frequency domain feature vector of the IMF component of the i-th filtered audio signal.

[0043] It is worth mentioning that in other signal processing schemes of the present application, the input audio can also be enhanced by other methods to obtain an enhanced audio signal, for example: input the input audio; use an audio processing library (such as Librosa, PyDub) to read the input audio file; convert the audio file into a digital signal form; extract basic parameters of the audio, such as sampling rate, number of channels, etc.; use a noise estimation algorithm (such as minimum statistical noise estimation) to estimate the characteristics of background noise; use short-time Fourier transform (STFT) to convert the audio signal from the time domain to the frequency domain; divide the audio signal into multiple frames and perform FFT transformation on each frame; suppress noise by subtracting the estimated noise spectrum from the spectrum; use a Wiener filter to weight the spectrum to reduce the impact of noise; use a deep learning model (such as C The invention relates to a method for suppressing noise by using a CNN (NN, RNN, Transformer); compressing the dynamic range of the audio signal to make the audio more uniform and clear; using a compressor to adjust the gain of the audio signal and reduce the dynamic range; using spectrum enhancement technology (such as MMSE-STSA) to further improve the clarity of the audio signal; enhancing the speech signal by adjusting certain frequency bands in the spectrum; using an equalizer to adjust the frequency response of the audio signal to make the audio more balanced and natural; using an inverse short-time Fourier transform (ISTFT) to convert the processed frequency domain signal back to the time domain; converting the frequency domain signal of each frame back to the time domain signal, and performing overlap-addition (OLA) to reconstruct the original audio signal to obtain the enhanced audio signal, which is not limited to the present application.

[0044] Furthermore, the signal restoration unit 324 is used to perform signal restoration based on the set of frequency domain feature vectors of the IMF component of the filtered audio signal to obtain the enhanced audio signal. In a specific example of the present application, the set of frequency domain feature vectors of the IMF component of the filtered audio signal is first cascaded to obtain the full-frequency domain cascade representation vector of the audio signal; the set of frequency domain feature vectors of the IMF component of the filtered audio signal is cascaded to integrate the key features of the audio signal in different frequency bands to form a full-frequency domain feature representation of the audio signal, that is, the full-frequency domain cascade representation vector of the audio signal, so as to fully reflect the overall characteristics of the audio signal. Subsequently, the full-frequency domain cascade representation vector of the audio signal is input into an audio signal restorer based on a decoder to obtain the enhanced audio signal. That is, in the technical solution of the present application, in order to restore the full-frequency domain cascade representation vector of the audio signal to the original audio signal form, the present application uses an audio signal restorer based on a decoder to reconstruct the full-frequency domain cascade representation vector of the audio signal to obtain an enhanced audio signal. The decoder can gradually map the deep features in the full-frequency domain cascade representation vector of the audio signal back to the original time-domain audio signal through multi-layer deconvolution operations, thereby achieving enhanced reconstruction of the audio signal.

[0045] Here, in the preferred example, each audio signal IMF component frequency domain feature vector in the set of audio signal IMF component frequency domain feature vectors respectively represents the frequency domain features of each audio signal IMF component of the input audio. Thus, when the set of audio signal IMF component frequency domain feature vectors is input into the feature filtering network based on pseudo-anchor center contribution semantic query, the feature filtering network based on pseudo-anchor center contribution semantic query performs feature filtering based on the audio signal IMF component frequency domain feature vector based on the pseudo-anchor center contribution, but considering that the pseudo-anchor center is not a real feature filtering anchor point, this will cause a contribution mismatch when calculating the pseudo-anchor center contribution, which will cause the full-frequency domain cascade representation vector of the audio signal obtained by cascading to have full-frequency domain aggregation differences based on node contribution mismatches. Therefore, it is expected to improve the detail semantic aggregation expression effect of the full-frequency domain cascade representation vector of the audio signal based on the local frequency domain feature aggregation differences in the full frequency domain.

[0046] Preferably, inputting the audio signal full frequency domain cascade representation vector into a decoder-based audio signal restorer to obtain an enhanced audio signal comprises:

[0047] Calculating the sum of the absolute values ​​of the eigenvalues ​​of the full-frequency domain cascade representation vector of the audio signal to obtain the full-frequency domain cascade and modulation value of the first audio signal, and calculating the square root of the sum of the squares of the eigenvalues ​​of the full-frequency domain cascade representation vector of the audio signal to obtain the full-frequency domain cascade and modulation value of the second audio signal;

[0048] After performing dot-wise subtraction on the audio signal full-frequency domain cascade representation vector and the second audio signal full-frequency domain cascade and modulation value, perform dot-wise multiplication on the number of eigenvalues ​​of the audio signal full-frequency domain cascade representation vector and the reciprocal of the first audio signal full-frequency domain cascade and modulation value, respectively, and take the reciprocal of each eigenvalue to obtain the first audio signal full-frequency domain cascade phase conversion vector;

[0049] After performing dot-wise subtraction on the audio signal full-frequency domain cascade representation vector and the first audio signal full-frequency domain cascade sum modulation value, perform dot-wise multiplication with the square root of the number of eigenvalues ​​of the audio signal full-frequency domain cascade representation vector and the reciprocal of the second audio signal full-frequency domain cascade sum modulation value, respectively, and take the reciprocal of each eigenvalue to obtain the second audio signal full-frequency domain cascade phase conversion vector; and

[0050] Calculating the dot product vector of the second audio signal full-frequency domain cascade phase conversion vector and the weighted hyperparameter, and subtracting the dot product vector from the first audio signal full-frequency domain cascade phase conversion vector to obtain an optimized audio signal full-frequency domain cascade representation vector; and

[0051] The optimized audio signal full-frequency domain cascade representation vector is input into a decoder-based audio signal restorer to obtain an enhanced audio signal.

[0052] Here, the optimized representation of the full frequency domain cascade representation vector of the audio signal, denoted as V, is:

[0053]

[0054]

[0055] v i ∈V∈R n

[0056] Where V is the full frequency domain cascade representation vector of the audio signal, v i are the eigenvalues ​​of the full-frequency domain cascade representation vector of the audio signal, α is the full-frequency domain cascade and modulation value of the first audio signal, and β is the full-frequency domain cascade and modulation value of the second audio signal, n is the number of eigenvalues ​​of the full-frequency domain cascade representation vector of the audio signal, V1 is the full-frequency domain cascade phase transition vector of the first audio signal, V2 is the full-frequency domain cascade phase transition vector of the second audio signal, ω is a weighted hyperparameter, θ is point subtraction, ⊙ is point multiplication, and V′ is the optimized full-frequency domain cascade representation vector of the audio signal.

[0057] Based on this, the difference in the difference and modulation representation of the eigenvalue of the full-frequency domain cascade representation vector of the audio signal relative to the feature set of the vector as a whole of the audio signal full-frequency domain cascade representation vector is used as the semantic change intensity information, and a phase-like transformation corresponding to the position-based intensity modulation is performed through different and modulation representation forms, so that the aggregation enhancement of the semantic change phase perception can improve the axial aggregation receptive field along the feature aggregation direction by performing a spatial translation operation based on alternating stacking under the scale equalization of the vector set of the full-frequency domain cascade representation vector of the audio signal, thereby improving the perception effect of the aggregated semantics of the full-frequency domain cascade representation vector of the audio signal for the detailed semantic changes, so as to improve the expression effect of the full-frequency domain cascade representation vector of the audio signal, and improve the signal quality and accuracy of the enhanced audio signal obtained by inputting it into the decoder-based audio signal restorer.

[0058] In particular, the power amplifier module 330 and the output module 340 are used to amplify the enhanced audio signal to obtain an amplified audio signal; and output the amplified audio signal. That is, after obtaining the enhanced audio signal, the power amplifier module will further perform power amplification processing on it to enhance the strength of the signal to ensure that it is clearly conveyed to the hearing-impaired in the rail transit environment. It is worth mentioning that the power amplifier module is responsible for amplifying the voice signal processed by the MCU, converting the voice signal into current, and outputting it to the loop.

[0059] In particular, the output module 340 is used to output the amplified audio signal. That is, the amplified audio signal is then output to the hearing-impaired through an output module, such as headphones, so that the hearing-impaired can clearly hear and understand important sound information even in a noisy rail transit environment. In one example, the output of the power amplifier is collected by a step-back circuit and fed back to the MCU. The MCU compares the collected power amplifier output with the original output, and makes corresponding feedback and records the output.

[0060] It is worth mentioning that in the technical solution of the present application, the voice sensing system 300 for assisting the hearing-impaired in rail transit also includes other necessary functional modules, such as a power supply module, etc., which are not the focus of the present application and will not be elaborated on.

[0061] As described above, the voice sensing system 300 for assisting hearing-impaired people in rail transit according to the embodiment of the present application can be implemented in various wireless terminals, such as a server having a voice sensing algorithm for assisting hearing-impaired people in rail transit. In one possible implementation, the voice sensing system 300 for assisting hearing-impaired people in rail transit according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the voice sensing system 300 for assisting hearing-impaired people in rail transit can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the voice sensing system 300 for assisting hearing-impaired people in rail transit can also be one of the many hardware modules of the wireless terminal.

[0062] Alternatively, in another example, the voice sensing system 300 for assisting the hearing impaired in rail transit and the wireless terminal may also be separate devices, and the voice sensing system 300 for assisting the hearing impaired in rail transit may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0063] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A voice sensing system for assisting hearing-impaired people in rail transit, characterized in that: include: Audio acquisition module, audio signal processing module, power amplifier module and output module; The audio acquisition module is used to receive input audio; The audio signal processing module is used to perform signal enhancement processing on the input audio to obtain an enhanced audio signal; The power amplifier module is used to amplify the enhanced audio signal to obtain an amplified audio signal; The output module is used to output the amplified audio signal; Among them, the audio signal processing module includes: a signal decomposition unit, used to perform CEEMDAN decomposition on the input audio to obtain a set of audio signal IMF components; a frequency domain feature extraction unit, used to respectively extract the frequency domain features of each audio signal IMF component in the set of audio signal IMF components to obtain a set of audio signal IMF component frequency domain feature vectors; a feature filtering unit, used to perform feature filtering based on semantic contribution on the set of audio signal IMF component frequency domain feature vectors to obtain a set of filtered audio signal IMF component frequency domain feature vectors; a signal restoration unit, used to perform signal restoration based on the set of filtered audio signal IMF component frequency domain feature vectors to obtain the enhanced audio signal.

2. The voice sensing system for assisting the hearing impaired in rail transit according to claim 1, characterized in that: The frequency domain feature extraction unit is used to: Each audio signal IMF component in the set of audio signal IMF components is input into an audio signal component frequency domain feature extractor based on a 1D-CNN model to obtain a set of audio signal IMF component frequency domain feature vectors.

3. The voice sensing system for assisting the hearing-impaired in rail transit according to claim 2, characterized in that: The audio signal component frequency domain feature extractor based on the 1D-CNN model includes an input layer, a one-dimensional convolution layer, an activation layer based on a ReLU activation function, a pooling layer and an output layer.

4. The voice sensing system for assisting the hearing-impaired in rail transit according to claim 3, characterized in that: The feature filtering unit comprises: A cluster analysis subunit, configured to perform cluster analysis on the set of frequency domain feature vectors of the IMF component of the audio signal to obtain a cluster center representation vector of the frequency domain feature of the IMF component of the audio signal as a pseudo anchor center; A centralized feature filtering subunit is used to perform feature filtering on the set of audio signal IMF component frequency domain feature vectors based on the semantic contribution of each audio signal IMF component frequency domain feature vector in the set of audio signal IMF component frequency domain feature vectors relative to the pseudo anchor center to obtain the set of filtered audio signal IMF component frequency domain feature vectors.

5. The voice sensing system for assisting the hearing-impaired in rail transit according to claim 4, characterized in that: The cluster analysis subunit is used for: A position-based mean vector of a set of frequency-domain feature vectors of the IMF component of the audio signal is calculated as a cluster center representation vector of frequency-domain features of the IMF component of the audio signal.

6. The voice sensing system for assisting the hearing-impaired in rail transit according to claim 5, characterized in that: The centralized feature filtering subunit includes: A semantic contribution query secondary subunit, used to calculate the semantic contribution representation matrix of each audio signal IMF component frequency domain feature vector in the set of audio signal IMF component frequency domain feature vectors relative to the pseudo anchor center to obtain a set of audio signal IMF component frequency domain feature semantic contribution representation matrices; The feature modulation secondary subunit is used to feature modulate the set of frequency domain feature vectors of the audio signal IMF component based on the set of frequency domain feature semantic contribution representation matrices of the audio signal IMF component to obtain the set of filtered frequency domain feature vectors of the audio signal IMF component.

7. The voice sensing system for assisting the hearing-impaired in rail transit according to claim 6, characterized in that: The semantic contribution query secondary sub-unit is used to: The frequency domain feature vector of the IMF component of the audio signal is multiplied by the transposed vector of the pseudo anchor center and then divided by the second norm of the pseudo anchor center to obtain the semantic contribution representation matrix of the frequency domain feature of the IMF component of the audio signal.

8. The voice sensing system for assisting the hearing-impaired in rail transit according to claim 7, characterized in that: The characteristic modulation secondary subunit is used for: Performing contribution mask sparse processing based on a random dropout mechanism on each audio signal IMF component frequency domain feature semantic contribution representation matrix in the set of audio signal IMF component frequency domain feature semantic contribution representation matrices to obtain a set of mask sparse audio signal IMF component frequency domain feature semantic contribution representation matrices; Taking each masked sparse audio signal IMF component frequency domain feature semantic contribution representation matrix in the set of masked sparse audio signal IMF component frequency domain feature semantic contribution representation matrices as a modulation matrix, the matrix product between it and each audio signal IMF component frequency domain feature vector is calculated respectively to obtain the set of filtered audio signal IMF component frequency domain feature vectors.

9. The voice sensing system for assisting the hearing-impaired in rail transit according to claim 8, characterized in that: The signal restoration unit comprises: A full frequency domain feature aggregation subunit, used for cascading the set of frequency domain feature vectors of the IMF component of the filtered audio signal to obtain a full frequency domain cascade representation vector of the audio signal; The decoding subunit is used to input the full-frequency domain cascade representation vector of the audio signal into a decoder-based audio signal restorer to obtain the enhanced audio signal.

Images