An audio recognition and noise reduction processing method and system for a power grid control workbench environment

Abstract

A method and system for audio recognition and noise reduction processing in a power grid control workstation environment includes: obtaining pure audio signals from all power grid control staff in different work scenarios and different background noises in the control station, cross-domain fusion of time and frequency domain features, and using them as a training set to train a speaker model; real-time acquisition of speech signals, and removal of pure noise and silence from the speech signals; weighted processing of the speech signals after pure noise and silence removal based on noise level estimation and noise power; identification and separation of human voice segments using a continuous downsampling and resampling algorithm of multi-resolution features; determination of whether a human voice segment contains a target speaker using a speaker model; generation of multiple speech segments based on the human voice segments, and extraction of the target speaker's voice through clustering and speaker modeling. The present invention adapts to environmental changes at control and control stations at all levels, can significantly reduce environmental noise, and achieve accurate recognition and extraction of human voices.

Description

Technical Field

[0001] The present invention belongs to the field of speech recognition technology, and more specifically, relates to an audio recognition and noise reduction processing method and system in a power grid control workbench environment. Background Art

[0002] Traditional audio acquisition methods are unable to capture clean audio in complex environments, due to the presence of numerous interference sources such as ringing phones, alarms, conversations with others, and equipment noise. Furthermore, power dispatch tasks typically involve multiple rounds of interaction, requiring continuous audio acquisition, which renders traditional endpoint detection methods ineffective.

[0003] Existing technologies for voice recognition include:

[0004] The patent with authorization announcement number CN109686377B discloses an audio recognition method and device, and a computer-readable storage medium. The method includes: obtaining a voiceprint vectorization model; obtaining multiple different first audio files of the same target speaker; vectorizing each first audio file using the voiceprint vectorization model; determining the center vectors of the voiceprint vectors of the multiple different first audio files using at least one method, and determining a similarity acceptance range using each center vector; obtaining the voiceprint vector of the audio file to be identified using the voiceprint vectorization model, and calculating the similarity between the voiceprint vector of the audio file to be identified and each center vector; for each center vector, determining whether the similarity between the voiceprint vector of the audio file to be identified and the center vector is within the similarity acceptance range determined using the center vector, and determining whether the audio file to be identified belongs to the target speaker based on the judgment result. Patent publication number CN116312570A discloses a method, apparatus, device, and medium for speech noise reduction based on voiceprint recognition. The method comprises: obtaining voiceprint template information of a designated person and scene audio including the designated person's voice; performing voice separation on the scene audio to obtain multiple individual audios corresponding to a single person, each of which includes scene noise; matching the individual audios with the voiceprint template information to determine the designated person's audio corresponding to the designated person; and performing noise reduction on the designated person's audio to obtain the target audio. By performing voice separation on the scene audio and matching the audio corresponding to the designated person among the multiple individual audios corresponding to the designated person, the audio corresponding to the designated person can be obtained. In a multi-speaker speech scene, all audio except that of the target speaker can be treated as noise, while retaining the target speaker's voice. However, the above patent cannot capture clean audio in complex environments with multiple interference sources, such as ringing phones, alarms, conversations between bystanders, and equipment noise. Summary of the Invention

[0005] In order to solve the deficiencies in the prior art, the present invention provides a comprehensive discrimination audio acquisition method that integrates endpoint detection, mixed audio separation, and voiceprint recognition and authentication.

[0006] The present invention adopts the following technical solutions.

[0007] A first aspect of the present invention provides an audio recognition and noise reduction processing method for a power grid control workstation environment, characterized by comprising:

[0008] The system obtains pure audio signals from all power grid control personnel in different working scenarios and different background noises in the dispatching console. It extracts the time domain waveform features and frequency domain features of the pure audio signals at different time scales. These two features are then cross-domain fused to serve as the training set for the speaker model. The system also collects voice signals in real time, performs endpoint detection on the voice signals, and removes pure noise and silence from the voice signals.

[0009] Based on the noise level estimation and the noise power of different background noises in the dispatching console, the weights of each time frame and different frequencies of the speech signal after pure noise and silence are removed are calculated, thereby performing weighted processing on the speech signal after pure noise and silence are removed;

[0010] The weighted speech signal is subjected to a continuous downsampling and resampling algorithm using pre-trained multi-resolution features to identify and separate vocal segments, including sidekicks and the target speaker's voice. The speaker model is used to determine whether the vocal segment contains the target speaker.

[0011] If the vocal segment contains the target speaker, the target speaker's voice is extracted through region proposal, clustering and speaker model, and a voiceprint mapping table for the staff's work scene and time period is established. Based on the table, a comprehensive loss function is constructed, and the speaker model is updated using it and the extracted target speaker's voice.

[0012] Preferably, different work scenarios for power grid control staff include: monitoring and dispatching, fault handling, and safety analysis;

[0013] Different background noises in the dispatching console include transformer noise, alarm noise, telephone ringing noise, server noise, and human noise;

[0014] The server noise includes the noise of air conditioners, fans, and power supplies, and the personnel noise includes the sounds of people walking, flipping through documents, and typing on keyboards.

[0015] Preferably, the time domain waveform features and frequency domain features of the pure audio signal are extracted from different time scales, specifically:

[0016] The audio signal is divided into a number of long-time window signals with a set time length T1 as a frame; each long-time window signal is divided into a number of medium-time window signals with a set time length T2 as a frame; each medium-time window signal is divided into a number of short-time window signals with a set time length T3 as a frame; the waveform skewness of all long-time window signals, the energy envelope of the medium-time window signal and the zero-crossing rate of the short-time window signal are calculated as time domain waveform features;

[0017] The Mel spectrum and phase derivative spectrum of these audio signals are extracted as frequency domain features.

[0018] Preferably, the cross-domain fusion of the two features is used as a training set to train the speaker model, specifically:

[0019] The speaker model is a convolutional neural network;

[0020] The cross-domain fusion of the two features is to respectively calculate the attention weight of the time domain waveform feature to the frequency domain feature, and the attention weight of the frequency domain feature to the time domain waveform feature;

[0021] When calculating the attention weight of the time domain waveform feature to the frequency domain feature, first obtain the query vector that maps the time domain waveform feature to the set dimension, and the key vector and value vector that map the frequency domain feature to the set dimension, and use them to calculate the attention weight of the time domain waveform feature to the frequency domain feature;

[0022] When calculating the attention weight of the frequency domain feature to the time domain waveform feature, first obtain the query vector that maps the frequency domain feature to the set dimension, and the key vector and value vector that map the time domain waveform feature to the set dimension, and use them to calculate the attention weight of the time domain waveform feature to the frequency domain feature; multiply the time domain waveform feature by the attention weight of the time domain waveform feature to the frequency domain feature, and multiply the frequency domain feature by the attention weight of the frequency domain feature to the time domain waveform feature, and concatenate the two calculation results to obtain the fused feature.

[0023] Preferably, the endpoint detection process is specifically as follows:

[0024] Step 1: Collect voice signals through a microphone. After collecting voice signals within a set time period, the collected voice signals are input into the voice activity detection algorithm model for judgment and output of the judged voice signal type;

[0025] Step 2: If the voice type is a non-voice signal, it is determined to be pure noise or silence, and the microphone stops collecting voice; if it is a voice signal, proceed to step 3 to continue using the microphone to collect voice signals;

[0026] Step 3: During the voice collection process, the voice signals collected in the next time period are sequentially passed through the voice activity detection algorithm to determine whether it detects non-voice signals. If no non-voice signals are detected, the signal collection continues. If non-voice signals are detected, the process proceeds to step 4.

[0027] Step 4: Continue to collect speech signals for the set M time periods. The speech signals collected in each time period are input into the voice activity detection algorithm for detection. If a speech signal is detected, the task is still in the speech process and returns to step 3. If there is no speech signal within M time periods, it is considered that the speech has ended and the microphone collection stops.

[0028] Preferably, the voice activity detection algorithm is specifically:

[0029] After the speech signal is pre-processed by pre-emphasis, framing, and windowing, it is converted into a frequency domain signal using Mel-frequency cepstral coefficients (MFCC).

[0030] The frequency domain signal is divided into six subbands according to the frequency. For each subband, its energy is calculated. The probability density function of the trained Gaussian model is used to calculate the energy of each of the six subbands and the weighted sum of the energy of the six subbands to obtain the corresponding log-likelihood ratio. The energy of the six subbands is calculated as the local log-likelihood function, and the weighted sum of the energy of the six subbands is calculated as the global log-likelihood ratio.

[0031] When making speech judgments, all local and global log-likelihood ratios are evaluated. If any local or global log-likelihood ratio exceeds the set log-likelihood ratio threshold, speech is considered to be present. The weights and thresholds of the six sub-band energy weightings are obtained through training. After each detection, the algorithm self-learns and updates the mean and variance parameters of the Gaussian model.

[0032] Preferably, the weights of each time frame and different frequencies of the speech signal after removing pure noise and silence are calculated based on the noise level estimation and the noise power of different background noises in the dispatching station, specifically:

[0033] The voice signal after pure noise and silence are removed, and the different background noises in the dispatching console are divided into frames with a set duration T4 as one frame, and decomposed into several frequency points through short-time Fourier transform;

[0034] Calculate the noise level estimate of the speech signal after removing pure noise and silence for each frame and noise power of different background noises in the dispatch console;

[0035] Calculate the speech signal after removing pure noise and silence Frame No. i The frequency corresponding to the frequency point Weight The formula is:

[0036]

[0037] in, 、 、 are regularization coefficients; is the noise level estimation threshold set, is the maximum noise level estimate, is the sum of the noise powers of the dominant background noise at different frequencies, 、 、 、 、 Respectively represent the noise power of transformer noise, alarm noise, telephone ringing noise, server noise, and personnel noise.

[0038] Preferably, the noise level estimate of the speech signal after removing pure noise and silence is calculated for each frame The noise power of the background noise different from that in the dispatching console is:

[0039] No. Noise level estimation of a frame The calculation formula is:

[0040]

[0041] in, is the total number of frequency points of short-time Fourier transform; For the Frame No. i The frequency corresponding to the frequency point The complex spectrum value of ;

[0042] The noise power of different background noises in the dispatching station is to calculate the noise level estimation of each frame of different background noises in the dispatching station, and calculate the average value as the noise power.

[0043] Preferably, the target speaker voice is extracted through region proposal, clustering and speaker model, specifically:

[0044] A region proposal algorithm is used to generate multiple non-overlapping speech segments based on the human voice segment. This algorithm is a fully convolutional network model that uses a 3*3 sliding window to slide the human voice segment to generate multiple speech segments.

[0045] Clustering is used to form multiple clusters from speech clips in multiple non-overlapping areas. Each speech clip in each cluster is input into the speaker model for recognition. The number of speech clips containing the target speaker in each cluster is counted, and all the speech clips in the cluster with the largest number are integrated into a speech signal, which is the extracted target speaker's voice.

[0046] Preferably, the voiceprint mapping table of the staff's working scene and time period is established, and a comprehensive loss function is constructed based on the table. The speaker model is updated using the comprehensive loss function and the extracted target speaker voice, specifically:

[0047] Based on the duty roster and historical work records of all power grid control staff, a voiceprint mapping table for staff working scenarios and periods is established. This table establishes the mapping relationship between the most common working scenarios in each working period of each power grid control staff, the cross-domain fused feature vector corresponding to the pure audio signal in the corresponding working scenario, and the short-time Fourier transform complex spectrum;

[0048] Extract the time domain waveform features and frequency domain features of the target speaker from different time scales respectively, and fuse them into a feature vector across domains. According to the staff working scene period voiceprint mapping table, obtain the characteristic vector of the pure audio signal of the target speaker in the working scene of the current period and short-time Fourier transform complex spectrum ;

[0049] The comprehensive loss function is:

[0050]

[0051] in, 、 are set parameters, and their values are all in the range of [0,1]. is the norm, for The transpose of is the short-time Fourier transform complex spectrum of the target speaker's voice.

[0052] The second aspect of the present invention provides an audio recognition and noise reduction processing system for a power grid control workbench environment using the audio recognition and noise reduction method described in the first aspect of the present invention, comprising: a speaker model construction module, an endpoint detection module, a voice segment separation module, a speech recognition module, and a target speaker voice extraction module, characterized in that:

[0053] The speaker model construction module obtains clean audio signals from all power grid control personnel in different working scenarios and different background noises in the dispatching console. It extracts the time domain waveform features and frequency domain features of the clean audio signals at different time scales. The two features are cross-domain fused and used as the training set to train the speaker model.

[0054] Endpoint detection module: collects voice signals in real time, performs endpoint detection on the voice signals, and removes pure noise and silence from the voice signals;

[0055] The Voice Segment Separation Module calculates the weights of each time frame and frequency of the speech signal after pure noise and silence are removed based on the noise level estimate and the noise power of different background noises in the dispatcher. This module then performs weighted processing on the speech signal after pure noise and silence are removed. The module then uses a pre-trained continuous downsampling and resampling algorithm using multi-resolution features to identify and separate the voice segments, including the voices of the target speaker and the surrounding speakers.

[0056] Speech recognition module: uses the speaker model to determine whether the vocal segment contains the target speaker;

[0057] Target speaker voice extraction module: If the voice clip contains the target speaker, the target speaker's voice is extracted through region proposal, clustering and speaker model, and a voiceprint mapping table for the staff's work scene and time period is established. Based on the table, a comprehensive loss function is constructed, and the speaker model is updated using it and the extracted target speaker's voice.

[0058] The beneficial effects of the present invention are that, compared with the prior art, the present invention takes into account the differences in the speech of different staff members in different working states at the control console, as well as the background noise of the control console in various frequency bands, extracts the time domain waveform features and frequency domain features of these audio signals from different time scales, and fuses these two features as a training set to train the speaker model. At the same time, the time domain and frequency domain features of the speech signal are utilized to make the model recognition more accurate; endpoint detection processing is performed through a voice activity detection algorithm; and to adapt to the environmental changes of control and dispatch consoles at all levels, the weights of different time frames and frequencies are weighted through noise level estimation and noise power calculation, thereby enhancing the noise characteristics and significantly reducing the environmental noise, especially the high-frequency noise in the main frequency band; the target speaker speech extraction based on speaker model and clustering is adopted to achieve accurate recognition and extraction of human voice, and by establishing a voiceprint mapping table for staff work scenes and time periods, self-learning and updating is performed according to the comprehensive loss function of the table, achieving environmental adaptation, and being able to accurately identify and extract target speaker segments, especially greatly improving the recognition rate for the control console environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1Flowchart of the method of the present invention. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. The embodiments described in this application are only part of the embodiments of the present invention, not all of them. Based on the spirit of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0061] like Figure 1 As shown, embodiment 1 of the present invention proposes an audio recognition and noise reduction processing method in a power grid control workbench environment, which is characterized by comprising:

[0062] Pure audio signals from all grid control staff in different work scenarios and different background noises in the dispatching desk are obtained, and the time domain waveform features and frequency domain features of the pure audio signals are extracted from different time scales respectively. The two features are cross-domain fused and used as the training set to train the speaker model. It should be noted that the features obtained after cross-domain fusion of each pure audio signal in different work scenarios of the same grid control staff are used as a training sample for the grid control staff.

[0063] Collect voice signals in real time, perform endpoint detection on the voice signals, and remove pure noise and silence from the voice signals;

[0064] Based on the noise level estimation and the noise power of different background noises in the dispatching console, the weights of each time frame and different frequencies of the speech signal after pure noise and silence are removed are calculated, thereby performing weighted processing on the speech signal after pure noise and silence are removed;

[0065] The weighted speech signal is subjected to a continuous downsampling and resampling algorithm using pre-trained multi-resolution features to identify and separate vocal segments, including sidekicks and the target speaker's voice. The speaker model is used to determine whether the vocal segment contains the target speaker.

[0066] If the vocal segment contains the target speaker, the target speaker's voice is extracted through region proposal, clustering and speaker model, and a voiceprint mapping table for the staff's work scene and time period is established. Based on the table, a comprehensive loss function is constructed, and the speaker model is updated using it and the extracted target speaker's voice.

[0067] Preferably, different work scenarios for power grid control staff include: monitoring and dispatching, fault handling, and safety analysis;

[0068] Different background noises in the dispatching console include transformer noise, alarm noise, telephone ringing noise, server noise, and human noise;

[0069] The server noise includes the noise of air conditioners, fans, and power supplies, and the personnel noise includes the sounds of people walking, flipping through documents, and typing on keyboards.

[0070] Preferably, the time domain waveform features and frequency domain features of these audio signals are extracted from different time scales, specifically:

[0071] The audio signal is divided into a number of long-time window signals with a set time length T1 as a frame; each long-time window signal is divided into a number of medium-time window signals with a set time length T2 as a frame; each medium-time window signal is divided into a number of short-time window signals with a set time length T3 as a frame; the waveform skewness of all long-time window signals, the energy envelope of the medium-time window signal and the zero-crossing rate of the short-time window signal are calculated as time domain waveform features;

[0072] The formula is as follows:

[0073]

[0074] in, When the function's independent variable is greater than 0, it outputs 1; when it is less than 0, it outputs -1; when it is equal to 0, it outputs 0; For the Zero-crossing rate of the frame short-time window signal; For the Energy envelope of the time window signal in the frame; For the The waveform skewness of the frame length time window signal; For the b The frame corresponds to the time window signal a Sample values; N is the total number of sampling values contained in a frame of short-time window signal, n is the first n sampling values; T2 / T3 is the number of short-time window signals contained in the medium-time window signal, and k is the kth short-time window signal contained in the medium-time window signal at this time; For the The set of all sample values contained in the frame length time window signal; and They are The mean and standard deviation of all sample values in ; For the The expectation of all sample values contained in the frame-length time window signal minus the cube of its mean;

[0075] The Mel spectrum and phase derivative spectrum of these audio signals are extracted as frequency domain features.

[0076] Preferably, the speaker model is specifically:

[0077] The speaker model is a convolutional neural network. When performing cross-domain attention fusion, the weighted fusion features of the waveform time features on the frequency domain features and the weighted fusion features of the frequency domain features on the waveform time features are calculated, and the two fusion features are spliced together.

[0078] When calculating the fusion feature after the multi-head attention weighting of feature A on feature B, first obtain the query vector that maps feature A to the set dimension , and a key vector that maps feature B to a set dimension Sum value vector ;

[0079] Calculate the query matrix of feature A to feature B , key matrix Sum Matrix :

[0080]

[0081] Calculate the multi-head attention weight of feature A to feature B. The formula is:

[0082]

[0083] in, is the vector composed of feature A; is the vector composed of feature B, for dimension.

[0084] Preferably, the endpoint detection process is specifically as follows:

[0085] Step 1: Collect voice signals through a microphone. After collecting voice signals within a set time period, the collected voice signals are input into the voice activity detection algorithm model for judgment and output of the judged voice signal type;

[0086] Step 2: If the voice type is a non-voice signal, it is determined to be pure noise or silence, and the microphone stops collecting voice; if it is a voice signal, proceed to step 3 to continue using the microphone to collect voice signals;

[0087] Step 3: During the voice collection process, the voice signals collected in the next time period are sequentially passed through the voice activity detection algorithm to determine whether it detects non-voice signals. If no non-voice signals are detected, the signal collection continues. If non-voice signals are detected, the process proceeds to step 4.

[0088] Step 4: Continue to collect speech signals for the set M time periods. The speech signals collected in each time period are input into the voice activity detection algorithm for detection. If a speech signal is detected, the task is still in the speech process and returns to step 3. If there is no speech signal within M time periods, it is considered that the speech has ended and the microphone collection stops.

[0089] Preferably, the voice activity detection algorithm is specifically:

[0090] After the speech signal is pre-processed by pre-emphasis, framing, and windowing, it is converted into a frequency domain signal using Mel-frequency cepstral coefficients (MFCC).

[0091] The frequency domain signal is divided into six sub-bands according to the frequency range. The frequency ranges of these sub-bands are 80Hz-250Hz, 250Hz-500Hz, 500Hz-1K, 1K-2K, 2K-3K, and 3K-4K. For each sub-band, its energy is calculated. The probability density function of the trained Gaussian model is used to calculate the energy of these six sub-bands and the weighted sum of the energy of the six sub-bands to obtain the corresponding log-likelihood ratio. Among them, the energy calculated for the six sub-bands is the local log-likelihood function, and the weighted sum of the energy of the six sub-bands is the global log-likelihood ratio.

[0092] When making speech judgments, all local and global log-likelihood ratios are evaluated. If any local or global log-likelihood ratio exceeds the set log-likelihood ratio threshold, speech is considered to be present. The weights and thresholds of the six sub-band energy weightings are obtained through training. After each detection, the algorithm self-learns and updates the mean and variance parameters of the Gaussian model.

[0093] Preferably, the weights of each time frame and different frequencies of the speech signal after removing pure noise and silence are calculated based on the noise level estimation and the noise power of different background noises in the dispatching station, specifically:

[0094] The voice signal after pure noise and silence are removed, and the different background noises in the dispatching console are divided into frames with a set duration T4 as one frame, and decomposed into several frequency points through short-time Fourier transform;

[0095] Calculate the noise level estimate of the speech signal after removing pure noise and silence for each frame and noise power of different background noises in the dispatch console;

[0096] Calculate the speech signal after removing pure noise and silence Frame No. i The frequency corresponding to the frequency point Weight The formula is:

[0097]

[0098] in, 、 、 are regularization coefficients; is the noise level estimation threshold set, is the maximum noise level estimate, is the sum of the noise powers of the dominant background noise at different frequencies, 、 、 、 、 Respectively represent the noise power of transformer noise, alarm noise, telephone ringing noise, server noise, and personnel noise.

[0099] Calculate the noise level estimate of the speech signal after removing pure noise and silence for each frame The noise power of the background noise different from that in the dispatching console is:

[0100] No. Noise level estimation of a frame The calculation formula is:

[0101]

[0102] in, is the total number of frequency points of short-time Fourier transform; For the Frame No. i The frequency corresponding to the frequency point The complex spectrum value of ;

[0103] The noise power of different background noises in the dispatching station is to calculate the noise level estimation of each frame of different background noises in the dispatching station, and calculate the average value as the noise power.

[0104] Preferably, the target speaker voice is extracted through region proposal, clustering and speaker model, specifically:

[0105] A region proposal algorithm is used to generate multiple non-overlapping speech segments based on the human voice segment. This algorithm is a fully convolutional network model that uses a 3*3 sliding window to slide the human voice segment to generate multiple speech segments.

[0106] Clustering is used to form multiple clusters from speech clips in multiple non-overlapping areas. Each speech clip in each cluster is input into the speaker model for recognition. The number of speech clips containing the target speaker in each cluster is counted, and all the speech clips in the cluster with the largest number are integrated into a speech signal, which is the extracted target speaker's voice.

[0107] Preferably, the voiceprint mapping table of the staff's working scene and time period is established, and a comprehensive loss function is constructed based on the table. The speaker model is updated using the comprehensive loss function and the extracted target speaker voice, specifically:

[0108] Based on the duty roster and historical work records of all power grid control staff, a voiceprint mapping table for staff working scenarios and periods is established. This table establishes the mapping relationship between the most common working scenarios in each working period of each power grid control staff, the cross-domain fused feature vector corresponding to the pure audio signal in the corresponding working scenario, and the short-time Fourier transform complex spectrum;

[0109] Extract the time domain waveform features and frequency domain features of the target speaker from different time scales respectively, and fuse them into a feature vector across domains. According to the staff working scene period voiceprint mapping table, obtain the characteristic vector of the pure audio signal of the target speaker in the working scene of the current period and short-time Fourier transform complex spectrum ;

[0110] The comprehensive loss function is:

[0111]

[0112] in, 、 are set parameters, and their values are all in the range of [0,1]. is the norm, for The transpose of is the short-time Fourier transform complex spectrum of the target speaker's voice.

[0113] Embodiment 2 of the present invention provides an audio recognition and noise reduction processing system for a power grid control workstation environment using the audio recognition and noise reduction method described in Embodiment 1 of the present invention, comprising: a speaker model construction module, an endpoint detection module, a voice segment separation module, a speech recognition module, and a target speaker voice extraction module, characterized in that:

[0114] The speaker model construction module obtains clean audio signals from all power grid control personnel in different working scenarios and different background noises in the dispatching console. It extracts the time domain waveform features and frequency domain features of the clean audio signals at different time scales. The two features are cross-domain fused and used as the training set to train the speaker model.

[0115] Endpoint detection module: collects voice signals in real time, performs endpoint detection on the voice signals, and removes pure noise and silence from the voice signals;

[0116] The Voice Segment Separation Module calculates the weights of each time frame and frequency of the speech signal after pure noise and silence are removed based on the noise level estimate and the noise power of different background noises in the dispatcher. This module then performs weighted processing on the speech signal after pure noise and silence are removed. The module then uses a pre-trained continuous downsampling and resampling algorithm using multi-resolution features to identify and separate the voice segments, including the voices of the target speaker and the surrounding speakers.

[0117] Speech recognition module: uses the speaker model to determine whether the vocal segment contains the target speaker;

[0118] Target speaker voice extraction module: If the voice clip contains the target speaker, the target speaker's voice is extracted through region proposal, clustering and speaker model, and a voiceprint mapping table for the staff's work scene and time period is established. Based on the table, a comprehensive loss function is constructed, and the speaker model is updated using it and the extracted target speaker's voice.

[0119] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A method for audio recognition and noise reduction in a power grid control workbench environment, characterized in that: include: The system obtains pure audio signals from all power grid control personnel in different working scenarios and different background noises in the dispatching console. It extracts the time domain waveform features and frequency domain features of all pure audio signals at different time scales. These two features are then cross-domain fused to serve as the training set for the speaker model. The system also collects voice signals in real time, performs endpoint detection on them, and removes pure noise and silence from the voice signals. The voice signal after pure noise and silence are removed, and the different background noises in the dispatching console are divided into frames with a set duration T4 as one frame, and decomposed into several frequency points through short-time Fourier transform; Calculate the noise level estimate of the speech signal after removing pure noise and silence for each frame and noise power of different background noises in the dispatch console; Calculate the speech signal after removing pure noise and silence Frame No. i The frequency corresponding to the frequency point Weight , the formula is: in, 、 、 are regularization coefficients; is the noise level estimation threshold set, is the maximum noise level estimate, is the sum of the noise powers of the dominant background noise at different frequencies, 、 、 、 、 Respectively represent the noise power of transformer noise, alarm noise, telephone ringing noise, server noise, and human noise; thereby performing weighted processing on the speech signal after removing pure noise and silence; Using a pre-trained multi-resolution feature continuous downsampling and resampling algorithm on the weighted speech signal to identify and separate vocal segments, the vocal segments including sidekicks and the target speaker's voice; Use the speaker model to determine whether the vocal segment contains the target speaker; If the vocal segment contains the target speaker, the target speaker's voice is extracted through region proposal, clustering and speaker model, and a voiceprint mapping table for the staff's work scene and time period is established. Based on the mapping table, a comprehensive loss function is constructed, and the speaker model is updated using it and the extracted target speaker's voice.

2. The method for audio recognition and noise reduction in a power grid control workbench environment according to claim 1, characterized in that: The different work scenarios for power grid control staff include: monitoring and dispatching, fault handling, and safety analysis; Different background noises in the dispatching console include transformer noise, alarm noise, telephone ringing noise, server noise, and human noise; The server noise includes the noise of air conditioners, fans, and power supplies, and the personnel noise includes the sounds of people walking, flipping through documents, and typing on keyboards.

3. The method for audio recognition and noise reduction in a power grid control workbench environment according to claim 2, characterized in that: Extract the time domain waveform features and frequency domain features of pure audio signals from different time scales, specifically: The audio signal is divided into a number of long-time window signals with a set time length T1 as a frame; each long-time window signal is divided into a number of medium-time window signals with a set time length T2 as a frame; each medium-time window signal is divided into a number of short-time window signals with a set time length T3 as a frame; the waveform skewness of all long-time window signals, the energy envelope of the medium-time window signal and the zero-crossing rate of the short-time window signal are calculated as time domain waveform features; The Mel spectrum and phase derivative spectrum of these audio signals are extracted as frequency domain features.

4. The method for audio recognition and noise reduction in a power grid control workbench environment according to claim 3, characterized in that: The two features are fused across domains and used as a training set to train the speaker model, specifically: The speaker model is a convolutional neural network; The cross-domain fusion of the two features is to respectively calculate the attention weight of the time domain waveform feature to the frequency domain feature, and the attention weight of the frequency domain feature to the time domain waveform feature; When calculating the attention weight of the time domain waveform feature to the frequency domain feature, first obtain the query vector that maps the time domain waveform feature to the set dimension, and the key vector and value vector that map the frequency domain feature to the set dimension, and use them to calculate the attention weight of the time domain waveform feature to the frequency domain feature; When calculating the attention weight of the frequency domain feature on the time domain waveform feature, first obtain the query vector that maps the frequency domain feature to the set dimension, and the key vector and value vector that map the time domain waveform feature to the set dimension, and use them to calculate the attention weight of the time domain waveform feature on the frequency domain feature; Multiply the time domain waveform feature by the attention weight of the time domain waveform feature to the frequency domain feature, multiply the frequency domain feature by the attention weight of the frequency domain feature to the time domain waveform feature, and concatenate the two calculation results to form the fused feature.

5. The method for audio recognition and noise reduction in a power grid control workbench environment according to claim 1, characterized in that: The endpoint detection process is as follows: Step 1: Collect voice signals through a microphone. After collecting voice signals within a set time period, the collected voice signals are input into the voice activity detection algorithm model for judgment and output of the judged voice signal type; Step 2: If the voice type is a non-voice signal, it is determined to be pure noise or silence, and the microphone stops collecting voice; if it is a voice signal, proceed to step 3 to continue using the microphone to collect voice signals; Step 3: During the voice collection process, the voice signals collected in the next time period are sequentially passed through the voice activity detection algorithm to determine whether it detects non-voice signals. If no non-voice signals are detected, the signal collection continues. If non-voice signals are detected, the process proceeds to step 4. Step 4: Continue to collect speech signals for the set M time periods. The speech signals collected in each time period are input into the voice activity detection algorithm for detection. If a speech signal is detected, the task is still in the speech process and returns to step 3. If there is no speech signal within M time periods, it is considered that the speech has ended and the microphone collection stops.

6. The method for audio recognition and noise reduction in a power grid control workstation environment according to claim 5, characterized in that: The voice activity detection algorithm is specifically: After the speech signal is pre-processed by pre-emphasis, framing, and windowing, it is converted into a frequency domain signal using Mel-frequency cepstral coefficients (MFCC). The frequency domain signal is divided into six subbands according to the frequency. For each subband, its energy is calculated. The probability density function of the trained Gaussian model is used to calculate the energy of each of the six subbands and the weighted sum of the energy of the six subbands to obtain the corresponding log-likelihood ratio. The energy of the six subbands is calculated as the local log-likelihood function, and the weighted sum of the energy of the six subbands is calculated as the global log-likelihood ratio. When making speech judgments, all local and global log-likelihood ratios are evaluated. If any local or global log-likelihood ratio exceeds the set log-likelihood ratio threshold, speech is considered to be present. The weights and thresholds of the six sub-band energy weightings are obtained through training. After each detection, the algorithm self-learns and updates the mean and variance parameters of the Gaussian model.

7. The method for audio recognition and noise reduction in a power grid control workstation environment according to claim 1, characterized in that: Calculate the noise level estimate of the speech signal after removing pure noise and silence for each frame The noise power of the background noise different from that in the dispatching console is: No. Noise level estimation of a frame The calculation formula is: in, is the total number of frequency points of short-time Fourier transform; For the Frame No. i The frequency corresponding to the frequency point The complex spectrum value of ; The noise power of different background noises in the dispatching station is to calculate the noise level estimation of each frame of different background noises in the dispatching station, and calculate the average value as the noise power.

8. The method for audio recognition and noise reduction in a power grid control workstation environment according to claim 1, characterized in that: The target speaker voice is extracted through region proposal, clustering and speaker model, specifically: A region proposal algorithm is used to generate multiple non-overlapping speech segments based on the human voice segment. This algorithm is a fully convolutional network model that uses a 3*3 sliding window to slide the human voice segment to generate multiple speech segments. Clustering is used to form multiple clusters from speech clips in multiple non-overlapping areas. Each speech clip in each cluster is input into the speaker model for recognition. The number of speech clips containing the target speaker in each cluster is counted, and all the speech clips in the cluster with the largest number are integrated into a speech signal, which is the extracted target speaker's voice.

9. The method for audio recognition and noise reduction in a power grid control workstation environment according to claim 2, characterized in that: The method is to establish a voiceprint mapping table for staff working scenes and periods, construct a comprehensive loss function based on the mapping table, and use the comprehensive loss function and the extracted target speaker voice to update the speaker model, specifically: Based on the duty roster and historical work records of all power grid control staff, a voiceprint mapping table for staff working scenarios and periods is established. This mapping table establishes the mapping relationship between the most common working scenarios in each working period of each power grid control staff, the cross-domain fused feature vector corresponding to the pure audio signal in the corresponding working scenario, and the short-time Fourier transform complex spectrum; Extract the time domain waveform features and frequency domain features of the target speaker from different time scales respectively, and fuse them into a feature vector across domains. According to the staff working scene period voiceprint mapping table, obtain the characteristic vector of the pure audio signal of the target speaker in the working scene of the current period and short-time Fourier transform complex spectrum ; Comprehensive loss function L for: in, 、 For the set 0-1 parameter, is the norm, for The transpose of is the short-time Fourier transform complex spectrum of the target speaker's voice.

10. An audio recognition and noise reduction processing system for a power grid control workstation environment based on the audio recognition and noise reduction processing method for a power grid control workstation environment according to any one of claims 1 to 9, comprising: The speaker model building module, endpoint detection module, voice segment separation module, speech recognition module, and target speaker voice extraction module are characterized by: The speaker model construction module obtains clean audio signals from all power grid control personnel in different working scenarios and different background noises in the dispatching console. It extracts the time domain waveform features and frequency domain features of the clean audio signals at different time scales. The two features are cross-domain fused and used as the training set to train the speaker model. Endpoint detection module: collects voice signals in real time, performs endpoint detection on the voice signals, and removes pure noise and silence from the voice signals; The Voice Segment Separation Module calculates the weights of each time frame and frequency of the speech signal after pure noise and silence are removed based on the noise level estimate and the noise power of different background noises in the dispatcher. This module then performs weighted processing on the speech signal after pure noise and silence are removed. The module then uses a pre-trained continuous downsampling and resampling algorithm using multi-resolution features to identify and separate the voice segments, including the voices of the target speaker and the surrounding speakers. Speech recognition module: uses the speaker model to determine whether the vocal segment contains the target speaker; Target speaker voice extraction module: If the voice clip contains the target speaker, the target speaker's voice is extracted through region proposal, clustering and speaker model, and a voiceprint mapping table for the staff's work scene and time period is established. Based on the mapping table, a comprehensive loss function is constructed, and the speaker model is updated using it and the extracted target speaker's voice.

Images