A method and system for controlling bathroom heater switch based on voice recognition

Abstract

The present invention relates to the field of speech recognition technology, and specifically to a method and system for controlling a bathroom heater switch based on speech recognition, including obtaining the speech energy at each moment in each historical IMF component and real-time IMF component, analyzing the distinguishing features between human voice and water sound, performing preliminary noise reduction on the real-time IMF component, and determining the first noise reduction speech energy at each moment in each real-time IMF component; analyzing the distinguishing features between human voice and external audio sound, performing further noise reduction on the real-time IMF component after preliminary noise reduction, and determining the second noise reduction speech energy at each moment in each real-time IMF component; obtaining the noise-reduced real-time speech signal segment according to the second noise reduction speech energy at each moment in each real-time IMF component, and then controlling the bathroom heater switch. The present invention analyzes the characteristics of water noise and audio noise, performs multiple noise reduction on the real-time IMF component, enhances the accuracy of speech recognition, and improves the accuracy of bathroom heater switch control.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a method and system for controlling a bathroom heater switch based on speech recognition. Background Art

[0002] At present, speech recognition technology has been widely used in smart homes, which can greatly improve the convenience of people's daily lives. For example, applying speech recognition technology to the switch control of bathroom heaters can make users more flexible in the use of bathroom heaters and reduce physical discomfort and colds caused by ambient temperature. The current speech recognition technology mainly extracts features from the received voice information, and then matches the extracted command-related features with the control commands in the model library. The central processing module then issues a control to achieve the purpose of controlling the switch of the bathroom heater.

[0003] The control of the bathroom heater is mainly carried out during the bathing process. During the bathing process, there is a large amount of water noise and possible noise from external audio playback, which will affect the accuracy of speech recognition. Therefore, before feature extraction, the audio will be initially denoised. The commonly used denoising method is to perform EMD (Empirical Mode Decomposition) decomposition on the audio, select the components containing effective information from the decomposed IMF components, and then superimpose them to obtain the denoised signal. However, the noise during the bathing process is large and the noise types are diverse. There are errors in the process of distinguishing IMF components, resulting in poor noise reduction effect, which in turn affects the accuracy of speech recognition. Summary of the invention

[0004] In order to solve the technical problem that the above-mentioned existing bathroom heater switch control process is affected by noise, resulting in low voice recognition accuracy, the purpose of the present invention is to provide a bathroom heater switch control method and system based on voice recognition, and the technical solution adopted is as follows:

[0005] An embodiment of the present invention provides a method for controlling a bathroom heater switch based on voice recognition, the method comprising the following steps:

[0006] Obtain several historical voice signal segments, real-time voice signal segments and voice segments of several operating users of the bathroom heater, and then determine the voice energy at each moment in each historical IMF component and real-time IMF component;

[0007] Analyze the distinguishing features between human voice and water sound according to the speech energy at each moment in each real-time IMF component and the historical IMF component, perform preliminary noise reduction on the real-time IMF component, and determine the first noise reduction speech energy at each moment in each real-time IMF component;

[0008] Analyze the distinguishing features of human voice and external audio sound according to the speech segment of each operating user and each real-time IMF component, perform re-noising on the real-time IMF component after preliminary noise reduction, and determine the second noise reduction speech energy at each moment in each real-time IMF component;

[0009] The real-time speech signal segment after noise reduction is obtained according to the second noise reduction speech energy at each moment in each real-time IMF component, and then the bathroom heater switch is controlled.

[0010] Furthermore, several historical voice signal segments and real-time voice signal segments of the bathroom heater are obtained, including:

[0011] First, obtain the historical sound signal of the bathroom heater in the preset historical period, and the real-time sound signal for voice recognition in the current period;

[0012] Then obtain several starting points and ending points of the historical sound signal, as well as the starting point and ending point of the real-time sound signal;

[0013] For the historical sound signal, any starting point, the first ending point on the right side of the starting point, and the sound signal contained therein are taken as a speech signal segment, which is recorded as a historical speech signal segment;

[0014] For a real-time sound signal, the starting point, the ending point and the sound signal contained therein are regarded as a speech signal segment and recorded as a real-time speech signal segment.

[0015] Further, according to the speech energy at each moment in each real-time IMF component and the historical IMF component, the distinguishing features between human voice and water sound are analyzed, the real-time IMF component is preliminarily denoised, and the first denoised speech energy at each moment in each real-time IMF component is determined, including:

[0016] Selecting the relevant IMF components of each real-time IMF component from all historical IMF components; determining the environmental noise intensity of the relevant IMF components of each real-time IMF component according to the speech energy at each moment in each historical IMF component;

[0017] The first noise reduction speech energy at each moment in each real-time IMF component is determined according to the energy intensity at each moment in each real-time IMF component and the ambient noise intensity of each related IMF component.

[0018] Further, according to the speech energy at each moment in each historical IMF component, the environmental noise intensity of the relevant IMF component of each real-time IMF component is determined, including:

[0019] For any historical speech signal segment, the water acoustic noise index of the historical speech signal segment is determined according to the speech energy extreme difference value in each historical IMF component corresponding to the historical speech signal segment and the difference between two speech energies at adjacent moments; the historical speech signal segment whose water acoustic noise index is greater than the preset noise index is taken as the target historical speech signal segment;

[0020] The environmental noise intensity of the relevant IMF components of each real-time IMF component is determined based on the correlation coefficient between each relevant IMF component and each historical IMF component corresponding to each target historical speech signal segment, the underwater acoustic noise index of each target historical speech signal segment and the speech energy at each moment in each corresponding historical IMF component.

[0021] Further, the determining of the environmental noise intensity of the relevant IMF components of each real-time IMF component includes:

[0022] For any relevant IMF component, a historical IMF component with a correlation coefficient greater than or equal to a preset correlation threshold is used as a target historical IMF component corresponding to the relevant IMF component;

[0023] The ambient noise intensity of the relevant IMF component is determined by combining the speech energy mean and the underwater noise index corresponding to each target historical IMF component corresponding to the relevant IMF component.

[0024] Further, according to the energy intensity of each real-time IMF component at each moment and the ambient noise intensity of each related IMF component, determining the first noise reduction speech energy at each moment in each real-time IMF component includes:

[0025] ; In the formula, represents the first denoised speech energy at the tth moment in the real-time IMF component, represents the speech energy at the tth moment in the real-time IMF component, represents the environmental noise intensity of the relevant IMF component of the real-time IMF component, T represents the time length of the real-time speech signal segment, represents the speech energy at the t+1th moment in the real-time IMF component, It represents the absolute value function, and exp represents the exponential function with the natural constant e as the base.

[0026] Further, according to the speech segment of each operating user and each real-time IMF component, the distinguishing features of human voice and external audio sound are analyzed, the real-time IMF component after preliminary noise reduction is subjected to further noise reduction, and the second noise reduction speech energy at each moment in each real-time IMF component is determined, including:

[0027] Acquire the fundamental frequency sequence and the representative frequency sequence corresponding to the real-time voice signal segment and the voice segment of each operating user;

[0028] In the pitch frequency sequence corresponding to the real-time voice signal segment, counting the number of pitch frequencies that are the same as the pitch frequency of each operating user, and counting the maximum number of pitch frequencies that are continuously the same as the pitch frequency of each operating user;

[0029] Determine the audio noise index of the real-time speech signal segment relative to each operating user according to the number of fundamental frequency, the maximum number of fundamental frequency, the number of real-time IMF components corresponding to the real-time speech signal segment, and the edit distance between the real-time speech signal segment and the representative frequency sequence corresponding to the speech segment of each operating user;

[0030] The second denoised speech energy at each moment in each real-time IMF component is determined based on the first denoised speech energy at each moment in each real-time IMF component, the audio noise index of the real-time speech signal segment relative to each operating user, and the representative frequency sequence corresponding to each frequency in each real-time IMF component and the speech segment of each operating user.

[0031] Further, determining the audio noise index of the real-time speech signal segment relative to each operating user according to the number of fundamental frequency, the maximum number of fundamental frequency, the number of real-time IMF components corresponding to the real-time speech signal segment, and the edit distance between the real-time speech signal segment and the representative frequency sequence corresponding to the speech segment of each operating user includes:

[0032] ; In the formula, represents the audio noise index of the real-time speech signal segment relative to the i-th operating user, norm represents the linear normalization function, Indicates the number of real-time IMF components corresponding to the real-time speech signal segment, represents the edit distance between the real-time speech signal segment and the representative frequency sequence corresponding to the speech segment of the i-th operating user, exp represents an exponential function with a natural constant as the base, Indicates the maximum number of fundamental frequencies, represents the number of fundamental frequency, and M represents the number of fundamental frequency in the fundamental frequency sequence corresponding to the real-time speech signal segment.

[0033] Further, according to the first noise reduction speech energy at each moment in each real-time IMF component, the audio noise index of the real-time speech signal segment relative to each operating user, and each frequency in each real-time IMF component, and the representative frequency sequence corresponding to the speech segment of each operating user, the second noise reduction speech energy at each moment in each real-time IMF component is determined, including:

[0034] The operating user corresponding to the largest audio noise index is selected as the matching user, and for any real-time IMF component, the difference between each frequency in the real-time IMF component and each representative frequency corresponding to the speech segment of the matching user is calculated;

[0035] Selecting the minimum value among all the differences, and determining the noise reduction correction coefficient according to the minimum value and the audio noise index of the matching user;

[0036] The first noise reduction speech energy at each moment in the real-time IMF component is corrected by using the noise reduction correction coefficient to obtain the second noise reduction speech energy at each moment in the real-time IMF component.

[0037] An embodiment of the present invention also provides a bathroom heater switch control system based on voice recognition, including a processor and a memory, wherein the processor is used to process instructions stored in the memory to implement a bathroom heater switch control method based on voice recognition.

[0038] The present invention has the following beneficial effects:

[0039] In view of the voice signal in the scenario of using a bathroom heater, in the process of determining the component signal of the voice signal to achieve noise reduction, it is difficult to accurately extract the effective IMF component due to the complex noise type and large impact. The present invention provides a bathroom heater switch control method and system based on speech recognition. The present invention first obtains the speech energy at each moment in each historical IMF component and real-time IMF component, combines the data characteristics of the voice signal and the component signal, analyzes the characteristics of the water noise and the audio noise, performs different degrees of noise reduction on different noises, and completes the bathroom heater switch control based on speech recognition. The present invention is conducive to improving the noise reduction effect of the voice signal during the bathroom heater switch control, enhancing the accuracy of speech recognition, and further improving the accuracy of the bathroom heater switch control. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 This is a flow chart of an implementation method of a bathroom heater switch control method based on voice recognition according to an embodiment of the present invention;

[0042] Figure 2 is a schematic diagram of empirical mode decomposition corresponding to a valid sound segment in an embodiment of the present invention;

[0043] Figure 3is a flowchart of the implementation of step S21 in an embodiment of the present invention;

[0044] Figure 4 4 is a flowchart for implementing step S3 in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the technical solutions proposed by the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0046] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0047] The specific scenarios targeted by the present invention may be:

[0048] In the use scenario of bathroom heater, the main sources of sound are human voice, external audio sound and water sound, and the purpose of noise reduction processing on sound data is to reduce the interference of external audio sound and water sound, so as to amplify the human voice and make it clearer. When the sound data is processed by empirical mode decomposition method for noise reduction, it is difficult to accurately extract the effective IMF (Intrinsic Mode Function) component signal because the control voice signal is greatly affected by noise, which leads to the problem of poor voice recognition accuracy when the bathroom heater switch is controlled. Among them, external audio sound refers to the audio played through mobile phones, speakers and other devices according to different user habits, including music, entertainment programs, etc.

[0049] In order to improve the accuracy of voice recognition when controlling the bathroom heater switch, the noise attribute of the IMF component is judged to improve the noise reduction effect. Specifically, this embodiment provides a bathroom heater switch control method based on voice recognition, such as Figure 1 As shown, the following steps are included:

[0050] S1, obtain several historical voice signal segments, real-time voice signal segments and several voice segments of operating users of the bathroom heater, and then determine the voice energy at each moment in each historical IMF component and real-time IMF component.

[0051] It should be noted that when controlling the switch of the bathroom heater, it is necessary to obtain the real-time voice to be recognized. At this time, the real-time voice obtained is disturbed by external factors such as the external environment, and there are many noises, such as water sound and external audio sound, which will affect the accuracy of the real-time voice, and then affect the accuracy of the bathroom heater switch control achieved through the voice recognition results. In order to improve the accuracy of the voice recognition results, the acquired real-time voice signal is subjected to signal decomposition processing, and for each decomposed component signal data, denoising processing is performed based on the noise attribute characteristics. Before implementing the denoising process, it is necessary to first obtain the voice signal data, then decompose the voice signal data, and perform data conversion on the decomposed signal to obtain the spectrogram of each component signal, and then perform data analysis and processing based on the data information in the spectrogram to obtain the real-time voice signal after noise reduction.

[0052] The above step S1 can be implemented through steps S11 to S12 (not shown in the figure):

[0053] S11, obtaining several historical voice signal segments, real-time voice signal segments and several voice segments of operating users of the bathroom heater.

[0054] Specifically, first obtain the historical sound signal of the bathroom heater in a preset historical time period, and the real-time sound signal used for voice recognition in the current time period; then obtain several starting points and ending points of the historical sound signal, and the starting point and ending point of the real-time sound signal; for the historical sound signal, take any starting point, the first ending point on the right side of the starting point, and the sound signal contained therein as a voice signal segment, recorded as a historical voice signal segment, and for the real-time sound signal, take the starting point, the ending point, and the sound signal contained therein as a voice signal segment, recorded as a real-time voice signal segment; collect 15 seconds of sound for each user who controls the bathroom heater switch, and obtain the voice segment of each operating user.

[0055] In this embodiment, the sound signal can be collected through a microphone, and the preset historical period can be one month. The purpose of collecting historical sound signals is to facilitate subsequent comparative analysis with real-time sound signals; the starting point and the ending point of the sound signal can be obtained using a double threshold endpoint detection algorithm; in order to segment the historical sound signal, in accordance with the data of the real-time sound signal, the historical sound signal is segmented with the determined starting point and the ending point as the segmentation point to obtain multiple historical voice signal segments corresponding to the historical voice signal, and at the same time obtain a single real-time voice signal segment corresponding to the real-time voice signal; the voice segment of the operating user is collected to match the correct operating user to the real-time voice signal segment. The higher the matching degree between a certain operating user and the real-time voice signal segment, the higher the possibility that the operating user is the main body of the human voice in the real-time voice signal segment. It is worth noting that the voice signal segment includes the voice signal data corresponding to human voice, external audio, water sound, etc. when they appear alone or simultaneously.

[0056] Among them, the sound signal collection device can also be other devices, which are not specifically limited here; the preset historical time period can be set by the implementer according to the specific actual situation, which is not limited here; the implementation process of the double threshold endpoint detection algorithm to obtain the starting point and the ending point recorded in this embodiment is the prior art, which is not within the protection scope of the present invention and will not be elaborated here; in addition, the duration of the voice segment can also be set by the implementer according to the specific actual situation, which is not specifically limited here.

[0057] S12, determining the speech energy at each moment in each historical IMF component and the real-time IMF component according to each historical speech signal segment and a single real-time speech signal segment.

[0058] It should be noted that the waveform of an effective sound segment is mainly reflected in the sound wave frequency, amplitude, and waveform shape. Among them, the frequency can determine the pitch. The denser the waveform, the higher the frequency and the higher the pitch. The amplitude can reflect the sound intensity and loudness. The larger the amplitude, the greater the sound intensity and loudness. The shape and structure of the waveform can reflect the timbre of the sound. However, in order to analyze the sound information more comprehensively, the sound waveform can be converted into the frequency domain, and the spectrogram can be used to show the energy distribution of each frequency of the effective speech segment in the corresponding time period.

[0059] Specifically, each historical speech signal segment and a single real-time speech signal segment are subjected to empirical mode decomposition to obtain a number of IMF components corresponding to the speech signal segment; the IMF components corresponding to the historical speech signal segment are called historical IMF components, and the IMF components corresponding to the real-time speech signal segment are called real-time IMF components. The historical speech signal segment and the real-time speech signal segment can be collectively referred to as effective sound segments. The schematic diagram of the empirical mode decomposition corresponding to the effective sound segment is shown in FIG. Figure 2As shown in the figure, the IMF components obtained after decomposition are converted into a spectrogram. The horizontal axis of the spectrogram is time, the vertical axis is frequency, and the coordinate point value is the energy of the speech data. The size of the energy value is represented by the depth of color. The darker the color, the stronger the speech energy.

[0060] Among them, the implementation process of obtaining the spectrogram is a prior art and is not within the scope of protection of the present invention, and will not be elaborated here.

[0061] So far, this embodiment has acquired the benchmark data for data analysis and processing, namely, the speech energy at each moment in the spectrogram corresponding to each historical IMF component and the real-time IMF component and the speech segment of each operating user.

[0062] S2, analyzing the distinguishing features between human voice and water sound according to the speech energy at each moment in each real-time IMF component and historical IMF component, performing preliminary noise reduction on the real-time IMF component, and determining the first noise reduction speech energy at each moment in each real-time IMF component.

[0063] It should be noted that human voice, external audio sound and water sound each have their own characteristics. In the effective sound segment, the human voice has a relatively simple timbre and is often short in duration. It has different frequencies and amplitudes depending on the content of the speech. It is possible to talk to oneself a lot or to only give short command statements. However, what remains unchanged is that the same human voice is a relatively stable feature in the spectrogram; for external audio sound, it may have a variable timbre, followed by a long duration, and there are obvious differences between it and the human voice in the spectrogram; for water sound, the sound of water during bathing is relatively stable, which can be compared to white noise. The characteristic of white noise is that the energy of different frequencies in the spectrogram is relatively uniform. Among them, different IMF components of the same effective sound segment can reflect the energy changes of different frequencies in the effective sound signal.

[0064] The above step S2 can be implemented through steps S21 to S22 (not shown in the figure):

[0065] S21, selecting the relevant IMF components of each real-time IMF component from all historical IMF components; and determining the environmental noise intensity of the relevant IMF components of each real-time IMF component according to the speech energy at each moment in each historical IMF component.

[0066] Here, the relevant IMF components are used as reference data. The relevant IMF components specifically refer to the historical IMF components that are strongly correlated with the real-time IMF components. Each real-time IMF component has its corresponding relevant IMF component. The ambient noise intensity is used to quantify the influence of the underwater noise corresponding to the relevant IMF components on speech recognition. The greater the ambient noise intensity, the greater the possibility that the data characteristics of the real-time IMF components that are strongly correlated with the relevant IMF components are manifested as ambient noise.

[0067] The above step S21 can be performed by Figure 3 The steps S211 to S213 shown implement:

[0068] S211, calculating the correlation coefficient between each real-time IMF component and each historical IMF component in sequence, and determining the relevant IMF components of each real-time IMF component according to a number of correlation coefficients corresponding to each real-time IMF component.

[0069] Specifically, for any real-time IMF component, the Pearson correlation coefficient between the real-time IMF component and each historical IMF component is calculated, and the historical IMF component corresponding to the largest Pearson correlation coefficient is used as the relevant IMF component. The calculation process of the Pearson correlation coefficient is a prior art and is not within the scope of protection of the present invention, and will not be elaborated here.

[0070] It should be noted that under normal circumstances, the real-time IMF components of the same bathroom heater have historical IMF components with strong correlation. The reason is that the historical IMF components are obtained by collecting the historical voice signals of the bathroom heater in the past month, and they contain a large amount of information. If the users and usage environment of the same bathroom heater are similar, then the voice signals collected by the same bathroom heater will also be relatively similar, so each real-time IMF component has at least one historical IMF component with a large correlation coefficient.

[0071] S212, for any historical speech signal segment, determine the water acoustic noise index of the historical speech signal segment according to the speech energy extreme difference values ​​in each historical IMF component corresponding to the historical speech signal segment and the difference in speech energies between two adjacent moments; and take the historical speech signal segment whose water acoustic noise index is greater than the preset noise index as the target historical speech signal segment.

[0072] Here, the water acoustic noise index can be used to measure whether there is water acoustic noise in the historical speech signal segment. The speech energy extreme value can characterize the intensity range of the corresponding historical IMF component. When the intensity range is smaller and the speech energy difference at all adjacent moments is smaller, it means that the energy distribution is more uniform and more consistent with the characteristics of the water acoustic spectrogram. Therefore, the corresponding historical IMF component has a larger water acoustic noise index.

[0073] As an example, the calculation formula of the underwater noise index of the historical speech signal segment can be:

[0074] ; In the formula, E represents the water noise index of the historical speech signal segment, norm represents the linear normalization function, C represents the number of historical IMF components corresponding to the historical speech signal segment, c represents the sequence number of the historical IMF component, T represents the time length of the historical speech signal segment, exp represents the exponential function with the natural constant e as the base, represents the difference between the maximum speech energy and the minimum speech energy in the cth historical IMF component of the historical speech signal segment, represents the speech energy at the tth moment in the cth historical IMF component of the historical speech signal segment, represents the speech energy at the t+1th moment in the cth historical IMF component of the historical speech signal segment, Represents the absolute value function.

[0075] In the calculation formula of underwater noise index, Represents the speech energy extreme difference value of the cth historical IMF component of the historical speech signal segment, It represents the intensity difference of the cth historical IMF component at adjacent moments; combined with the speech energy extreme difference value and the intensity difference at adjacent moments, it can show the degree to which the cth historical IMF component is affected by water noise. The numerical accuracy of the water noise index determined by it is higher, and it can be used to screen the historical speech signal segments with water noise; the exp function is used to realize the negative correlation between the speech energy extreme difference value, the speech energy difference at adjacent moments and the water noise index; the norm function is used to realize the normalization of the data and limit the value range of the water noise index to between 0 and 1.

[0076] After obtaining the water noise index of each historical voice signal segment, the size of the preset noise index is set, which can be set to 0.3. When the water noise index of any historical voice signal segment is greater than 0.3, it means that the historical voice signal segment is a voice signal segment affected by water noise, and the historical voice signal segment is recorded as the target historical voice signal segment. The size of the preset noise index can be set by the implementer according to the specific actual situation, and is not specifically limited here.

[0077] It should be noted that, under normal circumstances, there is water sound in the time period corresponding to the historical voice signal of the bathroom heater during operation, so this embodiment does not analyze the situation where the water sound noise index of all historical voice signal segments is not greater than the preset noise index.

[0078] S213, determining the environmental noise intensity of the relevant IMF components of each real-time IMF component based on the correlation coefficient between each relevant IMF component and each historical IMF component corresponding to each target historical voice signal segment, the underwater acoustic noise index of each target historical voice signal segment and the speech energy at each moment in each corresponding historical IMF component.

[0079] Here, the correlation coefficient is used to measure the correlation between the relevant IMF component and each historical IMF component. The higher the correlation, the more similar the speech energy distribution characteristics of the relevant IMF component and the historical IMF component with water acoustic noise are. The correlation coefficient can be used to screen out target historical speech signal segments with strong correlation from all target historical speech signal segments. Based on the water acoustic noise index of the screened target historical speech signal segments and the speech energy size of the corresponding historical IMF components with strong correlation, the environmental noise intensity of the relevant IMF components of each real-time IMF component can be determined.

[0080] Specifically, for any relevant IMF component, a historical IMF component with a correlation coefficient greater than or equal to a preset correlation threshold is used as a target historical IMF component corresponding to the relevant IMF component; and the environmental noise intensity of the relevant IMF component is determined by combining the speech energy mean and the water noise index corresponding to each target historical IMF component corresponding to the relevant IMF component. The correlation coefficient can be a Pearson correlation coefficient or a Spearman correlation coefficient.

[0081] As an example, the calculation formula of the ambient noise intensity of the relevant IMF component can be:

[0082] ; In the formula, represents the environmental noise intensity of the relevant IMF component, I represents the number of target historical IMF components corresponding to the relevant IMF component, i represents the sequence number of the target historical IMF component, represents the water acoustic noise index of the target historical speech signal segment to which the i-th target historical IMF component corresponding to the relevant IMF component belongs, Represents the mean value of the speech energy corresponding to the i-th target historical IMF component corresponding to the relevant IMF component.

[0083] In the calculation formula of environmental noise intensity, It represents the average value of the water noise index of the target historical speech signal segment to which all target historical IMF components belong. The average value can show the environmental noise intensity of the relevant IMF components. The larger the average value, the greater the possibility that the relevant IMF components are water noise. On this basis, multiplying the average value of the speech energy means corresponding to all target historical IMF components, it can be shown that the relevant IMF components are not only likely to have water noise, but also have a strong sound intensity, which is more likely to be the sound of shower water, because the sound of rain water is high-frequency.

[0084] It should be noted that the historical speech signal segment is used here to characterize the environmental noise intensity of the real-time IMF component, which can avoid the limitations of only analyzing the data characteristics of the real-time IMF component, thereby improving the accuracy of the environmental noise intensity analysis of the real-time IMF component.

[0085] S22, determining the first noise reduction speech energy at each moment in each real-time IMF component according to the energy intensity at each moment in each real-time IMF component and the ambient noise intensity of each related IMF component.

[0086] In this embodiment, the first noise reduction speech energy refers to the speech energy after removing the influence of water noise to a certain extent. When removing the influence of water noise, it is not only necessary to consider the ambient noise intensity determined by the water noise characteristics, but also the influence of other sounds. The influence of other sounds is characterized by the speech energy ratio at adjacent moments.

[0087] As an example, the calculation formula for the first noise reduction speech energy at the t-th moment in the real-time IMF component may be:

[0088] ; In the formula, represents the first denoised speech energy at the tth moment in the real-time IMF component, represents the speech energy at the tth moment in the real-time IMF component, represents the environmental noise intensity of the relevant IMF component of the real-time IMF component, T represents the time length of the real-time speech signal segment, represents the speech energy at the t+1th moment in the real-time IMF component, It represents the absolute value function, and exp represents the exponential function with the natural constant e as the base.

[0089] In the calculation formula of the first noise reduction speech energy, It means comparing the intensity of all adjacent moments of the real-time IMF component and finding the average value, which can characterize the degree of change of the average intensity of the real-time IMF component. The larger the value, the greater the degree of change of the real-time IMF component and the more frequent the change. The real-time IMF component not only represents the intensity of ambient noise, but also includes other sounds. At this time, the real-time IMF component is not only affected by the water noise, but also involves other sounds. Therefore, the smaller the degree of subtracting the ambient noise intensity, the Perform negative correlation normalization.

[0090] Right now , the normalized value is used as the weight to multiply the ambient noise intensity; , the first noise reduction speech energy can be obtained, which effectively weakens the influence of environmental noise.

[0091] It should be noted that, under normal circumstances, the speech energy corresponding to the speech signal segment will not be equal to 0. If there is an extreme case, in the calculation formula of the first noise reduction speech energy, A hyperparameter is added to the denominator. For example, the empirical value of the hyperparameter is 0.1, which can be used to avoid the possibility of the denominator of the fraction being zero.

[0092] So far, this embodiment has determined the first noise reduction speech energy at each moment in each real-time IMF component.

[0093] S3, analyzing the distinguishing features between human voice and external audio sound according to the speech segment of each operating user and each real-time IMF component, performing further noise reduction on the real-time IMF component after preliminary noise reduction, and determining the second noise reduction speech energy at each moment in each real-time IMF component.

[0094] It should be noted that the characteristics of external audio sound are complex sound, many changes in frequency and waveform, so there are many IMF components, which are reflected in the spectrogram as the resonance gap frequency value and the fundamental frequency may change, and are different from human voice. External audio sound mainly includes music and human voice, so it can be analyzed from two aspects: music noise degree and human voice noise degree. The music noise degree mainly lies in the complexity of the sound, while the human voice noise degree is mainly different from the human voice to be extracted, which leads to different resonance peak frequency and fundamental frequency.

[0095] Among them, the fundamental frequency (F0) sequence of speech signals is an important task in speech signal processing, which is usually used for speech analysis and speech recognition. The fundamental frequency refers to the periodic component in the speech signal, that is, the fundamental frequency of the sound, which determines the pitch of the speech; the formant frequency in the speech signal is determined by the shape and size of the vocal tract and is an important component of the acoustic characteristics of speech. Obtaining these frequencies can help analyze the working conditions of the vocal organs, or be used for tasks such as speech recognition and speech synthesis.

[0096] The above step S3 can be Figure 4 The steps S31 to S34 shown implement:

[0097] S31, obtaining a fundamental frequency sequence and a representative frequency sequence corresponding to a real-time voice signal segment and a voice segment of each operating user.

[0098] In this embodiment, the fundamental frequency sequence and the representative frequency sequence are the reference data for the subsequent analysis of the audio noise index, so the fundamental frequency sequence and the representative frequency sequence need to be obtained. For each operating user's voice segment, it is used to remove the human voice noise to determine the operating user to whom the real-time voice signal segment belongs.

[0099] First, a fundamental frequency sequence and a representative frequency sequence corresponding to a real-time speech signal segment are obtained.

[0100] A time window of 5 seconds and a step length of 5 seconds are set. The fundamental frequency within the time window is obtained from the start time of the real-time speech signal segment using the autocorrelation function method to obtain the fundamental frequency sequence of the real-time speech signal segment; the spectrogram of the real-time speech signal segment is obtained, and the spectrogram is decomposed at multiple scales using the existing technology to obtain a detail layer sub-graph, and the detail layer sub-graph is edge detected to obtain several edges, and the average frequency of any edge is recorded as the frequency corresponding to the detail layer sub-graph, and all frequencies of the detail layer sub-graph are used as the representative frequencies of the real-time speech signal segment, and all representative frequencies are sorted in order from small to large, so as to obtain a representative frequency sequence of the real-time speech signal segment. Among them, the size of the time window can be set by the implementer according to the specific actual situation, and is not specifically limited here.

[0101] Secondly, the fundamental frequency sequence and representative frequency sequence corresponding to the speech segment of each operating user are obtained.

[0102] The autocorrelation function is used to obtain the fundamental frequency sequence of the operating user, and then the linear prediction coding algorithm is used to obtain the formant frequency sequence of the operating user. The individual formant frequencies in the formant frequency sequence are sorted in ascending order to obtain the representative frequency sequence of each operating user. That is, the sorted formant frequency sequence is used as the representative frequency sequence of the corresponding operating user.

[0103] S32, counting the number of fundamental frequency sequences corresponding to the real-time voice signal segments that are identical to the fundamental frequency of each operating user, and counting the maximum number of fundamental frequency sequences that are consecutively identical to the fundamental frequency of each operating user.

[0104] In this embodiment, when determining the number of fundamental frequency and the maximum number of fundamental frequency, each operating user is determined in turn, that is, each operating user has its corresponding number of fundamental frequency and maximum number of fundamental frequency relative to the fundamental frequency sequence. The obtained number of fundamental frequency and maximum number of fundamental frequency are key factors for subsequent calculation of audio noise index, which can be used to quantify the performance characteristics of each operating user relative to the real-time speech signal segment.

[0105] S33, determining the audio noise index of the real-time speech signal segment relative to each operating user based on the number of fundamental frequency, the maximum number of fundamental frequency, the number of real-time IMF components corresponding to the real-time speech signal segment, and the editing distance between the real-time speech signal segment and the representative frequency sequence corresponding to the speech segment of each operating user.

[0106] As an example, the calculation formula of the audio noise index of the real-time speech signal segment relative to the i-th operating user may be:

[0107] ; In the formula, represents the audio noise index of the real-time speech signal segment relative to the i-th operating user, norm represents the linear normalization function, Indicates the number of real-time IMF components corresponding to the real-time speech signal segment, represents the edit distance between the real-time speech signal segment and the representative frequency sequence corresponding to the speech segment of the i-th operating user, exp represents an exponential function with a natural constant as the base, Indicates the maximum number of fundamental frequencies, represents the number of fundamental frequency, and M represents the number of fundamental frequency in the fundamental frequency sequence corresponding to the real-time speech signal segment.

[0108] In the calculation formula of audio noise index, It can be said that in the fundamental frequency sequence corresponding to the real-time voice signal segment, the larger the maximum number of consecutive fundamental frequencies that are the same as those of the i-th operating user, and the larger the proportion of the corresponding fundamental frequencies, the stronger the human voice performance of the i-th operating user in the real-time voice signal segment, and the smaller the audio noise index will be; the edit distance between the representative frequency sequence corresponding to the real-time voice signal segment and the voice segment of the i-th operating user Can represent the similarity of speech frequency distribution, edit distance The larger the value, the greater the difference in speech frequency distribution, and the greater the possibility of audio noise in the real-time speech signal segment; the number of real-time IMF components corresponding to the real-time speech signal segment Indicates the sound complexity of the real-time speech signal segment and the number of real-time IMF components The larger the value is, the more sound sources there are in the real-time speech signal segment, and the greater the possibility that audio noise exists in the real-time speech signal segment.

[0109] S34, determining the second denoised speech energy at each moment in each real-time IMF component according to the first denoised speech energy at each moment in each real-time IMF component, the audio noise index of the real-time speech signal segment relative to each operating user, and each frequency in each real-time IMF component and the representative frequency sequence corresponding to the speech segment of each operating user.

[0110] The above step S34 can be implemented through steps S341 to S343 (not shown in the figure):

[0111] S341, selecting the operating user corresponding to the largest audio noise index as the matching user, and for any real-time IMF component, calculating the difference between each frequency in the real-time IMF component and each representative frequency corresponding to the speech segment of the matching user.

[0112] It should be noted that by taking the voices of all users of bathroom heaters as reference sounds and matching the real-time voice signal segments with the reference sounds, matching users of the real-time voice signal segments can be obtained, which helps to achieve more personalized voice noise reduction.

[0113] S342, selecting the minimum value among all the differences, and determining a noise reduction correction coefficient according to the minimum value and the audio noise index of the matching user.

[0114] S343, using the noise reduction correction coefficient to correct the first noise reduction speech energy at each moment in the real-time IMF component to obtain the second noise reduction speech energy at each moment in the real-time IMF component.

[0115] As an example, the calculation formula of the second noise reduction speech energy at the t-th moment in the real-time IMF component may be:

[0116] ; In the formula, represents the second denoised speech energy at the tth moment in the real-time IMF component, represents the first denoised speech energy at the t-th moment in the real-time IMF component, norm represents the linear normalization function, F represents the audio noise index of the real-time speech signal segment relative to the matching user, min represents the minimum function, f represents the frequency of the real-time IMF component, G represents any representative frequency of the representative frequency sequence of the matching user of the real-time speech signal segment, Represents the absolute value function.

[0117] In the calculation formula of the second noise reduction speech energy, Indicates the difference between the real-time IMF component and the representative frequency with the smallest difference among all representative frequencies of the matching user. The larger the value is, the smaller the correlation between the real-time IMF component and the matched user is, and the greater the degree of noise reduction should be; As the weight of the first denoised speech energy, the speech energy at the tth moment in the real-time IMF component is reduced. It indicates the degree of noise reduction of the first noise reduction speech energy. The greater the degree of noise reduction, the greater the intensity reduction.

[0118] So far, this embodiment obtains the second noise reduction speech energy at each moment in each real-time IMF component.

[0119] S4, obtaining the real-time speech signal segment after noise reduction according to the second noise reduction speech energy at each moment in each real-time IMF component, and then controlling the bathroom heater switch.

[0120] Specifically, first, according to the second denoised speech energy at each moment in each real-time IMF component, the denoised spectrogram of each real-time IMF component can be obtained; secondly, the denoised spectrograms of all real-time IMF components corresponding to the real-time speech signal segment are superimposed to obtain the denoised spectrogram of the real-time speech signal segment; then, the denoised spectrogram is restored to time domain speech signal data through inverse Fourier transform, which is recorded as the real-time speech signal segment after denoising; the real-time speech signal segment after denoising is used for speech recognition, so as to complete more accurate switch control of the bathroom heater.

[0121] Among them, the superposition process of the spectrogram of the IMF component and the implementation process of the inverse Fourier transform are prior arts, which are no longer within the protection scope of the present invention and will not be elaborated in detail here.

[0122] At this point, this embodiment obtains a more accurate real-time voice signal segment after noise reduction processing, which can achieve more accurate bathroom heater switch control.

[0123] The present invention also provides a bathroom heater switch control system based on voice recognition, including a processor and a memory, wherein the processor is used to process instructions stored in the memory to implement a bathroom heater switch control method based on voice recognition.

[0124] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for controlling a bathroom heater switch based on voice recognition, characterized in that: The following steps are involved: Obtain several historical voice signal segments, real-time voice signal segments and voice segments of several operating users of the bathroom heater, and then determine the voice energy at each moment in each historical IMF component and real-time IMF component; Analyze the distinguishing features between human voice and water sound according to the speech energy at each moment in each real-time IMF component and the historical IMF component, perform preliminary noise reduction on the real-time IMF component, and determine the first noise reduction speech energy at each moment in each real-time IMF component; Analyze the distinguishing features of human voice and external audio sound according to the speech segment of each operating user and each real-time IMF component, perform re-noising on the real-time IMF component after preliminary noise reduction, and determine the second noise reduction speech energy at each moment in each real-time IMF component; According to the second noise reduction speech energy at each moment in each real-time IMF component, the real-time speech signal segment after noise reduction is obtained, and then the switch of the bathroom heater is controlled; According to the speech energy at each moment in each real-time IMF component and the historical IMF component, the distinguishing features between human voice and water sound are analyzed, the real-time IMF component is preliminarily denoised, and the first denoised speech energy at each moment in each real-time IMF component is determined, including: Selecting the relevant IMF components of each real-time IMF component from all historical IMF components; determining the environmental noise intensity of the relevant IMF components of each real-time IMF component according to the speech energy at each moment in each historical IMF component; Determine the first noise reduction speech energy at each moment in each real-time IMF component according to the energy intensity at each moment in each real-time IMF component and the ambient noise intensity of each related IMF component; According to the speech energy at each moment in each historical IMF component, the environmental noise intensity of the relevant IMF component of each real-time IMF component is determined, including: For any historical speech signal segment, the water acoustic noise index of the historical speech signal segment is determined according to the speech energy extreme difference value in each historical IMF component corresponding to the historical speech signal segment and the difference between two speech energies at adjacent moments; the historical speech signal segment whose water acoustic noise index is greater than the preset noise index is taken as the target historical speech signal segment; The environmental noise intensity of the relevant IMF components of each real-time IMF component is determined based on the correlation coefficient between each relevant IMF component and each historical IMF component corresponding to each target historical speech signal segment, the underwater acoustic noise index of each target historical speech signal segment and the speech energy at each moment in each corresponding historical IMF component.

2. According to the method of controlling the bathroom heater switch based on voice recognition according to claim 1, it is characterized in that: Get several historical and real-time voice signal segments of Yuba, including: First, obtain the historical sound signal of the bathroom heater in the preset historical period, and the real-time sound signal for voice recognition in the current period; Then obtain several starting points and ending points of the historical sound signal, as well as the starting point and ending point of the real-time sound signal; For the historical sound signal, any starting point, the first ending point on the right side of the starting point, and the sound signal contained therein are taken as a speech signal segment, which is recorded as a historical speech signal segment; For a real-time sound signal, the starting point, the ending point and the sound signal contained therein are regarded as a speech signal segment and recorded as a real-time speech signal segment.

3. According to the method of controlling the bathroom heater switch based on voice recognition according to claim 1, it is characterized in that: Determining the environmental noise intensity of the relevant IMF components of each real-time IMF component includes: For any relevant IMF component, a historical IMF component with a correlation coefficient greater than or equal to a preset correlation threshold is used as a target historical IMF component corresponding to the relevant IMF component; The ambient noise intensity of the relevant IMF component is determined by combining the speech energy mean and the underwater noise index corresponding to each target historical IMF component corresponding to the relevant IMF component.

4. According to the method of controlling a bathroom heater switch based on voice recognition according to claim 1, it is characterized in that: Determining the first noise reduction speech energy at each moment in each real-time IMF component according to the energy intensity at each moment in each real-time IMF component and the ambient noise intensity of each related IMF component, including: Where D' t represents the first denoised speech energy at the tth moment in the real-time IMF component, D t represents the speech energy at the tth moment in the real-time IMF component, K1 represents the environmental noise intensity of the related IMF component of the real-time IMF component, T represents the time length of the real-time speech signal segment, and D t+1 represents the speech energy at the t+1th moment in the real-time IMF component, || represents the absolute value function, and exp represents the exponential function with the natural constant e as the base.

5. According to the method of controlling a bathroom heater switch based on voice recognition according to claim 1, it is characterized in that: According to the speech segment of each operating user and each real-time IMF component, the distinguishing features of human voice and external audio sound are analyzed, the real-time IMF component after preliminary noise reduction is subjected to further noise reduction, and the second noise reduction speech energy at each moment in each real-time IMF component is determined, including: Acquire the fundamental frequency sequence and the representative frequency sequence corresponding to the real-time voice signal segment and the voice segment of each operating user; In the pitch frequency sequence corresponding to the real-time voice signal segment, counting the number of pitch frequencies that are the same as the pitch frequency of each operating user, and counting the maximum number of pitch frequencies that are continuously the same as the pitch frequency of each operating user; Determine the audio noise index of the real-time speech signal segment relative to each operating user according to the number of fundamental frequency, the maximum number of fundamental frequency, the number of real-time IMF components corresponding to the real-time speech signal segment, and the edit distance between the real-time speech signal segment and the representative frequency sequence corresponding to the speech segment of each operating user; The second denoised speech energy at each moment in each real-time IMF component is determined based on the first denoised speech energy at each moment in each real-time IMF component, the audio noise index of the real-time speech signal segment relative to each operating user, and the representative frequency sequence corresponding to each frequency in each real-time IMF component and the speech segment of each operating user.

6. According to the method of controlling the bathroom heater switch based on voice recognition according to claim 5, it is characterized in that: The step of determining the audio noise index of the real-time speech signal segment relative to each operating user according to the number of fundamental frequency, the maximum number of fundamental frequency, the number of real-time IMF components corresponding to the real-time speech signal segment, and the edit distance between the real-time speech signal segment and the representative frequency sequence corresponding to the speech segment of each operating user comprises: In the formula, F i represents the audio noise index of the real-time speech signal segment relative to the i-th operating user, norm represents the linear normalization function, C' represents the number of real-time IMF components corresponding to the real-time speech signal segment, d i represents the edit distance between the real-time speech signal segment and the representative frequency sequence corresponding to the speech segment of the i-th operating user, exp represents an exponential function with a natural constant as the base, and L f0 Indicates the maximum number of fundamental frequencies, m f0 represents the number of fundamental frequency, and M represents the number of fundamental frequency in the fundamental frequency sequence corresponding to the real-time speech signal segment.

7. According to the method of controlling a bathroom heater switch based on voice recognition according to claim 1, it is characterized in that: Determining the second denoised speech energy at each moment in each real-time IMF component according to the first denoised speech energy at each moment in each real-time IMF component, the audio noise index of the real-time speech signal segment relative to each operating user, and each frequency in each real-time IMF component and a representative frequency sequence corresponding to the speech segment of each operating user, including: The operating user corresponding to the largest audio noise index is selected as the matching user, and for any real-time IMF component, the difference between each frequency in the real-time IMF component and each representative frequency corresponding to the speech segment of the matching user is calculated; Selecting the minimum value among all the differences, and determining the noise reduction correction coefficient according to the minimum value and the audio noise index of the matching user; The first noise reduction speech energy at each moment in the real-time IMF component is corrected by using the noise reduction correction coefficient to obtain the second noise reduction speech energy at each moment in the real-time IMF component.

8. A bathroom heater switch control system based on voice recognition, characterized in that: It comprises a processor and a memory, wherein the processor is used to process instructions stored in the memory to implement a bathroom heater switch control method based on voice recognition as described in any one of claims 1-7.

Images