Anti-noise speech recognition method and system based on humidity sensitive memristor, terminal and medium

Abstract

The invention discloses an anti-noise voice recognition method and system based on a humidity sensitive memristor, a terminal and a medium, and the method comprises the steps: collecting a voice signal based on a microphone device, obtaining audio data, collecting humidity information based on an LIF neuron circuit, and outputting voltage pulse data based on the humidity information through the LIF neuron circuit, the LIF neuron circuit comprises a humidity sensitive memristor; applying random noise to the audio data, and fusing and encoding the audio data containing the random noise and the voltage pulse data to obtain a fused data vector group; and inputting the fused data vector group into a pre-trained neural network, and outputting a speech recognition result. According to the method, the recognition limitation of a single voice signal is broken through, and the voice recognition accuracy and robustness in a noise environment are remarkably improved by utilizing multi-modal information fusion.

Description

Technical Field

[0001] This invention relates to the field of electronic science and technology, and in particular to a noise-resistant speech recognition method, system, terminal, and medium based on humidity-sensitive memristor. Background Technology

[0002] Multimodal perception fusion plays a central role in human perception and decision-making, especially in complex or noisy environments. For example, when crossing a busy street, accurate judgments rely on the fusion of visual and auditory information. However, in acoustically chaotic environments, relying solely on auditory signals is susceptible to interference, leading to a decline in speech recognition performance.

[0003] Most existing speech recognition systems are based on single-modal audio signals, such as Mel-Frequency Cepstral Coefficients (MFCCs), which are extracted and processed by neural networks. These systems perform well in quiet environments, but their recognition accuracy drops significantly in noisy environments. Traditional methods lack utilization of environmental information accompanying speech generation (such as humidity changes in exhaled airflow during speaking), making it impossible to achieve multimodal information fusion and limiting their applicability in real-world environments.

[0004] Therefore, existing technologies still have shortcomings. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a noise-resistant speech recognition method, system, terminal, and medium based on humidity-sensitive memristor technology, addressing the aforementioned deficiencies of existing technologies. The technical solution adopted by this invention is as follows: In a first aspect, the present invention provides a noise-resistant speech recognition method based on humidity-sensitive memristor, wherein the method includes: The system collects voice signals using a microphone device to obtain audio data, and collects humidity information using a LIF neuron circuit, which then outputs voltage pulse data based on the humidity information. The LIF neuron circuit includes a humidity-sensitive memristor. Random noise is applied to the audio data, and the audio data containing random noise is fused and encoded with the voltage pulse data to obtain a fused data vector group; The fused data vector group is input into a pre-trained neural network, which outputs a speech recognition result. The neural network is a network model trained based on the correspondence between feature vector samples and speech recognition samples. The feature vector samples are vector samples obtained by fusing and encoding audio samples and voltage pulse samples in advance.

[0006] In one implementation, the LIF neuron circuit consists of the humidity-sensitive memristor, a current-limiting resistor, a load resistor, and a parallel capacitor. When the voice signal is accompanied by a humid airflow, the resistance of the humidity-sensitive memristor decreases as the humidity increases, which accelerates the charging process of the parallel capacitor, allowing the humidity-sensitive memristor to reach the threshold more quickly and trigger pulse discharge.

[0007] In one implementation, the fabrication process of the humidity-sensitive memristor includes: Measure 1 mL of Nafion solution, mix Nafion solution with 3.6 mL of anhydrous ethanol, and shake for 5 minutes to mix Nafion solution and anhydrous ethanol evenly to obtain Nafion diluted solution; The glass substrate with the 185 nm thick ITO electrode was ultrasonically cleaned with deionized water, then dried with nitrogen and dried at 120°C for 30 minutes. Take 200 μL of the Nafion diluent and spin-coat it onto a cleaned glass substrate. Then anneal at 100 degrees Celsius for 45 minutes to obtain a 30 nm thick Nafion polymer dielectric layer. Electrodes were fabricated using a metal mask with a pre-formed pattern to obtain a humidity-sensitive memristor.

[0008] In one implementation, the spin coating speed conditions are: first 500 rpm for 5 seconds, then 1500 rpm for 40 seconds.

[0009] In one implementation, electrodes are fabricated using a metal mask with a pre-patterned design to obtain a humidity-sensitive memristor, comprising: A metal mask is placed over a spin-coated glass substrate, and 50 nanometers of silver are deposited on the glass substrate by thermal evaporation. After removing the metal mask, a humidity-sensitive memristor is obtained.

[0010] In one implementation, audio data containing random noise is fused and encoded with the voltage pulse data to obtain a fused data vector group, including: The voltage pulse data is widened, and the widened voltage pulse data is multiplied by a scaling factor to obtain voltage pulse data with matching magnitude. Audio data containing random noise is added point by point to voltage pulse data that is matched in magnitude to obtain speech humidity fusion data. The voice humidity fusion data is MFCC encoded to obtain a fusion data vector group.

[0011] In one implementation, the method further includes: MFCC encoding is performed separately on the audio data containing random noise to obtain an audio data vector group; The audio data vector group and the fused data vector group are respectively input into the trained neural network to obtain a first recognition result corresponding to the audio data vector group and a second recognition result corresponding to the fused data vector group, wherein the accuracy of the second recognition result is higher than that of the first recognition result.

[0012] Secondly, embodiments of the present invention also provide a noise-resistant speech recognition system based on humidity-sensitive memristor, wherein the system is used to implement the steps of noise-resistant speech recognition based on humidity-sensitive memristor described in the above solution, and the system includes: A multimodal information acquisition module is used to acquire voice signals based on a microphone device to obtain audio data, and to acquire humidity information based on a LIF neuron circuit, and to output voltage pulse data based on the humidity information by the LIF neuron circuit, wherein the LIF neuron circuit includes a humidity-sensitive memristor. The fusion and encoding module is used to apply random noise to the audio data and fuse and encode the audio data containing random noise with the voltage pulse data to obtain a fused data vector group; The speech recognition module is used to input the fused data vector group into a pre-trained neural network and output the speech recognition result. The neural network is a network model trained based on the correspondence between feature vector samples and speech recognition samples. The feature vector samples are vector samples obtained by fusing and encoding audio samples and voltage pulse samples in advance.

[0013] Thirdly, embodiments of the present invention also provide a terminal, wherein the terminal includes a memory, a processor, and a noise-resistant speech recognition program based on humidity-sensitive memristor stored in the memory and executable on the processor. When the processor executes the noise-resistant speech recognition program based on humidity-sensitive memristor, it implements the steps of the noise-resistant speech recognition method based on humidity-sensitive memristor of any of the above-mentioned solutions.

[0014] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a noise-resistant speech recognition program based on humidity-sensitive memristor, the noise-resistant speech recognition program based on humidity-sensitive memristor implementing the steps of the noise-resistant speech recognition method based on humidity-sensitive memristor as described in any of the above schemes on the computer-readable storage medium.

[0015] Beneficial Effects: Compared with existing technologies, this invention provides a noise-resistant speech recognition method based on humidity-sensitive memristors. First, this invention acquires speech signals using a microphone device to obtain audio data, and acquires humidity information using a LIF neuron circuit, which outputs voltage pulse data based on the humidity information. The LIF neuron circuit includes a humidity-sensitive memristor. Then, random noise is applied to the audio data, and the audio data containing random noise is fused and encoded with the voltage pulse data to obtain a fused data vector group. Next, the fused data vector group is input into a pre-trained neural network to output the speech recognition result. The neural network is a network model trained based on the correspondence between feature vector samples and speech recognition samples. The feature vector samples are vector samples obtained by pre-fusing and encoding the audio samples and voltage pulse samples.

[0016] This invention utilizes the high sensitivity of humidity fluctuations to LIF neuron circuits based on humidity-sensitive memristors to convert humidity information into corresponding voltage pulse data, thereby providing complementary feature information for speech recognition, breaking through the recognition limitations of single speech signals, and significantly improving the accuracy and robustness of speech recognition in noisy environments by using multimodal information fusion. Attached Figure Description

[0017] Figure 1 This is a flowchart of a preferred embodiment of the noise-resistant speech recognition method based on humidity-sensitive memristor provided in this invention.

[0018] Figure 2 This is a schematic diagram illustrating the working principle of the noise-resistant speech recognition method based on humidity-sensitive memristor provided in an embodiment of the present invention.

[0019] Figure 3 This is a circuit diagram of a LIF neuron in an embodiment of the present invention.

[0020] Figure 4 This is a schematic diagram of the spike pulses output under different humidity levels according to an embodiment of the present invention.

[0021] Figure 5 This is a schematic diagram of the structure of the humidity-sensitive memristor in the noise-resistant speech recognition method based on humidity-sensitive memristor provided in the embodiments of the present invention.

[0022] Figure 6 This illustrates the changes in threshold voltage and current of the humidity-sensitive memristor under different humidity levels in this embodiment of the invention.

[0023] Figure 7 This is a schematic diagram illustrating the effect of humidity and voltage on the turn-on speed of a humidity-sensitive memristor.

[0024] Figure 8 A schematic diagram of the experimental results for programming the humidity field of a humidity-sensitive memristor.

[0025] Figure 9 This is a schematic diagram of a spiking neural network provided in an embodiment of the present invention.

[0026] Figure 10 This diagram illustrates the comparison of speech recognition accuracy under different noise levels.

[0027] Figure 11 This is a block diagram illustrating the principle of a noise-resistant speech recognition system based on humidity-sensitive memristors, as provided in an embodiment of the present invention.

[0028] Figure 12 A schematic diagram of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0030] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content, operations, or steps, nor does it require execution in the described order. For example, some operations or steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0031] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0032] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. For example, "first control information" and "second control information" are only used to distinguish different control information and do not limit their order.

[0033] Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or the order of execution, and that the words "first" and "second" do not necessarily imply that they are different.

[0034] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0035] To address the problems of existing technologies, this invention provides a noise-resistant speech recognition method based on a humidity-sensitive memristor. This method utilizes the high sensitivity of a humidity-sensitive memristor-based LIF (Leaky Integrate-and-Fire) neural circuit to humidity fluctuations, converting humidity information into corresponding voltage pulse data. This provides complementary feature information for speech recognition, overcoming the limitations of recognizing a single speech signal and improving the accuracy and robustness of speech recognition in noisy environments. In specific applications, this embodiment first acquires speech signals using a microphone to obtain audio data, and then acquires humidity information based on the LIF neural circuit, which outputs voltage pulse data based on the humidity information. The LIF neural circuit includes a humidity-sensitive memristor. Then, random noise is applied to the audio data, and the audio data containing random noise is fused and encoded with the voltage pulse data to obtain a fused data vector group. Next, the fused data vector group is input into a pre-trained neural network, which outputs the speech recognition result. The neural network is a network model trained based on the correspondence between feature vector samples and speech recognition samples. The feature vector samples are vector samples obtained by pre-fusing and encoding audio samples and voltage pulse samples. In this embodiment, a humidity-sensitive memristor LIF neuron circuit can be constructed. Based on this LIF neuron circuit, the humidity information of the airflow accompanying the speech signal can be detected, and voltage pulse data can be output. Then, by comprehensively analyzing the audio data and voltage pulse data, the speech recognition result is obtained. Compared with the existing technology that only analyzes audio data to achieve speech recognition, the speech recognition result of this embodiment is more accurate.

[0036] The humidity-sensitive memristor-based noise-resistant speech recognition method of this embodiment can be applied to a terminal, which processes and analyzes the collected audio data and voltage pulse data to achieve speech recognition. This terminal can be a mobile phone, computer, or other intelligent product terminal. Specifically, for example... Figure 1 As shown in the figure, the noise-resistant speech recognition method based on humidity-sensitive memristor in this embodiment includes the following steps: Step S100: Acquire voice signals based on a microphone device to obtain audio data, and acquire humidity information based on a LIF neuron circuit, and output voltage pulse data based on the humidity information by the LIF neuron circuit, wherein the LIF neuron circuit includes a humidity-sensitive memristor.

[0037] Traditional speech recognition technology primarily relies on audio data processing methods, such as extracting Mel-Frequency Cepstral Coefficients (MFCCs) and combining them with neural networks for recognition and classification. However, in noisy environments, background noise severely interferes with the quality of audio signals, leading to a significant decrease in recognition rates. Therefore, this embodiment introduces humidity information naturally accompanying the speech signal as an auxiliary perceptual modality, effectively overcoming the limitations of pure audio data analysis. Specifically, when a person speaks, they exhale humid air, and the humidity changes are highly correlated with the speech content in terms of timing and intensity. Since LIF neural circuits are sensitive to humidity fluctuations, they can convert humidity into corresponding voltage pulse data, thereby providing complementary feature information for speech recognition.

[0038] Therefore, this embodiment constructs a multimodal signal acquisition system, combined with Figure 2 As shown, the multimodal signal acquisition system includes a microphone device and a LIF neuron circuit. The microphone device is positioned 10 cm from the mouth to acquire speech signals and obtain audio data. The LIF neuron circuit is located 5 cm from the mouth to detect the humidity information of the exhaled moist airflow and output voltage pulse data. Specifically, as... Figure 3 As shown, the LIF neuron circuit in this embodiment consists of a humidity-sensitive memristor and a current-limiting resistor. (200k) ), load resistor (20k) and parallel capacitors (0.33 The structure consists of a humidity-sensitive memristor whose resistance decreases as humidity increases when a voice signal is accompanied by humid airflow. This accelerates the charging process of the parallel capacitor, allowing the humidity-sensitive memristor to reach its threshold and trigger pulse discharge more quickly. Figure 4 As shown, the higher the humidity, the greater the output pulse frequency and amplitude, enabling real-time encoding of humidity fluctuations related to speech. This LIF neuron circuit directly outputs voltage pulse data.

[0039] Traditional humidity sensing systems typically employ a discrete serial architecture for sensors, memory, and computing units, resulting in high energy consumption and latency during signal conversion and transmission. Existing nanowire-based humidity-sensitive memristors face the challenge of high integration complexity. This embodiment utilizes a highly humidity-sensitive polymer material to fabricate a humidity-sensitive memristor and constructs a LIF neuron circuit based on this memristor for humidity signal sensing and processing. When ambient humidity increases, the threshold voltage of the humidity-sensitive memristor decreases, the turn-on time shortens, and the device current increases significantly. Utilizing this humidity-dependent electrical characteristic, the constructed LIF neuron circuit can generate differentiated pulse discharge behavior according to humidity changes: under high humidity conditions, the output pulse frequency and amplitude increase; under low humidity conditions, they decrease accordingly. This embodiment utilizes the mapping relationship between humidity and pulse characteristic parameters to achieve direct sensing and preliminary processing of humidity signals. This humidity-sensitive memristor possesses advantages such as simple fabrication process, ease of large-scale integration, fast response speed, low threshold voltage, and low power consumption.

[0040] Specifically, the fabrication of a humidity-sensitive memristor includes the following steps: Step 1: Measure 1 ml of Nafion solution, mix Nafion solution with 3.6 ml of anhydrous ethanol, and shake for 5 minutes to mix Nafion solution and anhydrous ethanol evenly to obtain Nafion diluted solution.

[0041] Step 2: The glass substrate with the 185 nm thick ITO (Indium Tin Oxide) electrode is ultrasonically cleaned with deionized water for 10 minutes to remove surface contaminants. The surface is then dried with high-purity nitrogen and at 120°C for 30 minutes. In this embodiment, a silicon wafer or a flexible substrate can also be used as the glass substrate.

[0042] Step 3: Take 200 μL of the Nafion diluent and spin-coat it onto the cleaned glass substrate. The spin-coating program consists of two stages. In the first stage, the spin-coating speed is 500 rpm for 5 seconds to ensure solution spreading. In the second stage, the spin-coating speed is 1500 rpm for 40 seconds to form a uniform film. Subsequently, anneal at 100°C for 45 minutes to remove residual solvent and enhance film density, resulting in a 30 nm thick Nafion polymer dielectric layer.

[0043] Step 4: Fabricate electrodes using a pre-patterned metal mask to obtain a humidity-sensitive memristor. Specifically, this involves: covering the spin-coated glass substrate with a metal mask; depositing 50 nanometers of silver onto the glass substrate under vacuum conditions using thermal evaporation at a rate controlled at 0.1 Å / s; and removing the metal mask to obtain the humidity-sensitive memristor. A schematic diagram of the humidity-sensitive memristor is shown below. Figure 5 As shown. In other implementations, copper, platinum, nickel, and aluminum can also be used as electrode materials for humidity-sensitive memristors.

[0044] This embodiment studies the performance of a humidity-sensitive memristor, focusing on two main aspects: First, it investigates the threshold switching characteristics of the humidity-sensitive memristor under different humidity conditions, with a particular emphasis on analyzing the impact of humidity on the device's operating voltage and turn-on time; second, it studies a resistive-state programming method based on humidity field modulation. Specific experimental content and results are as follows: (1) The electrical performance of the device was tested using a semiconductor parameter analyzer and a self-built humidity control system. First, in environments with relative humidity of 30%, 50%, 70%, and 90%, a DC voltage of 0 V to 1.5 V was applied to the device for a scan at a rate of 0.01 V / s to monitor the resistive switching behavior of the humidity-sensitive memristor. Experimental results showed that at lower humidity (30%), no significant resistive switching occurred; however, under higher humidity conditions (50%, 70%, and 90%), the device exhibited typical resistive switching characteristics. Further analysis showed that the threshold voltage of the device decreased significantly with increasing ambient humidity, while the high-resistivity current increased with increasing humidity, specifically as follows: Figure 6 As shown.

[0045] (2) A pulse voltage signal was applied to the humidity-sensitive memristor using a semiconductor parameter analyzer to evaluate its fastest dynamic response characteristics. A 1-microsecond, 5-volt pulse signal was applied at 600 nanoseconds. The humidity-sensitive memristor generated a current response under the influence of the voltage. The fastest response time of the device was calculated with 70% of the maximum current as the turn-on point. Figure 7 As shown in the image.

[0046] (3) Humidity Field Programming Experiment: Under a fixed bias voltage of 0.3 V, the resistive state writing and erasing of the device were achieved by adjusting the ambient humidity. When the ambient humidity increased, 0.3 V exceeded the threshold voltage for the humidity-sensitive memristor to turn on, causing the humidity-sensitive memristor to switch from a high-resistance state to a low-resistance state; while when the humidity decreased, the voltage was lower than the threshold voltage for the humidity-sensitive memristor to turn on, and the humidity-sensitive memristor automatically returned to the high-resistance state. This result shows that reversible, non-volatile resistive state control can be achieved simply by adjusting the ambient humidity, thus verifying the feasibility of humidity field programming. The experimental results are as follows: Figure 8 As shown.

[0047] This embodiment uses a LIF neuron circuit based on a humidity-sensitive memristor to acquire humidity information of the airflow accompanying the speech signal. The resistance of the humidity-sensitive memristor decreases as humidity increases, accelerating the charging process of the parallel capacitor, thus allowing the humidity-sensitive memristor to reach its threshold and trigger pulse discharge more quickly. The higher the humidity, the greater the output pulse frequency and amplitude, thereby achieving real-time encoding of humidity fluctuations related to speech. This LIF neuron circuit directly outputs voltage pulse data, which is recorded using an oscilloscope. The voltage pulse data and audio data are used together to achieve speech recognition. Therefore, the LIF neuron circuit in this embodiment can not only sense humidity information but also convert the sensed results into spike pulse signals that are easier for the circuit to process.

[0048] Step S200: Apply random noise to the audio data, and fuse and encode the audio data containing random noise with the voltage pulse data to obtain a fused data vector group.

[0049] After obtaining the aforementioned audio data and voltage pulse data, this embodiment can apply random noise to the audio data. Then, the audio data containing random noise is fused and encoded with the voltage pulse data to obtain a fused data vector group. During fusion and encoding, this embodiment first broadens the voltage pulse data. Specifically, it can broaden the data dimension of the voltage pulse data using interpolation to ensure it is fully aligned with the audio data on the time axis. For example, if the audio data has 1000 sampling points, the voltage pulse data should be broadened to 1000 corresponding points. Next, this embodiment multiplies the broadened voltage pulse data by a scaling factor to obtain voltage pulse data with matched magnitudes. The scaling factor in this embodiment can be set and adjusted according to the amplitude range of the voltage pulse data and the audio data. For example, the scaling factor can be set to 0.1 or 0.2. By multiplying the broadened voltage pulse data by the scaling factor, the magnitudes of the broadened voltage pulse data and the audio data can be matched, preventing the magnitude of the broadened voltage pulse data from being too large or too small, which could cause it to be masked by the audio data or mask the audio data during subsequent fusion. Then, in this embodiment, the audio data containing random noise is added point-by-point to the voltage pulse data after matching the magnitude to obtain the speech-humidity fusion data. Next, the speech-humidity fusion data is MFCC encoded to achieve feature extraction, resulting in a fused data vector group. The MFCC encoding in this embodiment is a commonly used feature extraction method in the field of speech recognition technology, mainly used to convert time-domain signals into frequency-domain feature vectors. This embodiment adopts the method of fusing data first and then encoding, which not only simplifies the data processing flow, but also reduces encoding time and complexity.

[0050] Step S300: Input the fused data vector group into the pre-trained neural network and output the speech recognition result.

[0051] After obtaining the aforementioned fused data vector set, this embodiment can input the fused data vector set into a pre-trained neural network to output the speech recognition result. In this embodiment, the neural network is a network model trained based on the correspondence between feature vector samples and speech recognition samples. The feature vector samples are vector samples obtained by fusing and encoding audio samples and voltage pulse samples in advance. During the training process, several audio samples are first collected. The audio samples can be audio recorded by multiple volunteers in a quiet environment. While collecting the audio samples, humidity information is collected through a LIF neuron circuit, and voltage pulse samples are output. Then, random noise is added to the collected audio samples. For example, the noise level can be set to 10%, 20%, and 30% respectively, so that three sets of audio samples containing different noise can be generated to simulate different levels of background interference in reality. Next, the audio samples and voltage pulse samples are fused and encoded to obtain feature vector samples. During fusion and encoding, the voltage pulse samples are first broadened, then the broadened voltage pulse samples are multiplied by a scaling factor, and then added point by point with the audio samples to obtain fused sample data. Finally, the fused sample data is MFCC encoded to obtain the feature vector samples. Next, in this embodiment, audio samples can be pre-identified to determine the corresponding recognition results, thereby obtaining speech recognition samples. Then, a correspondence is established between feature vector samples and speech recognition samples. Finally, the neural network is trained based on this correspondence to obtain a trained neural network.

[0052] Based on this, when the fused data vector group is input into the trained neural network in this embodiment, the trained neural network can automatically recognize the fused data vector group and thus output the speech recognition result. In one implementation, the neural network in this embodiment can be a spiking neural network, such as... Figure 9 As shown in the figure, this spiking neural network can identify and classify fused data vector groups, and output speech classification results, such as... Figure 9 The system outputs category 1, category 2, and category 3 to achieve speech recognition. In other implementations, the neural network in this embodiment can also be replaced with a binary neural network, a feedforward neural network, a recurrent neural network, etc., and this embodiment is not limited to this.

[0053] Furthermore, to verify that humidity information has an optimizing effect on speech recognition in noisy environments, combined with Figure 2 As shown in the illustration, this embodiment can perform MFCC encoding on audio data containing random noise separately to extract features and obtain an audio data vector group. Then, the audio data vector group and the fused data vector group are respectively input into a trained neural network to obtain a first recognition result corresponding to the audio data vector group and a second recognition result corresponding to the fused data vector group. Specifically, as follows... Figure 10 As shown, from Figure 10 As can be seen, under different noise levels, the accuracy of speech recognition based on both audio and humidity data is higher than that based solely on audio. Therefore, it can be concluded that the accuracy of the second recognition result in this embodiment is higher than that of the first recognition result. Similarly, if the neural network is trained based solely on audio data, the recognition accuracy of the resulting neural network will also be lower than that of the neural network trained based on the fusion of audio data and humidity information in this embodiment.

[0054] In summary, this embodiment first acquires speech signals using a microphone device to obtain audio data, and then acquires humidity information using a LIF neuron circuit, which outputs voltage pulse data based on the humidity information. The LIF neuron circuit includes a humidity-sensitive memristor. Next, random noise is applied to the audio data, and the audio data containing random noise is fused and encoded with the voltage pulse data to obtain a fused data vector set. Then, the fused data vector set is input into a pre-trained neural network to output the speech recognition result. This embodiment utilizes the high sensitivity of the humidity-sensitive memristor-based LIF neuron circuit to humidity fluctuations, converting humidity information into corresponding voltage pulse data, thereby providing complementary feature information for speech recognition. This overcomes the limitations of recognizing a single speech signal and significantly improves the accuracy and robustness of speech recognition in noisy environments through multimodal information fusion.

[0055] Based on the above embodiments, the present invention also provides a noise-resistant speech recognition system based on humidity-sensitive memristor. The system in this embodiment is used to implement the steps in the above method embodiments. Specifically, as... Figure 11 As shown, the system includes a multimodal information acquisition module 10, a fusion and encoding module 20, and a speech recognition module 30. Specifically, the multimodal information acquisition module is used to acquire speech signals based on a microphone device to obtain audio data, and to acquire humidity information based on a LIF neuron circuit, and the LIF neuron circuit outputs voltage pulse data based on the humidity information, wherein the LIF neuron circuit includes a humidity-sensitive memristor. The fusion and encoding module 20 is used to apply random noise to the audio data, and to fuse and encode the audio data containing random noise with the voltage pulse data to obtain a fused data vector group. The speech recognition module 30 is used to input the fused data vector group into a pre-trained neural network and output a speech recognition result, wherein the neural network is a network model trained based on the correspondence between feature vector samples and speech recognition samples, and the feature vector samples are vector samples obtained by pre-fusing and encoding audio samples and voltage pulse samples.

[0056] The humidity-sensitive memristor-based noise-resistant speech recognition system in this embodiment is based on the same principle as the steps in the above method embodiments, and will not be repeated here.

[0057] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 12 As shown. The terminal may include one or more processors 100 ( Figure 12 (Only one is shown in the diagram), memory 101, and computer program 102 stored in memory 101 and executable on one or more processors 100. For example, a noise-resistant speech recognition program based on humidity-sensitive memristors. When one or more processors 100 execute computer program 102, they can implement various steps in the embodiments of the noise-resistant speech recognition method based on humidity-sensitive memristors. Alternatively, when one or more processors 100 execute computer program 102, they can implement the functions of various modules / units in the embodiments of the noise-resistant speech recognition system based on humidity-sensitive memristors, which is not limited here.

[0058] In one embodiment, the processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0059] In one embodiment, memory 101 may be an internal storage unit of an electronic device, such as a hard drive or RAM. Memory 101 may also be an external storage device of the electronic device, such as a plug-in hard drive, smart media card (SMC), secure digital card (SD), flash card, etc. Furthermore, memory 101 may include both internal and external storage units. Memory 101 is used to store computer programs and other programs and data required by the terminal. Memory 101 can also be used to temporarily store data that has been output or will be output.

[0060] Those skilled in the art will understand that Figure 12The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0061] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, operational databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual operating data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A noise-resistant speech recognition method based on humidity-sensitive memristor, characterized in that, The method includes: The system collects voice signals using a microphone device to obtain audio data, and collects humidity information using a LIF neuron circuit, which then outputs voltage pulse data based on the humidity information. The LIF neuron circuit includes a humidity-sensitive memristor. Random noise is applied to the audio data, and the audio data containing random noise is fused and encoded with the voltage pulse data to obtain a fused data vector group; The fused data vector group is input into a pre-trained neural network, which outputs a speech recognition result. The neural network is a network model trained based on the correspondence between feature vector samples and speech recognition samples. The feature vector samples are vector samples obtained by fusing and encoding audio samples and voltage pulse samples in advance.

2. The noise-resistant speech recognition method based on humidity-sensitive memristor according to claim 1, characterized in that, The LIF neuron circuit consists of a humidity-sensitive memristor, a current-limiting resistor, a load resistor, and a parallel capacitor. When the voice signal is accompanied by a humid airflow, the resistance of the humidity-sensitive memristor decreases as the humidity increases, which accelerates the charging process of the parallel capacitor, allowing the humidity-sensitive memristor to reach the threshold more quickly and trigger pulse discharge.

3. The noise-resistant speech recognition method based on humidity-sensitive memristor according to claim 2, characterized in that, The fabrication process of the humidity-sensitive memristor includes: Measure 1 mL of Nafion solution, mix Nafion solution with 3.6 mL of anhydrous ethanol, and shake for 5 minutes to mix Nafion solution and anhydrous ethanol evenly to obtain Nafion diluted solution; The glass substrate with the 185 nm thick ITO electrode was ultrasonically cleaned with deionized water, then dried with nitrogen and dried at 120°C for 30 minutes. Take 200 μL of the Nafion diluent and spin-coat it onto a cleaned glass substrate. Then anneal at 100 degrees Celsius for 45 minutes to obtain a 30 nm thick Nafion polymer dielectric layer. Electrodes were fabricated using a metal mask with a pre-formed pattern to obtain a humidity-sensitive memristor.

4. The noise-resistant speech recognition method based on humidity-sensitive memristor according to claim 3, characterized in that, The spin coating speed conditions are: first 500 rpm for 5 seconds, then 1500 rpm for 40 seconds.

5. The noise-resistant speech recognition method based on humidity-sensitive memristor according to claim 3, characterized in that, Electrodes are fabricated using a metal mask with a pre-formed pattern to obtain a humidity-sensitive memristor, including: A metal mask is placed over a spin-coated glass substrate, and 50 nanometers of silver are deposited on the glass substrate by thermal evaporation. After removing the metal mask, a humidity-sensitive memristor is obtained.

6. The noise-resistant speech recognition method based on humidity-sensitive memristor according to claim 1, characterized in that, The audio data containing random noise is fused and encoded with the voltage pulse data to obtain a fused data vector group, including: The voltage pulse data is widened, and the widened voltage pulse data is multiplied by a scaling factor to obtain voltage pulse data with matching magnitude. Audio data containing random noise is added point by point to voltage pulse data that is matched in magnitude to obtain speech humidity fusion data. The voice humidity fusion data is MFCC encoded to obtain a fusion data vector group.

7. The noise-resistant speech recognition method based on humidity-sensitive memristor according to claim 1, characterized in that, The method further includes: MFCC encoding is performed separately on the audio data containing random noise to obtain an audio data vector group; The audio data vector group and the fused data vector group are respectively input into the trained neural network to obtain a first recognition result corresponding to the audio data vector group and a second recognition result corresponding to the fused data vector group, wherein the accuracy of the second recognition result is higher than that of the first recognition result.

8. A noise-resistant speech recognition system based on humidity-sensitive memristor, characterized in that, The system is used to implement the steps of noise-resistant speech recognition based on humidity-sensitive memristor as described in any one of claims 1-7, and the system comprises: A multimodal information acquisition module is used to acquire voice signals based on a microphone device to obtain audio data, and to acquire humidity information based on a LIF neuron circuit, and to output voltage pulse data based on the humidity information by the LIF neuron circuit, wherein the LIF neuron circuit includes a humidity-sensitive memristor. The fusion and encoding module is used to apply random noise to the audio data and fuse and encode the audio data containing random noise with the voltage pulse data to obtain a fused data vector group; The speech recognition module is used to input the fused data vector group into a pre-trained neural network and output the speech recognition result. The neural network is a network model trained based on the correspondence between feature vector samples and speech recognition samples. The feature vector samples are vector samples obtained by fusing and encoding audio samples and voltage pulse samples in advance.

9. A terminal, characterized in that, The terminal includes a memory, a processor, and a humidity-sensitive memristor-based noise-resistant speech recognition program stored in the memory and executable on the processor. When the processor executes the humidity-sensitive memristor-based noise-resistant speech recognition program, it implements the steps of the humidity-sensitive memristor-based noise-resistant speech recognition method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a noise-resistant speech recognition program based on humidity-sensitive memristor, the noise-resistant speech recognition program based on humidity-sensitive memristor implementing the steps of the noise-resistant speech recognition method based on humidity-sensitive memristor as described in any one of claims 1-7 on the computer-readable storage medium.

Images