Intelligent voice mouse control method, mouse, terminal and Internet of Things system

Abstract

The embodiment of the present invention provides a control method, a mouse, a terminal and an Internet of Things system for an intelligent voice mouse, firstly collecting an original voice signal; then extracting its signal-to-noise ratio; finally, when the signal-to-noise ratio is greater than a preset threshold, according to a lightweight neural network, determining the target information in the original voice signal and transmitting it to a target terminal for identification; when the signal-to-noise ratio is not greater than the preset threshold, inputting the original voice signal into the lightweight neural network, obtaining the first output value of the noise processing module at the same time, inputting the first output value into the lightweight neural network, obtaining the target information and transmitting it to the target terminal for identification; wherein the input of the noise processing module is determined by the lightweight neural network and the original voice signal. Based on the original technology, the present invention adds a noise processing module to the terminal part, uses the intermediate value of the lightweight neural network to remove noise, and ensures the recognition accuracy under complex background noise.

Description

Technical Field

[0001] The present invention belongs to the technical field of electronic digital data processing, and in particular relates to a control method of an intelligent voice mouse, a mouse, a terminal and an Internet of Things system. Background Art

[0002] At present, the mouse is a commonly used external input device for computers, mainly used in the office field. The basic functions of the mouse are also reflected in text input, translation and other functions. However, the functions of mouse buttons are often simple and users cannot customize the functions of mouse buttons. As a computer accessory, the mouse will lose most of its functions when it is separated from the computer.

[0003] In the IoT scenario, the mouse and the computer it is connected to usually work together as a gateway to control various devices in the computer IoT. The prior art provides a technical means of setting a lightweight neural network in the mouse for voiceprint recognition, while the terminal completes the remaining recognition. However, after the neural network is lightweight, the recognition effect of voice is poor, especially under the premise of complex background noise, it is difficult to achieve effective recognition of user voice. Summary of the invention

[0004] In view of this, the present invention provides a control method for an intelligent voice mouse, a mouse, a terminal and an Internet of Things system, aiming to solve the problem in the prior art that it is difficult to effectively recognize user voice.

[0005] A first aspect of an embodiment of the present invention provides a control method for an intelligent voice mouse, which is applied to the mouse; the mouse is wirelessly connected to a target terminal; a lightweight neural network is stored in the mouse; a noise processing module and a recognition module are provided in the target terminal; the method comprises:

[0006] Collecting original speech signals;

[0007] Extract the signal-to-noise ratio of the original speech signal;

[0008] When the signal-to-noise ratio is greater than a preset threshold, the target information in the original speech signal is determined according to the lightweight neural network and transmitted to the target terminal so that the recognition module recognizes the target information;

[0009] When the signal-to-noise ratio is not greater than a preset threshold, the original speech signal is input into the lightweight neural network, and the first output value of the noise processing module is obtained at the same time, and the first output value is input into the lightweight neural network, and the obtained target information is transmitted to the target terminal so that the recognition module recognizes the target information; wherein, the input of the noise processing module is determined by the lightweight neural network and the original speech signal.

[0010] In a possible implementation, the lightweight neural network includes an encoder and a decoder; an original speech signal is input into the lightweight neural network, a first output value of a noise processing module is obtained, the first output value is input into the lightweight neural network, and target information is obtained and transmitted to a target terminal so that a recognition module recognizes the target information, including:

[0011] Inputting the original speech signal into the decoder to obtain the first feature;

[0012] Transmitting the first feature to the target terminal as an input of the noise processing module;

[0013] Obtaining a first output value of a noise processing module;

[0014] The first feature and the first output value are input into the decoder to obtain the target information and transmit it to the target terminal so that the recognition module can recognize the target information.

[0015] In a possible implementation, the lightweight neural network includes an encoder and a decoder; a second neural network model is also provided in the target terminal;

[0016] Inputting the original speech signal into the lightweight neural network, obtaining the first output value of the noise processing module, inputting the first output value into the lightweight neural network, and transmitting the obtained target information to the target terminal so that the recognition module recognizes the target information, including:

[0017] Inputting the original speech signal into the decoder to obtain the first feature;

[0018] The signal-to-noise ratio is transmitted to the target terminal, so that the target terminal determines the adaptive parameters of the noise processing module according to the signal-to-noise ratio and the second neural network;

[0019] Transmitting the first feature to the target terminal as an input of the noise processing module;

[0020] Obtaining a first output value of a noise processing module;

[0021] The first feature and the first output value are input into the decoder to obtain the target information and transmit it to the target terminal so that the recognition module can recognize the target information.

[0022] In a possible implementation, the noise processing module includes a feature extraction module and a masking module; an output end of the feature extraction module is connected to an input end of the masking module;

[0023] Inputting the original speech signal into the lightweight neural network, obtaining the first output value of the noise processing module, inputting the first output value into the lightweight neural network, and transmitting the obtained target information to the target terminal so that the recognition module recognizes the target information, including:

[0024] Inputting the original speech signal into the decoder to obtain the first feature;

[0025] Transmitting the signal-to-noise ratio to the target terminal, so that the target terminal determines a first adaptive parameter of the feature extraction module and a second adaptive parameter of the masking module according to the signal-to-noise ratio and the second neural network;

[0026] Transmitting the first feature to the target terminal as input of the feature extraction module;

[0027] Obtaining a first output value of the masking module;

[0028] The first feature and the first output value are input into the decoder to obtain the target information and transmit it to the target terminal so that the recognition module can recognize the target information.

[0029] In a possible implementation, the lightweight neural network includes an encoder and a decoder; the target terminal is also provided with a first neural network;

[0030] The method further includes:

[0031] When the signal-to-noise ratio is not greater than a preset threshold, determining the type of the target terminal and the current working mode of the mouse;

[0032] If the type of the target terminal is a temporary device or the current working mode is a normal mode, the original voice signal is input into the lightweight neural network, and the first output value of the noise processing module is obtained, and the first output value is input into the lightweight neural network, and the target information obtained is transmitted to the target terminal so that the recognition module recognizes the target information; wherein the input of the noise processing module is determined by the lightweight neural network;

[0033] If the type of the target terminal is a trusted device and / or the current working mode is an energy-saving mode, the original voice signal is transmitted to the target terminal so that the target terminal obtains the target information according to the first neural network recognition, and the recognition module recognizes the target information.

[0034] The second aspect of an embodiment of the present invention provides a mouse, including a voice acquisition module, a control module, and a communication module; the voice acquisition module is connected to the control module; the control module is connected to the communication module; the control module includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the control method of the intelligent voice mouse of the first aspect are implemented.

[0035] A third aspect of an embodiment of the present invention provides a terminal control method, which is applied to a target terminal; a mouse is wirelessly connected to the target terminal; a lightweight neural network is stored in the mouse; a noise processing module and a recognition module are provided in the target terminal; the method comprises:

[0036] receiving target information transmitted by the mouse, and identifying the target information according to the recognition module; or transmitting the first output value of the noise processing module to the mouse, and receiving the target information transmitted by the mouse, and identifying the target information according to the recognition module;

[0037] Among them, the input of the noise processing module is determined by the lightweight neural network and the original voice signal; the mouse is used to collect the original voice signal; when the signal-to-noise ratio is greater than a preset threshold, the target information is determined based on the lightweight neural network and the original voice signal; when the signal-to-noise ratio is not greater than the preset threshold, the target information is determined based on the lightweight neural network, the first output value of the noise processing module and the original voice signal.

[0038] In a possible implementation, a first neural network is further provided in the target terminal;

[0039] The method further includes:

[0040] In the debugging mode, the first neural network is updated according to the user's input information;

[0041] The updated first neural network is compressed, and the compressed network is transmitted to the mouse to update the lightweight neural network.

[0042] The fourth aspect of an embodiment of the present invention provides a target terminal, including a voice acquisition module, a control module, and a communication module; the voice acquisition module is connected to the control module; the control module is connected to the communication module; the control module includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the terminal control method of the third aspect above are implemented.

[0043] A fifth aspect of an embodiment of the present invention provides an Internet of Things system, comprising the mouse of the second aspect and the target terminal of the fourth aspect.

[0044] The control method, mouse, terminal and Internet of Things system of the intelligent voice mouse provided by the embodiment of the present invention first collect the original voice signal; then extract the signal-to-noise ratio of the original voice signal; finally, when the signal-to-noise ratio is greater than a preset threshold, according to the lightweight neural network, the target information in the original voice signal is determined and transmitted to the target terminal, so that the recognition module recognizes the target information; when the signal-to-noise ratio is not greater than the preset threshold, the original voice signal is input into the lightweight neural network, and the first output value of the noise processing module is obtained at the same time, and the first output value is input into the lightweight neural network, and the target information obtained is transmitted to the target terminal, so that the recognition module recognizes the target information; wherein, the input of the noise processing module is determined by the lightweight neural network and the original voice signal. On the basis of the original technology, the present invention adds a noise processing module to the terminal part, uses the intermediate value of the lightweight neural network to remove noise, and ensures the recognition accuracy under complex background noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0046] Figure 1 Schematic diagram of the structure of the Internet of Things system provided by an embodiment of the present invention;

[0047] Figure 2 is a flow chart of an implementation method of a control method of an intelligent voice mouse provided by an embodiment of the present invention;

[0048] Figure 3 It is a schematic diagram of the structure of a first neural network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0050] Figure 1 Schematic diagram of the structure of the Internet of Things system provided by the embodiment of the present invention. Figure 1 As described above, in some embodiments, the Internet of Things system includes a mouse 11 and a target terminal 12 .

[0051] Among them, the mouse 11 includes a voice acquisition module, a control module, and a communication module; the voice acquisition module is connected to the control module; the control module is connected to the communication module; the control module can be any type of MCU, and the communication mode of the communication module can include but is not limited to at least one of the following: WIFI communication, Bluetooth communication, Zigbee communication. The target terminal 12 can be a computer, a tablet, a smart TV, a smart gateway, etc., which is not limited here. When the mouse 11 is connected to the target terminal 12, if it is detected that the corresponding app has not been downloaded in the target terminal 12, it will be displayed that all function configurations are not completed, and at the same time, the target terminal is instructed to send a download request to the server to download the corresponding app, if it is detected that the corresponding app has been downloaded in the target terminal 12.

[0052] In order to better illustrate the present invention, the technical background of this solution is described below:

[0053] The wireless mouse in the prior art is usually only equipped with common functions of a mouse, and other complex control functions and Internet of Things functions are usually realized by the computer to which the mouse is connected.

[0054] For example, when the user is in the office and wants to turn on the air conditioner at home remotely, the user can long press the designated button of the mouse and say the voice message "turn on the air conditioner to 25℃". The mouse then sends the voice message to the connected terminal, which processes the voice message and sends it to the remote server. The server instructs the device to complete the corresponding operation to complete the IoT control.

[0055] This prior art has the following disadvantages:

[0056] 1. The corresponding app or voice processing algorithm must be pre-downloaded in the terminal. If the wireless mouse is connected to a terminal that does not have the corresponding app or voice processing algorithm installed, the IoT control process cannot be completed.

[0057] 2. For temporarily connected devices, there is no corresponding app or voice processing algorithm inside, and they need to be temporarily installed. During this process, the user's voiceprint characteristics and other information will inevitably be transmitted to the temporary device, which may easily cause user privacy leakage and low security.

[0058] Another existing technology has improved this, that is, a lightweight neural network is set in the mouse main control chip to perform voiceprint recognition, and the terminal is responsible for subsequent recognition. This method can avoid the leakage of the user's voiceprint privacy, but this method still has defects, as follows:

[0059] 1. In a noisy environment (interference from television, other people's voices, etc.), the recorded voice signals usually contain a lot of noise. Ordinary neural network speech recognition systems hope that the signal-to-noise ratio is above 20dB, otherwise the recognition accuracy will be poor or even unrecognizable.

[0060] 2. Wireless mice are limited by their size and weight. The battery capacity they deploy is small, and the memory and computing power of the main control chip are also small. This makes it difficult for the mouse to execute algorithms that require large memory, large computing power, or high power consumption. Therefore, lightweight neural networks are used in the above-mentioned technologies. The original background noise will interfere with the recognition of the neural network to a certain extent. After lightweight processing, the model becomes smaller, which also leads to weaker noise resistance and lower recognition accuracy in complex backgrounds.

[0061] In view of this, the present invention improves the prior art:

[0062] Figure 2 FIG. 1 is a flow chart of the control method of the intelligent voice mouse provided by an embodiment of the present invention. Figure 2 As shown, in some embodiments, a lightweight neural network is stored in the mouse; a noise processing module and a recognition module are provided in the target terminal; and a control method of the intelligent voice mouse includes:

[0063] S210, collecting original speech signals;

[0064] S220, extracting the signal-to-noise ratio of the original speech signal;

[0065] S230, when the signal-to-noise ratio is greater than a preset threshold, determining target information in the original speech signal according to the lightweight neural network and transmitting the target information to the target terminal so that the recognition module recognizes the target information;

[0066] S240, when the signal-to-noise ratio is not greater than a preset threshold, the original speech signal is input into the lightweight neural network, and the first output value of the noise processing module is obtained, and the first output value is input into the lightweight neural network, and the obtained target information is transmitted to the target terminal so that the recognition module recognizes the target information; wherein, the input of the noise processing module is determined by the lightweight neural network and the original speech signal.

[0067] In the embodiment of the present invention, the target information is instruction information extracted from the original voice signal, and the instruction information is a control instruction issued by the user to the computer or the target device.

[0068] In the embodiment of the present invention, the original speech signal is framed, and then the signal-to-noise ratio of each frame is calculated by the mean square method, and the signal-to-noise ratio of the original speech signal is obtained by removing the average value. Alternatively, the power spectrum of the original speech signal can be estimated by fast Fourier transform to determine the signal-to-noise ratio.

[0069] When the signal-to-noise ratio is greater than the preset threshold, the noise has little effect on the recognition of voice commands, so it is still processed according to the existing method, that is, the lightweight neural network in the mouse recognizes the voiceprint in the signal, and then the terminal completes the subsequent voice recognition. However, in a noisy environment, that is, when the signal-to-noise ratio is not greater than the preset threshold, the original lightweight neural network is no longer sufficient to complete the processing of the voice signal, and the performance of the mouse does not allow the installation of a more complex algorithm. Therefore, the present invention adds a noise processing module in the terminal to perform denoising.

[0070] Figure 3 is a schematic diagram of the structure of the first neural network provided by an embodiment of the present invention. Figure 3 As shown, the first neural network is composed of an encoder, a decoder and a noise processing module. The first neural network has a good recognition effect on speech signals in a noisy background, but it is limited by memory and computing power and cannot be deployed in a mouse. Therefore, the present invention divides it, deploys the noise processing module after division in the terminal, and deploys the encoder and decoder on the mouse. Compared with the original technology, the mouse only adds a first feature and a first output value transmission, which has little effect on the original memory and computing power, and the power consumption will not increase significantly, but the recognition effect is greatly improved. In addition, in the lightweight process, some unimportant intermediate feature mapping values ​​of the encoder and decoder can also be dynamically pruned according to the remaining memory and computing power of the mouse to adapt to the performance of the mouse.

[0071] In some embodiments, the lightweight neural network includes an encoder and a decoder; the original speech signal is input into the lightweight neural network, and the first output value of the noise processing module is obtained at the same time, and the first output value is input into the lightweight neural network to obtain the target information and transmit it to the target terminal so that the recognition module recognizes the target information, including: inputting the original speech signal into the decoder to obtain the first feature; transmitting the first feature to the target terminal as the input of the noise processing module; obtaining the first output value of the noise processing module; inputting the first feature and the first output value into the decoder to obtain the target information and transmit it to the target terminal so that the recognition module recognizes the target information.

[0072] In an embodiment of the present invention, the encoder in the mouse performs convolution, normalization, activation and other processing on the original speech signal to generate a first feature of a smaller scale, which is transmitted to the decoder in the mouse and the target terminal respectively. The target terminal recognizes the impact of noise on the speech through the first feature, and gives a corresponding first output value for adjustment so that the decoder can output the target information to the target terminal, and the target terminal further recognizes it to complete the user's instructions.

[0073] In an embodiment of the present invention, the target terminal noise processing module includes a feature extraction module and a masking module. The feature extraction module can extract local and global context information. Specifically, a two-stage I-Conformer block can be used, that is, local and global context information can be extracted respectively through a local I-Conformer and a global I-Conformer. In the spectrum of the speech signal, the energy of speech and noise is distributed in different frequency regions. When the energy of the noise is lower than the energy of the speech in a certain frequency region to a certain extent, the interference of the noise on the speech is small. The masking module uses this principle to determine the masking threshold by calculating the energy difference between the speech signal and the noise signal in the spectrum. For example, for a pure speech signal, its spectrum has obvious peaks in certain frequency bands (corresponding to the resonance peaks of the speech and other features). When noise is present, the masking module can set the masking threshold according to the energy near the peak frequency of the speech, so that the noise frequency components below the threshold are suppressed.

[0074] The first feature of the input terminal is extracted through the local I-Conformer to extract the local feature, and then input into the global I-Conformer to extract the global feature. The extracted feature is input into the masking module to obtain the first output value, which is used for element-by-element multiplication between the masked first output value and the first feature of the encoder output to obtain the final mask encoder feature.

[0075] In some embodiments, the lightweight neural network includes an encoder and a decoder; a second neural network model is also provided in the target terminal; the original speech signal is input into the lightweight neural network, and the first output value of the noise processing module is obtained at the same time, and the first output value is input into the lightweight neural network, and the target information obtained is transmitted to the target terminal so that the recognition module recognizes the target information, including: inputting the original speech signal into the decoder to obtain a first feature; transmitting the signal-to-noise ratio to the target terminal so that the target terminal determines the adaptive parameters of the noise processing module according to the signal-to-noise ratio and the second neural network; transmitting the first feature to the target terminal as an input of the noise processing module; obtaining the first output value of the noise processing module; inputting the first feature and the first output value into the decoder to obtain the target information and transmit it to the target terminal so that the recognition module recognizes the target information.

[0076] In the embodiment of the present invention, it can be seen from the above analysis that since the network is lightweight, the greater the noise, the worse the recognition effect. After the noise processing module is installed in the terminal, although the noise can be processed, its effect can still be further improved. The present invention further improves it, that is, the parameters of the noise processing module are adaptively adjusted using the estimated signal-to-noise ratio to enhance the capability of the noise processing module.

[0077] In some embodiments, the noise processing module includes a feature extraction module and a masking module; the output end of the feature extraction module is connected to the input end of the masking module; the original speech signal is input into the lightweight neural network, and the first output value of the noise processing module is obtained at the same time, and the first output value is input into the lightweight neural network to obtain the target information and transmit it to the target terminal so that the recognition module recognizes the target information, including: inputting the original speech signal into the decoder to obtain the first feature; transmitting the signal-to-noise ratio to the target terminal so that the target terminal determines the first adaptive parameter of the feature extraction module and the second adaptive parameter of the masking module according to the signal-to-noise ratio and the second neural network; transmitting the first feature to the target terminal as the input of the feature extraction module; obtaining the first output value of the masking module; inputting the first feature and the first output value into the decoder to obtain the target information and transmit it to the target terminal so that the recognition module recognizes the target information.

[0078] In an embodiment of the present invention, the first adaptive parameter in the feature extraction module can dynamically adjust the gain of the signal according to the noise level of the input signal. When high noise (low signal-to-noise ratio) is detected, the signal gain is reduced to reduce the impact of noise on subsequent layers; when the noise is low (high signal-to-noise ratio), the gain is appropriately increased to better utilize the signal information. Specifically, the weight, step size and padding method of the convolution kernel can be changed to achieve this.

[0079] In the masking module, the masking threshold is a key parameter. The noise adaptive layer can adjust the masking threshold according to the noise situation of the input signal. In the case of high signal-to-noise ratio, since the signal is relatively strong, the noise adaptive layer can appropriately lower the masking threshold. This is because the signal is easier to distinguish at this time, and a strong masking is not needed to highlight the signal. For example, in the masking module of speech enhancement, if the signal-to-noise ratio is 30dB, the noise adaptive layer can set the masking threshold to a lower value so that only the obvious noise component is masked without over-suppressing the signal part.

[0080] On the contrary, in the case of low signal-to-noise ratio, the masking threshold needs to be increased to better suppress noise. For example, when the signal-to-noise ratio is 0dB, the noise adaptive layer can increase the masking threshold so that more parts that are likely to be noise are masked, thereby improving signal quality. The adjustment of the masking threshold can be based on the estimation of the signal-to-noise ratio of the input signal, and a mapping function is used to determine the appropriate masking threshold. This mapping function can be linear or nonlinear, depending on the specific application scenario and experimental results.

[0081] In some embodiments, the lightweight neural network includes an encoder and a decoder; a first neural network is also provided in the target terminal; the method further includes: when the signal-to-noise ratio is not greater than a preset threshold, determining the type of the target terminal and the current working mode of the mouse; if the type of the target terminal is a temporary device or the current working mode is a normal mode, the original voice signal is input into the lightweight neural network, and the first output value of the noise processing module is obtained at the same time, and the first output value is input into the lightweight neural network, and the target information obtained is transmitted to the target terminal so that the recognition module recognizes the target information; wherein, the input of the noise processing module is determined by the lightweight neural network; if the type of the target terminal is a trusted device and / or the current working mode is an energy-saving mode, the original voice signal is transmitted to the target terminal so that the target terminal obtains the target information according to the first neural network recognition, and the recognition module recognizes the target information at the same time.

[0082] In an embodiment of the present invention, the type of the target terminal may include a trusted device and a temporary device. When the mouse is connected to any target terminal, the user can set it as a trusted device or a temporary device. At this time, the mouse will record its device ID and the type of device, and there is no need to set the device again when it is connected to the target terminal next time. Among them, after the first connection to the target terminal, a window with device type options can be displayed on the display screen of the target terminal, waiting for the user to complete the selection. Alternatively, after the first connection to the target terminal, if the user operates the mouse according to a preset action (such as long pressing a key), it is set as a trusted device, otherwise it is a temporary device. It is also possible to prompt the user "Please confirm whether the device is a trusted device" in the mouse according to the broadcast module after the first connection to the target terminal. If the user replies "Confirm", the device type is corrected from a temporary device to a trusted device.

[0083] In the embodiment of the present invention, the current working mode of the mouse may include a normal mode and an energy-saving mode, and may also include a plurality of modes set according to an energy consumption gradient, which is not limited here.

[0084] In the embodiment of the present invention, if the terminal is a temporary device / the mouse is in a normal working mode, the processing is performed in the manner shown in the above embodiment. When the mouse needs to save energy and / or the terminal is a trusted device, in order to reduce the amount of calculation of the mouse, the first neural network can be downloaded in the terminal, and the processing of the voice signal is entirely handed over to the terminal.

[0085] In some embodiments, a terminal control method includes:

[0086] receiving target information transmitted by the mouse, and identifying the target information according to the recognition module; or transmitting the first output value of the noise processing module to the mouse, and receiving the target information transmitted by the mouse, and identifying the target information according to the recognition module;

[0087] Among them, the input of the noise processing module is determined by the lightweight neural network and the original voice signal; the mouse is used to collect the original voice signal; when the signal-to-noise ratio is greater than a preset threshold, the target information is determined based on the lightweight neural network and the original voice signal; when the signal-to-noise ratio is not greater than the preset threshold, the target information is determined based on the lightweight neural network, the first output value of the noise processing module and the original voice signal.

[0088] In some embodiments, a first neural network is also provided in the target terminal; the terminal control method further includes: in a debugging mode, updating the first neural network according to user input information; compressing the updated first neural network, and transmitting the compressed network to the mouse to update the lightweight neural network.

[0089] In an embodiment of the present invention, if the user finds that there are multiple problems with inaccurate voice recognition, the user can enter the debugging mode. In the debugging mode, the user can issue voice commands in various environments and input text information corresponding to the voice commands to the terminal, so that the terminal can retrain the first neural network. After the training is completed, the first neural network is re-segmented and compressed, the segmented noise processing module is retained in the terminal, and the compressed encoder and decoder are transmitted to the mouse.

[0090] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0091] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A method for controlling an intelligent voice mouse, characterized in that: Applied to a mouse; the mouse is wirelessly connected to a target terminal; a lightweight neural network is stored in the mouse; a noise processing module and a recognition module are provided in the target terminal; the method comprises: Collecting original speech signals; Extracting the signal-to-noise ratio of the original speech signal; When the signal-to-noise ratio is greater than a preset threshold, determining target information in the original speech signal according to the lightweight neural network and transmitting the target information to the target terminal so that the recognition module recognizes the target information; When the signal-to-noise ratio is not greater than a preset threshold, the original speech signal is input into the lightweight neural network, and a first output value of the noise processing module is obtained, and the first output value is input into the lightweight neural network to obtain target information and transmit it to the target terminal, so that the recognition module recognizes the target information; The lightweight neural network includes an encoder and a decoder; the target terminal is also provided with a second neural network model; Inputting the original speech signal into the lightweight neural network, acquiring a first output value of the noise processing module, inputting the first output value into the lightweight neural network, and transmitting the obtained target information to the target terminal so that the recognition module recognizes the target information, including: Inputting the original speech signal into an encoder to obtain a first feature; Transmitting the signal-to-noise ratio to the target terminal, so that the target terminal determines an adaptive parameter of a noise processing module according to the signal-to-noise ratio and the second neural network; Transmitting the first feature to the target terminal as an input of the noise processing module; Obtaining a first output value of the noise processing module; The first feature and the first output value are input into a decoder to obtain target information and transmit it to a target terminal so that the recognition module can recognize the target information.

2. The control method of the intelligent voice mouse according to claim 1, characterized in that: The noise processing module includes a feature extraction module and a masking module; the output end of the feature extraction module is connected to the input end of the masking module; Inputting the original speech signal into the lightweight neural network, acquiring a first output value of the noise processing module, inputting the first output value into the lightweight neural network, and transmitting the obtained target information to the target terminal so that the recognition module recognizes the target information, including: Inputting the original speech signal into an encoder to obtain a first feature; Transmitting the signal-to-noise ratio to the target terminal, so that the target terminal determines a first adaptive parameter of a feature extraction module and a second adaptive parameter of a masking module according to the signal-to-noise ratio and the second neural network; Transmitting the first feature to the target terminal as input of the feature extraction module; Obtaining a first output value of the masking module; The first feature and the first output value are input into a decoder to obtain target information and transmit it to a target terminal so that the recognition module can recognize the target information.

3. The control method of the intelligent voice mouse according to claim 1, characterized in that: The lightweight neural network includes an encoder and a decoder; the target terminal is also provided with a first neural network; The method further comprises: When the signal-to-noise ratio is not greater than a preset threshold, determining the type of the target terminal and the current working mode of the mouse; If the type of the target terminal is a temporary device or the current working mode is a normal mode, the original voice signal is input into the lightweight neural network, and the first output value of the noise processing module is obtained, and the first output value is input into the lightweight neural network, and the target information obtained is transmitted to the target terminal so that the recognition module recognizes the target information; wherein the input of the noise processing module is determined by the lightweight neural network; If the type of the target terminal is a trusted device and / or the current working mode is an energy-saving mode, the original voice signal is transmitted to the target terminal so that the target terminal obtains the target information according to the first neural network recognition, and the recognition module recognizes the target information.

4. A mouse, characterized in that: It includes a voice acquisition module, a control module, and a communication module; the voice acquisition module is connected to the control module; the control module is connected to the communication module; the control module includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the control method of the intelligent voice mouse as described in any one of claims 1 to 3 are implemented.

5. A terminal control method, characterized in that: The method is applied to a target terminal and adapted to the mouse as claimed in claim 4 above; the mouse as claimed in claim 4 above is wirelessly connected to the target terminal; a lightweight neural network is stored in the mouse; a noise processing module and a recognition module are provided in the target terminal; the method comprises: receiving target information transmitted by the mouse, and identifying the target information according to the identification module; or transmitting the first output value of the noise processing module to the mouse, and receiving the target information transmitted by the mouse, and identifying the target information according to the identification module; Among them, the input of the noise processing module is determined by a lightweight neural network and an original voice signal; the mouse is used to collect the original voice signal; when the signal-to-noise ratio is greater than a preset threshold, the target information is determined based on the lightweight neural network and the original voice signal; when the signal-to-noise ratio is not greater than the preset threshold, the target information is determined based on the lightweight neural network, the first output value of the noise processing module and the original voice signal.

6. The terminal control method according to claim 5, characterized in that: The target terminal is also provided with a first neural network; The method further comprises: In debugging mode, updating the first neural network according to user input information; The updated first neural network is compressed, and the compressed network is transmitted to the mouse to update the lightweight neural network.

7. A target terminal, characterized in that: It includes a voice acquisition module, a control module, and a communication module; the voice acquisition module is connected to the control module; the control module is connected to the communication module; the control module includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the terminal control method described in claim 5 are implemented.

8. An Internet of Things system, characterized in that: It comprises the mouse as described in claim 4 and the target terminal as described in claim 7.

Images