Internet of Things mouse and terminal control method and system based on Internet of Things mouse

Abstract

The invention discloses an Internet of Things mouse and a terminal control method and system based on the Internet of Things mouse, and relates to the technical field of the Internet of Things. In the Internet of Things mouse, a voice acquisition module is used for acquiring current voice information and judging whether a user-defined instruction exists in the current voice information or not; the instruction matching module is used for matching an original instruction combination corresponding to the user-defined instruction in a preset user-defined instruction library if the user-defined instruction exists; the control module is used for rewriting the current voice information according to the original instruction combination to obtain target instruction information corresponding to each execution device; and the communication module is used for sending the target instruction information to the control terminal, so that the control terminal controls each execution device to complete a corresponding instruction according to the target instruction information. Based on the Internet of Things mouse, a user can replace a complex combined instruction with a simple user-defined instruction, and inconvenience brought to the user due to the fact that a long instruction is repeatedly dictated for multiple times is avoided while the periodic control requirement of the user is met.

Description

Technical Field

[0001] The present application relates to the technical field of Internet of Things, and in particular to an Internet of Things mouse, and a terminal control method and system based on the Internet of Things mouse. Background Art

[0002] The voice-controlled IoT mouse can be used to form a gateway with various control terminals, so as to control various home appliances through the control terminals, thereby realizing the control function of the IoT. When the user controls the target execution device based on the IoT mouse, the control command issued needs to include the device name, and only a single device can be controlled at a time.

[0003] However, considering that users may have periodic needs to control multiple target devices at once, for example, when returning home from get off work every day, users need to execute the following commands: turn on the water heater, turn on the air conditioner to 25 degrees, and turn on the TV. Therefore, when executing the above control requirements based on the existing IoT mouse, it is necessary to repeat the long-segment command information multiple times, which brings inconvenience to the user. Summary of the invention

[0004] The main purpose of this application is to provide an Internet of Things mouse, a terminal control method and system based on the Internet of Things mouse, aiming to solve the technical problem that when users use the existing Internet of Things mouse to meet periodic control needs, they need to repeatedly orally state long command information, which affects the user experience.

[0005] To achieve the above objectives, the present application proposes an Internet of Things mouse, which includes: Voice collection module, command matching module, control module and communication module; The voice collection module is used to obtain current voice information and determine whether there is a user-defined instruction in the current voice information; The instruction matching module is used to match the original instruction combination corresponding to the user-defined instruction in the preset custom instruction library when there is a user-defined instruction in the current voice information; The control module is used to rewrite the current voice information according to the original instruction combination to obtain target instruction information corresponding to each execution device; The communication module is used to send the target instruction information to the control terminal, so that the control terminal controls each execution device to complete the instruction operation corresponding to the target instruction information according to the target instruction information.

[0006] In one embodiment, the voice collection module includes: a voice collection unit, a voice conversion unit and a command recognition unit; The speech collection unit is used to collect speech segments and pre-process the speech segments; The speech conversion unit is used to convert the pre-processed speech segment into text to obtain current speech information; The instruction recognition unit is used to determine whether there is a user-defined instruction in the current voice information.

[0007] In one embodiment, the voice collection module further includes: an identity recognition unit; The identity recognition unit is used to extract features from the preprocessed voice segment through a preset user recognition model to obtain a user voiceprint feature; The identity recognition unit is further used to authenticate the current user according to the user voiceprint feature; The voice conversion unit is further used to convert the pre-processed voice segment into text to obtain current voice information when the current user identity authentication is passed.

[0008] In one embodiment, the control module includes: an instruction rewriting unit and a device confirmation unit; The instruction rewriting unit is used to rewrite the current voice information according to the original instruction combination to obtain complete instruction information, and extract the device name keyword and the corresponding operation keyword in the complete instruction information based on a preset instruction template; The device confirmation unit is used to integrate the operation keywords based on the device name keywords, and obtain target instruction information corresponding to each execution device according to the integration result.

[0009] In one embodiment, the instruction rewriting unit is further used to extract the device name keyword and the corresponding operation keyword in the current voice information based on a preset instruction template when there is no user-defined instruction in the current voice information.

[0010] In one embodiment, the IoT mouse further comprises: a mode switching module; The mode switching module is used to switch the working mode according to the mode switching instruction when receiving the mode switching instruction triggered by the current user, and the working modes include: general working mode, Internet of Things working mode and instruction entry mode; The mode switching module is also used to receive the custom instructions input by the current user and the corresponding original instruction combination when the working mode after switching is the instruction entry mode, and update the preset custom instruction library.

[0011] In one embodiment, the communication module is further used to obtain a list of currently connectable terminals when receiving the target instruction information; The communication module is further used to determine the target control terminal in the current connectable terminal list based on a preset priority order, wherein the preset priority order is pre-configured; The communication module is further used to send the target instruction information to the target control terminal, so that the target control terminal controls each execution device to complete the instruction operation corresponding to the target instruction information according to the target instruction information.

[0012] In addition, to achieve the above purpose, the present application also proposes a terminal control method based on an Internet of Things mouse, the method comprising: Acquire current voice information, and determine whether there is a user-defined instruction in the current voice information; When there is a user-defined instruction in the current voice information, matching the original instruction combination corresponding to the user-defined instruction in a preset custom instruction library; rewriting the current voice information according to the original instruction combination to obtain target instruction information, and determining each execution device according to the target instruction information; The target instruction information is sent to the control terminal, so that the control terminal controls each execution device to complete the instruction operation corresponding to the target instruction information according to the target instruction information.

[0013] In one embodiment, before the step of determining whether there is a user-defined instruction in the current voice information, the step includes: Collecting speech fragments and preprocessing the speech fragments; Extracting features from the preprocessed voice segment using a preset user recognition model to obtain user voiceprint features; Performing identity authentication on the current user according to the user's voiceprint features; When the current user identity authentication is passed, the pre-processed voice segment is converted into text to obtain current voice information.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a terminal control system based on an Internet of Things mouse, the system comprising: a control terminal, a server and the Internet of Things mouse as described above; The control terminal is used to determine the execution device corresponding to each target instruction information when receiving the target instruction information, and control each execution device through the server to complete the instruction operation corresponding to each target instruction information.

[0015] The present application provides an Internet of Things mouse, a terminal control method and system based on the Internet of Things mouse, and the Internet of Things mouse includes: a voice acquisition module, an instruction matching module, a control module and a communication module; the voice acquisition module is used to obtain the current voice information and determine whether there is a user-defined instruction in the current voice information; the instruction matching module is used to match the original instruction combination corresponding to the user-defined instruction in the preset custom instruction library when there is a user-defined instruction in the current voice information; the control module is used to rewrite the current voice information according to the original instruction combination to obtain the target instruction information corresponding to each execution device; the communication module is used to send the target instruction information to the control terminal, so that the control terminal controls each execution device to complete the instruction operation corresponding to the target instruction information according to the target instruction information. In the present application, the Internet of Things mouse can identify the user-defined instruction in the current voice information, and then restore it to the original instruction combination to rewrite the current voice information, and finally obtain the target instruction information corresponding to each execution device based on the rewriting result, so that the control terminal executes the instruction to each execution device. Based on the Internet of Things mouse of the present application, the user can replace the complex combination instruction with a concise and easy-to-remember user-defined instruction, and realize the same multi-device control requirements at different times, avoiding the inconvenience caused by the user repeatedly speaking long instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 This is a schematic diagram of the module structure of the first embodiment of the Internet of Things mouse of this application; Figure 2 This is a schematic diagram of the module structure of the second embodiment of the Internet of Things mouse of this application; Figure 3 This is a schematic diagram of the module structure of the third embodiment of the Internet of Things mouse of this application; Figure 4 This is a flow chart of the first embodiment of the terminal control method based on the Internet of Things mouse of the present application; Figure 5 This is a flow chart of the second embodiment of the terminal control method based on the Internet of Things mouse of the present application; Figure 6 This is a schematic diagram of the structure of the terminal control system based on the Internet of Things mouse in this application.

[0019] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0021] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0022] Reference Figure 1 , Figure 1 This is a schematic diagram of the module structure of the first embodiment of the Internet of Things mouse of this application.

[0023] like Figure 1 As shown, in this embodiment, the Internet of Things mouse includes: a voice acquisition module 10, a command matching module 20, a control module 30, a communication module 40 and a mode switching module 50.

[0024] The voice collection module 10 is used to obtain current voice information and determine whether there is a user-defined instruction in the current voice information.

[0025] It should be noted that the current voice information may be the text conversion of the collected voice clips after the IoT mouse switches to the IoT working mode. The current voice information may include the current user's demand instructions for one or more execution devices, which are expressed in the form of standard instructions: for example, "turn on the air conditioner to 25 degrees", "turn on the bedroom Bluetooth speaker", etc. The current user is the voice user corresponding to the voice clip received by the IoT mouse.

[0026] It should be understood that user-defined instructions are customized instructions compared to standard instructions, and are used to replace multiple standard instructions that need to be executed continuously. Standard instructions need to include the name keywords of the device that the user needs to control and the corresponding operation keywords, while user-defined instructions do not need to mention the above device name keywords and corresponding operation keywords.

[0027] For example, the current user can pre-set the user-defined instruction to, for example, "Execute daily homecoming strategy", and configure the original instruction combination corresponding to the user-defined instruction to be: "Turn on the air conditioner to 25 degrees" + "Turn on the water heater" + "Turn on the TV in the living room".

[0028] It is understandable that the above-mentioned setting process of user-defined instructions can be implemented based on the IoT mouse in the instruction input mode. The current user can switch the working mode of the IoT mouse by operating the mouse according to the preset action (for example, long pressing the left button, right button or scroll wheel of the mouse).

[0029] The mode switching module 50 is used to switch the working mode according to the mode switching instruction when receiving the mode switching instruction triggered by the current user. The working modes include: general working mode (I gear), Internet of Things working mode (II gear) and instruction entry mode (III gear).

[0030] It should be understood that the user generates a mode switching instruction by operating the IoT mouse according to a preset action. When the mode switching module 50 receives the mode switching instruction, it switches the working mode and generates vibration feedback to prompt the current user to switch the working mode category.

[0031] It should be noted that the IoT mouse can be a Bluetooth wireless mouse, the general working mode is the mode in which the IoT mouse performs conventional mouse functions, the IoT working mode is the mode in which the IoT mouse is used to realize the control function of the IoT, and the command entry mode is the mode in which the IoT mouse is used to configure custom mouse commands before performing IoT control.

[0032] For example, if the user long presses the left mouse button to trigger a mode switching command and switches the working mode of the IoT mouse to the command entry mode, the IoT mouse can perform three consecutive vibration feedbacks to prompt the current user that the mode has been switched to the command entry mode (Gear III).

[0033] The mode switching module 50 is further used to receive the custom instruction input by the current user and the corresponding original instruction combination, and update the preset custom instruction library when the working mode after switching is the instruction input mode.

[0034] It should be understood that the communication module 40 in the IoT mouse can maintain the connection relationship between the IoT mouse and a control terminal (a terminal device with a display screen such as a computer, tablet, notebook, smart TV, etc.). When the working mode of the IoT mouse is the command input mode, the current user can voice input the custom command and the corresponding original command combination based on the guide input information shown on the display screen of the control terminal, and then the IoT mouse can bind and store the corresponding text information and the user voiceprint feature.

[0035] It is understandable that a preset custom instruction library may be maintained in the instruction matching module 20, and the preset custom instruction library may use the voiceprint features of each user as an index to store the custom instructions entered by each user and the corresponding original instruction combination.

[0036] It should also be noted that when the mode switching module 50 switches the working mode to the IoT working mode, the IoT mouse can obtain current voice information through the voice acquisition module 10 and determine whether there is a user-defined instruction in the current voice information.

[0037] The instruction matching module 20 is used to match the original instruction combination corresponding to the user-defined instruction in the preset custom instruction library when there is a user-defined instruction in the current voice information.

[0038] It should be noted that, in the instruction matching module 20, the custom instructions entered by each user in the preset custom instruction library can be traversed to determine the matching item corresponding to the user-defined instruction of the current user, and the original instruction combination in the matching item can be obtained.

[0039] The control module 30 is used to rewrite the current voice information according to the original instruction combination to obtain target instruction information corresponding to each execution device.

[0040] In a specific implementation, the control module 30 can restore the user-defined instructions in the current voice information to the original instruction combination, and then determine the target device (ie, the execution device) that the current user needs to control and the corresponding target instruction information based on the standard instructions in the original instruction combination.

[0041] The communication module 40 is used to send the target instruction information to the control terminal, so that the control terminal controls each execution device to complete the instruction operation corresponding to the target instruction information according to the target instruction information.

[0042] It should be understood that the IoT mouse sends the target instruction information to the control terminal based on its own communication module 40, and the control terminal transmits the target instruction information to the corresponding execution device through the server so that the execution device completes the instruction operation corresponding to the target instruction information, thereby realizing the control of home devices, which can effectively control the devices in each room at home and can also be remotely controlled through the mouse when going out. The server is a physical server or a cloud server, which is not limited in this embodiment.

[0043] Furthermore, since the communication module 40 in the Internet of Things mouse can be connected to different types of control terminals, the communication module 40 can first obtain a list of currently connectable terminals when receiving the target instruction information; then determine the target control terminal in the list of currently connectable terminals based on a preset priority order; and finally send the target instruction information to the target control terminal, so that the target control terminal controls each of the execution devices according to the target instruction information to complete the instruction operation corresponding to the target instruction information.

[0044] It should be noted that the communication method adopted by the communication module 40 can be at least one of WIFI communication, Bluetooth communication or Zigbee communication, and the current connectable terminal list can include all terminals that the IoT mouse can detect and communicate with based on the above communication methods.

[0045] It should be understood that the above-mentioned preset priority order can be pre-configured by the current user, or can be automatically generated based on the number of connections of each terminal in the historical communication record of the IoT mouse, for example, a terminal with a large number of connections is configured as a higher priority. In addition, the preset priority order can also be generated based on the network quality of each terminal in the current connectable terminal list or the network location distance of each terminal to the IoT mouse and the server, for example, a terminal with better network quality or closer network location is configured as a higher priority. This embodiment is not limited to this.

[0046] In this embodiment, the Internet of Things mouse realizes the control function of the Internet of Things by forming a gateway with each control terminal, and in the voice recognition process completed by the Internet of Things mouse, it can recognize user-defined instructions in the current voice information, and then restore it to the original instruction combination to rewrite the current voice information, and finally obtain the target instruction information corresponding to each execution device based on the rewriting result, so that the control terminal executes the instructions for each execution device. Based on the Internet of Things mouse of this embodiment, the user can replace the complex combination instructions with concise and easy-to-remember user-defined instructions, and realize the same multi-device control requirements at different times, avoiding the inconvenience caused to the user by repeatedly reciting long instructions.

[0047] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , Figure 2 This is a schematic diagram of the module structure of the second embodiment of the Internet of Things mouse of this application.

[0048] refer to Figure 2 In this embodiment, the voice collection module 10 includes: a voice collection unit 101, a voice conversion unit 102, a command recognition unit 103 and an identity recognition unit 104.

[0049] The speech collection unit 101 is used to collect speech segments and pre-process the speech segments.

[0050] It should be noted that, considering the diversity of the environment in which the IoT mouse is located, in order to improve the accuracy of subsequent speech-to-text conversion, the collected voice fragments can be pre-processed such as denoising and echo cancellation to remove noise interference in the voice fragments.

[0051] The speech conversion unit 102 is used to convert the pre-processed speech segment into text to obtain current speech information.

[0052] It should be understood that, through text conversion, invalid information in the voice segment can be further removed, and the denoised voice segment can be converted into text information with a simpler structure.

[0053] It should also be noted that, considering that the voice clips collected by the IoT mouse in a noisy environment may come from different users, before converting the voice clips into text, it is possible to first identify whether the speaking user corresponding to the voice clip is an authenticated user with the corresponding user voiceprint feature in the preset custom instruction library of the IoT mouse.

[0054] The identity recognition unit 104 is used to extract features from the preprocessed voice segment using a preset user recognition model to obtain user voiceprint features.

[0055] It should be understood that the preset user recognition model can be a lightweight neural network, which can reduce the size of the model and the running memory through the corresponding neural network compression algorithm and memory compression algorithm so that it can run on the MCU deployed in the mouse.

[0056] The identity recognition unit 104 is further configured to perform identity authentication on the current user according to the user voiceprint feature.

[0057] Specifically, the IoT mouse calculates the similarity between the user voiceprint feature of the current user and the voiceprint features of each user in the preset custom instruction library, and determines whether the highest similarity calculated exceeds the preset value. If it exceeds, the current user passes the identity authentication, otherwise the current user fails the identity authentication.

[0058] The voice conversion unit 102 is further configured to convert the pre-processed voice segment into text to obtain current voice information when the current user identity authentication is passed.

[0059] It should be understood that the text conversion process of the voice segment is performed when the current user passes the identity authentication, thereby avoiding energy waste of the IoT mouse caused by text conversion of invalid voice segments when the current user fails the identity authentication.

[0060] The instruction recognition unit 103 is used to determine whether there is a user-defined instruction in the current voice information.

[0061] It should be understood that since the current voice information is in text format, it is possible to determine whether the voice information is a corresponding text representation of a user-defined instruction, or to determine whether there is a text representation corresponding to a user-defined instruction in the voice information by traversing the text format voice information.

[0062] It should also be noted that since there may be confusion between homophones and similar sounds in the process of converting voice fragments into text, when a text representation similar to a user-defined instruction is recognized in the voice information, it can be directly determined that a user-defined instruction exists in the current voice information.

[0063] In this embodiment, the pre-processed voice segment is feature extracted by a preset user recognition model to obtain the user voiceprint feature; the current user is authenticated according to the user voiceprint feature; when the current user passes the authentication, the pre-processed voice segment is converted into text to obtain the current voice information; and then the pre-processed voice segment is converted into text to obtain the current voice information to determine whether there is a user-defined instruction in the current voice information. In this embodiment, a lightweight neural network model can be used to extract the user voiceprint feature before the voice segment is converted into text, so as to authenticate the identity based on the user voiceprint feature, and the voice segment is converted into text when the current user passes the authentication, thereby avoiding the energy waste of the IoT mouse caused by the text conversion of invalid voice segments when the current user fails the authentication.

[0064] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those of the first and second embodiments can be referred to the above description, and will not be described in detail later. Figure 3 , Figure 3 This is a schematic diagram of the module structure of the third embodiment of the Internet of Things mouse of the present application.

[0065] refer to Figure 3 In this embodiment, the control module 30 includes: an instruction rewriting unit 301 and a device confirmation unit 302.

[0066] The instruction rewriting unit 301 is used to rewrite the current voice information according to the original instruction combination to obtain complete instruction information, and extract the device name keyword and the corresponding operation keyword in the complete instruction information based on a preset instruction template.

[0067] It should be understood that the complete instruction information may include each standard instruction in the original instruction combination, and other instruction information except the customized user instruction in the current voice information.

[0068] It is understandable that the preset instruction template may be in the format of “verb+device noun+instruction noun.” Through the preset instruction template, keywords can be extracted from the current voice information to ensure the accuracy of subsequent instruction execution.

[0069] For example, if the current voice message is "Execute the daily homecoming strategy and turn on the air conditioner to swing left and right", where "Execute the daily homecoming strategy" is a user-defined instruction, and "turn on the air conditioner to swing left and right" is other instruction information except the customized user instruction. Then "Execute the daily homecoming strategy" can be restored to "turn on the air conditioner to 25 degrees, turn on the water heater, and turn on the TV in the living room", and the complete instruction information is "turn on the air conditioner to 25 degrees, turn on the water heater, turn on the TV in the living room, and turn on the air conditioner to swing left and right".

[0070] Next, keywords are extracted from the above complete command information through the preset command template, and the results are “turn on + air conditioning + 25 degrees, turn on + water heater, turn on + living room TV, turn on + air conditioning + left and right swing”, among which the equipment keywords are: (air conditioning; water heater; living room; air conditioning), and the operation keywords are: (turn on 25 degrees; turn on; turn on; turn on left and right swing).

[0071] It should also be noted that, further taking into account the situation where there are no user-defined voice instructions in the current voice information, the instruction rewriting unit 301 can also be used to directly extract the device name keywords and corresponding operation keywords in the current voice information based on a preset instruction template when there are no user-defined instructions in the current voice information.

[0072] The device confirmation unit 302 is used to integrate the operation keywords based on the device name keywords, and obtain target instruction information corresponding to each execution device according to the integration result.

[0073] It should be understood that, as can be seen from the above device keywords, there are different operation instructions for the same execution device. Therefore, in order to facilitate subsequent instruction execution, the operation keywords with the same device keywords can be integrated to determine the target instruction information corresponding to each execution device.

[0074] In a specific implementation, based on the above example, the integration result can be: the device keywords are: (air conditioner; water heater; living room TV), and the operation keywords are: (open 25 degrees, swing air left and right; open; open). Therefore, it can be determined that the execution devices are air conditioners, water heaters, and object TVs, as well as the target instruction information corresponding to each execution device.

[0075] This embodiment rewrites the current voice information according to the original command combination to obtain complete command information, and extracts the device name keyword and the corresponding operation keyword in the complete command information based on the preset command template; integrates each operation keyword based on each device name keyword, and obtains the target command information corresponding to each execution device according to the integration result. This embodiment merges the instructions to be executed by the same execution device, so that the execution device does not need to process each independent instruction separately, thereby simplifying the execution process and improving the efficiency of command response.

[0076] This application also provides a terminal control method based on the Internet of Things mouse, refer to Figure 4 , Figure 4 This is a flow chart of the first embodiment of the terminal control method based on the Internet of Things mouse of the present application.

[0077] In this embodiment, the terminal control method based on the Internet of Things mouse includes: steps S10 to S40: Step S10: Acquire current voice information, and determine whether there is a user-defined instruction in the current voice information.

[0078] It should be noted that the method of this embodiment can be applied to the scenario where the user effectively controls the devices in each room of the home through the Internet of Things, and can also be applied to the scenario where the user performs remote Internet of Things control through a mouse when going out. The execution subject of the method of this embodiment can be the Internet of Things mouse described above.

[0079] It should be understood that the current voice information can be obtained by converting the collected voice fragments into text after the IoT mouse switches to the IoT working mode. The current voice information can include the current user's demand instructions (standard instructions) for one or more execution devices. User-defined instructions are customized instructions compared to standard instructions, which are used to replace multiple standard instructions that need to be executed continuously.

[0080] It should be understood that the above user-defined instructions can be set by the IoT mouse in the instruction input mode. The current user can switch the working mode of the IoT mouse by operating the mouse according to the preset action (such as long pressing the left button, right button or scroll wheel of the mouse).

[0081] Therefore, before step S10, the following steps are also included: steps S01-S02.

[0082] Step S01: upon receiving a mode switching instruction triggered by the current user, switching the working mode according to the mode switching instruction, the working modes including: general working mode, Internet of Things working mode and instruction entry mode.

[0083] Step S02: When the switched working mode is the instruction input mode, the custom instruction input by the current user and the corresponding original instruction combination are received, and a preset custom instruction library is updated.

[0084] It is understandable that the preset custom instruction library can use the voiceprint features of each user as an index to store the custom instructions entered by each user and the corresponding original instruction combination.

[0085] Step S20: When there is a user-defined instruction in the current voice information, an original instruction combination corresponding to the user-defined instruction is matched in a preset custom instruction library.

[0086] In a specific implementation, the custom instructions entered by each user in the preset custom instruction library can be traversed to determine the matching item corresponding to the user-defined instruction of the current user, and the original instruction combination in the matching item can be obtained.

[0087] Step S30: rewrite the current voice information according to the original instruction combination to obtain target instruction information, and determine each execution device according to the target instruction information.

[0088] It should be understood that the IoT mouse can restore the user-defined instructions in the current voice information to the original instruction combination, and then determine the target device (i.e., the execution device) that the current user needs to control and the corresponding target instruction information based on the standard instructions in the original instruction combination.

[0089] In a specific implementation, the IoT mouse can rewrite the current voice information according to the original command combination to obtain complete command information, and extract the device name keywords and corresponding operation keywords in the complete command information based on a preset command template; and integrate each operation keyword based on the device name keywords, and obtain the target command information corresponding to each execution device based on the integration result.

[0090] In addition, further taking into account the situation that there are no user-defined voice instructions in the current voice information, the method also includes: when there are no user-defined instructions in the current voice information, directly extracting the device name keywords and corresponding operation keywords in the current voice information based on a preset instruction template; and integrating each of the operation keywords based on the device name keywords, and obtaining the target instruction information corresponding to each execution device based on the integration result.

[0091] Step S40: Send the target instruction information to the control terminal, so that the control terminal controls each execution device to complete the instruction operation corresponding to the target instruction information according to the target instruction information.

[0092] Furthermore, since the Internet of Things mouse can be connected to different types of control terminals (computers, tablets, notebooks, smart TVs, etc.), step S40 specifically includes: steps S401 to S403: Step S401: Obtain a list of currently connectable terminals.

[0093] Step S402: determining a target control terminal in the current connectable terminal list based on a preset priority order, wherein the preset priority order is pre-configured.

[0094] Step S403: Send the target instruction information to the target control terminal, so that the target control terminal controls each execution device to complete the instruction operation corresponding to the target instruction information according to the target instruction information.

[0095] In this embodiment, the Internet of Things mouse realizes the control function of the Internet of Things by forming a gateway with each control terminal, and in the voice recognition process completed by the Internet of Things mouse, it can recognize user-defined instructions in the current voice information, and then restore it to the original instruction combination to rewrite the current voice information, and finally obtain the target instruction information corresponding to each execution device based on the rewriting result, so that the control terminal executes the instructions for each execution device. Based on the Internet of Things mouse of this embodiment, the user can replace the complex combination instructions with concise and easy-to-remember user-defined instructions, and realize the same multi-device control requirements at different times, avoiding the inconvenience caused to the user by repeatedly reciting long instructions.

[0096] Based on the first embodiment of the terminal control method based on the Internet of Things mouse of the present application, a second embodiment of the terminal control method based on the Internet of Things mouse of the present application is proposed, with reference to Figure 5 , Figure 5 This is a flow chart of the second embodiment of the terminal control method based on the Internet of Things mouse of the present application.

[0097] In this embodiment, before step S10, steps S01 to S04 are also included: Step S01: Collect voice clips and pre-process the voice clips.

[0098] It should be noted that, considering the diversity of the environment in which the IoT mouse is located, in order to improve the accuracy of subsequent speech-to-text conversion, the collected voice fragments can be pre-processed such as denoising and echo cancellation to remove noise interference in the voice fragments.

[0099] Step S02: extracting features from the preprocessed speech segment using a preset user recognition model to obtain user voiceprint features.

[0100] It should also be noted that, considering that the voice clips collected by the IoT mouse in a noisy environment may come from different users, before converting the voice clips into text, it is possible to first identify whether the speaking user corresponding to the voice clip is an authenticated user with the corresponding user voiceprint feature in the preset custom instruction library of the IoT mouse.

[0101] It should be understood that the preset user recognition model can be a lightweight neural network, which can reduce the size of the model and the running memory through the corresponding neural network compression algorithm and memory compression algorithm so that it can run on the MCU deployed in the mouse.

[0102] Step S03: authenticating the current user according to the user's voiceprint features.

[0103] Specifically, the IoT mouse calculates the similarity between the user voiceprint feature of the current user and the voiceprint features of each user in the preset custom instruction library, and determines whether the highest similarity calculated exceeds the preset value. If it exceeds, the current user passes the identity authentication, otherwise the current user fails the identity authentication.

[0104] Step S04: when the current user identity authentication is passed, the pre-processed voice segment is converted into text to obtain current voice information.

[0105] In this embodiment, a lightweight neural network model is used to extract user voiceprint features before converting the voice fragment into text to perform identity authentication based on the user voiceprint features, and the voice fragment is converted into text when the current user passes the identity authentication, thereby avoiding energy waste of the IoT mouse caused by converting invalid voice fragments into text when the current user fails the identity authentication.

[0106] In addition, the present application embodiment also proposes a terminal control system based on an Internet of Things mouse, referring to Figure 6 , Figure 6 This is a schematic diagram of the structure of the terminal control system based on the Internet of Things mouse in this application.

[0107] exist Figure 6 In the embodiment, the system includes: a control terminal 22, a server 23 and the Internet of Things mouse 21 as described above.

[0108] In this embodiment, the Internet of Things mouse 21 is connected to the control terminal 22, the control terminal 22 is connected to the server 23, and the server 23 is connected to execution devices such as air conditioners and projectors.

[0109] Among them, the control terminal 22 can be a computer, a tablet, a notebook, a smart TV, etc., and the server 23 can be a physical server or a cloud server, which is not limited here.

[0110] Based on the embodiment of the present application, the IoT mouse 21 processes the collected voice clips of the current user and sends them to the control terminal 22, thereby determining the control instructions, transmitting them to the corresponding execution device via the server 23, and completing the terminal control, thereby achieving effective control of the equipment in each room in the home, as well as remote control through the IoT mouse and the control terminal when going out.

[0111] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other elements in the process, method, article or system including the element.

[0112] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0114] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An Internet of Things mouse, characterized in that: The Internet of Things mouse includes: a voice acquisition module, a command matching module, a control module and a communication module; The voice collection module is used to obtain current voice information and determine whether there is a user-defined instruction in the current voice information; The instruction matching module is used to match the original instruction combination corresponding to the user-defined instruction in the preset custom instruction library when there is a user-defined instruction in the current voice information; The control module is used to rewrite the current voice information according to the original instruction combination to obtain target instruction information corresponding to each execution device; The communication module is used to send the target instruction information to the control terminal, so that the control terminal controls each execution device to complete the instruction operation corresponding to the target instruction information according to the target instruction information.

2. The Internet of Things mouse as claimed in claim 1, characterized in that: The voice collection module includes: a voice collection unit, a voice conversion unit and a command recognition unit; The speech collection unit is used to collect speech segments and pre-process the speech segments; The speech conversion unit is used to convert the pre-processed speech segment into text to obtain current speech information; The instruction recognition unit is used to determine whether there is a user-defined instruction in the current voice information.

3. The Internet of Things mouse as claimed in claim 2, characterized in that: The voice collection module also includes: an identity recognition unit; The identity recognition unit is used to extract features from the preprocessed voice segment through a preset user recognition model to obtain a user voiceprint feature; The identity recognition unit is further used to authenticate the current user according to the user voiceprint feature; The voice conversion unit is further used to convert the pre-processed voice segment into text to obtain current voice information when the current user identity authentication is passed.

4. The Internet of Things mouse as claimed in claim 2, characterized in that: The control module includes: an instruction rewriting unit and a device confirmation unit; The instruction rewriting unit is used to rewrite the current voice information according to the original instruction combination to obtain complete instruction information, and extract the device name keyword and the corresponding operation keyword in the complete instruction information based on a preset instruction template; The device confirmation unit is used to integrate the operation keywords based on the device name keywords, and obtain target instruction information corresponding to each execution device according to the integration result.

5. The Internet of Things mouse as claimed in claim 4, characterized in that: The instruction rewriting unit is further used to extract the device name keyword and the corresponding operation keyword in the current voice information based on a preset instruction template when there is no user-defined instruction in the current voice information.

6. The Internet of Things mouse as claimed in claim 1, characterized in that: The Internet of Things mouse also includes: a mode switching module; The mode switching module is used to switch the working mode according to the mode switching instruction when receiving the mode switching instruction triggered by the current user, and the working modes include: general working mode, Internet of Things working mode and instruction entry mode; The mode switching module is also used to receive the custom instructions input by the current user and the corresponding original instruction combination when the working mode after switching is the instruction entry mode, and update the preset custom instruction library.

7. The Internet of Things mouse as claimed in claim 1, characterized in that: The communication module is further used to obtain a list of currently connectable terminals when receiving target instruction information; The communication module is further used to determine the target control terminal in the current connectable terminal list based on a preset priority order, wherein the preset priority order is pre-configured; The communication module is further used to send the target instruction information to the target control terminal, so that the target control terminal controls each execution device to complete the instruction operation corresponding to the target instruction information according to the target instruction information.

8. A terminal control method based on an Internet of Things mouse, characterized in that: The method comprises: Acquire current voice information, and determine whether there is a user-defined instruction in the current voice information; When there is a user-defined instruction in the current voice information, matching the original instruction combination corresponding to the user-defined instruction in a preset custom instruction library; rewriting the current voice information according to the original instruction combination to obtain target instruction information, and determining each execution device according to the target instruction information; The target instruction information is sent to the control terminal, so that the control terminal controls each execution device to complete the instruction operation corresponding to the target instruction information according to the target instruction information.

9. The method according to claim 8, characterized in that Before the step of determining whether there is a user-defined instruction in the current voice information, the method includes: Collecting speech fragments and preprocessing the speech fragments; Extracting features from the preprocessed voice segment using a preset user recognition model to obtain user voiceprint features; Performing identity authentication on the current user according to the user's voiceprint features; When the current user identity authentication is passed, the pre-processed voice segment is converted into text to obtain current voice information.

10. A terminal control system based on an Internet of Things mouse, characterized in that: The system comprises: a control terminal, a server and an Internet of Things mouse as claimed in any one of claims 1 to 7; The control terminal is used to determine the execution device corresponding to each target instruction information when receiving the target instruction information, and control each execution device through the server to complete the instruction operation corresponding to each target instruction information.

Images