Intelligent voice recognition and dynamic sensing mouse system

Abstract

According to the intelligent voice recognition and dynamic sensing mouse system, an instruction conflict switching control module is arranged, whether gesture action data sent by a sensor meet switching conditions or not is judged, if yes, a mechanical instruction control mode is switched, and if not, a voice instruction control mode is maintained; the voice command termination caused by wrong switching is prevented, and the system is ensured to accurately execute the voice command of the user.

Description

Technical Field

[0001] The present application relates to the field of voice mouse technology, and specifically to an intelligent voice recognition and dynamic sensing mouse system. Background Art

[0002] The intelligent voice mouse is a traditional mechanical mouse with a voice command control function. It can collect the voice commands issued by the user and convert these commands into text information, which is then matched with the preset command library and converted into control signals that can be recognized by the terminal. The processed control signal will be transmitted to the terminal via a wired (such as USB connection) or wireless (such as Bluetooth, 2.4GHz wireless technology). After receiving the signal, the terminal will perform the corresponding operation, such as moving the cursor on the screen, opening a specified file, searching for keywords in the browser, etc., to complete the task given by the user through voice. However, when performing voice command control, the voice command control mode is easily switched to mechanical command control due to external factors (such as bumps or tilts on the platform where the mouse is placed) or daily usage habits such as touching the mouse to move the mouse or pressing buttons, resulting in the termination of the voice command. Summary of the Invention

[0003] In view of the above problems, the present application provides an intelligent voice recognition and motion sensing mouse system to solve the above problems.

[0004] To achieve the above objectives, the inventors provide an intelligent speech recognition and motion sensing mouse system, which includes:

[0005] The voice command control module receives the voice data collected by the microphone and converts it into control signals that can be recognized by the terminal;

[0006] The mechanical command control module receives the gesture action data sent by the sensor and converts it into a control signal that can be recognized by the terminal;

[0007] The control mode switching module is used to switch between the voice command control mode and the mechanical command control mode, and transmit the converted control signal to the communication module according to the control module corresponding to the current switching mode instruction;

[0008] A communication module transmits control signals that can be recognized by the terminal to the terminal to perform corresponding operations; and

[0009] The command conflict switching control module receives gesture action data sent by the sensor when switching to the voice command control mode; and performs the following steps:

[0010] Determine whether the gesture action data sent by the sensor meets the switching conditions. If so, switch to the mechanical command control mode; otherwise, maintain the voice command control mode;

[0011] The switching conditions include meeting the following conditions within a preset window time:

[0012] Posture condition: The user's hand is recognized to be covering the mouse through the image captured by the camera on the mouse;

[0013] Gesture conditions: The gesture action data sent by the sensor belongs to mechanical command control.

[0014] Furthermore, the gesture action data sent by the sensor belongs to mechanical instruction control, including:

[0015] Determine whether there is movement in the gesture action data. If so:

[0016] The movement amplitude includes a movement displacement in at least one direction within a preset movement displacement range and a movement angle within a preset movement angle range; and / or

[0017] The acceleration and angular velocity are obtained from the gesture action data; the acceleration in any direction is greater than a preset acceleration threshold; and the angular velocity in any direction is less than a preset angular velocity threshold.

[0018] Furthermore, the movement displacement range is 2cm-10cm, and the movement angle range is 0°-45°.

[0019] Furthermore, the gesture action data sent by the sensor belongs to mechanical instruction control, including:

[0020] Determine whether there is a click button in the gesture action data. If so:

[0021] The duration of a button click is obtained from the gesture action data; the duration of the button click is less than a preset button duration threshold.

[0022] Furthermore, the gesture action data sent by the sensor belongs to mechanical instruction control, including:

[0023] An action frequency is obtained from the gesture action data, where the action frequency is greater than a preset frequency threshold.

[0024] Furthermore, the gesture action data sent by the sensor belongs to mechanical instruction control, including:

[0025] The gesture action data is input into a gesture motion classification model to determine whether it belongs to mechanical instruction manipulation; the gesture motion classification model is trained by inputting the mistouch gesture action data and the normal gesture action data into the classification model.

[0026] Furthermore, the order of determining whether the gesture action data sent by the sensor meets the switching condition is: first determining whether the position condition is met, and then determining whether the gesture condition is met.

[0027] Furthermore, the switching condition includes satisfying a voice condition within a preset window time: there is no abnormal change in the voice features of the voice data collected by the receiving microphone.

[0028] Furthermore, the voice features include volume and speaking speed.

[0029] Furthermore, after the step of satisfying the switching condition, a voice reminder is provided to confirm whether to switch to the mechanical command control mode.

[0030] Different from the existing technology, the above technical solution sets up a command conflict switching control module to determine whether the gesture action data sent by the sensor meets the switching conditions. If so, it switches to the mechanical command control mode, otherwise it maintains the voice command control mode; prevents the voice command from being terminated due to erroneous switching, and ensures that the system accurately executes the user's voice commands.

[0031] The above-mentioned records related to the content of the invention are only an overview of the technical solution of this application. In order to enable ordinary technicians in this field to understand the technical solution of this application more clearly, and then implement it according to the text of the specification and the contents recorded in the drawings, and to make the above-mentioned purposes and other purposes, features and advantages of this application easier to understand, the following is an explanation in combination with the specific implementation methods and drawings of this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings are only used to illustrate the principles, implementation methods, applications, features and effects of the specific embodiments of the present invention and other related contents, and are not to be considered as limiting the present application.

[0033] In the drawings of the specification:

[0034] Figure 1 This is a module diagram of the intelligent voice recognition and motion sensing mouse system described in the specific implementation method.

[0035] The reference numerals in the above drawings are described as follows:

[0036] 10. Control mode switching module;

[0037] 20. Mechanical instruction control module;

[0038] 30. Voice command control module;

[0039] 40. Communication module;

[0040] 50. Command conflict switching control module;

[0041] 60. Sensor;

[0042] 70. Microphone;

[0043] 80. Terminal. DETAILED DESCRIPTION

[0044] In order to explain in detail the possible application scenarios, technical principles, specific solutions that can be implemented, and the purpose and effects of this application, the following is a detailed description of the specific embodiments listed in conjunction with the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of this application and are therefore only examples and are not intended to limit the scope of protection of this application.

[0045] References to "embodiments" herein mean that the specific features, structures, or characteristics described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the word "embodiment" in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or relevance to other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the various technical features mentioned in the embodiments can be combined in any manner to form a corresponding implementable technical solution.

[0046] Unless otherwise defined, the technical terms used herein have the same meanings as those generally understood by those skilled in the art to which this application belongs; the use of relevant terms herein is only for describing specific embodiments and is not intended to limit this application.

[0047] In the description of this application, the term "and / or" is used to describe a logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and both A and B exist. In addition, the character " / " in this document generally indicates that the objects before and after are in a logical "or" relationship.

[0048] In this application, terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual quantity, priority or sequence relationship between these entities or operations.

[0049] Without further limitations, in this application, the words "include", "comprise", "have" or other similar open-ended expressions used in sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product that includes the elements, so that the process, method or product that includes a series of elements may include not only those defined elements, but also other elements that are not explicitly listed, or also include elements inherent to such process, method or product.

[0050] Consistent with the understanding in the Examination Guidelines, in this application, expressions such as "greater than," "less than," and "exceed" are understood to exclude the number itself; expressions such as "above," "below," and "within" are understood to include the number itself. Furthermore, in the description of the embodiments of this application, "multiple" means more than two (including two), and similar expressions related to "multiple" are also understood in this manner, such as "multiple groups," "multiple times," etc., unless otherwise specifically defined.

[0051] In the description of the embodiments of the present application, the space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present application or facilitating the reader's understanding, and do not indicate or imply that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation. Therefore, it should not be understood as a limitation on the embodiments of the present application.

[0052] The processor described in the embodiments of the present application can be implemented by hardware, firmware, software or a combination thereof, and can use a circuit, a single or multiple application-specific integrated circuits (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, or at least one of a microprocessor. It also includes other physical, biological or chemical structures that can achieve similar or equivalent functions to the processors listed above, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute some or all of the steps in the computer program or method involved in the various embodiments of the present application, or any combination of the steps mentioned therein.

[0053] The computer program involved in the embodiment can be stored in a computer device readable storage medium, which includes but is not limited to a disk, a tape, a magnetic card, a floppy disk, a flash memory, an optical disc, an optical card, a read-only memory (ROM), a random access memory (RAM), an erasable programmable ROM (EPROM) and an electrically erasable programmable ROM (EEPROM), etc., and also includes other biological, physical or chemical structures that can achieve similar or equivalent functions to the storage media listed above, such as DNA, RNA, protein and other units with information storage capabilities. In a specific embodiment, the storage medium involved can be one of the above-mentioned media types or a combination of the above-mentioned media types. In different embodiments, the computer program involved in the embodiment can be stored in a single medium in a centralized manner or in a distributed manner on multiple media. The memory containing the computer device readable storage medium can be a non-volatile memory or a random access memory. These computer device readable storage media can be built into the device or connected to the device involved in the embodiment as an external device or part of an external device. In some embodiments, a memory having a computer-readable storage medium is deployed locally; in other embodiments, a solution of deploying the memory away from the processor may also be adopted, such as a network-attached memory accessed via an RF circuit or an external port and a communication network, wherein the communication network may be the Internet, one or more intranets, a local area network (LAN), a wide area network (WLAN), a storage area network (SAN), etc., or a suitable combination thereof, as long as the computer device can access the memory. In addition, the computer program involved in the embodiment can be stored in plaintext / ciphertext form, or can be designed as training data, which can be integrated and reorganized through model training and implicitly stored in the parameter state of a deep neural network or other machine learning model.

[0054] See also Figure 1 As shown, the intelligent voice recognition and motion sensing mouse system can be applied to an intelligent voice mouse. The intelligent voice mouse is equipped with a microphone 70 for collecting user voice data, a sensor 60 (such as a velocity sensor 60, a gyroscope, etc.) for collecting user gesture data, and a camera for collecting images of the mouse surface. The system uses a command conflict switching control module 50 to determine whether the gesture data sent by the sensor 60 meets the switching conditions. If so, it switches to the mechanical command control mode; otherwise, it maintains the voice command control mode. This prevents voice commands from being terminated due to erroneous switching, ensuring that the system accurately executes the user's voice commands.

[0055] The following combination Figure 1 , providing an implementation method of an intelligent voice recognition and motion sensing mouse system, including:

[0056] The voice command control module 30 receives the voice data collected by the microphone 70 and converts it into a control signal that can be recognized by the terminal 80;

[0057] The mechanical command control module 20 receives the gesture action data sent by the sensor 60 and converts it into a control signal that can be recognized by the terminal 80;

[0058] The control mode switching module 10 is used to switch between the voice command control mode and the mechanical command control mode, and transmits the converted control signal to the communication module 40 according to the control module corresponding to the current switching mode instruction;

[0059] The communication module 40 transmits a control signal recognizable by the terminal 80 to the terminal 80 to perform a corresponding operation; and

[0060] The command conflict switching control module 50 receives the gesture action data sent by the sensor 60 when switching to the voice command control mode; and performs the following steps:

[0061] Determine whether the gesture action data sent by the sensor 60 meets the switching conditions. If so, switch to the mechanical command control mode; otherwise, maintain the voice command control mode;

[0062] The switching conditions include meeting the following conditions within a preset window time:

[0063] Posture condition: The user's hand is recognized to be covering the mouse through the image captured by the camera on the mouse;

[0064] Gesture condition: The gesture action data sent by the sensor 60 belongs to mechanical instruction control.

[0065] The above-mentioned sensor 60 is used to collect gesture action data (such as movement, clicking buttons, etc.) so as to subsequently analyze the gesture action data and determine whether the gesture action data belongs to mechanical instruction manipulation. The sensor 60 includes an acceleration sensor 60, a gyroscope, etc. The above-mentioned terminal receives the control signal transmitted by the communication module and performs corresponding operations. The terminal includes a computer, a pad, a central control screen, etc. that can be connected to the mouse for communication. The above-mentioned preset window time is a time interval used to analyze the changes in the user's gesture actions within this time interval.

[0066] The installation position of the above-mentioned camera must ensure that it can capture whether the user's hand covers the key area of ​​the mouse. For example, the camera can be set on the upper surface of the mouse, close to the button area (such as between two buttons), and the lens is facing the end away from the button area to capture images of whether the index finger and middle finger cover the mouse, so as to identify whether the user's hand covers the mouse; of course, if the camera is set at a high enough height, the lens can be tilted vertically downward or downward at a certain angle (such as 5°-15°) to capture images of whether the user's entire hand covers the mouse, so as to identify whether the user's hand covers the mouse. Or the camera can be set on the side of the mouse close to the thumb grip, and the lens is facing the thumb grip area, and the user's thumb and part of the palm are captured from the side to see whether they cover the side of the mouse, so as to identify whether the user's hand covers the mouse.

[0067] After the camera captures the image, it is necessary to perform feature extraction on the image to identify whether the user's hand is covering the mouse. Specifically:

[0068] In some embodiments, an edge detection algorithm (such as Canny edge detection) is used to extract the contour shape, such as calculating the contour's perimeter, area, circularity and other geometric parameters, and by calculating the overlapping area and overlapping ratio of the hand contour and the mouse contour, the relative position relationship between the hand edge and the mouse edge is analyzed to determine whether the hand is above the mouse. Specifically, a series of judgment thresholds can be set. For example, when the overlapping area of ​​the hand contour and the mouse contour accounts for more than 50% of the mouse area, and the hand remains within a certain range above the mouse in three consecutive frames of images, it is determined that the user's hand covers the mouse. If these threshold conditions are not met, it is determined that the hand does not cover the mouse.

[0069] In some embodiments, texture information is extracted from the hand's skin texture using methods such as a gray-level co-occurrence matrix. This information is then compared and matched against a pre-stored database of hand texture features. Texture feature similarity is then used to determine whether the user's hand is covering the mouse. Specifically, if the texture feature similarity is ≥ 0.75, the user's hand is considered to be covering the mouse; if the texture feature similarity is < 0.75, the user's hand is considered to be not covering the mouse.

[0070] In some embodiments, the hand region can also be segmented based on the color characteristics of the hand skin, taking advantage of the fact that human skin has a relatively fixed color range in a specific color space (such as the HSV color space) by setting an appropriate color threshold. Specifically, the image is converted from the common RGB color space to the HSV color space. Then, based on the set HSV color threshold (skin hue (H) between 0-30 and 150-180, saturation (S) between 20-255, and brightness (V) between 30-255), the converted HSV image is processed using an image segmentation algorithm. The HSV value of each pixel in the image is compared with the set threshold range. If the HSV value of the pixel is within the threshold range, the pixel is marked as part of the hand region; otherwise, it is marked as a non-hand region. In this way, the hand region can be preliminarily segmented to obtain a binary image, in which the white (or other set color) area represents the hand and the black area represents the background (including other objects such as the mouse).

[0071] In actual applications, in order to more accurately identify whether the user's hand is covering the mouse, one or more of the above embodiments can be combined and applied according to specific application scenarios and needs. For example, a judgment can be made based on the overlap rate between the hand outline and the mouse area. If the texture feature similarity is ≥0.75 and the edge detection shows that the overlap rate between the hand outline and the mouse area is ≥50%, then the user's hand is determined to be covering the mouse; if the texture feature similarity is <0.75 or the edge detection shows that the overlap rate between the hand outline and the mouse area is <30%, then the hand is determined not to be covering the mouse.

[0072] Gesture action data can be used to determine whether it belongs to mechanical command control based on the difference characteristics in dimensions such as action amplitude, action frequency, and duration.

[0073] In some embodiments, whether the gesture is a mechanical command operation is determined by the amplitude of the action. Only when the amplitude of the action is within a preset range, the gesture action data is considered to be a mechanical command operation. Specifically, the gesture action data sent by the sensor 60 is considered to be a mechanical command operation, including:

[0074] Determine whether there is movement in the gesture action data. If so:

[0075] Obtaining motion amplitude from gesture motion data;

[0076] The movement amplitude includes a movement displacement in at least one direction within a preset movement displacement range and a movement angle within a preset movement angle range;

[0077] In specific application scenarios, properly setting the preset motion displacement range and the preset motion angle range plays a key role in accurately identifying user operations and reducing accidental switching. The preset motion displacement range can be 2cm-10cm. In daily use, slight hand tremors or unintentional mouse touches generally cause mouse displacements of less than 2cm, while large, accidental collisions often cause mouse displacements far exceeding 10cm. Using 2cm-10cm as the judgment range for mechanical command control effectively filters out both unintentional small displacements and accidental large displacements, accurately identifying intentional user operations. Similarly, the preset motion angle range can be set to 0°-45°. During normal operation, when the user has a clear intention to operate, the mouse motion angle is mostly within the 0°-45° range. However, when the mouse deviates by a large angle, exceeding this range, it can generally be assumed that this is not an active user operation. This setting significantly reduces the probability of accidentally switching to mechanical command control mode due to unintentional circumstances.

[0078] Of course, acceleration and angular velocity can also be obtained from gesture action data; the acceleration in any direction is greater than a preset acceleration threshold; the angular velocity in any direction is less than a preset angular velocity threshold. The above-mentioned determination of motion displacement and motion angle, as well as the determination of acceleration and angular velocity, can be used separately or in combination.

[0079] In some embodiments, the duration of a key press is used to determine whether it is a mechanical command control. If the key press duration is too long, it can be considered as an accidental touch. Generally speaking, under mechanical command control, the duration of a single key press will be within a reasonable range. For example, when using a mouse to click, the normal single-click operation duration is usually between 0.1 and 0.5 seconds. If the key press duration exceeds 1 second, it may not be a normal instruction from the user. Specifically, the gesture action data sent by the sensor 60 belongs to mechanical command control, including:

[0080] Determine whether there is a click button in the gesture action data. If so:

[0081] The duration of a button click is obtained from the gesture action data; the duration of the button click is less than a preset button duration threshold.

[0082] In some embodiments, the frequency of the action is used to determine whether it belongs to mechanical command manipulation. When the action frequency is too high or too low, it may not fall within the scope of mechanical command manipulation. Mechanical command manipulation often has a certain range of action frequencies. Taking mouse operation as an example, in daily office scenarios, when users perform operations such as clicking and dragging, the click frequency is usually between 1 and 5 times per second. If it is detected that the click frequency of the mouse is too low, such as clicking once every few minutes, after excluding the special case where the user is indeed performing extremely slow operations, it can be determined that it is an accidental action caused by the user inadvertently touching the mouse. Specifically, the gesture action data sent by the sensor 60 belongs to mechanical command manipulation, including:

[0083] An action frequency is obtained from the gesture action data, where the action frequency is greater than a preset frequency threshold.

[0084] In practical applications, in order to more accurately determine whether the collected gesture action data belongs to mechanical command manipulation, one or more of the above embodiments may be combined and applied according to specific application scenarios and requirements.

[0085] In some embodiments, a training gesture motion classification model is further provided to determine whether the gesture motion data sent by the sensor 60 belongs to mechanical command manipulation. Specifically, the gesture motion data sent by the sensor 60 belongs to mechanical command manipulation, including:

[0086] The gesture action data is input into a gesture motion classification model to determine whether it belongs to mechanical instruction manipulation; the gesture motion classification model is trained by inputting the mistouch gesture action data and the normal gesture action data into the classification model.

[0087] Specifically, when collecting accidental touch gesture data, various scenarios that may cause accidental touches can be simulated, such as natural hand tremors and gestures caused by accidentally touching the device. When collecting normal gesture data, various normal mechanical control gestures, such as clicking, sliding, and zooming, can be collected. As users perform these actions, the corresponding gesture data is collected simultaneously. Similarly, it is important to ensure that the collected data is diverse, reflecting the differences in user habits. Preprocess the collected gesture data and normal gesture data. Select an appropriate classification model based on the data characteristics and task requirements. Common classification models include decision trees and support vector machines (SVMs). The preprocessed collected gesture data and normal gesture data are input into the selected classification model for training. During training, the model continuously adjusts its parameters to minimize the difference between the predicted results and the true annotations, ultimately forming a gesture classification model. When determining whether new gesture data represents mechanical control, the gesture data is processed using the data preprocessing method and then input into the trained gesture classification model. The gesture motion classification model outputs a prediction result of whether the gesture action data belongs to "mechanical command control" or "non-mechanical command control" based on the learned features and patterns.

[0088] Whether the gesture action data sent by the sensor 60 meets the switching condition can be judged at the same time or one by one. In some embodiments, the order of judging whether the gesture action data sent by the sensor 60 meets the switching condition is: first judge whether the posture condition is met, and then judge whether the gesture condition is met. Posture is the basic feature of gestures, and the user's hand covering the mouse is a prerequisite for mechanical command control. Judging the posture condition first can effectively filter out false touch gestures caused by false touches, natural shaking of the hand, etc. If the posture condition is not met, there is no need to perform subsequent gesture condition judgment, which reduces unnecessary calculations and judgment time. In actual application scenarios, a large number of false touch gestures can be quickly filtered out through posture judgment, which improves the response speed of the system and enables the system to process valid gesture instructions more promptly.

[0089] In certain embodiments, certain characteristics of the voice input process, such as voice continuity and volume changes, are used to assist in determining whether the current gesture data represents an unintended touch. If the voice input is not accompanied by noticeable pauses or anomalies, and the sensor 60 captures gesture data, the voice characteristics are used to infer that the gesture data represents an unintended touch, thereby ignoring the gesture and not terminating the voice input. Specifically, the switching condition includes meeting the voice condition within a preset window time and ensuring that the voice characteristics of the voice data captured by the receiving microphone 70 do not exhibit any unusual changes. These voice characteristics include, but are not limited to, volume and speech rate.

[0090] In some embodiments, after the switching condition is met, a voice prompt is provided to confirm the switch to mechanical command control mode. The voice prompt allows the user to promptly notify and confirm the operation, further avoiding accidental entry into mechanical command control mode and preventing erroneous execution of commands in this mode, thereby protecting data security and operational accuracy.

[0091] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of this application, this does not limit the scope of patent protection of this application. All technical solutions generated by replacing or modifying equivalent structures or equivalent processes based on the essential concepts of this application using the contents recorded in the specification and drawings of this application, as well as directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, are included in the scope of patent protection of this application.

Claims

1. Intelligent voice recognition and dynamic sensing mouse system, characterized by: include: The voice command control module receives the voice data collected by the microphone and converts it into control signals that can be recognized by the terminal; The mechanical command control module receives the gesture action data sent by the sensor and converts it into a control signal that can be recognized by the terminal; The control mode switching module is used to switch between the voice command control mode and the mechanical command control mode, and transmit the converted control signal to the communication module according to the control module corresponding to the current switching mode instruction; A communication module transmits control signals that can be recognized by the terminal to the terminal to perform corresponding operations; and The command conflict switching control module receives the gesture action data sent by the sensor when switching to the voice command control mode; Perform the following steps: Determine whether the gesture action data sent by the sensor meets the switching conditions. If so, switch to the mechanical command control mode; otherwise, maintain the voice command control mode; The switching conditions include meeting the following conditions within a preset window time: Posture condition: The user's hand is recognized to be covering the mouse through the image captured by the camera on the mouse; Gesture conditions: The gesture action data sent by the sensor belongs to mechanical command control.

2. The intelligent speech recognition and motion sensing mouse system according to claim 1, characterized in that: The gesture action data sent by the sensor belongs to mechanical command control, including: Determine whether there is movement in the gesture action data. If so: The movement amplitude includes a movement displacement in at least one direction within a preset movement displacement range and a movement angle within a preset movement angle range; and / or The acceleration and angular velocity are obtained from the gesture action data; the acceleration in any direction is greater than a preset acceleration threshold; and the angular velocity in any direction is less than a preset angular velocity threshold.

3. The intelligent speech recognition and motion sensing mouse system according to claim 2, characterized in that: The movement displacement range is 2cm-10cm, and the movement angle range is 0°-45°.

4. The intelligent speech recognition and motion sensing mouse system according to claim 1, characterized in that: The gesture action data sent by the sensor belongs to mechanical command control, including: Determine whether there is a click button in the gesture action data. If so: The duration of a button click is obtained from the gesture action data; the duration of the button click is less than a preset button duration threshold.

5. The intelligent speech recognition and motion sensing mouse system according to claim 1, characterized in that: The gesture action data sent by the sensor belongs to mechanical command control, including: An action frequency is obtained from the gesture action data, where the action frequency is greater than a preset frequency threshold.

6. The intelligent speech recognition and motion sensing mouse system according to claim 1, characterized in that: The gesture action data sent by the sensor belongs to mechanical command control, including: The gesture action data is input into a gesture motion classification model to determine whether it belongs to mechanical instruction manipulation; the gesture motion classification model is trained by inputting the mistouch gesture action data and the normal gesture action data into the classification model.

7. The intelligent speech recognition and motion sensing mouse system according to claim 1, characterized in that: The order of determining whether the gesture action data sent by the sensor meets the switching condition is: first determine whether the position condition is met, and then determine whether the gesture condition is met.

8. The intelligent speech recognition and motion sensing mouse system according to claim 1, characterized in that: The switching condition includes that within a preset window time, a voice condition is also satisfied: there is no abnormal change in the voice features of the voice data collected by the receiving microphone.

9. The intelligent speech recognition and motion sensing mouse system according to claim 1, characterized in that: The speech features include volume and speaking speed.

10. The intelligent speech recognition and motion sensing mouse system according to claim 1, characterized in that: After the step of satisfying the switching conditions, a voice reminder is also included to confirm whether to switch to the mechanical command control mode.

Images