Mouse and eye movement cursor interaction system and method

Abstract

The invention provides a mouse and an eye movement cursor interaction system and method. A mouse comprises a mouse body and further comprises an eye movement detection module, and the mouse body is configured to keep the eye movement detection module so that the eye movement detection module can be in a stored state; when the eye movement detection module is in the storage state, the eye movement detection module is in signal connection with the mouse main body; and when the eye movement detection module is in a taking-out state of being taken out from the mouse main body, the eye movement detection module is configured to be capable of being in signal connection with the electronic equipment, so that a user controls a mouse cursor to operate on the electronic equipment through eye movement. According to the mouse, fatigue damage of the wrist of a user due to long-time use can be relieved, and the use requirements of common users and special users can be met. According to the mouse, a more natural and smoother interaction mode is provided for the user by transferring the cooperative operation of the eyes and the hands of the user, and the use efficiency and experience are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of mouse technology, and in particular to a mouse, eye-motion cursor interaction system and method. Background Art

[0002] As an external input device for computers, the mouse is widely used. The current structure of the mouse generally includes a mouse body and a left button, a right button and a scroll wheel installed on the mouse body. When operating a computer, users habitually use a mouse to operate the display interface. However, long-term use can easily cause wrist fatigue and affect hand health.

[0003] For different users, the current ordinary mouse cannot meet the usage requirements of different scenarios at the same time. For example, in gaming scenarios, the ordinary mouse structure cannot meet the requirements of gaming scenarios because it has too few functions. Additional equipment needs to be purchased separately to cooperate with the mouse operation, which undoubtedly increases the user's usage cost. Summary of the invention

[0004] The present disclosure provides a mouse, eye-movement cursor interaction system and method to at least solve one of the technical problems existing in the prior art.

[0005] According to a first aspect of the present disclosure, there is provided a mouse, comprising a mouse body, and an eye movement detection module, wherein the mouse body is configured to hold the eye movement detection module so that the eye movement detection module is in a storage state;

[0006] Wherein, when the eye movement detection module is in the storage state, the eye movement detection module is connected to the mouse body signal;

[0007] When the eye movement detection module is in a state of being removed from the mouse body, the eye movement detection module is configured to be able to connect with the electronic device signal so that the user controls the mouse cursor to operate on the electronic device through eye movement.

[0008] In one possible implementation, the eye movement detection module is configured to allow manual or electronic control to switch between the storage state and the removal state.

[0009] In one possible implementation manner, a receiving cavity with an opening is formed on the mouse body, and the eye movement detection module is accommodated in the receiving cavity and is detachably and elastically connected to the mouse body to enable the eye movement detection module to switch between a storage state and a removal state.

[0010] In one embodiment, the mouse body further includes:

[0011] A limiting groove is arranged below the accommodating cavity and communicated with the accommodating cavity;

[0012] An elastic arm assembly is movably connected between the accommodating cavity and the limiting groove, wherein a first side of the elastic arm assembly abuts against the eye movement detection module in the accommodating cavity, and a second side is connected to the limiting groove through an elastic member;

[0013] A first signal connection module is movably mounted on the cavity wall of the accommodating cavity;

[0014] When the eye movement detection module is in a stored state, the elastic arm assembly is in a first position, and when the eye movement detection module is in a removed state, the elastic arm assembly is in a second position; when the elastic arm assembly is in the first position, the elastic arm assembly forms an extrusion force on the first signal connection module to push the first signal connection module from an initial position toward the eye movement detection module, thereby establishing a signal connection with the second signal connection module of the eye movement detection module; when the elastic arm assembly is in the second position, the elastic arm assembly releases the extrusion of the first signal connection module to allow the first signal connection module to return to the initial position.

[0015] In one embodiment, the elastic arm assembly includes:

[0016] A first elastic arm portion is disposed below the accommodating cavity, wherein a first side of the first elastic arm portion abuts against the eye movement detection module, and a second side of the first elastic arm portion is connected to the limiting groove through an elastic member;

[0017] A second elastic arm portion is disposed in the interlayer space between the accommodating cavity and the limiting groove, and a first end of the second elastic arm portion is connected to the first elastic arm portion;

[0018] The limiting portion is arranged on the outer side of the second elastic arm portion, and the limiting portion is accommodated in the limiting hole opened in the limiting groove. The limiting portion is used to limit the elastic arm assembly at the second position when the elastic arm assembly moves from the first position to the second position.

[0019] In one possible implementation manner, the mouse body further includes a first magnetic attraction component, and the first magnetic attraction component is installed on the first signal connection module; the second signal connection module is provided with a second magnetic attraction component at a corresponding position;

[0020] The mouse body also includes:

[0021] An activation button, mounted on the shell of the mouse body;

[0022] A touch operation module is installed on the shell of the mouse body;

[0023] A main control board is installed in the shell of the mouse body, and the main control board is electrically connected to the activation button, the touch operation module, the first signal connection module and the eye movement detection module.

[0024] In one possible implementation, the eye movement detection module includes:

[0025] The first detection module body,

[0026] The second detection module body is rotatably connected to the first detection module body,

[0027] A first camera module is installed on the second detection module body;

[0028] An infrared sensing module, installed on the second detection module body;

[0029] The control unit is installed in the main body of the first detection module, and is used for signal connection with the first camera module, the infrared sensing module, the main control board and the electronic equipment.

[0030] According to a second aspect of the present disclosure, an eye-movement cursor interaction system is provided, comprising an electronic device having a second camera module and the mouse, wherein the eye-movement detection module can be detachably installed on the electronic device when the eye-movement detection module is in a removed state.

[0031] According to a third aspect of the present disclosure, a method is provided, which is applied to the eye-movement cursor interaction system, and the method comprises:

[0032] Get the position of the user's pupil on the display screen;

[0033] Synchronize the user's pupil and mouse cursor lock connection;

[0034] Locate the user's head position and calibrate the user's pupil center position;

[0035] Determine whether to change the user usage mode;

[0036] If yes, the electronic device enters the fine operation mode; if no, the electronic device enters the normal mode;

[0037] When the electronic device is in the fine operation mode, the application software icon on the display screen is in a state that can be activated by the eye movement cursor. If the eye movement control cursor stays on a certain application software and exceeds the threshold, the application software is activated and the eye movement cursor range is controlled within the display interface range of the application software;

[0038] In response to a touch operation input by a user via a touch operation module, the touch operation is implemented on the display interface.

[0039] In one possible implementation, obtaining the position of the user's pupil on the display screen includes: obtaining a first pupil center position input by an eye movement detection module, taking a user's corneal reflection point as a base point for the relative position of the first camera module and the eyeball, obtaining a sight line vector coordinate according to the first pupil center position, thereby determining a gaze point of the human eye, wherein the position of the gaze point is the position of the user's pupil on the display screen;

[0040] The positioning of the user's head position and the calibration of the user's pupil center position include:

[0041] Acquire a first image captured by the second camera module when the user's head is gazing at a calibration point in the center of the display screen, perform facial recognition on the first image, and obtain a pupil position;

[0042] Determine a second image and determine the second pupil center position of the user at this time according to the second image; the second image is an image of the infrared light reflected by the eyes when the user's head remains still and the user's eyes are focused on the calibration point at the center of the display screen, taken by the first camera module;

[0043] Projecting the second pupil center position onto the display screen area to coincide with the pupil position, and performing center position calibration;

[0044] Determine the control path of the eye-controlled cursor;

[0045] When the electronic device is in normal mode, eye movement controls the cursor to browse pages and files, and the change and movement of the browsing content of the current page are controlled by the direction of the eye movement trajectory.

[0046] Compared with the prior art, the advantages of the present application are: the mouse of the present application, while maintaining the functions of the current ordinary mouse, is also provided with an eye movement detection module. When the eye movement detection module is stored, the mouse maintains ordinary functions, such as the mouse wheel function, the left and right mouse button functions. When the eye movement detection module is in the removed state, it can be connected to the electronic device signal to realize eye movement control of the cursor. The mouse can be more flexibly controlled by combining eye movement with finger operation, which can not only relieve the fatigue damage of the user's wrist caused by long-term use, but also take into account the use needs of ordinary users and specialized users. In addition, the mouse of the present application provides users with a more natural and smoother interaction method by mobilizing the coordinated operation of the user's eyes and hands, thereby improving the use efficiency and experience.

[0047] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0049] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0050] Figure 1 The structure of the mouse of the embodiment of the present disclosure is shown. Figure 1 ;

[0051] Figure 2 The structure of the mouse of the embodiment of the present disclosure is shown. Figure 2 ;

[0052] Figure 3 The structure of the mouse of the embodiment of the present disclosure is shown. Figure 3 ;

[0053] Figure 4 The mouse of the present invention is partially disassembled. Figure 1 ;

[0054] Figure 5 The mouse of the present invention is partially disassembled. Figure 2 ;

[0055] Figure 6 The mouse of the present invention is partially disassembled. Figure 3 ;

[0056] Figure 7 The explosion diagram of the mouse of the embodiment of the present disclosure is shown Figure 1 ;

[0057] Figure 8 The explosion diagram of the mouse of the embodiment of the present disclosure is shown Figure 2 ;

[0058] Fig. 9 The structure diagram of the mouse according to the embodiment of the present disclosure when the elastic arm assembly is in the first position is shown. Figure 1 (wherein the eye movement detection module is not shown);

[0059] Fig.10 The structure diagram of the mouse according to the embodiment of the present disclosure when the elastic arm assembly is in the first position is shown. Figure 2 (wherein the eye movement detection module is not shown);

[0060] Fig.11 A schematic diagram of the structure of the mouse according to the embodiment of the present disclosure when the elastic arm assembly is located in the second position (wherein the eye movement detection module is removed);

[0061] Fig.12A cross-sectional view of the eye movement detection module of the mouse according to the embodiment of the present disclosure when it is in a storage state is shown;

[0062] Fig.13 A cross-sectional view of the eye movement detection module of the mouse according to the embodiment of the present disclosure when it is in a removed state is shown;

[0063] Fig.14 The structure of the eye movement detection module of the mouse of the embodiment of the present disclosure is shown in the storage state. Figure 1 ;

[0064] Fig.15 The structure of the eye movement detection module of the mouse of the embodiment of the present disclosure is shown in the storage state. Figure 2 ;

[0065] Fig.16 A schematic diagram showing the structure of an eye movement detection module of a mouse according to an embodiment of the present disclosure is shown;

[0066] Fig.17 A flow chart of a method according to an embodiment of the present disclosure is shown.

[0067] Description of reference numerals in the figure: 1-mouse body, 11-accommodating cavity, 12-limiting groove, 13-elastic arm assembly, 14-elastic member, 15-first signal connection module, 16-first magnetic member, 121-limiting hole, 131-first elastic arm part, 132-second elastic arm part, 133-limiting part, 134-elastic structure, 151-first signal base, 152-first signal connector;

[0068] 2-eye movement detection module, 21-second signal connection module, 22-first detection module body, 23-second detection module body, 24-infrared sensing module, 25-first camera module, 211-second signal base, 212-second signal connector, 213-second magnetic attraction member;

[0069] 3-Activation button, 4-Touch operation module, 5-Left mouse button, 6-Right mouse button, 7-Mouse wheel. DETAILED DESCRIPTION

[0070] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0071] According to one embodiment of the present disclosure, the present invention provides a mouse. Figure 1-16As shown, a mouse comprises a mouse body 1 and an eye movement detection module 2, wherein the mouse body 1 is configured to hold the eye movement detection module 2 so that the eye movement detection module 2 is in a storage state;

[0072] When the eye movement detection module 2 is in the storage state, the eye movement detection module 2 is connected to the mouse body 1 by signal;

[0073] When the eye movement detection module 2 is removed from the mouse body 1 , the eye movement detection module 2 is configured to be able to connect to the electronic device signal so that the user controls the mouse cursor to operate on the electronic device through eye movement.

[0074] The mouse of the present application, while maintaining the functions of the current ordinary mouse, is also provided with an eye movement detection module 2. When the eye movement detection module 2 is stored, the mouse maintains ordinary functions, such as the mouse wheel function, the left and right mouse button functions. When the eye movement detection module is in the removed state, it can be connected to the electronic device signal to realize eye movement control of the cursor. The mouse can be more flexibly operated and controlled by combining eye movement with finger operation, which can not only relieve the fatigue damage of the user's wrist caused by long-term use, but also take into account the use needs of ordinary users and specialized users. In addition, the mouse of the present application provides users with a more natural and smoother interaction method by mobilizing the coordinated operation of the user's eyes and hands, thereby improving the use efficiency and experience.

[0075] As an optional implementation, the eye movement detection module 2 is configured to allow manual or electronic control to switch between the storage state and the removal state. The eye movement detection module 2 can be removed from the mouse body 1 manually or electronically, which is simple and convenient.

[0076] As an optional implementation, when the eye movement detection module 2 is configured to allow manual switching between the storage state and the removal state, it can be achieved in the following manner: a receiving cavity 11 with an opening is provided on the mouse body 1, the eye movement detection module 2 is received in the receiving cavity 11 and is detachably elastically connected to the mouse body 1, so as to achieve the switching of the eye movement detection module 2 between the storage state and the removal state.

[0077] Among them, Figure 7-15 As shown, the mouse body 1 also includes:

[0078] The limiting groove 12 is disposed below the accommodating cavity 11 and communicates with the accommodating cavity 11;

[0079] The elastic arm assembly 13 is movably connected between the accommodating cavity 11 and the limiting groove 12, wherein a first side of the elastic arm assembly 13 abuts against the eye movement detection module 2 in the accommodating cavity 11, and a second side is connected to the limiting groove 12 via an elastic member 14;

[0080] The first signal connection module 15 is movably mounted on the cavity wall of the accommodating cavity 11;

[0081] When the eye movement detection module 2 is in the storage state, the elastic arm assembly 13 is in the first position. Fig.12 As shown, when the eye movement detection module 2 is in the removed state, the elastic arm assembly 13 is in the second position, as shown in FIG. Fig.13 As shown; when the elastic arm assembly 13 is in the first position, the elastic arm assembly 13 forms an extrusion force on the first signal connection module 15 to push the first signal connection module 15 to move from the initial position toward the eye movement detection module 2, thereby establishing a signal connection with the second signal connection module 21 of the eye movement detection module 2; when the elastic arm assembly 13 is in the second position, the elastic arm assembly 13 releases the extrusion on the first signal connection module 15 to allow the first signal connection module to return to the initial position.

[0082] In this embodiment, Figure 7-11 As shown, the limiting groove 12 is generally a concave structure, and the opening of the limiting groove 12 faces and the limiting groove 12 is connected to the accommodating cavity 11. The elastic arm assembly 13 is generally a concave structure, and the elastic arm assembly 13 is arranged between the accommodating cavity 11 and the limiting groove 12. The elastic arm assembly 13 is half-covered from the bottom of the accommodating cavity 11 to the outside of the accommodating cavity. The elastic arm assembly has elastic reset ability, so the elastic arm assembly can be made of materials such as plastic and rubber. The elastic member 14 includes but is not limited to a spring.

[0083] Further, such as Figure 9-13 As shown, the elastic arm assembly 13 includes:

[0084] The first elastic arm portion 131 is disposed below the accommodating cavity 11 and above the limiting groove 12. The first side of the first elastic arm portion 131 abuts against the eye movement detection module 2, and the second side is connected to the limiting groove 12 through the elastic member 14.

[0085] The second elastic arm portion 132 is disposed in the interlayer space between the accommodating cavity 11 and the limiting groove 12, and the first end of the second elastic arm portion 132 is connected to the first elastic arm portion 131;

[0086] The limiting portion 133 is disposed on the outer side of the second elastic arm portion 132 and is received in the limiting hole 121 formed in the limiting groove 12. The limiting portion 133 is used to limit the elastic arm assembly at the second position when the elastic arm assembly moves from the first position to the second position.

[0087] In this embodiment, the upper and lower ends of the accommodating cavity 11 are both provided with openings, and an elastic structural member 134 is provided on the first side of the first elastic arm portion 131, and the elastic structural member is provided in the opening at the lower end of the accommodating cavity and can be moved up and down. Two second elastic arm portions 132 are provided, which are respectively provided in the middle of the interlayer between the accommodating cavity and the limiting groove, and one end of the two second elastic arm portions 132 is connected to the first elastic arm portion, so that the elastic arm assembly forms a concave structure. Figure 9-10 , Fig.12 As shown, when the eye movement detection module 2 is in the storage state, the second end of the second elastic arm 132 on the left side forms a squeezing force on the first signal connection module 15 on the cavity wall of the accommodating cavity, so that it is connected to the second signal connection module 21 on the eye movement detection module 2 by signal. Therefore, when the eye movement detection module 2 is put back into the accommodating cavity 11, the bottom of the eye movement detection module 2 first contacts the elastic structure 134, and then when the pressure continues, the elastic structure drives the first elastic arm 131 and the second elastic arm 132 to move downward to the first position. At this time, the second end of the second elastic arm 132 forms a squeezing force on the first signal connection module 15 to push the first signal connection module 15 from the initial position to the direction of the eye movement detection module 2, so as to connect the signal with the second signal connection module 21 of the eye movement detection module 2. As a result, the control unit of the eye movement detection module 2 is electrically connected to the main control board through the second signal connection module and the first signal connection module.

[0088] When the eye movement detection module 2 needs to be taken out, the top of the eye movement detection module 2 is pressed to release the spring at the bottom of the module, driving the first elastic arm and the second elastic arm to move upward. As the module moves upward, the pressure at the second end of the second elastic arm decreases, and the signal connector is partially disconnected, so that a part of the whole is ejected for easy removal. At this time, the elastic arm assembly moves to the second position, such as Fig.11 , Fig.13 shown.

[0089] In the present application embodiment, Fig.10As shown, the first signal connection module 15 includes a first signal base 151 and a first signal connector 152 mounted on the first signal base 151. Further, the first signal base 151 is an insulator and can be made of materials such as plastic and rubber. The first signal base 151 and the first signal connector 152 can be integrally formed using a mold. The first signal connection module 15 can be movably installed on the cavity wall of the accommodating cavity 11 by using well-known technology (for example, an opening is opened on the cavity wall of the accommodating cavity 11, and the first signal base 151 is installed in the opening. Since the elastic arm assembly 13 moves from the first position to the second position or from the second position to the first position, the first signal base 151 is restricted between the second elastic arm part 132 and the outer wall of the shell of the eye movement detection module 2, preventing the first signal base 151 from falling off from the opening. When the elastic arm assembly is in the second position, the second end of the second elastic arm part releases the squeezing of the first signal connection module, and the first signal connection module returns to the initial position. Of course, in order to prevent the first signal base 151 from falling off, the left and right side edges of the first signal base 151 of the first signal connection module 15 can be formed with protrusions upward to restrict the first signal connection module 15 to the opening in the cavity wall of the accommodating cavity). When the elastic arm assembly 13 is in the first position, the elastic arm assembly 13 (mainly the second elastic arm portion 132) exerts a squeezing force on the first signal connection module 15, so that the first signal connection module 15 is connected to the second signal connection module 21 by signal, and the first signal connection module 15 is limited between the second signal connection module 21 and the elastic arm assembly 13. When the elastic arm assembly moves from the first position to the second position, the first signal connection module moves in a direction away from the eye movement detection module 2 under the squeezing of the eye movement detection module 2, until the first signal connection module 15 returns to the initial position when the elastic arm assembly 13 is in the second position.

[0090] Furthermore, in order to enable the eye movement detection module 2 to maintain a stable signal connection with the mouse body 11 and remain in the first position when in the storage state, the mouse body 1 of the present application also includes a first magnetic member 16, which is installed on the first signal connection module 15; the second signal connection module 21 is provided with a second magnetic member at the corresponding position. Specifically, there are two first magnetic members 16, which are respectively provided at the two ends of the first signal base 151. Therefore, when the elastic arm assembly 13 moves from the second position to the first position, the end of the second elastic arm portion 132 of the elastic arm assembly 13 pushes the first signal base 151 of the first signal connection module toward the eye movement detection module, so that the first signal connector 152 of the first signal connection module is connected to the second signal connection module 21 by signal. At this time, the first magnetic member 16 and the second magnetic member of the second signal connection module are magnetically connected. Therefore, the stability of the signal connection is ensured by the provision of the magnetic member, and the alignment and fixation of the eye movement detection module and the mouse body are ensured.

[0091] As an optional implementation, the mouse body of the present application further includes:

[0092] The activation button 3 is installed on the shell of the mouse body 1.

[0093] The touch operation module 4 is installed on the shell of the mouse body 1.

[0094] The main control board is installed in the shell of the mouse body 1 , and the main control board is electrically connected to the activation button, the touch operation module, the first signal connection module and the eye movement detection module 2 .

[0095] Among them, the activation button 3 and the touch operation module 4 are also electrically connected to the main control board provided in the mouse body. Then the activation button and the touch operation module are connected to the electronic device signal through the main control board. The activation button 3 is the fine touch module function activation button, and the electronic device is used to respond to the activation signal input by the user through the activation button 3, so as to determine whether to enter the fine touch operation mode (fine operation mode). When the electronic device enters the fine touch operation mode, in response to the touch operation input by the user through the touch operation module 4, the touch operation is implemented on the display interface of the electronic device.

[0096] The mouse body of the present application further includes a left mouse button 5, a right mouse button 6 and a mouse wheel 7, which are mounted on the housing of the mouse body 1 and are electrically connected to the main control board. Figure 1-3 As shown, the mouse body 1 is an irregular structure formed by a flat bottom surface and a curved side surface. The left mouse button, the right mouse button and the mouse wheel are arranged on one side surface of the mouse body, the activation button and the touch operation module are arranged on the other side surface, and the eye movement detection module is arranged in the middle position to facilitate the removal or replacement of the eye movement detection module.

[0097] As an optional implementation, Figure 11-16 As shown, the eye movement detection module 2 includes:

[0098] The first detection module body 22,

[0099] The second detection module body 23 is rotatably connected to the first detection module body 22.

[0100] The first camera module 25 is installed on the second detection module body 23.

[0101] The infrared sensing module 24 is installed on the second detection module body 23.

[0102] The second signal connection module 21 is installed on the first detection module body 22;

[0103] The control unit is installed in the first detection module body 22, and is used for signal connection with the first camera module 25, the infrared sensing module 24, the second signal connection module 21, the main control board and the electronic equipment.

[0104] In the present application, the first detection module body 22 and the second detection module body 23 are rotatably connected via a rotating shaft. The first detection module body 22 has a cavity inside, and a control unit is installed in the cavity. The control unit is electrically connected to the infrared sensing module 24, the first camera module 25, and the second signal connection module 21. The first camera module 25 is an infrared camera. The second signal connection module 21 includes a second signal base 211 and a second signal connector 212 installed on the second signal base 211, and a second magnetic suction member 213 respectively installed at both ends of the second signal base 211. Furthermore, the second signal base 211, the second signal connector 212 and the second magnetic suction member 213 can be made into an integrally formed structure by mold injection.

[0105] The second signal connection module 21 is installed on the wall of the first detection module body 22, and the second magnetic attraction member 213 and the second signal connector 212 protrude from the surface of the second signal base 211. The second signal connector 212 is electrically connected to the control unit.

[0106] When the eye movement detection module 2 is in the storage state, the eye movement detection module 2 is connected to the first signal connector 152 of the mouse body through the second signal connector 212 provided thereon, and then realizes electrical connection with the main control board (main control circuit board, using the control circuit board of the current conventional mouse) provided inside the mouse body. Thus, the first camera module 25, the infrared sensing module 24, and the control unit of the eye movement detection module are powered by the mouse body. Among them, when the eye movement detection module is in the storage state, the eye movement detection module can be powered by a physical connection between the mouse body and an electronic device, such as a laptop computer (for example, through a USB interface). Of course, it is also possible that a rechargeable battery is detachably installed in the mouse body, and the battery is electrically connected to the main control board in the mouse body, thereby powering the eye movement detection module 2 in the storage state.

[0107] Among them, when the eye movement detection module is in the storage state, since the eye movement detection module 2 and the main control board of the mouse body 1 are electrically connected through the second signal connector 212 and the first signal connector 152 (for example, to realize the main function-charging), at this time, the elastic arm assembly 13 is in the first position, and the second elastic arm portion forms a squeezing force on the first signal connection module, so that the first signal connector on it can be electrically connected with the second signal connector of the eye movement detection module. Therefore, in order to enable the first signal connector to be electrically connected with the main control board, a terminal assembly of metal material is embedded inside the second elastic arm portion 132 close to the first signal connection module side, and the two ends of the terminal assembly are exposed on the surface of the second elastic arm portion. Therefore, when the end of the second elastic arm portion abuts on the first signal connection module, the first end of the terminal assembly exposed on its surface abuts against the first signal connector 152 on the first signal connection module, and the second end of the terminal assembly is electrically connected to the main control board through a connecting line. Therefore, when the eye movement detection module is in the removal state, when the second end of the elastic arm assembly is in the second position, the second elastic arm portion moves to the second position, so that the first end of the terminal assembly is disconnected from the first signal connector.

[0108] When the eye movement detection module 2 is in the removed state, since the first detection module body 22 and the second detection module body 23 are rotatably connected, they can be clamped on the casing of an electronic device (such as a laptop computer) after adjusting the angle between the two, which is convenient for users to adjust the angle when interacting with the user, and since the second signal connection module 21 of the first detection module 22 is also provided with a second magnetic component 213, the second magnetic component 213 can be magnetically fixedly connected to the casing of the electronic device.

[0109] When the eye movement detection module 2 is in the removed state, it can be connected to the electronic device with a signal extension function. Specifically, the second signal connector 212 on the eye movement detection module can be connected to the electronic device by wire through an external connector, thereby realizing signal interaction between the eye movement detection module and the electronic device. Of course, it is also possible that a wireless module (including but not limited to a Bluetooth module) electrically connected to its control unit is integrated inside the eye movement detection module, and a wireless signal connection with the electronic device is realized by setting the wireless module.

[0110] According to one embodiment of the present disclosure, the present invention also provides an eye-movement cursor interaction system, including an electronic device, the electronic device has a second camera module, and also includes the mouse described above, wherein when the eye-movement detection module 2 is in a removed state, the eye-movement detection module can be detachably installed on the electronic device.

[0111] Wherein, electronic equipment includes but is not limited to a laptop computer with a signal extension option, and the laptop computer comes with a camera (i.e., a second camera module). Due to the rotational connection between the first detection module body 22 and the second detection module body 23 of the eye movement detection module 2, the angle can be adjusted between the first / second detection module body, so as to be clamped on the housing of the laptop computer, and because the eye movement detection module has a magnetic suction piece, it can be further magnetically fixedly connected with the metal housing of the laptop computer. The eye movement cursor interaction system with the mouse of the present application and the electronic device can also be realized, and when the eye movement detection module is in a taken-out state, it can be connected with the electronic device signal to realize the eye movement control cursor, and the mouse can be more flexibly operated and controlled by combining the eye movement with the finger operation mode, which can not only alleviate the fatigue damage of the user's wrist for a long time, but also take into account the use needs of ordinary users and specialized users. In addition, the mouse of the present application provides the user with a more natural and smoother interaction mode by mobilizing the coordinated operation of the user's eyes and hands, thereby improving the use efficiency and experience.

[0112] According to an embodiment of the present disclosure, the present invention further provides a method, which is applied to the eye-movement cursor interaction system, and the method comprises:

[0113] Step S100, obtaining the position of the user's pupil on the display screen;

[0114] Step S200, locking and connecting the user's pupil and the mouse cursor in sync;

[0115] Step S300, locating the user's head position and calibrating the user's pupil center position;

[0116] Step S400, determining whether to change the user usage mode;

[0117] If yes, step S500, the electronic device executes the fine operation mode; if no, step S600, the electronic device continues to execute the normal mode;

[0118] In step S510, when the electronic device is in the fine operation mode, the application software icon on the display screen is in a state that can be activated by the eye movement cursor, and if the eye movement control cursor stays on a certain application software and exceeds a threshold, the application software is activated and the eye movement cursor range is controlled within the display interface range of the application software;

[0119] Step S520 , in response to a touch operation input by the user via the touch operation module, implementing the touch operation on the display interface.

[0120] For example, in the present application, the electronic device includes but is not limited to a laptop computer. The present application provides a novel method for interacting and controlling the screen interface information between a user and a laptop computer by combining an active eye movement detection module with the operation of a mouse button. After the eye movement detection module is removed from the mouse body, the second signal connector of the eye movement detection module is connected to the display part by signal, and the control unit operates to display the pixel information of the display area and the maximum size of the displayable area.

[0121] In this application, before obtaining the position of the user's pupil on the display screen, it is necessary to determine that the eye movement detection module is connected to the extended signal of the electronic device and to the second camera module of the electronic device. The eye movement detection module is directly connected to the screen signal to realize the eye tracking cursor control of activating the display screen. This application uses an infrared light source to cause a dark pupil effect. At this time, the pupil is darker and the iris is brighter, which makes it easy to segment the pupil area to achieve eye tracking. The camera of the eye movement detection module itself is used to capture the eye image, and the camera of the electronic device is used to locate the head position to calibrate the eye's gaze point and cursor position on the screen.

[0122] In this application, the user's pupil activity is detected by an eye movement detection module to achieve natural eye movement to directly control the position of the mouse cursor on the screen, replacing the traditional mouse photoelectric sensor displacement. In addition to eye movement control of cursor movement, the electronic device also has eye movement control combined with the conventional operation of the left and right buttons and scroll wheel of the mouse body.

[0123] The components of the eye movement detection module of the present application are described as follows: a first camera module (i.e., an infrared camera (infrared optical sensor)), an infrared sensing module (with an infrared light source emitter) and a system signal control unit (i.e., a control unit), wherein the infrared camera of the eye detection module needs to be matched with the display screen camera of the electronic device (i.e., the second camera module) to set the eye movement control and set the correspondence between the pupil position and the screen pixel position.

[0124] Background of eye tracking technology: Infrared light source is used to induce dark pupil effect, in which the pupil is darker and the iris is brighter, making it easier to segment the pupil area and thus achieve eye tracking. The camera of the module itself is used to capture the eye image, and the display screen camera is used to locate the head position to calibrate the eye's gaze point on the screen and the cursor position.

[0125] In this embodiment, step S100, obtaining the position of the user's pupil on the display screen includes: obtaining the first pupil center position input by the eye movement detection module, taking the user's corneal reflection point as the base point of the relative position of the first camera module and the eyeball, and obtaining the line of sight vector coordinates according to the first pupil center position, thereby determining the human eye gaze point, and the position of the gaze point is the position of the user's pupil on the display screen.

[0126] Specifically, the first camera module is used to capture the user's eye image, and the control unit is used to process the image to obtain the first pupil center position; then the corneal reflection point is used as the base point of the relative position of the first camera module and the eyeball, and the line of sight vector coordinates can be obtained based on the first pupil center position, thereby determining the gaze point of the human eye.

[0127] In this embodiment, step S300, locating the user's head position and calibrating the user's pupil center position includes:

[0128] Step S310, obtaining a first image captured by a second camera module when the user's head is gazing at a calibration point in the center of the display screen, performing facial recognition on the first image, and obtaining a pupil position;

[0129] Step S320, determining a second image and determining the second pupil center position of the user at this time according to the second image; the second image is an image taken by the first camera module when the user's head remains still and the user's eyes are focused on the calibration point at the center of the display screen, reflecting the infrared light source;

[0130] Step S330, projecting the second pupil center position onto the display screen area to coincide with the pupil position, and performing center position calibration;

[0131] Step S340, determining a control path of the eye-movement controlled cursor.

[0132] In this embodiment, in step S330, during position calibration, the user keeps his head still and looks at the calibration point in the center of the screen. The infrared camera of the eye movement detection module is started, and an infrared light source is emitted into the user's pupil. The light source reflected from the eye is identified by the camera infrared sensor to determine the position of the user's pupil at this time. The control unit performs signal conversion, and projects the center position of the pupil to the display screen area through the internal signal control software and performs debugging to make it coincide with the pupil position of the facial features captured by the second camera module, and perform center position calibration.

[0133] In this embodiment, step S340, determining the control path of the eye-control cursor includes: recording the time taken by the eye-control cursor to scan from the center position to the edges of the display screen, thereby determining the time setting for the eye-control cursor to move.

[0134] For example, in step S340, the user scans from the center point, up, down, left, and right in accordance with the habitual eye movement, records the time taken for the action, sets the time for the eye movement control cursor to move from the center position to the top edge of the display screen as t1, the time for the eye movement control cursor to move from the center position to the bottom edge of the display screen as t2, the time for the eye movement control cursor to move from the center position to the left edge of the display screen as t3, and the time for the eye movement control cursor to move from the center position to the right edge of the display screen as t4; according to the user's eye habits, the time taken for the eye movement control cursor to move is set and the operation is performed. For example, the distance the cursor moves toward the top of the display screen is set to d1, and the actual corresponding screen pixel relationship Px pixel = a (d1 / t1), where a is a fixed coefficient.

[0135] For example, in step 500, when the electronic device is in the fine operation mode, in response to the button start signal input by the user via the activation button, the professional user fine operation mode is started, the electronic device controls the switching of the eye movement signal, the eye movement cursor controls the movement range conversion, the eye movement cursor controls the application software interface activation area of ​​the display screen, and in response to the touch operation input by the user via the touch fine operation module, the touch operation is implemented on the application software interface.

[0136] When the electronic device is in fine operation mode, in addition to maintaining the function of the right mouse function button operation area in the ordinary user mode, the user mode switching is activated by pressing the activation button, and the eye control method is changed through the internal software. The eye control cursor movement is only effective for the current professional software in use and the operation interface of the screen interface. At the same time, the finger is used in the touch area of ​​the left touch operation module for fine operation control. The user operates the touch area of ​​the left touch operation module to perform specific professional software operations. When it is detected that the user's finger has no operation gesture in the left touch area, the display interface system prompts whether to switch the eye control mode, and the user clicks the OK button of the mouse to make a selection.

[0137] In this embodiment, in step S600, when the electronic device is in the normal mode, the cursor is controlled by eye movement to browse pages and files, and the change and movement of the browsing content of the current page are controlled by the direction of the eye movement track.

[0138] Among them, step S600, when the electronic device is in normal mode, the user controls the cursor to browse the display area by eye movement, and the left and right mouse buttons are operated by finger clicking according to the needs of use. Therefore, when the electronic device is in normal mode, in conjunction with the left / right mouse button, the mouse cursor is locked and connected synchronously with the user's eye pupil, and the eye movement controls the cursor movement. When operation is required, the left / right mouse button is used to confirm. For example, the user's eye movement control cursor mode can be set to the default state, and pages and files are browsed, mainly by controlling the changes and movements of the current page browsing content as a whole through the direction of the eye movement trajectory. In this mode, the cursor position defaults to the upper left area of ​​the open page. If the user's eye movement trajectory is to the right, in this case, the screen cursor controls the browsing page content information on the right side to be gradually displayed, and other viewing angles are similar. The information display pixel changes are based on the corresponding relationship between the eye movement and the cursor movement distance statistically calculated under the initial setting conditions.

[0139] In this embodiment, it is initially set that, based on the user's eye habits, when the user's eyes continue to gaze at a certain area of ​​the display page for a period of tx, the cursor position control is triggered. Once the user confirms the command information, the screen cursor will be fixed in the command control state, and the eye movement detection signal connection will be invalid. During this period, the user's eye movement is in a non-detected state until the user gazes at another menu command area of ​​the page again for a period of time exceeding the set period tx, the cursor position control is triggered, and the cursor signal control is implemented, the cursor of the previous step is moved to the position where the user is looking and assists the mouse operation.

[0140] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0141] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0142] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A mouse, comprising a mouse body, characterized in that: It also includes an eye movement detection module, and the mouse body is configured to hold the eye movement detection module so that the eye movement detection module is in a storage state; Wherein, when the eye movement detection module is in the storage state, the eye movement detection module is connected to the mouse body signal; When the eye movement detection module is in a state of being removed from the mouse body, the eye movement detection module is configured to be able to connect with the electronic device signal so that the user controls the mouse cursor to operate on the electronic device through eye movement.

2. The mouse according to claim 1, characterized in that: The eye movement detection module is configured to allow manual or electronic control to switch between a storage state and a removal state.

3. The mouse according to claim 2, characterized in that: The mouse body is formed with a receiving cavity with an opening, the eye movement detection module is received in the receiving cavity and is detachably and elastically connected to the mouse body, so as to realize the switching of the eye movement detection module between a storage state and a removal state.

4. The mouse according to claim 3, characterized in that: The mouse body also includes: A limiting groove is arranged below the accommodating cavity and communicated with the accommodating cavity; An elastic arm assembly is movably connected between the accommodating cavity and the limiting groove, wherein a first side of the elastic arm assembly abuts against the eye movement detection module in the accommodating cavity, and a second side is connected to the limiting groove through an elastic member; A first signal connection module is movably mounted on the cavity wall of the accommodating cavity; When the eye movement detection module is in a stored state, the elastic arm assembly is in a first position, and when the eye movement detection module is in a removed state, the elastic arm assembly is in a second position; when the elastic arm assembly is in the first position, the elastic arm assembly forms an extrusion force on the first signal connection module to push the first signal connection module from an initial position toward the eye movement detection module, thereby establishing a signal connection with the second signal connection module of the eye movement detection module; when the elastic arm assembly is in the second position, the elastic arm assembly releases the extrusion of the first signal connection module to allow the first signal connection module to return to the initial position.

5. The mouse according to claim 4, characterized in that: The elastic arm assembly comprises: A first elastic arm portion is disposed below the accommodating cavity, wherein a first side of the first elastic arm portion abuts against the eye movement detection module, and a second side of the first elastic arm portion is connected to the limiting groove through an elastic member; A second elastic arm portion is disposed in the interlayer space between the accommodating cavity and the limiting groove, and a first end of the second elastic arm portion is connected to the first elastic arm portion; The limiting portion is arranged on the outer side of the second elastic arm portion, and the limiting portion is accommodated in the limiting hole opened in the limiting groove. The limiting portion is used to limit the elastic arm assembly at the second position when the elastic arm assembly moves from the first position to the second position.

6. The mouse according to claim 4, characterized in that: The mouse body further comprises a first magnetic attraction member, wherein the first magnetic attraction member is mounted on the first signal connection module, and correspondingly, the second signal connection module is provided with a second magnetic attraction member at a corresponding position; The mouse body also includes: An activation button, mounted on the shell of the mouse body; A touch operation module is installed on the shell of the mouse body; A main control board is installed in the shell of the mouse body, and the main control board is electrically connected to the activation button, the touch operation module, the first signal connection module and the eye movement detection module.

7. The mouse according to any one of claims 1 to 6, characterized in that: The eye movement detection module comprises: The first detection module body, The second detection module body is rotatably connected to the first detection module body, A first camera module is installed on the second detection module body; An infrared sensing module, installed on the second detection module body; A second signal connection module, installed on the first detection module body; The control unit is installed in the main body of the first detection module, and is used for signal connection with the first camera module, the infrared sensing module, the second signal connection module, the main control board and the electronic equipment.

8. An eye-movement cursor interaction system, comprising an electronic device, wherein the electronic device has a second camera module, characterized in that: It also includes the mouse according to any one of claims 1 to 7, wherein when the eye movement detection module is in a removed state, the eye movement detection module can be detachably installed on the electronic device.

9. A method, applied to the eye-cursor interaction system of claim 8, characterized in that: The method comprises, Get the position of the user's pupil on the display screen; Synchronize the user's pupil and mouse cursor lock connection; Locate the user's head position and calibrate the user's pupil center position; Determine whether to change the user usage mode; If yes, the electronic device enters the fine operation mode; if no, the electronic device enters the normal mode; When the electronic device is in the fine operation mode, the application software icon on the display screen is in a state that can be activated by the eye movement cursor. If the eye movement control cursor stays on a certain application software and exceeds the threshold, the application software is activated and the eye movement cursor range is controlled within the display interface range of the application software; In response to a touch operation input by a user via a touch operation module, the touch operation is implemented on the display interface.

10. The method according to claim 9, characterized in that: The obtaining of the position of the user's pupil on the display screen comprises: obtaining a first pupil center position input by an eye movement detection module, taking a user's corneal reflection point as a base point of the relative position of the first camera module and the eyeball, obtaining a sight line vector coordinate according to the first pupil center position, thereby determining a gaze point of the human eye, wherein the position of the gaze point is the position of the user's pupil on the display screen; The positioning of the user's head position and the calibration of the user's pupil center position include: Acquire a first image captured by the second camera module when the user's head is gazing at a calibration point in the center of the display screen, perform facial recognition on the first image, and obtain a pupil position; Determine a second image and determine the second pupil center position of the user at this time according to the second image; the second image is an image of the infrared light reflected by the eyes when the user's head remains still and the user's eyes are focused on the calibration point at the center of the display screen, taken by the first camera module; Projecting the second pupil center position onto the display screen area to coincide with the pupil position, and performing center position calibration; Determine the control path of the eye-controlled cursor; When the electronic device is in normal mode, eye movement controls the cursor to browse pages and files, and the change and movement of the browsing content of the current page are controlled by the direction of the eye movement trajectory.

Images