An Unmanned Equipment Control Method and System for Hybrid Eye-Teeth Interaction

Abstract

The present invention discloses a control method and system for an unmanned equipment with eye-tooth hybrid interaction. The method includes: matching eye movement action data with a preset eye movement interaction rule to obtain an interface control instruction associated with the eye movement action data, so as to move a cursor within a corresponding area of a user interaction interface; obtaining eyelid data of the current user's eyes; judging whether the user has a normal eye-closed state within a preset time and whether the number of normal eye-closed times corresponding to the normal eye-closed state is greater than a preset threshold; if the user has a normal eye-closed state within the preset time and the number of normal eye-closed times corresponding to the normal eye-closed state is greater than the preset threshold, stopping generating the interface control instruction to make the cursor stagnate at a target position in the corresponding area; generating a control instruction for the target position based on the obtained tooth movement of the current user. The problem that the use scenario is limited due to the cooperation through operations such as gestures or remote controls during the interaction process is solved.

Description

Technical Field

[0001] The present invention belongs to the field of virtual control technology, and in particular relates to an unmanned equipment control method and system with eye-tooth hybrid interaction. Background Art

[0002] The eye-movement interaction method of augmented reality glasses can select, move and mark targets through the eyes, but further operations on the targets are limited and need to be coordinated through gestures or remote controls. In some special scenarios (such as a single soldier holding a gun with both hands), it is difficult to perform quick operations. Summary of the invention

[0003] The present invention provides an unmanned equipment control method and system with eye-teeth hybrid interaction, which is used to solve the technical problem of limited usage scenarios caused by cooperation through operations such as gestures or remote controls during the interaction process.

[0004] In a first aspect, the present invention provides an unmanned equipment control method for eye-teeth mixed interaction, comprising: obtaining eye movement action data of the current user's eyes; matching the eye movement action data with preset eye movement interaction rules to obtain interface control instructions associated with the eye movement action data, so as to move a cursor in a corresponding area of the user interaction interface; obtaining eyelid data of the current user's eyes within a preset time of obtaining the interface control instructions, wherein the eyelid data includes the user's normal eye closing state and the number of normal eye closing times corresponding to the normal eye closing state; judging whether the user has a normal eye closing state within the preset time and whether the number of normal eye closing times corresponding to the normal eye closing state is greater than a preset threshold; if the user has a normal eye closing state within the preset time and the number of normal eye closing times corresponding to the normal eye closing state is greater than a preset threshold, stopping generating interface control instructions and causing the cursor to stay at a target position in the corresponding area; generating control instructions for the target position based on the acquired tooth movements of the current user.

[0005] Furthermore, the obtaining of the eye movement data of the current user's eyes includes: obtaining three-dimensional eye movement data information of the user during the interaction process; mapping the three-dimensional eye movement data information to obtain two-dimensional eye movement data information corresponding to the three-dimensional eye movement data information; processing the two-dimensional eye movement data according to preset judgment rules to obtain eye movement data.

[0006] Further, the preset eye movement interaction rules include generating a cursor up control instruction when the eyeball moves upward, generating a cursor down control instruction when the eyeball moves downward, generating a cursor left control instruction when the eyeball moves left, and generating a cursor right control instruction when the eyeball moves right; matching the eye movement action data with the preset eye movement interaction rules to obtain an interface control instruction associated with the eye movement action data, including: matching the upward eye movement action data with the preset eye movement interaction rules to obtain a cursor up control instruction associated with the upward eye movement action data; matching the downward eye movement action data with the preset eye movement interaction rules to obtain a cursor down control instruction associated with the downward eye movement action data; matching the left eye movement action data with the preset eye movement interaction rules to obtain a cursor left control instruction associated with the left eye movement action data; matching the right eye movement action data with the preset eye movement interaction rules to obtain a cursor right control instruction associated with the right eye movement action data.

[0007] Further, after determining whether the user has a normal eye-closed state within a preset time and whether the number of normal eye-closed times corresponding to the normal eye-closed state is greater than a preset threshold, the method further includes: if the user has a normal eye-closed state within the preset time and the number of normal eye-closed times corresponding to the normal eye-closed state is not greater than the preset threshold, suspending the generation of interface control instructions, and continuing to generate interface control instructions based on subsequent eye movement action data after a preset time interval.

[0008] Further, after determining whether the user has a normal eye-closed state within a preset time and whether the number of normal eye-closed times corresponding to the normal eye-closed state is greater than a preset threshold, the method further includes: if the user does not have a normal eye-closed state within the preset time, directly generating a control instruction for the target position based on the obtained tooth movement of the current user.

[0009] Further, the control instructions include a determination instruction, a cancellation instruction, an identification instruction, a maneuver instruction, and a reconnaissance instruction; generating a control instruction for the target position based on the obtained tooth movement of the current user includes: generating a determination instruction for the target position based on the obtained left tooth bite movement of the current user; generating an identification instruction for the target position based on the obtained continuous fast tooth bite movement of the current user; generating a maneuver instruction for the target position based on the obtained long-time slow tooth bite movement of the current user; generating a reconnaissance instruction for the target position based on the obtained intermittent fast tooth bite movement of the current user.

[0010] Second aspect, the present invention provides an unmanned equipment control system for eye-tooth hybrid interaction, including: a first acquisition module configured to acquire eye movement action data of the current user's eyes; a movement module configured to match the eye movement action data with a preset eye movement interaction rule to obtain an interface control instruction associated with the eye movement action data, so as to move a cursor within a corresponding area of the user interface; a second acquisition module configured to acquire eyelid data of the current user's eyes within a preset time after obtaining the interface control instruction, where the eyelid data includes the user's normal closed-eye state and the corresponding number of normal eye closures; a judgment module for judging whether the user has a normal closed-eye state within a preset time and whether the number of normal eye closures corresponding to the normal closed-eye state is greater than a preset threshold; a generation module configured to, if the user has a normal closed-eye state within a preset time and the number of normal eye closures corresponding to the normal closed-eye state is greater than a preset threshold, stop generating interface control instructions, so that the cursor stays at a target position in the corresponding area; and a control module configured to generate a control instruction for the target position based on the acquired tooth action of the current user.

[0011] Third aspect, there is provided an electronic device, which includes: at least one processor, and a memory communicatively connected to the at least one processor, where the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps of an unmanned equipment control method for eye-tooth hybrid interaction according to any embodiment of the present invention.

[0012] Fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program instructions are executed by a processor, the processor is enabled to execute the steps of an unmanned equipment control method for eye-tooth hybrid interaction according to any embodiment of the present invention.

[0013] An unmanned equipment control method and system for eye-tooth hybrid interaction according to the present application controls the movement of the cursor on the user interface through eye movement actions, and at the same time generates corresponding control instructions according to tooth actions, solving the problem of limited usage scenarios caused by cooperation through gestures or remote controls during the interaction process. And if the user has a normal closed-eye state within a preset time and the number of normal eye closures corresponding to the normal closed-eye state is greater than a preset threshold, the generation of interface control instructions is stopped, and the cursor stays at the target position in the corresponding area. At this time, the cursor will not move again with the movement of the eyeball, achieving the purpose of pre-selecting the target position, so that the convenience of interaction can be improved by increasing the time of eye-tooth hybrid interaction. Description of the Drawings

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required in the description of the embodiments. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0015] Figure 1 It is a flowchart of a method for controlling an unmanned equipment with eye-tooth hybrid interaction provided by an embodiment of the present invention;

[0016] Figure 2 It is a flowchart of a specific embodiment of a method for controlling an unmanned equipment with eye-tooth hybrid interaction provided by an embodiment of the present invention;

[0017] Figure 3 It is a structural block diagram of a control system for an unmanned equipment with eye-tooth hybrid interaction provided by an embodiment of the present invention;

[0018] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Specific Embodiments

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention fall within the scope of protection of the present invention.

[0020] Embodiment 1

[0021] Please refer to Figure 1 , which shows a flowchart of a method for controlling an unmanned equipment with eye-tooth hybrid interaction of the present application.

[0022] As Figure 1 shown, the method for controlling an unmanned equipment with eye-tooth hybrid interaction of the present invention specifically includes Step S101 - Step S106.

[0023] Step S101, obtain the eye movement action data of the current user's eyes.

[0024] In this embodiment, obtain the three-dimensional eye movement data information of the user during the interaction; perform mapping processing on the three-dimensional eye movement data information to obtain the two-dimensional eye movement data information corresponding to the three-dimensional eye movement data information; process the two-dimensional eye movement data according to a preset judgment rule to obtain the eye movement action data.

[0025] Specifically, the two-dimensional eye movement data information refers to the screen fixation point coordinate information of the user during the interaction process. In addition to the coordinate information of the user's screen fixation point in the screen coordinate system, the two-dimensional eye movement data information also includes time scale information, where the time scale information refers to the time when the two-dimensional eye movement data information is obtained.

[0026] Since it is difficult for the user to fixate on a single point every time, and even if the user fixates on a point, the time scale information in the two-dimensional eye movement data information will change at this time, that is, it is impossible to judge the eye movement action information through a single two-dimensional eye movement data information; therefore, the eye movement trajectory information of the user at this time can be judged based on multiple two-dimensional eye movement data information, and then the eye movement trajectory information is matched with the preset eye movement action information database to obtain the eye movement action information corresponding to the eye movement trajectory information.

[0027] The preset judgment rule specifically refers to judging the eye movement trajectory information jointly composed of multiple two-dimensional eye movement data information based on multiple two-dimensional eye movement data information of the user during the interaction process, and then matching the eye movement trajectory information with the preset eye movement action information database to obtain the eye movement action information.

[0028] For example, multiple two-dimensional eye movement data information obtained during the interaction process of the user are respectively 、 , then the trajectory formed by in the screen coordinate system and in the screen coordinate system is the eye movement trajectory information jointly composed of the multiple two-dimensional eye movement data information.

[0029] Step S102, match the eye movement action data with the preset eye movement interaction rules to obtain an interface control instruction associated with the eye movement action data, and move the cursor within the corresponding area of the user interaction interface.

[0030] In this embodiment, the preset eye movement interaction rules include generating a cursor up control instruction when the eyeball moves up, generating a cursor down control instruction when the eyeball moves down, generating a cursor left control instruction when the eyeball moves left, and generating a cursor right control instruction when the eyeball moves right.

[0031] It should be noted that, based on the eyeball up movement action data and the preset eye movement interaction rules, a cursor up control instruction associated with the eyeball up movement action data is obtained; based on the eyeball down movement action data and the preset eye movement interaction rules, a cursor down control instruction associated with the eyeball up movement action data is obtained; based on the eyeball left movement action data and the preset eye movement interaction rules, a cursor left control instruction associated with the eyeball up movement action data is obtained; based on the eyeball right movement action data and the preset eye movement interaction rules, a cursor right control instruction associated with the eyeball up movement action data is obtained.

[0032] Accordingly, the cursor is controlled to move within the corresponding area of the user interaction interface according to the cursor up control instruction, the cursor down control instruction, the cursor left control instruction, and / or the cursor right control instruction.

[0033] Step S103: Obtain the eyelid data of the current user's eyes within a preset time after obtaining the interface control instruction, where the eyelid data includes the user's normal closed-eye state and the corresponding number of normal eye closures.

[0034] In this embodiment, when the cursor is controlled to move within the corresponding area of the user interaction interface by the generated cursor up control instruction, cursor down control instruction, cursor left control instruction, and / or cursor right control instruction, the eyelid data of the current user's eyes is obtained in real time.

[0035] Step S104: Determine whether the user has a normal closed-eye state within the preset time and whether the corresponding number of normal eye closures is greater than a preset threshold.

[0036] In this embodiment, by obtaining the eyelid data of the user's eyes, it is determined whether the user has a normal closed-eye state within the preset time and whether the corresponding number of normal eye closures is greater than a preset threshold.

[0037] Specifically, if the user has a normal closed-eye state within the preset time and the corresponding number of normal eye closures is not greater than the preset threshold, generating the interface control instruction is paused, and after a preset time interval, the interface control instruction is continued to be generated based on subsequent eye movement action data.

[0038] For example, when the user's eyeball moves to the right, at this time, the eyeball right movement action data is generated. The eyeball right movement action data is matched with the preset eye movement interaction rule to obtain the cursor right control instruction associated with the eyeball right movement action data, so that the cursor moves to the right. Within 1 s after the cursor moves, the user has a normal closed-eye state, but the number of normal eye closures is only 1 time, which is less than the preset threshold of 3 times. At this time, generating the cursor right control instruction is paused, and at the same time, the cursor stays at the current position. The user's eyeball can return to the normal position, and the cursor will not move synchronously. After 1 s, the user's eyeball moves to the right again, and the corresponding cursor right control instruction is generated, so that the cursor moves to the right again. This facilitates the user to control the cursor movement by eyeball movement, thereby reducing the fatigue caused by continuous rotation of the user's eyeball.

[0039] Further, if the user does not have a normal closed-eye state within the preset time, the control instruction for the target position is directly generated based on the obtained tooth movement of the current user.

[0040] Step S105, if the user has a normal eye-closed state within a preset time and the number of normal eye-closed times corresponding to the normal eye-closed state is greater than a preset threshold, stop generating an interface control instruction and make the cursor stay at the target position in the corresponding area.

[0041] In this embodiment, if the user has a normal eye-closed state within a preset time and the number of normal eye-closed times corresponding to the normal eye-closed state is greater than a preset threshold, stop generating an interface control instruction and make the cursor stay at the target position in the corresponding area. At this time, the cursor will not move again with the movement of the eyeball, achieving the purpose of pre-selecting the target position. Thus, by increasing the time of eye-tooth hybrid interaction, the convenience of interaction can be improved.

[0042] For example, in an application scenario, it is required that the user generate a confirmation instruction at the target position at 11 o'clock. At this time, the user uses eye movement interaction at 10:55 to make the cursor stay at the target position in the corresponding area. Five minutes later, the user generates a confirmation instruction for the target position through tooth movement. In this way, although the preparation time for interaction is increased, the accuracy and convenience of interaction can be improved.

[0043] Step S106, based on the obtained tooth movement of the current user, generate a control instruction for the target position.

[0044] In this embodiment, the control instructions include a confirmation instruction, a cancellation instruction, an identification instruction, a maneuver instruction, and a reconnaissance instruction.

[0045] Specifically, generating a control instruction for the target position based on the obtained tooth movement of the current user includes: generating a confirmation instruction for the target position based on the obtained left tooth-biting movement of the current user; generating an identification instruction for the target position based on the obtained continuous fast tooth-biting movement of the current user; generating a maneuver instruction for the target position based on the obtained long-time slow tooth-biting movement of the current user; generating a reconnaissance instruction for the target position based on the obtained intermittent fast tooth-biting movement of the current user.

[0046] In summary, for the method of this embodiment, if the user has a normal eye-closed state within a preset time and the number of normal eye-closed times corresponding to the normal eye-closed state is greater than a preset threshold, stop generating an interface control instruction and make the cursor stay at the target position in the corresponding area. At this time, the cursor will not move again with the movement of the eyeball, achieving the purpose of pre-selecting the target position. Thus, by increasing the time of eye-tooth hybrid interaction, the convenience of interaction can be improved.

[0047] Embodiment Two

[0048] The second embodiment of the present invention also proposes a control method for unmanned equipment with eye-tooth hybrid interaction. The control method for unmanned equipment with eye-tooth hybrid interaction is implemented by software and / or hardware, and the method specifically includes steps S201 - S203.

[0049] As Figure 2 shown, in step S201, use eye movements to quickly identify the target on the augmented / virtual reality glasses, and send relevant instructions through tooth movements. The operation of the unmanned system / equipment can be realized without too much operation;

[0050] In step S202, feedback can be given on the augmented / virtual reality glasses and the intelligent dental retainer when receiving special information.

[0051] Especially, the vibration feedback on the dental retainer can inform the user of the feedback in the first time, which has good timeliness and concealment in military applications;

[0052] In step S203, through different combinations of movements, an instruction set can be formed, and the system can be quickly controlled through facial movements to complete specified tasks.

[0053] Unmanned equipment mainly refers to unmanned robotic dogs, unmanned vehicles, unmanned aerial vehicles, unmanned boats, unmanned robots and other equipment; unmanned systems mainly refer to systems composed of multiple unmanned equipment, where the unmanned equipment can be one or more types. After receiving the control instruction, the entire system is controlled as a whole according to the instruction;

[0054] Eye movements mainly include up, down, left, right, etc.

[0055] Tooth movements mainly include: left bite, right bite, simultaneous bite, continuous fast bite, long slow bite, etc.

[0056] Unmanned system / equipment control instructions mainly include: confirmation instruction, cancellation instruction, identification instruction, maneuver instruction, reconnaissance instruction, strike instruction, etc.

[0057] Intelligent wearable glasses: AR glasses (augmented reality glasses) or MR glasses (mixed reality glasses).

[0058] Intelligent dental retainer: A dental device that can monitor the biting movements of different parts of the teeth and convert them into digital instructions, and can be interconnected with the intelligent wearable glasses in a wireless or wired form.

[0059] Communication equipment: A ground station or a handheld / backpack radio that can be interconnected with the intelligent wearable glasses or the unmanned system / equipment in a wired or wireless form. The communication equipment can communicate over long distances to meet the requirements of long-distance operation of the unmanned system / equipment.

[0060] Emergency state feedback: There are two feedback methods. One is to display through the user interface of the smart wearable glasses, and the other is to display through different vibration states of the smart tooth protector. The emergency states mainly include: attack, retreat, network failure, equipment failure, etc.

[0061] User interface: Display of the status of unmanned devices, position identification, display of operation buttons (maneuvering buttons, task buttons, etc.), situation display, etc.

[0062] In summary, the method of this embodiment can be applied to the single-soldier combat system in the military. Without affecting the soldiers' performance of conventional tactical actions, it can realize the control of unmanned systems and equipment. At the same time, it can timely sense and transmit information in a silent situation. And it can be applied to medical auxiliary communication and is suitable for communication occasions where speech and gestures are inconvenient.

[0063] Embodiment 3

[0064] Please refer to Figure 3 , which shows the structural block diagram of an unmanned equipment control system with eye-tooth hybrid interaction of the present application.

[0065] As Figure 3 shown, the unmanned equipment control system 200 includes a first acquisition module 210, a movement module 220, a second acquisition module 230, a judgment module 240, a generation module 250, and a control module 260.

[0066] Among them, the first acquisition module 210 is configured to acquire the eye movement action data of the current user's eyes; the movement module 220 is configured to match the eye movement action data with a preset eye movement interaction rule to obtain an interface control instruction associated with the eye movement action data, so as to move the cursor within a corresponding area of the user interface; the second acquisition module 230 is configured to acquire the eyelid data of the current user's eyes within a preset time after obtaining the interface control instruction, where the eyelid data includes the user's normal closed-eye state and the corresponding normal number of closed-eye times; the judgment module 240 judges whether the user has a normal closed-eye state within a preset time and whether the corresponding normal number of closed-eye times of the normal closed-eye state is greater than a preset threshold; the generation module 250 is configured to stop generating the interface control instruction if the user has a normal closed-eye state within a preset time and the corresponding normal number of closed-eye times of the normal closed-eye state is greater than the preset threshold, so that the cursor stays at the target position in the corresponding area; the control module 260 is configured to generate a control instruction for the target position based on the acquired tooth movement of the current user.

[0067] It should be understood that Figure 3 the modules recorded in Figure 1corresponds to each step in the described method. Thus, the operations, features, and corresponding technical effects described above for the method also apply to Figure 3 the various modules in, which will not be elaborated here.

[0068] Embodiment 4

[0069] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program instructions are executed by a processor, the processor is caused to execute the eye-teeth hybrid interaction-based unmanned equipment control method in any of the above method embodiments;

[0070] As an implementation manner, the computer-readable storage medium of the present invention stores computer-executable instructions, which are set as:

[0071] Obtain the eye movement action data of the current user's eyes;

[0072] Based on the matching of the eye movement action data with a preset eye movement interaction rule, obtain an interface control instruction associated with the eye movement action data, so as to move the cursor within a corresponding area of the user interface;

[0073] Obtain the eyelid data of the current user's eyes within a preset time after obtaining the interface control instruction, where the eyelid data includes the user's normal closed-eye state and the corresponding number of normal eye closures;

[0074] Determine whether the user has a normal closed-eye state within a preset time and whether the number of normal eye closures corresponding to the normal closed-eye state is greater than a preset threshold;

[0075] If the user has a normal closed-eye state within a preset time and the number of normal eye closures corresponding to the normal closed-eye state is greater than a preset threshold, stop generating the interface control instruction and make the cursor stay at the target position in the corresponding area;

[0076] Generate a control instruction for the target position based on the obtained tooth actions of the current user.

[0077] A computer-readable storage medium may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function. The data storage area may store data created according to the use of the unmanned equipment control system for eye-tooth hybrid interaction, etc. In addition, the computer-readable storage medium may include a high-speed random access memory, and may also include a memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the computer-readable storage medium may optionally include a memory remotely provided with respect to the processor, and these remote memories may be connected to the unmanned equipment control system for eye-tooth hybrid interaction through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0078] Figure 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As Figure 4 shown, the device includes: a processor 310 and a memory 320. The electronic device may further include: an input device 330 and an output device 340. The processor 310, the memory 320, the input device 330, and the output device 340 may be connected through a bus or other means. Figure 4 Taking the connection through the bus as an example. The memory 320 is the above-mentioned computer-readable storage medium. The processor 310 executes various functional applications and data processing of the server by running non-volatile software programs, instructions, and modules stored in the memory 320, that is, implements the unmanned equipment control method for eye-tooth hybrid interaction in the above method embodiment. The input device 330 may receive input digital or character information, and generate key signal inputs related to user settings and function controls of the unmanned equipment control system for eye-tooth hybrid interaction. The output device 340 may include a display device such as a display screen.

[0079] The above electronic device may execute the method provided by the embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method. For technical details not described in detail in this embodiment, reference may be made to the method provided by the embodiment of the present invention.

[0080] As an implementation manner, the above electronic device is applied to the unmanned equipment control system for eye-tooth hybrid interaction and is used for a client, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0081] Obtain eye movement action data of the current user's eyes;

[0082] Match the eye movement action data with a preset eye movement interaction rule to obtain an interface control instruction associated with the eye movement action data, so as to move the cursor within a corresponding area of the user interaction interface;

[0083] Obtain the eyelid data of the current user's eyes within a preset time after obtaining the interface control instruction, where the eyelid data includes the user's normal closed-eye state and the corresponding number of normal eye closures;

[0084] Determine whether the user has a normal closed-eye state within the preset time and whether the number of normal eye closures corresponding to the normal closed-eye state is greater than a preset threshold;

[0085] If the user has a normal closed-eye state within the preset time and the number of normal eye closures corresponding to the normal closed-eye state is greater than the preset threshold, stop generating the interface control instruction and make the cursor stay at the target position in the corresponding area;

[0086] Generate a control instruction for the target position based on the obtained tooth movements of the current user.

[0087] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solutions, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0088] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A control method for an unmanned equipment with hybrid eye-tooth interaction, characterized in that Including: Obtaining eye movement action data of the current user's eyes; Based on the matching of the eye movement action data with a preset eye movement interaction rule, obtaining an interface control instruction associated with the eye movement action data, so as to move the cursor within a corresponding area of the user interaction interface; Obtaining eyelid data of the current user's eyes within a preset time after obtaining the interface control instruction, where the eyelid data includes the user's normal closed-eye state and the corresponding number of normal eye closures corresponding to the normal closed-eye state; Judging whether the user has a normal closed-eye state within a preset time and whether the number of normal eye closures corresponding to the normal closed-eye state is greater than a preset threshold; If the user has a normal closed-eye state within a preset time and the number of normal eye closures corresponding to the normal closed-eye state is greater than a preset threshold, stop generating the interface control instruction, and make the cursor stagnate at the target position in the corresponding area; Based on the obtained tooth movement of the current user, generating a control instruction for the target position; Wherein, after judging whether the user has a normal closed-eye state within a preset time and whether the number of normal eye closures corresponding to the normal closed-eye state is greater than a preset threshold, the method further includes: If the user has a normal closed-eye state within a preset time and the number of normal eye closures corresponding to the normal closed-eye state is not greater than a preset threshold, pause generating the interface control instruction, and continue to generate the interface control instruction based on subsequent eye movement action data after a preset time interval; If the user does not have a normal closed-eye state within a preset time, directly generate a control instruction for the target position based on the obtained tooth movement of the current user.

2. The control method for an unmanned equipment with eye-tooth hybrid interaction according to claim 1, characterized in that, The obtaining of the eye movement action data of the current user's eyes includes: Obtaining three-dimensional eye movement data information during the interaction process of the user; Performing a mapping process on the three-dimensional eye movement data information to obtain two-dimensional eye movement data information corresponding to the three-dimensional eye movement data information; Processing the two-dimensional eye movement data according to a preset judgment rule to obtain eye movement action data.

3. A method for controlling an unmanned equipment with eye-tooth hybrid interaction according to claim 1, characterized in that, Wherein, The preset eye movement interaction rule includes that an upward movement of the eyeball generates a cursor upward movement control instruction, a downward movement of the eyeball generates a cursor downward movement control instruction, a leftward movement of the eyeball generates a cursor leftward movement control instruction, and a rightward movement of the eyeball generates a cursor rightward movement control instruction; The matching of the eye movement action data with the preset eye movement interaction rule to obtain an interface control instruction associated with the eye movement action data includes: Based on the matching of the upward movement action data of the eyeball with the preset eye movement interaction rule, obtaining a cursor upward movement control instruction associated with the upward movement action data of the eyeball; Based on the matching of the downward movement action data of the eyeball with the preset eye movement interaction rule, obtaining a cursor downward movement control instruction associated with the downward movement action data of the eyeball; Based on the matching of the leftward movement action data of the eyeball with the preset eye movement interaction rule, obtaining a cursor leftward movement control instruction associated with the leftward movement action data of the eyeball; Based on the matching of the rightward movement action data of the eyeball with the preset eye movement interaction rule, obtaining a cursor rightward movement control instruction associated with the rightward movement action data of the eyeball.

4. A method for controlling an unmanned equipment with hybrid eye-teeth interaction according to claim 1, characterized in that, Wherein, The control instructions include a determination instruction, a cancellation instruction, an identification instruction, a maneuver instruction, and a reconnaissance instruction; Generating a control instruction for the target position based on the obtained tooth actions of the current user, includes: Generating a determination instruction for the target position based on the obtained left tooth biting action of the current user; Generating an identification instruction for the target position based on the obtained continuous fast tooth biting action of the current user; Generating a maneuver instruction for the target position based on the obtained long-term slow tooth biting action of the current user; Generating a reconnaissance instruction for the target position based on the obtained intermittent fast tooth biting action of the current user.

5. An unmanned equipment control system for hybrid eye-tooth interaction, characterized in that, Includes: A first acquisition module configured to acquire eye movement action data of the eyes of the current user; A movement module configured to match the eye movement action data with a preset eye movement interaction rule to obtain an interface control instruction associated with the eye movement action data, and move a cursor within a corresponding area of the user interaction interface; A second acquisition module configured to acquire eyelid data of the eyes of the current user within a preset time after obtaining the interface control instruction, where the eyelid data includes the normal eye closing state of the user and the number of normal eye closings corresponding to the normal eye closing state; A judgment module for judging whether the user has a normal eye closing state within a preset time and whether the number of normal eye closings corresponding to the normal eye closing state is greater than a preset threshold; A generation module configured to, if the user has a normal eye closing state within a preset time and the number of normal eye closings corresponding to the normal eye closing state is greater than a preset threshold, stop generating the interface control instruction and make the cursor stop at a target position in the corresponding area; A control module configured to generate a control instruction for the target position based on the obtained tooth actions of the current user; Wherein, after judging whether the user has a normal eye closing state within a preset time and whether the number of normal eye closings corresponding to the normal eye closing state is greater than a preset threshold, the system further includes: If the user has a normal eye closing state within a preset time and the number of normal eye closings corresponding to the normal eye closing state is not greater than a preset threshold, pause generating the interface control instruction, and continue to generate the interface control instruction based on subsequent eye movement action data after a preset time interval; If the user does not have a normal eye closing state within a preset time, directly generate a control instruction for the target position based on the obtained tooth actions of the current user.

6. An electronic device, characterized in that, Includes: At least one processor, and a memory communicatively connected to the at least one processor, where the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program, when executed by a processor, implements the method according to any one of claims 1 to 4.

Images