An operating method, system, medium, and computing device

Abstract

This application discloses an operation method, system, medium, and computing device. The operation method includes: acquiring the user's current partial iris image; determining the user's current gaze focus based on the current partial iris image; and if the user blinks, sending an operation command to a second terminal based on the blinking action information and the current gaze focus. In this embodiment, the user's current partial iris image is acquired using a first terminal, the user's current gaze focus is determined based on the image, and then, in conjunction with the user's blinking action, control operations on the second terminal are achieved. The entire interaction process is fast and convenient.

Description

Technical Field

[0001] This application relates to the field of interactive systems, and more particularly to an operating method, system, medium, and computing device. Background Technology

[0002] Currently, more and more smart terminals are appearing in people's daily lives, such as smart bracelets, smart glasses, smartphones, laptops, etc. Smart glasses can be worn directly on the user's eyes to intuitively display the screen. Therefore, smart bracelets, smartphones, laptops and other terminals can be connected to smart glasses to communicate and control smart bracelets, smartphones, laptops and other terminals through smart glasses, so as to provide users with a better user experience.

[0003] Therefore, current methods include: one is to install physical buttons on the smart glasses for control; the other is to use a voice system. Both of these methods are inconvenient for controlling other devices through smart glasses.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide an operating method, system, medium, and computing device, which aims to solve the problem mentioned in the background art that current methods for controlling other terminals through smart glasses are inconvenient.

[0006] To achieve the above objectives, this application provides an operating method applied to a first terminal, wherein the first terminal and a second terminal are communicatively connected, the first terminal includes a display device, and the first terminal is worn on the user's eyes. The operating method includes:

[0007] Obtain the user's current local iris image;

[0008] Based on the current local iris image, determine the user's current gaze focus;

[0009] If the user blinks, an operation command is sent to the second terminal based on the blinking action information and the current gaze focus.

[0010] In this embodiment of the application, obtaining the user's current local iris image includes:

[0011] The current local iris image is obtained by capturing images of the user's eyeballs within a preset area from a fixed direction.

[0012] In this embodiment of the application, determining the user's current gaze focus based on the current local iris image includes:

[0013] Based on the current local iris image, obtain the target image features;

[0014] The user's current gaze focus is determined based on the preset matching relationship between iris features and gaze focus, as well as the target image features.

[0015] In this embodiment of the application, the preset matching relationship between the iris features and the displayed image is obtained in the following way:

[0016] When the user's eyes scan the boundary of the display area of ​​the display device, the iris of the user within the preset area is captured from the fixed direction to obtain multiple boundary iris images;

[0017] Acquire an image of the iris region formed by the enclosed portions of the iris images;

[0018] The iris region image is divided into grids of a preset size to obtain multiple iris grid images;

[0019] Obtain the iris grid image features of each iris grid image;

[0020] The display area is divided into multiple display grids according to the number of iris grid images;

[0021] Obtain the center coordinates of each of the displayed grids;

[0022] The iris grid image features are matched with the center coordinates of the display grids to obtain the preset matching relationship between the iris features and the gaze focus;

[0023] Among them, the mutually matching iris grid image features and the center coordinates of the display grid represent the iris features of the current local iris image when the user's gaze is focused on the center coordinates, which are the iris grid image features that match the center coordinates.

[0024] In this embodiment of the application, determining the user's current gaze focus based on a preset matching relationship between iris features and gaze focus, and the target image features, includes:

[0025] From each of the iris grid image features, a target grid image feature that is identical to the target image feature is obtained;

[0026] Based on the target grid image features, determine the target coordinates in the display area that match the target grid image features;

[0027] Based on the target coordinates, the user's current gaze focus is determined.

[0028] In this embodiment of the application, the blinking action is determined based on the following method:

[0029] If the signal strength of the current local iris image decreases to below a preset threshold within a preset time period and then recovers, it is determined that the user blinked.

[0030] In this embodiment of the application, before sending the operation instruction to the second terminal, the operation method further includes:

[0031] Determine whether a preset wake-up command has been received;

[0032] If the preset wake-up command is received, an operation command is sent to the second terminal based on the blinking motion information and the current gaze focus;

[0033] After sending the operation command to the second terminal, the operation method further includes:

[0034] Determine whether a preset sleep command has been received. If the preset sleep command has been received, stop the action information based on the blinking action and the current gaze focus, and send an operation command to the second terminal.

[0035] This application also proposes an operating system for use in a first terminal, wherein the first terminal and a second terminal are communicatively connected, the first terminal includes a display device, and the first terminal is worn on a user's eye. The operating system includes:

[0036] The acquisition module is used to acquire the user's current local iris image;

[0037] The processing module is used to determine the user's current gaze focus based on the current local iris image;

[0038] Determine whether the user blinks. If the user blinks, send an operation command to the second terminal based on the blinking action information and the current gaze focus.

[0039] In this embodiment of the application, the acquisition module is further used for:

[0040] The current local iris image is obtained by capturing images of the user's eyeballs within a preset area from a fixed direction.

[0041] In this embodiment of the application, the processing module is further configured to:

[0042] Based on the current local iris image, obtain the target image features of the current local iris image;

[0043] The user's current gaze focus is determined based on the preset matching relationship between iris features and gaze focus, as well as the target image features.

[0044] In this embodiment of the application, the preset matching relationship between iris features and gaze focus is obtained in advance through the following initialization configuration:

[0045] When the user's eyes scan the boundary of the display area of ​​the display device, multiple boundary iris images are obtained by the acquisition module from the fixed direction of the user's iris within the preset area.

[0046] Acquire an image of the iris region formed by the enclosed portions of the iris images;

[0047] The iris region image is divided into grids of a preset size to obtain multiple iris grid images;

[0048] Obtain the iris grid image features of each of the aforementioned iris grid images;

[0049] The display area is divided into multiple display grids according to the number of iris grid images;

[0050] Obtain the center coordinates of each of the displayed grids;

[0051] The iris grid image features are matched with the center coordinates of the display grids to obtain the preset matching relationship between the iris features and the gaze focus;

[0052] Among them, the mutually matching iris grid image features and the center coordinates of the display grid represent the iris features of the current local iris image when the user's gaze is focused on the center coordinates, which are the iris grid image features that match the center coordinates.

[0053] In this embodiment of the application, the processing module is further configured to:

[0054] Target grid image features that are identical to the target image features are obtained by matching each of the iris grid image features;

[0055] Based on the target grid image features, determine the target coordinates in the display area that match the target grid image features;

[0056] Based on the target coordinates, determine the user's current gaze focus.

[0057] In this embodiment of the application, the processing module is further configured to:

[0058] If the signal strength of the current local iris image decreases to below a preset threshold within a preset time period and then recovers, it is determined that the user blinked.

[0059] In this embodiment of the application, before sending the operation command to the second terminal, the processing module is further configured to:

[0060] Determine whether a preset wake-up command has been received;

[0061] If the preset wake-up command is received, an operation command is sent to the second terminal based on the blinking motion information and the current gaze focus;

[0062] After sending the operation command to the second terminal, the processing module is further configured to:

[0063] Determine whether a preset sleep command has been received. If the preset sleep command has been received, stop the action information based on the blinking action and the current gaze focus, and send an operation command to the second terminal.

[0064] This application also proposes a storage medium on which a computer program is stored, which, when executed by a processor, implements the above-described method.

[0065] This application also proposes a computing device including a processor for implementing the above-described method when executing a computer program stored in a memory.

[0066] In this embodiment, the user's current local iris image is collected based on the first terminal, the user's current gaze focus is determined based on the current local iris image, and then the user's blinking action is combined to realize the control operation of the second terminal. The whole interaction process is fast and convenient. Attached Figure Description

[0067] Figure 1 This is a step diagram of an operation method in one embodiment of this application;

[0068] Figure 2 This is a schematic diagram of the structure of smart glasses in one embodiment of this application;

[0069] Figure 3a This is a schematic diagram of an infrared camera acquiring a user's target pupil image at a certain moment in one embodiment of this application;

[0070] Figure 3b This is a schematic diagram of an infrared camera acquiring a user's target pupil image at another moment in one embodiment of this application;

[0071] Figure 3c This is a schematic diagram of an infrared camera acquiring a user's target pupil image at another moment in one embodiment of this application;

[0072] Figure 4a This is a schematic diagram of the display area of ​​the display device of the first terminal in one embodiment of this application;

[0073] Figure 4b Focusing the user's attention Figure 4a A schematic diagram showing the infrared camera acquiring the user's pupil image in region S1.

[0074] Figure 4c Focusing the user's attention Figure 4a A schematic diagram showing the infrared camera acquiring the user's pupil image in region S2;

[0075] Figure 4d Focusing the user's attention Figure 4a A schematic diagram showing the infrared camera acquiring the user's pupil image in region S3.

[0076] Figure 4e Focusing the user's attention Figure 4a A schematic diagram showing the infrared camera acquiring the user's pupil image in region S4;

[0077] Figure 5a This is a schematic diagram of grid division of the display area of ​​the display device of the first terminal in one embodiment of this application;

[0078] Figure 5b This is a schematic diagram of a user's pupil region image in one embodiment of this application;

[0079] Figure 5c yes Figure 5b A magnified schematic diagram of the pupil region in the image;

[0080] Figure 6 This is a schematic diagram of the structure of an operating system according to an embodiment of this application;

[0081] Figure 7 This is a schematic diagram of the structure of a storage medium according to this application;

[0082] Figure 8 This is a schematic diagram of the structure of a computing device according to this application. Detailed Implementation

[0083] The principles and spirit of this application will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement this application, and are not intended to limit the scope of this application in any way. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0084] Those skilled in the art will understand that embodiments of this application can be implemented as an apparatus, system, device, method, or computer program product. Therefore, this application can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0085] According to embodiments of this application, an operating method, system, medium, and computing device are proposed.

[0086] In this article, it is important to understand that any number of elements in the accompanying figures is for illustrative purposes and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0087] The principles and spirit of this application will be explained in detail below with reference to several representative embodiments.

[0088] Exemplary methods

[0089] like Figure 1 As shown in the embodiments of this application, an operating method is proposed for use on a first terminal, wherein the first terminal and a second terminal are communicatively connected, the first terminal includes a display device, and the first terminal is worn on the user's eyes.

[0090] The first terminal can be smart glasses, worn on the user's eyes. The second terminal can be a smart bracelet, smartwatch, smartphone, tablet, laptop, or other smart device. The second terminal and the first terminal are communicatively connected. The first terminal can receive the display images sent by the second terminal and display them on its own display device. The display images sent by the second terminal can be displayed in front of the user through the display device (physical screen) on the first terminal, or through VR (Virtual Reality) technology. Thus, the images displayed by the second terminal can be displayed on the first terminal (such as smart glasses) for the user to view.

[0091] In addition, communication between the first terminal and the second terminal can be achieved based on WiFi Direct (wireless connection system). The first terminal can act as a WiFi Direct data receiver. The WiFi Direct connection parameters can be stored in advance on the NFC (Near Field Communication) chips of the first terminal and the second terminal. When the first terminal and the second terminal are close together, the display screen of the second terminal can be transmitted to the first terminal in real time and then displayed on the display device on the first terminal.

[0092] The specific process for establishing a communication connection is as follows:

[0093] The NFC and WiFi Direct functions of both the first and second terminals are activated, and the WiFi Direct device parameters (WiFi Direct name and MAC address) are stored in the simulated NFC fields of both terminals. The first and second terminals transfer these parameters to each other when brought close together. Specifically, the second terminal is simulated as an NFC card with WiFi Direct link data, the first terminal acts as a card reader, and acquires the connection information from the second terminal. The second terminal, as the sender, initiates a connection request to the first terminal. After the user confirms and establishes the connection on the second terminal, the display screen on the second terminal can be transmitted to the first terminal via WiFi Direct.

[0094] Using this communication method, the first and second terminals can easily and quickly establish a connection during use, and the information exchange latency between them is low.

[0095] After the first terminal and the second terminal establish a communication connection, this operation method can be used to send operation commands from the first terminal to the second terminal, such as... Figure 1 As shown in the embodiments of this application, the operation method includes the following steps:

[0096] Step S100: Obtain the current local iris image of the user.

[0097] like Figure 2 As shown, Figure 2 This is a structural diagram of a smart glasses device as the first terminal. The smart glasses are equipped with a data acquisition module that can collect information about the user's eyes, such as taking photos or videos of the user's eyes.

[0098] For example, in one embodiment, such as Figure 2 As shown, the acquisition module may include an infrared transmitter and an infrared receiver. The infrared transmitter can be an infrared LED light source, and the infrared receiver can be an infrared camera. Both the infrared transmitter and receiver can be installed on the side of the smart glasses closest to the user's eyes. The infrared transmitter emits infrared light into the user's eyes; the reflected infrared light is then received by the infrared receiver. It should be noted that... Figure 2 In the smart glasses shown, both the infrared LED light source and the infrared camera are located on the right eye side. In other embodiments, they may both be located on the left eye side.

[0099] In this embodiment, the acquisition module can acquire images of the user's eyeball within a preset area from a fixed direction to obtain the current local iris image. As shown in Figure 3, which is a schematic diagram of the infrared signal received by the infrared receiver during the user's eyeball rotation, the infrared transmitter continuously emits infrared signals towards the user's eyeball, and the infrared receiver continuously receives infrared signals reflected from a fixed area (preset area) from a fixed direction. Taking an infrared camera as an example, after the smart glasses are worn on the user's eyes, the smart glasses remain relatively stationary with respect to the user's head. The infrared camera always faces one direction, and its infrared receiving area remains unchanged, i.e., the focal length remains constant. During the eyeball rotation, the infrared camera continuously receives the infrared signals reflected from the eyeball according to a fixed direction and a fixed focal length.

[0100] As shown in Figure 3. Figure 3a The middle eyeball is facing straight forward. Figure 3b The middle eyeball turns to the left. Figure 3c As the eye moves to the right, the eyeball moves relative to the infrared camera. Therefore, the infrared signal reflected from the eyeball changes as the eyeball moves. For example... Figure 3a , 3b In the 3C models, the area of ​​the eye reflecting infrared light into the infrared camera is different at each of the three moments when the eye looks in different directions.

[0101] As shown in Figure 3, the area of ​​the eyeball reflecting infrared light into the infrared camera varies depending on the direction of the gaze. Furthermore, the iris, a ring-shaped structure located between the black pupil and the white sclera in the human eye, exhibits different texture features at various locations, such as spots, filaments, and stripes. Each location of the iris is unique, and the iris of each individual is also unique. Therefore, in this embodiment, the infrared camera can be directed towards the user's iris area. When the user's eyeball moves, the position of the reflected infrared light from the iris entering the infrared receiver changes, thus altering the infrared signal received by the infrared camera. This allows the focus of the user's current gaze to be determined based on the current local iris image captured by the infrared camera, i.e., step S200: Determine the user's current gaze focus based on the current local iris image.

[0102] When the infrared camera is installed on the smart glasses, its installation direction and position are preset so that when the smart glasses are worn on the user's eyes, the infrared camera faces the iris area of ​​the user's eyeball. After the user wears the smart glasses, the infrared camera on the smart glasses can receive the infrared signals reflected by the iris of the user's eyeball, and the infrared camera has also been preset with a focal length. Therefore, in step S100, when the user wears the smart glasses, the infrared camera can capture the user's current local iris image as the user's eyeball moves. The current local iris image is the iris image that the infrared camera can capture when the user's eyeball is focused on the current focal point; when the user moves their eyeball to change the focal point, the position of the iris relative to the infrared camera changes, and the area of ​​the iris that reflects the infrared signal into the infrared camera also changes, so the current local iris image captured by the infrared camera will also change accordingly.

[0103] In this embodiment of the application, after obtaining the current local iris image, the user's current gaze focus can be determined based on the following steps S210-S220:

[0104] Step S210: Based on the current local iris image, obtain the target image features of the current local iris image.

[0105] In this embodiment, after capturing a user's current local iris image using an infrared camera, target image features can be obtained based on the current local iris image. Specifically, this can be done based on an image processing model. For example, the image recognition model can be one or more pre-prepared neural network models with different structures. Through a convolutional neural network, a spatial vector map can be extracted from the input current local iris image, and the extracted spatial vector map is the target image feature of the current local iris image.

[0106] Step S220: Determine the user's current gaze focus based on the preset matching relationship between iris features and gaze focus, and the target image features.

[0107] In this embodiment of the application, the preset matching relationship between iris features and gaze focus can be pre-initialized, specifically including the following steps S221-S227:

[0108] Step S221: When the user's eyes scan the boundary of the display area of ​​the display device, multiple boundary iris images are acquired from the user's iris within the preset area from a fixed direction.

[0109] like Figures 4a-4e As shown, where, Figure 4aFor the display screen of the smart glasses (taking the display device as an example), S1, S2, S3, and S4 are the four corners of the display area edge. After the user puts on the smart glasses, calibration is performed first. The user's gaze scans the edge of the display area, for example, from S1 to S2, then to S3, then to S4, and finally back to S1. When the user's gaze is focused on position S1, such as... Figure 4b As shown, the infrared camera acquires the iris image at position t1; when the user's gaze is focused on position S2, as... Figure 4c As shown, the infrared camera acquires the iris image at position t2; when the user's gaze is focused on position S3, as... Figure 4d As shown, the infrared camera acquires the iris image at position t3; when the user's gaze is focused on position S4, as... Figure 4e As shown, the infrared camera acquires the iris image at position t4; when the user's gaze refocuses on position S1, as... Figure 4a As shown, the infrared camera once again acquired the iris image of t1.

[0110] Therefore, when a user is viewing the display area, t1, t2, t3, and t4 represent the outermost boundary of the iris image that the infrared camera can capture. When the user focuses on other parts of the display area, the iris image captured by the infrared camera will necessarily be within the area enclosed by t1, t2, t3, and t4. Thus, the area enclosed by t1, t2, t3, and t4 is the area that the infrared camera can capture.

[0111] Step S222: Obtain an iris region image formed by the enclosed area of ​​the plurality of boundary iris images.

[0112] In step S221, four boundary images t1, t2, t3, and t4 are obtained, and the enclosed area is the iris region image. In this embodiment, after determining the four boundary images t1, t2, t3, and t4, the coordinates of the four boundary images can be further determined. For example, the coordinates of the lower left corner of t1, the lower right corner of t2, the upper right corner of t3, and the upper left corner of t4 can be obtained respectively. Then, an eye image containing the above t1, t2, t3, and t4 is obtained, and the iris region image is obtained by cutting the eye image according to the above coordinates.

[0113] Step S223: Divide the iris region image into a grid according to a preset size to obtain multiple iris grid images.

[0114] like Figures 5a-5c As shown, Figure 5b This is a diagram illustrating the user's eyeball. Figure 5bThe grid region in the image is the iris region image obtained in step S222. Each grid is obtained by dividing the image into grids according to a preset size, and each grid represents an iris grid image.

[0115] like Figure 5c As shown, Figure 5c for Figure 5b In this embodiment of the application, the iris region image is divided into a 5*11 grid, which means there are a total of 55 iris grid images.

[0116] Step S224: Obtain the iris grid image features of each of the iris grid images.

[0117] In this embodiment of the application, after determining the iris region image and dividing it into iris grid images, an image recognition model can be used to obtain the iris grid image features of each iris grid image, such as... Figure 5c As shown, assume that the iris grid image features of the 55 iris grid images are N1-N55 respectively.

[0118] Step S225: Divide the display area into multiple display grids according to the number of iris grid images.

[0119] In step S223, the iris region image is divided into 5*11 grids, totaling 55 grids. Therefore, in this embodiment, the display area can be divided into 5*11 grids, such as... Figure 5a As shown.

[0120] Step S226: Obtain the center coordinates of each of the display grids.

[0121] In step S225, the display area of ​​the screen is divided into a 5*11 display grid. In step S226, the center coordinates of each display grid can be obtained, such as... Figure 5a As shown, assuming the center points of each display grid are M1-M55, then the center coordinates of each display grid are the coordinates of each point M1-M55.

[0122] Step S227: Match each of the iris grid image features with the center coordinates of each of the display grids to obtain the preset matching relationship between iris features and gaze focus. The mutually matched iris grid image features and the center coordinates of the display grids represent the iris features of the current local iris image that match the center coordinates when the user's gaze focus is on those coordinates.

[0123] When the user's gaze is focused on the first grid in the first row of the display area, the infrared camera can acquire the iris grid image in the 11th grid in the 5th row of the iris region image. The image recognition model can identify the iris grid image feature N1. Therefore, the coordinates of iris grid image feature N11 match those of M11. Similarly, when the user's gaze is focused on the 11th grid in the first row of the display area, the infrared camera can acquire the iris grid image in the first grid in the 5th row of the iris region image. The image recognition model can identify the iris grid image feature N11. Therefore, the coordinates of iris grid image feature N1-N55 match those of M1-M55 respectively.

[0124] After steps S221-227 above, the preset matching relationship between iris features and gaze focus can be obtained, that is, the matching relationship between each grid image feature in N1-N55 and each point in M1-M55. After obtaining this matching relationship, the user's current gaze focus can be determined through the following steps S228-S230:

[0125] Step S228: From each of the iris grid image features, a target grid image feature that is the same as the target image feature is obtained.

[0126] After the infrared camera acquires the user's current local iris image, it uses an image recognition model to obtain the target image features of the current local iris image. At this time, it can match an iris grid feature that is the same as the target image feature from each iris grid feature in N1-N55. Assuming that the target image feature at the current moment is the same as N38, then N38 is the target grid image feature.

[0127] Step S229: Based on the target grid image features, determine the target coordinates in the display area that match the target grid image features.

[0128] The center coordinates that match N38 are the coordinates of M38, that is, the target coordinates are the coordinates of M38.

[0129] Step S229: Determine the user's current gaze focus based on the target coordinates.

[0130] If the target coordinates are M38, then it can be concluded that the user's gaze is currently focused on the display area centered on M38.

[0131] Step S300: If the user blinks, an operation command is sent to the second terminal based on the blinking action information and the current gaze focus.

[0132] In this embodiment of the application, the user's blinking action can be determined based on the following method:

[0133] If the intensity of the infrared signal received by the infrared receiver decreases below a preset threshold within a preset time period and then recovers, it is determined that the user blinked. The infrared LED light source continuously emits infrared signals towards the user's eyeball. If the user does not blink, the infrared camera will continuously receive the infrared signal reflected from the user's eyeball, and the intensity of this infrared signal can remain stable within a certain range. When the user blinks, the eyelid covers the eyeball, and the eyelid can absorb the infrared signal, thus reflecting almost no infrared signal, or only a portion of the infrared signal is reflected. At this time, the infrared signal received by the infrared camera is significantly weaker. When the user opens their eyes, the intensity of the infrared signal reflected from the eyeball returns to normal. Therefore, an infrared signal intensity threshold can be set. If the intensity of the infrared signal received by the infrared camera is lower than the threshold within a preset time period (e.g., lower than the threshold within 0.1 seconds), it indicates that the user blinked, and the eyelid closure time during the blink was 0.1 seconds. Additionally, it should be noted that, assuming the preset time period is set to 0.1 seconds, if the intensity of the infrared signal received by the infrared camera is below the threshold for more than 0.1 seconds, it will not be considered a blinking action, but rather an eye-closing action. Furthermore, the length of the preset time period and the threshold for the intensity of the infrared signal can be determined based on the user's typical blinking time and the strength of the infrared signal reflected by the eyeball in a normal open-eye state.

[0134] Furthermore, for the user, the blinking action may be an unconscious blink, rather than an intentional blinking to control the second terminal. To avoid accidental operation of the second terminal due to unconscious blinking, in this embodiment of the application, the operation method further includes the following before sending the operation command to the second terminal:

[0135] Determine whether a preset wake-up command has been received;

[0136] If the preset wake-up command is received, an operation command is sent to the second terminal based on the blinking action information and the current gaze focus. The wake-up module can be installed on the smart glasses, for example, on the temple of the glasses. The wake-up module can be in the form of a physical button or a touch sensor. When the user wants to send an operation command to the second terminal based on a blinking action, they can press the physical button or touch the touch-sensitive area. When the wake-up module receives the wake-up command, the user's blinking action will be interpreted as an operation command to the second terminal, thus avoiding accidental operation caused by unconscious blinking.

[0137] In addition, to avoid accidental operation caused by users blinking unconsciously, in this embodiment of the application, after sending the operation command to the second terminal, the operation method further includes:

[0138] Determine whether a preset sleep command has been received. If the preset sleep command has been received, stop the action information based on the blinking action and the current gaze focus, and send an operation command to the second terminal.

[0139] In this embodiment, after the user completes the operation on the second terminal, they can press the physical button again or touch the sensing area to send a sleep command to the second terminal. When the wake-up module receives the sleep command, the user's blinking will be considered an unconscious blinking action and will not send operation commands to the second terminal.

[0140] It should be noted that the display screen sent from the second terminal to the first terminal can be a document, video, image, etc. When the user's gaze is focused on the end of the document in the current display screen, a blink can send a page-turning or scrolling command, achieving automatic page turning based on the user's gaze, eliminating the need for user operation. Similarly, when the user's gaze is focused on the fast-forward or rewind button in the progress bar of the video in the current display screen, a blink can send a fast-forward or rewind command, achieving automatic fast-forwarding and rewinding based on the user's gaze. In this embodiment, the operation commands sent to the second terminal based on the user's gaze focus and blinking are implemented according to the operation buttons on the display screen at the user's gaze focus position, such as close, switch, zoom in, or zoom out. That is, whatever operation button is displayed at the gaze focus position, the command to perform the relevant operation is sent to the second terminal.

[0141] In another embodiment, the number of blinks can be used as an operation command to avoid accidental operation. For example, two or three blinks within a preset time period can be set to represent blinks indicating conscious operation by the user. Assuming the preset blink count is two, if the intensity of the infrared signal first falls below the threshold range and then returns to normal intensity twice consecutively within the preset time period, it is determined that the user blinked twice consecutively and that the blinking action was conscious. At this time, an operation command is sent to the second terminal. The length of the preset time period can be set according to the time period of a single blink by the user. For example, if a single blink takes 0.1 seconds, the preset time period can be set to 0.25 seconds. That is, if two blinks are detected within 0.25 seconds, it is determined that the blinking action was conscious.

[0142] Additionally, it should be noted that the above embodiment illustrates dividing the display area and iris region images into 55 grids. The 55 grids are merely for illustrative purposes and do not represent that 55 grids are the only possible division method. The size of the grid represents the precision of the operation; the smaller the grid, the more precise the control operation based on the user's gaze focus. Therefore, in other embodiments, the display area and iris region images can also be divided into grids at the pixel level.

[0143] In another embodiment of this application, before the user operates the second terminal based on the first terminal, iris verification can be performed on the user to determine the user's identity, as follows:

[0144] First, during the initialization phase, users can record and store their iris features using an infrared camera. In subsequent use, iris authentication is performed before each use. The authentication process is as follows:

[0145] It should be noted that the iris image obtained by the infrared camera is not limited to a preset area size, but can be obtained by adjusting the focal length of the infrared camera to obtain the complete iris image of the user.

[0146] After obtaining the user's iris image, the iris features of the iris image are obtained using an image recognition model and compared with the iris features pre-recorded during the initialization phase. If they match, the user authentication is successful; otherwise, the authentication fails.

[0147] In this embodiment, the user's current local iris image is collected based on the first terminal, the user's current gaze focus is determined based on the current local iris image, and then the user's blinking action is combined to realize the control operation of the second terminal. The whole interaction process is fast and convenient.

[0148] Exemplary device

[0149] like Figure 6 As shown, this exemplary embodiment proposes an operating system 100, applied to a first terminal, wherein the first terminal and a second terminal are communicatively connected, the first terminal includes a display device, and the first terminal is worn on the user's eyes. The operating system 100 includes:

[0150] Acquisition module 110 is used to acquire the current local iris image of the user;

[0151] Processing module 120 is used to determine the user's current gaze focus based on the current local iris image;

[0152] Determine whether the user blinks. If the user blinks, send an operation command to the second terminal based on the blinking action information and the current gaze focus.

[0153] In this embodiment of the application, the acquisition module 110 is further configured to:

[0154] The current local iris image is obtained by capturing images of the user's eyeballs within a preset area from a fixed direction.

[0155] In this embodiment of the application, the processing module 120 is further configured to:

[0156] Based on the current local iris image, obtain the target image features of the current local iris image;

[0157] The user's current gaze focus is determined based on the preset matching relationship between iris features and gaze focus, as well as the target image features.

[0158] In this embodiment of the application, the preset matching relationship between iris features and gaze focus is obtained in advance through the following initialization configuration:

[0159] When the user's eyes scan the boundary of the display area of ​​the display device, the acquisition module 110 acquires multiple boundary iris images of the user's iris within the preset area from the fixed direction.

[0160] Acquire an image of the iris region formed by the enclosed portions of the iris images;

[0161] The iris region image is divided into grids of a preset size to obtain multiple iris grid images;

[0162] Obtain the iris grid image features of each of the aforementioned iris grid images;

[0163] The display area is divided into multiple display grids according to the number of iris grid images;

[0164] Obtain the center coordinates of each of the displayed grids;

[0165] The iris grid image features are matched with the center coordinates of the display grids to obtain the preset matching relationship between the iris features and the gaze focus;

[0166] Among them, the mutually matching iris grid image features and the center coordinates of the display grid represent the iris features of the current local iris image when the user's gaze is focused on the center coordinates, which are the iris grid image features that match the center coordinates.

[0167] In this embodiment of the application, the processing module 120 is further configured to:

[0168] Target grid image features that are identical to the target image features are obtained by matching each of the iris grid image features;

[0169] Based on the target grid image features, determine the target coordinates in the display area that match the target grid image features;

[0170] Based on the target coordinates, determine the user's current gaze focus.

[0171] In this embodiment of the application, the processing module 120 is further configured to:

[0172] If the signal strength of the current local iris image decreases to below a preset threshold within a preset time period and then recovers, it is determined that the user blinked.

[0173] In this embodiment of the application, before sending the operation command to the second terminal, the processing module 120 is further configured to:

[0174] Determine whether a preset wake-up command has been received;

[0175] If the preset wake-up command is received, an operation command is sent to the second terminal based on the blinking motion information and the current gaze focus;

[0176] After sending the operation command to the second terminal, the processing module 120 is further configured to:

[0177] Determine whether a preset sleep command has been received. If the preset sleep command has been received, stop the action information based on the blinking action and the current gaze focus, and send an operation command to the second terminal.

[0178] In this embodiment, the user's current local iris image is collected based on the first terminal, and the processing module 120 determines the user's current gaze focus based on the current local iris image. Then, in conjunction with the user's blinking action, the control operation of the second terminal is realized. The entire interaction process is fast and convenient.

[0179] Exemplary media

[0180] After introducing the methods, media, and systems of exemplary embodiments of this application, the following references are made. Figure 7 The computer-readable storage medium of exemplary embodiments of this application will be described, please refer to... Figure 7The computer-readable storage medium shown is an optical disc 70, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it implements the steps described in the above method implementation, such as acquiring the user's current local iris image; determining the user's current gaze focus based on the current local iris image; and if the user blinks, sending an operation command to the second terminal based on the blinking action information and the current gaze focus. The specific implementation of each step will not be repeated here.

[0181] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM) and static random access memory.

[0182] Memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, will not be described in detail here.

[0183] Exemplary computing device

[0184] After introducing the methods, systems, and media of exemplary embodiments of this application, the following references are made. Figure 8 Computing devices according to exemplary embodiments of this application.

[0185] Figure 8 A block diagram is shown of an exemplary computing device 80 suitable for implementing embodiments of the present application. The computing device 80 may be a computer system or a server. Figure 8 The computing device 80 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0186] like Figure 8 As shown, the components of the computing device 80 may include, but are not limited to: one or more processors or processing units 801, system memory 802, and bus 803 connecting different system components (including system memory 802 and processing unit 801).

[0187] The computing device 80 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computing device 80, including volatile and non-volatile media, removable and non-removable media.

[0188] System memory 802 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 8021 and / or cache memory 8022. Computing device 70 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, ROM 8023 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 8 Not shown in the image (usually referred to as a "hard drive"). Although not shown in Figure 8 The diagram illustrates that disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disc drives for reading and writing to removable non-volatile optical discs (e.g., CD-ROMs, DVD-ROMs, or other optical media) can be provided. In these cases, each drive can be connected to bus 803 via one or more data media interfaces. System memory 802 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0189] A program / utility 8025 having a set (at least one) of program modules 8024 may be stored, for example, in system memory 802, and such program modules 8024 include, but are not limited to, an operating system 100, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment. Program modules 8024 typically perform the functions and / or methods described in the embodiments of this application.

[0190] The computing device 80 can also communicate with one or more external devices 804 (such as a keyboard, pointing device, display, etc.). This communication can be performed through an input / output (I / O) interface. Furthermore, the computing device 80 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 806. Figure 8 As shown, network adapter 806 communicates with other modules of computing device 80 (such as processing unit 801) via bus 803. It should be understood that, although... Figure 8 As not shown, it can be used in conjunction with computing device 80 with other hardware and / or software modules.

[0191] The processing unit 801 executes various functional applications and data processing by running programs stored in the system memory 802. For example, it acquires the user's current local iris image; determines the user's current gaze focus based on the current local iris image; and if the user blinks, sends an operation command to the second terminal based on the blinking action information and the current gaze focus. The specific implementation of each step will not be repeated here. It should be noted that although several units / modules or sub-units / sub-modules of the facial recognition device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0192] In the description of this application, it should be noted that the terms "first", "second", and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0193] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, units, and processes described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0194] In the several embodiments provided in this application, it should be understood that the disclosed apparatus, system, and method can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces. Indirect couplings or communication connections between systems or units may be electrical, mechanical, or other forms.

[0195] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0196] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0197] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computing device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0198] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the technical scope disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

[0199] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0200] Based on the above description, the embodiments of this application provide at least the following technical solutions, but are not limited thereto:

[0201] 1. An operating method applied to a first terminal, wherein the first terminal and a second terminal are communicatively connected, the first terminal includes a display device, and the first terminal is worn on a user's eye, the operating method comprising:

[0202] Obtain the user's current local iris image;

[0203] Based on the current local iris image, determine the user's current gaze focus;

[0204] If the user blinks, an operation command is sent to the second terminal based on the blinking action information and the current gaze focus.

[0205] 2. The operation method as described in technical solution 1, wherein acquiring the user's current local iris image includes:

[0206] The current local iris image is obtained by capturing images of the user's eyeballs within a preset area from a fixed direction.

[0207] 3. The operation method as described in technical solution 1 or 2, wherein determining the user's current gaze focus based on the current local iris image includes:

[0208] Based on the current local iris image, obtain the target image features;

[0209] The user's current gaze focus is determined based on the preset matching relationship between iris features and gaze focus, as well as the target image features.

[0210] 4. The operating method as described in any one of technical solutions 1-3, wherein the preset matching relationship between the iris feature and the displayed image is obtained in the following manner:

[0211] When the user's eyes scan the boundary of the display area of ​​the display device, the iris of the user within the preset area is captured from the fixed direction to obtain multiple boundary iris images;

[0212] Acquire an image of the iris region formed by the enclosed portions of the iris images;

[0213] The iris region image is divided into grids of a preset size to obtain multiple iris grid images;

[0214] Obtain the iris grid image features of each iris grid image;

[0215] The display area is divided into multiple display grids according to the number of iris grid images;

[0216] Obtain the center coordinates of each of the displayed grids;

[0217] The iris grid image features are matched with the center coordinates of the display grids to obtain the preset matching relationship between the iris features and the gaze focus;

[0218] Among them, the mutually matching iris grid image features and the center coordinates of the display grid represent the iris features of the current local iris image when the user's gaze is focused on the center coordinates, which are the iris grid image features that match the center coordinates.

[0219] 5. The operation method as described in any one of technical solutions 1-4, wherein determining the user's current gaze focus based on a preset matching relationship between iris features and gaze focus, and the target image features, includes:

[0220] From each of the iris grid image features, a target grid image feature that is identical to the target image feature is obtained;

[0221] Based on the target grid image features, determine the target coordinates in the display area that match the target grid image features;

[0222] Based on the target coordinates, the user's current gaze focus is determined.

[0223] 6. The operating method as described in any one of technical solutions 1-5, wherein the blinking action is determined based on the following method:

[0224] If the signal strength of the current local iris image decreases to below a preset threshold within a preset time period and then recovers, it is determined that the user blinked.

[0225] 7. The operation method as described in any one of technical solutions 1-6, further comprising, before sending the operation instruction to the second terminal:

[0226] Determine whether a preset wake-up command has been received. If the preset wake-up command has been received, then send an operation command to the second terminal based on the blinking action information and the current gaze focus.

[0227] After sending the operation command to the second terminal, the operation method further includes:

[0228] Determine whether a preset sleep command has been received. If the preset sleep command has been received, stop the action information based on the blinking action and the current gaze focus, and send an operation command to the second terminal.

[0229] 8. An operating system applied to a first terminal, the first terminal and a second terminal being communicatively connected, the first terminal including a display device, the first terminal being worn on a user's eyes, the operating system comprising:

[0230] The acquisition module is used to acquire the user's current local iris image;

[0231] The processing module is used to determine the user's current gaze focus based on the current local iris image;

[0232] Determine whether the user blinks. If the user blinks, send an operation command to the second terminal based on the blinking action information and the current gaze focus.

[0233] 9. The operating system as described in technical solution 8, wherein the acquisition module is further configured to:

[0234] The current local iris image is obtained by capturing images of the user's eyeballs within a preset area from a fixed direction.

[0235] 10. The operating system as described in technical solution 8 or 9, wherein the processing module is further configured to:

[0236] Based on the current local iris image, obtain the target image features of the current local iris image;

[0237] The user's current gaze focus is determined based on the preset matching relationship between iris features and gaze focus, as well as the target image features.

[0238] 11. In the operating system described in any one of technical solutions 8-10, the preset matching relationship between iris features and gaze focus is obtained in advance through the following initialization configuration:

[0239] When the user's eyes scan the boundary of the display area of ​​the display device, multiple boundary iris images are obtained by the acquisition module from the fixed direction of the user's iris within the preset area.

[0240] Acquire an image of the iris region formed by the enclosed portions of the iris images;

[0241] The iris region image is divided into grids of a preset size to obtain multiple iris grid images;

[0242] Obtain the iris grid image features of each of the aforementioned iris grid images;

[0243] The display area is divided into multiple display grids according to the number of iris grid images;

[0244] Obtain the center coordinates of each of the displayed grids;

[0245] The iris grid image features are matched with the center coordinates of the display grids to obtain the preset matching relationship between the iris features and the gaze focus;

[0246] Among them, the mutually matching iris grid image features and the center coordinates of the display grid represent the iris features of the current local iris image when the user's gaze is focused on the center coordinates, which are the iris grid image features that match the center coordinates.

[0247] 12. The operating system as described in any one of technical solutions 8-11, wherein the processing module is further configured to:

[0248] Target grid image features that are identical to the target image features are obtained by matching each of the iris grid image features;

[0249] Based on the target grid image features, determine the target coordinates in the display area that match the target grid image features;

[0250] Based on the target coordinates, determine the user's current gaze focus.

[0251] 13. The operating system as described in any one of technical solutions 8-12, wherein the processing module is further configured to:

[0252] If the signal strength of the current local iris image decreases to below a preset threshold within a preset time period and then recovers, it is determined that the user blinked.

[0253] 14. The operating system as described in any one of technical solutions 8-13, before sending the operation command to the second terminal, the processing module is further configured to:

[0254] Determine whether a preset wake-up command has been received;

[0255] If the preset wake-up command is received, an operation command is sent to the second terminal based on the blinking motion information and the current gaze focus;

[0256] After sending the operation command to the second terminal, the processing module is further configured to:

[0257] Determine whether a preset sleep command has been received. If the preset sleep command has been received, stop the action information based on the blinking action and the current gaze focus, and send an operation command to the second terminal.

[0258] 15. A storage medium having a computer program stored thereon, characterized in that: when the computer program is executed by a processor, it implements the method as described in any one of technical solutions 1-7.

[0259] 16. A computing device, characterized in that: the computing device includes a processor, which, when executing a computer program stored in a memory, implements the method as described in any one of technical solutions 1-7.

Claims

1. An operating method applied to a first terminal, wherein the first terminal and a second terminal are communicatively connected, the first terminal includes a display device, and the first terminal is worn on a user's eye, the operating method comprising: Obtain the user's current local iris image; Based on the current local iris image, determine the user's current gaze focus; If the user blinks, an operation command is sent to the second terminal based on the blinking action information and the current gaze focus. Determining the user's current gaze focus based on the current local iris image includes: Based on the current local iris image, obtain the target image features; The user's current gaze focus is determined based on the preset matching relationship between iris features and gaze focus, as well as the target image features; The preset matching relationship between iris features and gaze focus is obtained through the following method: When the user's eyes scan the boundary of the display area of ​​the display device, the iris of the user within the preset area is captured from a fixed direction to obtain multiple boundary iris images; Acquire an image of the iris region formed by the enclosed portions of the iris images; The iris region image is divided into grids of a preset size to obtain multiple iris grid images; Obtain the iris grid image features of each iris grid image; The display area is divided into multiple display grids according to the number of iris grid images; Obtain the center coordinates of each of the displayed grids; The iris grid image features are matched with the center coordinates of the display grids to obtain the preset matching relationship between the iris features and the gaze focus; Among them, the mutually matching iris grid image features and the center coordinates of the display grid represent the iris features of the current local iris image when the user's gaze is focused on the center coordinates, which are the iris grid image features that match the center coordinates.

2. The operation method as described in claim 1, acquiring the user's current local iris image, includes: The current local iris image is obtained by capturing images of the user's eyeballs within a preset area from a fixed direction.

3. The operation method as described in claim 1, wherein determining the user's current gaze focus based on a preset matching relationship between iris features and gaze focus, and the target image features, includes: From each of the iris grid image features, a target grid image feature that is identical to the target image feature is obtained; Based on the target grid image features, determine the target coordinates in the display area that match the target grid image features; Based on the target coordinates, the user's current gaze focus is determined.

4. The operation method as described in claim 1, wherein the blinking action is determined based on the following method: If the signal strength of the current local iris image decreases to below a preset threshold within a preset time period and then recovers, it is determined that the user blinked.

5. The operation method as described in claim 1, further comprising, before sending the operation instruction to the second terminal: Determine whether a preset wake-up command has been received. If the preset wake-up command has been received, then send an operation command to the second terminal based on the blinking action information and the current gaze focus. After sending the operation command to the second terminal, the operation method further includes: Determine whether a preset sleep command has been received. If the preset sleep command has been received, stop the action information based on the blinking action and the current gaze focus, and send an operation command to the second terminal.

6. An operating system applied to a first terminal, the first terminal and a second terminal being communicatively connected, the first terminal including a display device, the first terminal being worn on a user's eyes, the operating system comprising: The acquisition module is used to acquire the user's current local iris image; The processing module is used to determine the user's current gaze focus based on the current local iris image; Determine whether the user blinks. If the user blinks, send an operation command to the second terminal based on the blinking action information and the current gaze focus. Based on the current local iris image, obtain the target image features of the current local iris image; The user's current gaze focus is determined based on the preset matching relationship between iris features and gaze focus, as well as the target image features; The preset matching relationship between iris features and gaze focus is obtained through the following initialization configuration: When the user's eyes scan the boundary of the display area of ​​the display device, multiple boundary iris images are obtained by the acquisition module from a fixed direction by acquiring the user's iris within the preset area; Acquire an image of the iris region formed by the enclosed portions of the iris images; The iris region image is divided into grids of a preset size to obtain multiple iris grid images; Obtain the iris grid image features of each of the aforementioned iris grid images; The display area is divided into multiple display grids according to the number of iris grid images; Obtain the center coordinates of each of the displayed grids; The iris grid image features are matched with the center coordinates of the display grids to obtain the preset matching relationship between the iris features and the gaze focus; Among them, the mutually matching iris grid image features and the center coordinates of the display grid represent the iris features of the current local iris image when the user's gaze is focused on the center coordinates, which are the iris grid image features that match the center coordinates.

7. The operating system as described in claim 6, wherein the acquisition module is further configured to: The current local iris image is obtained by capturing images of the user's eyeballs within a preset area from a fixed direction.

8. The operating system as described in claim 6, wherein the processing module is further configured to: Target grid image features that are identical to the target image features are obtained by matching each of the iris grid image features; Based on the target grid image features, determine the target coordinates in the display area that match the target grid image features; Based on the target coordinates, determine the user's current gaze focus.

9. The operating system of claim 6, wherein the processing module is further configured to: If the signal strength of the current local iris image decreases to below a preset threshold within a preset time period and then recovers, it is determined that the user blinked.

10. The operating system as described in claim 9, wherein before sending the operation command to the second terminal, the processing module is further configured to: Determine whether a preset wake-up command has been received; If the preset wake-up command is received, an operation command is sent to the second terminal based on the blinking motion information and the current gaze focus; After sending the operation command to the second terminal, the processing module is further configured to: Determine whether a preset sleep command has been received. If the preset sleep command has been received, stop the action information based on the blinking action and the current gaze focus, and send an operation command to the second terminal.

11. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

12. A computing device, characterized in that: The computing device includes a processor, which, when executing a computer program stored in a memory, implements the method as described in any one of claims 1-5.

Images