Control method and device

Abstract

The embodiment of the invention provides a control method and device, computer equipment, a computer readable storage medium and a computer program product, and belongs to the field of human-computer interaction. The control method comprises the following steps: acquiring eye movement data of a target object under the condition of running the target application program; determining a target fixation point of the target object on the client screen according to the eye movement data; generating a target control instruction according to the target fixation point; and in response to the target control instruction, controlling the target application program to execute a corresponding target operation. According to the technical scheme, the immersion and the convenience degree of using the target application program by the target object can be improved, and the use experience of the target object is enhanced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of human-computer interaction technology, and in particular, to a control method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] The comics industry is booming and has become a vital component of the global cultural and creative landscape. With the advancement of digitalization, comics have evolved from print publications to online platforms and finally to mobile applications. More and more users prefer to read comics anytime, anywhere through comic reading apps on mobile devices like phones and tablets.

[0003] However, some operations of comic reading applications (such as turning pages) have poor human-computer interaction efficiency and low convenience, which affects users' reading immersion.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the Invention

[0005] The embodiments of the present application provide a control method, an apparatus, a computer device, a computer-readable storage medium, and a computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides a control method for a client having a target application installed thereon, the method comprising: While running the target application, acquiring eye movement data of the target object; determining a target gaze point of the target object on the client screen according to the eye movement data; Generate a target control instruction according to the target gaze point; and In response to the target control instruction, the target application is controlled to perform a corresponding target operation.

[0007] Optionally, determining a target gaze point of the target object on the client screen according to the eye movement data includes: Obtaining a target direction vector of the target object's eye gaze according to the eye movement data; Acquire a normal vector of the screen and binocular fitting points of the target object; The target gaze point is determined according to the target direction vector, the binocular fitting point, and the normal vector.

[0008] Optionally, obtaining a target direction vector of the target object's eye gaze according to the eye movement data includes: Obtaining an initial direction vector based on a first coordinate system according to the eye movement data; wherein the first coordinate system is a coordinate system with the center of the target object's head as the origin, and the initial direction vector is an initial direction vector of the target object's eye gaze; The target direction vector is determined based on the second coordinate system and the initial direction vector, where the target direction vector is a representation of the initial direction vector based on the second coordinate system; wherein the second coordinate system is a coordinate system with the client front camera as its origin.

[0009] Optionally, obtaining the normal vector of the screen and the binocular fitting points of the target object includes: Obtaining a deflection angle of the client based on a second coordinate system, where the second coordinate system takes the front camera of the client as an origin; acquiring a screen plane based on the second coordinate system according to the deflection angle; A normal vector based on the second coordinate system is determined according to the screen plane.

[0010] Optionally, determining the target gaze point according to the target direction vector, the binocular fitting point, and the normal vector includes: Determining a target distance parameter from the binocular fitting point to the screen according to the binocular fitting point, the target direction vector, and the normal vector; The target gaze point is determined according to the target distance parameter, the target direction vector and the binocular fitting point.

[0011] Optionally, generating a target control instruction according to the target gaze point includes: Obtaining the screen size of the client; determining, according to the target gaze point, the screen plane, and the screen size, whether the target gaze point is located in a target area on the screen; When the target gaze point is located in the target area, a corresponding target control instruction is generated.

[0012] Optionally, when the target gaze point is located in the target area, generating a corresponding target control instruction includes: In a case where the target gaze point is located in the target area, determining a stay time of the target gaze point in the target area; When the stay time exceeds a preset threshold, a corresponding target control instruction is generated.

[0013] Optionally, the target application is a comic reading application, and there are multiple target areas, including a first area located at the left edge, a second area at the right edge, and / or a third area at the upper edge and a fourth area at the lower edge; When the target gaze point is located in the target area, generating a corresponding target control instruction includes: When the target gaze point is located in the first area, generating a left page turning instruction; If the target gaze point is located in the second area, generating a right page turning instruction; and / or generating a page-up instruction when the target gaze point is located in the third area; When the target gaze point is located in the fourth area, a page-down instruction is generated.

[0014] Optionally, when the target gaze point is located in the target area, generating a corresponding target control instruction includes: When the target gaze point is located in the target area, obtaining a page turning record of a previous page turning; Determining a page turning interval according to the page turning record and the current time; determining whether the target gaze point has left the historical page turning area corresponding to the page turning record within the page turning interval; In the case where the target gaze point leaves the history page turning area, generating a corresponding target control instruction; and If the target gaze point has not left the historical page turning area, the target control instruction is refused to be generated.

[0015] Another aspect of an embodiment of the present application provides a control device, the device comprising: an acquisition module, configured to acquire eye movement data of a target object while running the target application; A determination module, configured to determine a target gaze point of the target object on the client screen based on the eye movement data; A generating module, configured to generate a target control instruction according to the target gaze point; and The control module is configured to control the target application to execute a corresponding target operation in response to the target control instruction.

[0016] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0017] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0018] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the above-mentioned method when executed by a processor.

[0019] The embodiments of the present application adopting the above-mentioned technical solution may include the following advantages: performing target operations based on the target gaze point determined from the target object's eye movement data can reduce the manual operations required by the target object, thereby improving the efficiency of human-computer interaction, improving the immersion and convenience of the target object in using the target application, and enhancing the target object's usage experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings illustrate exemplary embodiments and constitute a part of the specification. Together with the description of the specification, they serve to explain exemplary implementation of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals designate similar, but not necessarily identical, elements.

[0021] Figure 1 The following schematically shows an operating environment diagram of the control method according to the first embodiment of the present application; Figure 2 The flowchart of the control method according to the first embodiment of the present application is schematically shown; Figure 3 Schematically shows Figure 2 Flowchart of sub-steps of step S202; Figure 4 Schematically shows Figure 3 Flowchart of sub-steps of step S300; Figure 5 Schematically shows Figure 3 Flowchart of sub-steps of step S302; Figure 6 Schematically shows Figure 3 Flowchart of sub-steps of step S304; Figure 7 Schematically shows Figure 2 Flowchart of sub-steps of step S204; Figure 8 Schematically shows Figure 7Flowchart of sub-steps of step S704; Figure 9 Schematically shows Figure 7 Another sub-step flow chart of step S704; Figure 10 Schematically shows Figure 7 Another sub-step flow chart of step S704; Figure 11 The following schematically shows an exemplary application flow chart of the control method according to the first embodiment of the present application; Figure 12 A schematic diagram of a first coordinate system of a control method according to the first embodiment of the present application is schematically shown; Figure 13 A schematic diagram of a second coordinate system of the control method according to the first embodiment of the present application is schematically shown; Figure 14 A schematic diagram schematically shows the positional relationship between the target object and the client screen according to the control method of the first embodiment of the present application; Figure 15 Schematically shows an example diagram of setting the first to fourth areas on the screen according to the control method of the first embodiment of the present application; Figure 16 A block diagram schematically shows a control device according to the second embodiment of the present application; and Figure 17 The following schematically shows a hardware architecture diagram of a computer device according to the third embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of this application more clear, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] It should be noted that the descriptions of "first", "second", etc. in the embodiments of the present application are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0024] It should be noted that in all aspects of this application involving data collection, storage, use, transmission, and processing, all data must strictly adhere to the laws, regulations, industry standards, and regulatory requirements of the data's source and use locations, as well as the relevant countries and regions, to ensure the legality and compliance of data activities. During the collection phase, the purpose, method, and scope of collection will be clearly stated to the data subject in a conspicuous manner. Collection will be carried out only after obtaining the data subject's legal authorization, ensuring that the collection process adheres to the "minimum necessary" principle and does not exceed the scope of data collection. During storage, the storage period will be limited, and after the storage purpose has been achieved, data will be promptly deleted, anonymized, or encrypted. During the use phase, strict data security protection mechanisms will be implemented, using field-level desensitization technology to process raw data according to pre-set desensitization rules. A variety of desensitization strategies, such as data generalization, data anonymization, and data encryption, will be employed for different data types to effectively mitigate the risk of sensitive information leakage and ensure that the data ultimately used is securely desensitized data, fully protecting the rights and interests of data subjects and data security. During the transmission and processing phases, the confidentiality and security of data will be ensured during transmission and processing.

[0025] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed. They are only used to facilitate the description of this application and to distinguish each step. Therefore, they cannot be understood as limitations on this application.

[0026] First, an explanation of the terms used in this application is provided: Augmented Reality (AR) is a technology that uses computer technology to overlay virtual information onto real-world scenes in real time. Using hardware such as cameras, sensors, and display devices, combined with algorithms such as image recognition and spatial positioning, it allows users to integrate virtual images, text, 3D models, and other elements into their real-world field of view.

[0027] Kalman filter algorithm: It is a self-recursive filter that can estimate the dynamic state of the system from a series of noisy measurement data.

[0028] Stereo vision algorithm: A computer vision technology that simulates the principle of human binocular vision, uses two or more cameras to shoot the same scene from different angles, obtains multiple images, and then processes and analyzes these images.

[0029] Triangulation: It is a measurement technique based on geometric principles. By measuring the angle or distance between the target point and two known positions, the precise position of the target point is calculated using trigonometric functions.

[0030] Facial landmark detection algorithm: It is a computer vision technology used to accurately locate a series of key points in facial images, such as the center of the eyes, corners of the mouth, tip of the nose, etc.

[0031] Compensation algorithm: It is an algorithm used to correct and optimize data processing results. The compensation algorithm can effectively reduce errors and improve the accuracy and reliability of the system.

[0032] Kinematic model: It is a mathematical model that studies the laws of motion of objects. It mainly focuses on the relationship between the changes in kinematic quantities such as the object's position, velocity, acceleration, etc. over time, but does not involve the cause of the motion, that is, it does not consider the effect of force.

[0033] Electronic compass: A sensor designed based on the principle of the geomagnetic field that can measure and determine the direction of the device relative to the earth's magnetic field in real time.

[0034] An accelerometer is a sensor that measures the acceleration of an object. When an object is subjected to an external force and experiences acceleration, the sensitive element within the accelerometer (such as the mass) will shift relative to the sensor body due to inertia, or produce changes in other physical quantities. By detecting this change and converting it into an electrical signal, the magnitude and direction of the object's acceleration can be determined.

[0035] Gyroscope: A sensor that can measure the angular velocity of an object's rotation. It obtains rotation angle and speed information by detecting the object's rotational motion in space.

[0036] Complementary filtering algorithm: decomposes the input signal into high-frequency and low-frequency parts, processes them separately through different filters, and then recombine the processed signals.

[0037] Tracking algorithm: Real-time positioning and tracking of specific targets (such as objects, people, feature points, etc.) in a continuous sequence of images or videos. By analyzing the target's appearance characteristics, movement pattern, and relationship with the surrounding environment, the target's position in the next frame is predicted, thereby achieving continuous tracking of the target. Feature point matching: This technique is used to determine the positional relationship between corresponding points in two images (or versions of the same image at different viewing angles, times, and other conditions). First, representative feature points are extracted from the images. These feature points are often unique, such as corners and edges. Descriptors for these feature points are then calculated using algorithms such as SIFT, SURF, and ORB. These descriptors can be thought of as the "fingerprint" of a feature point, containing characteristic information about the image area surrounding that point.

[0038] Inertial Measurement Unit (IMU): A device used to measure and report information about an object's motion in three-dimensional space, such as acceleration and angular velocity. It typically consists of multiple sensors, including an accelerometer, a gyroscope, and sometimes a magnetometer.

[0039] Binocular Fit Point: In eye tracking technology, it refers to a virtual center point obtained by calculating the gaze direction and position of the user's eyes.

[0040] Screen pixel size refers to the relationship between the physical size of a screen and its resolution, typically measured in pixels per inch (PPI). It reflects the screen's pixel density, or the number of pixels a screen can accommodate per unit length.

[0041] Feature model: A model used to describe and manage the relationships between features in a software product line. It organizes the various features of a software product line in a tree-like structure, where the root node typically represents the entire product line, while the other nodes represent different functional or attribute features.

[0042] Secondly, to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the following describes the relevant technologies: More and more users prefer to read comics anytime, anywhere through comic reading apps on mobile devices such as mobile phones and tablets. However, the inventors have found that the interactive methods of some operations (such as page turning) in comic reading apps still have many limitations. Manual touch page turning requires users to move their fingers frequently, which can easily interrupt the immersive reading experience. Voice control, gesture recognition and other methods are limited by environmental conditions such as light and noise, have a high false trigger rate and insufficient reliability. Although automatic page turning can free users' hands, it cannot adapt to the characteristics of different content concentrations and required dwell time on each page of comics, making it difficult to meet users' personalized reading needs.

[0043] To this end, the present invention provides a control technology solution. This technology solution (1) directly maps the target object's reading intent through eye tracking, achieving "what you see is what you read"; and (2) reduces eye movement recognition errors by combining the client's deflection angle. See below for details.

[0044] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0045] like Figure 1 As shown, the operating environment diagram includes: computer equipment 10000.

[0046] Computer devices 10000 include various types of electronic devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service kiosks, service robots, gaming systems, thin clients, various messaging devices, sensors, or other electronic devices. These computer devices can run various types and versions of software applications and operating systems, such as Windows, iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (such as Chrome OS); or various mobile operating systems, such as Windows, Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants, etc. Wearable devices may include head-mounted displays (such as smart glasses), etc. Gaming systems may include various handheld gaming devices and internet-enabled gaming devices. Client devices are capable of executing a variety of different applications, such as various internet-related applications, communication applications (such as email applications), and short message service (SMS) applications, and may utilize various communication protocols.

[0047] Computer device 10000 may include input / output interfaces. The input interface may include a camera, touchpad, touch screen, mouse, keyboard, or other sensing elements. The input interface may be configured to receive control instructions that cause computer device 10000 to perform various operations, such as turning pages. The output interface is used to output information to a target object, such as display information.

[0048] Based on the operating environment described above, the computer device 10000 may also be installed with one or more application programs, such as a comic reading program.

[0049] The following describes the technical solutions of the present application through multiple embodiments using the computer device 10000 as the execution subject. It should be noted that these embodiments can be implemented in various forms and should not be construed as being limited to the embodiments described herein.

[0050] Example 1 Figure 2 The flowchart of the control method according to the first embodiment of the present application is schematically shown.

[0051] like Figure 2 As shown, the control method may include steps S200 to S206, which are used in a client having a target application installed, wherein: Step S200: acquiring eye movement data of a target object while running the target application.

[0052] Step S202: determining a target gaze point of the target object on the client screen according to the eye movement data.

[0053] Step S204: generating a target control instruction according to the target gaze point.

[0054] Step S206 , in response to the target control instruction, controlling the target application to execute a corresponding target operation.

[0055] The control method provided in this embodiment performs target operations based on the target gaze point determined from the target object's eye movement data, which can reduce the manual operations required by the target object, improve the efficiency of human-computer interaction, increase the immersion and convenience of the target object in using the target application, and enhance the target object's usage experience.

[0056] The following combination Figure 2 , each step in steps S200~S206 and other optional steps are described in detail.

[0057] Step S200 , while running the target application, acquiring eye movement data of the target object.

[0058] Cameras in augmented reality frameworks (e.g., ARKit) can be used to capture eye images to obtain eye movement data. This eye movement data can include information such as eye position and orientation. In some embodiments, other sensors such as gyroscopes and accelerometers can be combined to assist in determining head posture, thereby obtaining more comprehensive eye movement data.

[0059] In some embodiments, the collected raw eye movement data may be preprocessed to remove noise and interference signals, for example, by using a Kalman filter algorithm to smooth the data so that it more realistically reflects the gaze behavior of the target object.

[0060] In some embodiments, an appropriate eye movement data sampling frequency may be set based on the actual application scenario of the target application and the requirements for response speed. For example, in scenarios such as comic reading that require a quick response to the target object's intentions, a higher sampling frequency (such as 60 frames per second or higher) may be used.

[0061] In some embodiments, before using the target application, a mapping relationship between eye movement data and screen coordinates can be established by having the target object look at multiple calibration points on the screen to perform data calibration and compensate for the impact of factors such as camera position deviation and the target object wearing glasses on the eye movement data.

[0062] Step S202 , determining a target gaze point of the target object on the client screen based on the eye movement data.

[0063] In some embodiments, the eye gaze direction can be calculated by substituting eye movement parameters (such as pupil center position and corneal reflection point position) from the eye movement data into a pre-established geometric or physical model. The gaze direction of both eyes can then be used to determine the location of the target gaze point on the screen using triangulation or stereo vision algorithms.

[0064] In some embodiments, eye movement data can be trained using a machine learning algorithm to construct a mapping model, mapping the features of the eye movement data to the position of the target gaze point in the screen coordinate system. By continuously training and optimizing the model parameters, the accuracy and adaptability of the target gaze point determination can be improved.

[0065] In some embodiments, the target subject's head posture can be monitored in real time. Sensors such as gyroscopes and accelerometers can be used to obtain head movement information. This information, combined with a model of the relationship between head posture and eye movement data, can dynamically compensate for gaze position, reducing gaze deviations caused by head shaking. Alternatively, the client's deflection angle can be monitored to obtain the client's posture, and eye movement data can be corrected based on the client's posture to improve recognition accuracy.

[0066] In some embodiments, when the target gaze point has a continuous moving trajectory, the position of the next target gaze point can be estimated through a prediction algorithm, and corrected based on the actual data obtained to improve the real-time and smoothness of the target gaze point determination.

[0067] As mentioned above, there are many methods for determining the target gaze point. An exemplary method for determining the target gaze point is provided below.

[0068] In an optional embodiment, if Figure 3 As shown, step S202 includes: S300: Acquire a target direction vector of the target object's eye gaze according to the eye movement data.

[0069] S302: Acquire the normal vector of the screen and binocular fitting points of the target object.

[0070] S304: Determine the target gaze point according to the target direction vector, the binocular fitting point, and the normal vector.

[0071] In some embodiments, the target direction vector can be determined based on the movement information of the eyeball in three-dimensional space (such as the rotation angle and rotation speed of the eyeball) and the data of multiple dimensions such as the horizontal rotation angle and vertical rotation angle of the eyeball.

[0072] In some embodiments, a large amount of labeled eye movement data can be trained based on a neural network to establish a prediction model, through which a more accurate target direction vector can be predicted based on the eye movement data obtained in real time.

[0073] In some embodiments, a posture sensor (such as a gyroscope, an accelerometer, etc.) of the client can be used to detect in real time whether the screen is deflected or moved, and the normal vector of the screen can be dynamically obtained according to the posture change of the screen.

[0074] The eye positions of the target subject can be determined using image processing algorithms, such as deep learning-based facial landmark detection algorithms, to further determine the binocular fitting points. In some embodiments, the binocular fitting points can also be modified by combining head posture data (such as head pitch angle, left and right tilt angle, etc.).

[0075] In some embodiments, the target direction vector, binocular fitting point, and normal vector may be converted into the coordinates of the target gaze point through a matrix operation method or a geometric calculation model.

[0076] In this embodiment, by obtaining the target direction vector, screen normal vector and binocular fitting point of the target object's gaze direction to calculate the target gaze point, the gaze position of the target object on the screen can be accurately calculated, thereby accurately determining the target object's gaze intention, so that more accurate control instructions can be generated subsequently, improving the accuracy and precision of the control.

[0077] As mentioned above, the target direction vector can be obtained through various methods. An exemplary method for obtaining the target direction vector is provided below.

[0078] In an optional embodiment, if Figure 4 As shown, step S300 includes: S400, obtaining an initial direction vector based on a first coordinate system according to the eye movement data; wherein the first coordinate system is a coordinate system with the center of the target object's head as the origin, and the initial direction vector is used to represent the eye gaze direction of the target object.

[0079] S402: Determine the target direction vector based on the second coordinate system and the initial direction vector, where the target direction vector is a representation of the initial direction vector based on the second coordinate system; wherein the second coordinate system is a coordinate system with the client front camera as its origin.

[0080] In the first coordinate system, the direction from the center of the head to the left can be set as the positive direction of the x-axis, the direction from the center of the head upward can be set as the positive direction of the y-axis, and the direction from the center of the head forward can be set as the positive direction of the z-axis. Figure 12In some embodiments, key feature points of the head can be captured in real time by multiple sensors such as a depth sensor and an infrared sensor, and the center position of the head can be calculated based on the coordinates of the multiple feature points.

[0081] In the second coordinate system, the x-axis positive direction can be set downward from the front camera, the y-axis positive direction can be set leftward from the front camera, and the z-axis positive direction can be set forward from the front camera. Figure 13 shown.

[0082] In some embodiments, an initial direction vector based on a first coordinate system can be converted into a target direction vector based on a second coordinate system using a matrix-based coordinate transformation algorithm or a combination of machine learning methods. The coordinate system transformation matrix can also be dynamically updated based on client movement or changes in the target object's posture to adjust the coordinate transformation relationship in real time.

[0083] In this embodiment, by converting the coordinates of the initial direction vector to the target direction vector, the eye movement data can be corresponded to the physical position and direction of the client screen, thereby more accurately calculating the target gaze point where the eye gaze direction intersects the screen plane, thereby improving the accuracy and reliability of the control.

[0084] As mentioned above, there are also multiple optional methods for obtaining the normal vector of the screen. An exemplary method is provided below.

[0085] In an optional embodiment, if Figure 5 As shown, step S302 includes: S500: Obtain a deflection angle of the client based on a second coordinate system, where the second coordinate system takes the front camera of the client as an origin.

[0086] S502: Acquire a screen plane based on the second coordinate system according to the deflection angle.

[0087] S504: Determine a normal vector based on the second coordinate system according to the screen plane.

[0088] By obtaining the client's deflection angle to determine the screen normal, we can dynamically compensate for the impact of client deflection on eye movement recognition and target gaze point determination, thereby more accurately determining the target gaze point and achieving high-precision eye tracking. Incorporating the client's deflection angle can reduce eye movement recognition errors, mitigate misjudgments caused by client deflection, and improve control accuracy and adaptability.

[0089] There are also many methods for determining the target gaze point. An exemplary method is provided below.

[0090] In an optional embodiment, if Figure 6As shown, step S304 includes: S600 : Determine a target distance parameter from the binocular fitting point to the screen according to the binocular fitting point, the target direction vector, and the normal vector.

[0091] S602: Determine the target gaze point according to the target distance parameter, the target direction vector, and the binocular fitting point.

[0092] The target distance parameter may represent how far along the target direction vector from the binocular fitting point is required to reach the screen plane. In some embodiments, the target distance parameter may be determined by calculating the ratio of the signed distance from the binocular fitting point to the screen plane to the projection of the target direction vector in the normal direction. This calculation process may be implemented using the following formula:

[0093] Among them, t is the target distance parameter, p eye are the coordinates of the eye fitting points, is the normal vector, is the target direction vector.

[0094] In other embodiments, a spherical model or ellipsoidal model in three-dimensional space may be used to approximately simulate the gaze scene of the human eye, and the target distance parameter may be determined through geometric operations in combination with binocular fitting points, target direction vectors, and normal vectors.

[0095] In some embodiments, the target gaze point can be obtained by adding the binocular fitting point to the product of the target distance parameter and the target direction vector. This calculation process can be implemented by the following formula:

[0096] Where D is the coordinate of the target gaze point, p eye is the coordinate of the binocular fitting point, t is the target distance parameter, is the target direction vector. This calculation process can be expressed as starting from the binocular fitting point, moving the target distance parameter along the target direction vector by a unit length to determine the position of the target gaze point. The position relationship diagram of the target object and the client screen can be shown as follows: Figure 14 shown.

[0097] In other embodiments, a machine learning algorithm can be trained on a large amount of known target gaze point data to construct a target gaze point prediction model. When determining the target gaze point, the target distance parameter, target direction vector, and binocular fitting points are used as input features, and the target gaze point is obtained through the target gaze point prediction model.

[0098] In this embodiment, by calculating the target distance parameters from the binocular fitting point to the screen and combining the target distance parameters, the target direction vector and the binocular fitting point, the target gaze point can be accurately determined, thereby achieving precise interactive control based on the target gaze point, enhancing the accuracy and reliability of the control.

[0099] Step S204 , generating a target control instruction according to the target gaze point.

[0100] Target control instructions can include page turning instructions, continuous page turning instructions, gaze duration statistics and processing instructions, etc. For example, when the target gaze point is in the hot zone at the left edge of the screen, a right page turning instruction can be generated. For another example, when the target gaze point is in the hot zone at the left edge of the screen and blinks twice, an instruction to continuously turn multiple pages can be generated. For another example, when the target gaze point stays on a certain page for too long (such as more than 1 minute), a gaze duration statistics and processing instruction can be generated, and a prompt window can pop up to ask the target object whether he needs to view the relevant explanation or information of the page.

[0101] In some embodiments, the client screen can be divided into different areas, each area corresponds to a different operation instruction weight or priority, and when the target gaze point is located in a specific area, a target control instruction is generated according to the corresponding rules.

[0102] In some embodiments, the movement trajectory of the target gaze point on the screen can also be tracked, and different control instructions can be generated based on the shape, direction, speed, and other characteristics of the trajectory. For example, if the target gaze point moves along a specific straight or curved trajectory and conforms to a preset page-turning gesture pattern, a target control instruction for turning the page will be generated.

[0103] In this embodiment, the target gaze point is converted into a target control instruction, and the operation intention of the target object contained in the target gaze point can be converted into a clear operation instruction without the need for the target object to perform additional physical operations, thereby improving the intelligence and convenience of the control and enhancing the immersion and usage experience of the target object when using the target application.

[0104] As mentioned above, target control instructions can be generated by various methods. An exemplary generation method is provided below.

[0105] In an optional embodiment, if Figure 7 As shown, step S204 includes: S700: Acquire the screen size of the client.

[0106] S702: Determine whether the target gaze point is located in a target area on the screen according to the target gaze point, the screen plane, and the screen size.

[0107] S704: When the target gaze point is located in the target area, generate a corresponding target control instruction.

[0108] The screen size can be expressed in terms of pixel size (ppi) or the actual screen size calculated based on the ppi. In some embodiments, it can first be determined whether the screen is currently in landscape or portrait mode. For different screen orientations, the corresponding screen width and height are obtained, and the coordinates of the target gaze point are calculated based on the change in screen orientation.

[0109] The target area can be a fixed-width area at the edge of the screen, a custom-shaped hotspot, or other areas. Different target areas can be configured based on different application scenarios and target objects. For example, in a comic reading app, the target areas can be set at the left and right edges of the screen to trigger page turning.

[0110] In some embodiments, the position, size, and shape of the target area on the screen can be adjusted in real time based on dynamic changes in screen content (such as plot climaxes and areas with frequent screen switching). For example, when a comic strip is being displayed on the screen, the target area can be appropriately expanded during exciting plot sections to make it easier for the target object to trigger corresponding actions through eye movements. When there are multiple target areas, weights can be assigned based on their importance, prioritizing gaze operations on high-weighted target areas, thereby improving the flexibility and adaptability of the interaction.

[0111] In this embodiment, whether to generate a target control instruction is determined based on the relative positional relationship between the target gaze point and the target area, which can achieve accurate recognition and control of the target object's eye movement intention, thereby accurately judging whether the target object has the intention to trigger the target control operation, thereby improving the reliability and practicality of the control.

[0112] As mentioned above, there are many optimization methods in the process of generating target control instructions. Several exemplary optimization methods are provided below.

[0113] Optimization method 1: Figure 8 As shown, step S704 includes: S800: When the target gaze point is located in the target area, determine the residence time of the target gaze point in the target area.

[0114] S802: When the stay time exceeds a preset threshold, generate a corresponding target control instruction.

[0115] In some embodiments, a timestamp can be used to record the precise time when the target gaze point enters the target area, and its residence time in the target area can be monitored in real time. A permissible interruption time threshold can also be set. If the target gaze point briefly leaves the target area, as long as it returns within the interruption time threshold, it will still be considered as a continuous residence time for cumulative calculation.

[0116] In some embodiments, the preset threshold can also be dynamically adjusted based on the target subject's behavior pattern. For example, by analyzing the target subject's reading habits and eye movement patterns over a period of time, if the target subject generally gazes longer when reading text-intensive areas and shorter when reading pages dominated by images, a longer preset threshold can be set when the target area contains more text, and a shorter preset threshold can be set when the target area contains less text but more images.

[0117] In some embodiments, a misoperation feature model can also be established based on the historical misoperation records of the target object (such as frequently turning pages back and forth in a short period of time). When similar features are detected, a secondary confirmation mechanism is triggered or a preset threshold is adjusted.

[0118] In this embodiment, the target control instruction is generated only when the target gaze point stays in the target area for more than a preset threshold, which can further determine the operation intention of the target object, reduce the triggering of erroneous operations, and improve the target object's user experience when using the target application.

[0119] As mentioned above, the target area can have many different settings, and the target control instructions also have many different types. The following provides an exemplary case.

[0120] In an optional embodiment, the target application may be a comic reading application, and there are multiple target areas, including a first area at the left edge, a second area at the right edge, and / or a third area at the upper edge and a fourth area at the lower edge. Figure 9 As shown, step S704 includes: S900: When the target gaze point is located in the first area, generate a left page turning instruction.

[0121] S902: When the target gaze point is located in the second area, generate a right page turning instruction.

[0122] S904: When the target gaze point is located in the third area, generate a page-up instruction.

[0123] S906: When the target gaze point is located in the fourth area, generate a next page turning instruction.

[0124] The settings of the first to fourth areas on the screen can be as follows Figure 15 The target application may also be an electronic book reading program, a video player program, a map navigation program, a slide show player program, etc. Different target control instruction combinations may be set for different target applications.

[0125] In some embodiments, the target area can be further subdivided within the first to fourth areas. For example, within the first area on the left edge and the second area on the right edge, sub-areas near the top and bottom can be further divided, so that the generated page turning instructions can not only control left and right page turning, but also combine the difference in top and bottom directions to achieve functions such as adjusting the view position while turning the page.

[0126] In some embodiments, the target area can also be set to a size, shape, and position that can be customized by the target subject based on personal habits and preferences. For example, the target subject may be accustomed to looking at the corner of the screen to trigger page turning, so the target area can be set to a smaller angular area.

[0127] In some embodiments, the target control instructions corresponding to each target area can also be other settings or customized by the target object. For example, the third area can correspond to the current page enlargement instruction, and the fourth area can correspond to the current page reduction instruction.

[0128] In some embodiments, after a page-turning command is generated, a brief dynamic effect (such as a page corner flipping animation) or sound prompt may appear at the edge of the screen, letting the target user know that the target control command has been recognized and executed. An undo function may also be configured. If the target user accidentally triggers a page-turning command or wishes to return to the previous page, they can quickly undo the previous page-turning operation by, for example, gazing at a specific undo area or using other pre-defined input methods.

[0129] In this embodiment, a corresponding page-turning instruction is generated based on the specific area where the target's gaze is located. This accurately determines the target's intention to turn the page and replaces the target's manual operation to turn the page. By directly mapping the target's reading intention through eye tracking, a "what you see is what you turn" approach can be achieved, improving the efficiency of human-computer interaction and the convenience of comic reading, and enhancing the target's immersive experience and experience using the comic reading program.

[0130] Optimization method 2: Figure 10 As shown, step S704 includes: S1000 , when the target gaze point is located in a target area on the screen, obtaining a page turning record of a previous page turning.

[0131] S1002: Determine a page turning interval according to the page turning record and the current time.

[0132] S1004: Determine whether the target gaze point has left the historical page turning area corresponding to the page turning record within the page turning interval.

[0133] S1006 , when the target gaze point has left the historical page turning area, generating a corresponding target control instruction.

[0134] S1008: Refuse to generate a target control instruction if the target gaze point has not left the historical page turning area.

[0135] For example, if the target gaze point is currently detected at the bottom edge of the screen (corresponding to the "page down" operation), and the "page down" operation is triggered 2 seconds ago (i.e., the page turning interval), and the target gaze point has not left the bottom edge area of ​​the screen within these 2 seconds, the "page down" instruction (i.e., the target control instruction) will be refused to generate, limiting automatic page turning.

[0136] The page turning record may include a timestamp, page turning direction, dwell time, etc. In some embodiments, a time threshold may be set for the page turning interval, and the generation of the target control instruction may be triggered only when the page turning interval exceeds the time threshold and the target gaze point leaves the historical page turning area.

[0137] In some embodiments, a weighted calculation may be performed on the time the target gaze point stays in the historical page turning area and the time it stays in other areas of the screen after leaving the area, and whether to trigger a page turning operation may be determined based on the comprehensive weight.

[0138] For example, if the target gaze point stays in the history page turning area for a long time, but stays in other areas of the screen for a short time after leaving, it may be considered that the target gaze point has only temporarily left and will not trigger the generation of the target control instruction. Conversely, if the target gaze point stays in other areas of the screen for a certain period of time, it can trigger the generation of the target control instruction.

[0139] In some embodiments, the target subject may be allowed to choose whether to enable the eye movement page turning function, record or upload eye movement data, etc. The target subject may also set the eye movement data to be used only locally and not uploaded to the cloud.

[0140] In this embodiment, the generation of the target control instruction can only be triggered when the target gaze point leaves the historical page turning area, which can reduce abnormal page turning triggered by erroneous operation of the target object, improve the accuracy and reliability of eye movement recognition and control, and enhance the control experience of the target object.

[0141] Step S206 , in response to the target control instruction, controlling the target application to perform a corresponding target operation.

[0142] The target control instruction may be a page-up instruction, a page-down instruction, a page-left instruction, or a page-right instruction, and the corresponding target operation may be a page-up operation, a page-down operation, a page-left operation, or a page-right operation. In some embodiments, the target control instruction may also be an available instruction such as a page zoom instruction or a content display area switching instruction, and the corresponding target operation may also be adaptively set.

[0143] In some embodiments, an operation frequency limit can be set to restrict the number of target operations that can be performed per unit time. Visual feedback animations can also be displayed at the corresponding screen location after the target object successfully triggers the target operation. For example, when a page is successfully turned, a dynamic page turning effect appears at the edge of the page, accompanied by light and shadow changes; when a zoom operation is successful, a border halo appears on the magnified area and gradually expands to the final magnified size.

[0144] In order to make this application easier to understand, the following Figure 11 An exemplary application is provided, wherein: S11, user U (i.e., target subject) opens comics software B (i.e., target application) on mobile phone (i.e., client) and browses comics M1; S12, monitoring the eye movement data of user U through the camera of the mobile phone; S13. Determine the gaze direction vector A (i.e., the initial direction vector) of the user U based on the facial coordinate system (i.e., the first coordinate system) based on the acquired eye movement data. S14, the sight direction vector of the user U based on the facial coordinate system , converted into a direction vector based on the world coordinate system (ie, the second coordinate system) (i.e. target direction vector); S15. Calculate the plane P (i.e., screen plane) of the mobile phone screen in the world coordinate system and the normal vector of plane P based on the deflection angle of the mobile phone. ; S16, according to the direction vector , normal vector and user U's binocular fitting point p eye , calculate the parameter t (i.e., target distance parameter) that represents the distance between the binocular fitting point and the mobile phone screen; S17, according to the parameter t, direction vector and user U's binocular fitting point p eye , calculate the gaze point D (i.e., target gaze point) of user U on the mobile phone screen based on the world coordinate system; S18. Determine, based on the plane P, the gaze point D, and the size of the mobile phone screen, that the gaze point D is within the lower edge hot zone (i.e., the target area) of the mobile phone screen; S19: If it is detected that the gaze point D has stayed in the lower edge hot zone for more than 500ms (i.e., the preset threshold), and the time of the last page turn is 2s (i.e., the page turn interval), and the gaze point D has left the lower edge hot zone within 2s, then a "page turn down" instruction (i.e., the target control instruction) is generated; S20. According to the "page down" instruction, the comic M1 currently being browsed is turned down (ie, the target operation is performed).

[0145] Example 2 Figure 16 The block diagram of the control device according to the second embodiment of the present application is schematically shown. It is used in a client with a target application installed. The device can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Figure 16 As shown, the apparatus 1000 may include: an acquisition module 1100, a determination module 1200, a generation module 1300, and a control module 1400, wherein: An acquisition module 1100 is configured to acquire eye movement data of a target object while running the target application; A determination module 1200 is configured to determine a target gaze point of the target object on the client screen based on the eye movement data; A generating module 1300 is configured to generate a target control instruction according to the target gaze point; and The control module 1400 is configured to control the target application to execute a corresponding target operation in response to the target control instruction.

[0146] As an optional embodiment, the determining module 1200 is further configured to: Obtaining a target direction vector of the target object's eye gaze according to the eye movement data; Acquire a normal vector of the screen and binocular fitting points of the target object; The target gaze point is determined according to the target direction vector, the binocular fitting point, and the normal vector.

[0147] As an optional embodiment, the determining module 1200 is further configured to: Obtaining an initial direction vector based on a first coordinate system according to the eye movement data; wherein the first coordinate system is a coordinate system with the center of the target object's head as the origin, and the initial direction vector is used to represent the gaze direction of the target object's eyes; The target direction vector is determined based on the second coordinate system and the initial direction vector, where the target direction vector is a representation of the initial direction vector based on the second coordinate system; wherein the second coordinate system is a coordinate system with the client front camera as its origin.

[0148] As an optional embodiment, the determining module 1200 is further configured to: Obtaining a deflection angle of the client based on a second coordinate system, where the second coordinate system takes the front camera of the client as an origin; acquiring a screen plane based on the second coordinate system according to the deflection angle; A normal vector based on the second coordinate system is determined according to the screen plane.

[0149] As an optional embodiment, the determining module 1200 is further configured to: Determining a target distance parameter from the binocular fitting point to the screen according to the binocular fitting point, the target direction vector, and the normal vector; The target gaze point is determined according to the target distance parameter, the target direction vector and the binocular fitting point.

[0150] As an optional embodiment, the generating module 1300 is further configured to: Obtaining the screen size of the client; determining, according to the target gaze point, the screen plane, and the screen size, whether the target gaze point is located in a target area on the screen; When the target gaze point is located in the target area, a corresponding target control instruction is generated.

[0151] As an optional embodiment, the generating module 1300 is further configured to: In a case where the target gaze point is located in the target area, determining a stay time of the target gaze point in the target area; When the stay time exceeds a preset threshold, a corresponding target control instruction is generated.

[0152] As an optional embodiment, the target application is a comic reading application, and there are multiple target areas, including a first area located at the left edge, a second area at the right edge, and / or a third area at the upper edge and a fourth area at the lower edge. The generating module 1300 is further configured to: When the target gaze point is located in the first area, generating a left page turning instruction; If the target gaze point is located in the second area, generating a right page turning instruction; and / or generating a page-up instruction when the target gaze point is located in the third area; When the target gaze point is located in the fourth area, a page-down instruction is generated.

[0153] As an optional embodiment, the generating module 1300 is further configured to: When the target gaze point is located in the target area, obtaining a page turning record of a previous page turning; Determining a page turning interval according to the page turning record and the current time; determining whether the target gaze point has left the historical page turning area corresponding to the page turning record within the page turning interval; In the case where the target gaze point leaves the history page turning area, generating a corresponding target control instruction; and If the target gaze point has not left the historical page turning area, the target control instruction is refused to be generated.

[0154] Example 3 Figure 17 The following schematically shows a hardware architecture diagram of a computer device 10000 suitable for implementing the control method according to the third embodiment of the present application. In some embodiments, the computer device 10000 can be a terminal device such as a smart phone, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. Figure 17 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other via a system bus. Memory 10010 includes at least one type of computer-readable storage medium, including flash memory, a hard disk, a multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, and the like. In some embodiments, memory 10010 may be an internal storage module of computer device 10000, such as a hard disk or memory of computer device 10000. In other embodiments, memory 10010 may also be an external storage device of computer device 10000, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, and the like equipped on computer device 10000. Of course, memory 10010 may also include both internal storage modules and external storage devices of computer device 10000. In this embodiment, the memory 10010 is generally used to store an operating system and various application software installed on the computer device 10000, such as program codes of control methods, etc. In addition, the memory 10010 can also be used to temporarily store various data that has been output or is to be output.

[0155] In some embodiments, processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. Processor 10020 is typically used to control the overall operation of computer device 10000, such as performing control and processing related to data exchange or communication with computer device 10000. In this embodiment, processor 10020 is used to execute program code stored in memory 10010 or process data.

[0156] Network interface 10030 may include a wireless network interface or a wired network interface. Network interface 10030 is typically used to establish a communication link between computer device 10000 and other computer devices. For example, network interface 10030 is used to connect computer device 10000 to an external terminal via a network, establishing a data transmission channel and a communication link between computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile Communications (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth, or Wi-Fi.

[0157] It should be pointed out that Figure 17 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementing all of the shown components is not a requirement, and more or fewer components may alternatively be implemented.

[0158] In this embodiment, the control method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0159] Example 4 An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the control method in the embodiment when executed by a processor.

[0160] In this embodiment, computer-readable storage media include flash memory, hard disks, multimedia cards, card-type memories (e.g., SD or DX memories), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disks, optical disks, and the like. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as the computer device's hard disk or memory. In other embodiments, the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, and the like. Of course, the computer-readable storage medium may also include both the internal storage unit and external storage devices of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the control method described in the embodiment. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that has been output or is about to be output.

[0161] Example 5 An embodiment of the present application further provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0162] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented using general-purpose computer devices. They can be concentrated on a single computer device or distributed on a network composed of multiple computer devices. Alternatively, they can be implemented using program codes executable by the computer device, so that they can be stored in a storage device and executed by the computer device. In some cases, the steps shown or described can be performed in a different order than herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0163] It should be noted that the above are only preferred embodiments of the present application and do not limit the scope of patent protection of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the scope of patent protection of the present application.

Claims

1. A control method, characterized in that: Used in a client having a target application installed, the method includes: While running the target application, acquiring eye movement data of the target object; determining a target gaze point of the target object on the client screen according to the eye movement data; Generate a target control instruction according to the target gaze point; and In response to the target control instruction, the target application is controlled to perform a corresponding target operation.

2. The method according to claim 1, characterized in that Determining a target gaze point of the target object on the client screen according to the eye movement data includes: Obtaining a target direction vector of the target object's eye gaze according to the eye movement data; Acquire a normal vector of the screen and binocular fitting points of the target object; The target gaze point is determined according to the target direction vector, the binocular fitting point, and the normal vector.

3. The method according to claim 2, characterized in that Obtaining a target direction vector of the target object's eye gaze according to the eye movement data, including: Obtaining an initial direction vector based on a first coordinate system according to the eye movement data; wherein the first coordinate system is a coordinate system with the center of the target object's head as the origin, and the initial direction vector is used to represent the gaze direction of the target object's eyes; The target direction vector is determined based on the second coordinate system and the initial direction vector, where the target direction vector is a representation of the initial direction vector based on the second coordinate system; wherein the second coordinate system is a coordinate system with the client front camera as its origin.

4. The method according to claim 2, characterized in that Acquiring a normal vector of the screen and binocular fitting points of the target object, including: Obtaining a deflection angle of the client based on a second coordinate system, where the second coordinate system takes the front camera of the client as an origin; acquiring a screen plane based on the second coordinate system according to the deflection angle; A normal vector based on the second coordinate system is determined according to the screen plane.

5. The method according to claim 2, characterized in that Determining the target gaze point according to the target direction vector, the binocular fitting point, and the normal vector includes: Determining a target distance parameter from the binocular fitting point to the screen according to the binocular fitting point, the target direction vector, and the normal vector; The target gaze point is determined according to the target distance parameter, the target direction vector and the binocular fitting point.

6. The method according to claim 4, characterized in that Generating a target control instruction according to the target gaze point includes: Obtaining the screen size of the client; determining, according to the target gaze point, the screen plane, and the screen size, whether the target gaze point is located in a target area on the screen; When the target gaze point is located in the target area, a corresponding target control instruction is generated.

7. The method according to claim 6, characterized in that When the target gaze point is located in the target area, generating a corresponding target control instruction includes: In a case where the target gaze point is located in the target area, determining a stay time of the target gaze point in the target area; When the stay time exceeds a preset threshold, a corresponding target control instruction is generated.

8. The method according to claim 6, characterized in that The target application is a comic reading application, and there are multiple target areas, including a first area at the left edge, a second area at the right edge, and / or a third area at the upper edge and a fourth area at the lower edge; When the target gaze point is located in the target area, generating a corresponding target control instruction includes: When the target gaze point is located in the first area, generating a left page turning instruction; If the target gaze point is located in the second area, generating a right page turning instruction; and / or generating a page-up instruction when the target gaze point is located in the third area; When the target gaze point is located in the fourth area, a page-down instruction is generated.

9. The method according to claim 6, characterized in that When the target gaze point is located in the target area, generating a corresponding target control instruction includes: When the target gaze point is located in the target area, obtaining a page turning record of a previous page turning; Determining a page turning interval according to the page turning record and the current time; determining whether the target gaze point has left the historical page turning area corresponding to the page turning record within the page turning interval; In the case where the target gaze point leaves the history page turning area, generating a corresponding target control instruction; and If the target gaze point has not left the historical page turning area, the target control instruction is refused to be generated.

10. A control device, characterized in that: Used in a client having a target application installed, the device includes: an acquisition module, configured to acquire eye movement data of a target object while running the target application; A determination module, configured to determine a target gaze point of the target object on the client screen based on the eye movement data; A generating module, configured to generate a target control instruction according to the target gaze point; and The control module is configured to control the target application to execute a corresponding target operation in response to the target control instruction.

11. A computer device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 9 are implemented.

Images