Control method of intelligent terminal and intelligent terminal

Abstract

This invention discloses a control method and a smart terminal, relating to the field of smart terminal technology. The control steps are as follows: S1, acquiring the user's eye gaze position based on a camera and converting the gaze direction into specific position coordinates on the screen; S2, detecting the user's touch position on the screen based on a touchscreen sensor and acquiring the coordinates of the touch position; S3, comparing the acquired gaze position coordinates with the touch position coordinates, determining whether the gaze position is within the range defined by the touch position, establishing a judgment range threshold, and performing the comparison; S4, if the gaze position and the touch position are consistent, responding to the user's touch and executing operation control. In terms of interaction accuracy, this invention integrates the gaze position coordinates acquired by the camera and the touch position coordinates of the touchscreen, solving the problem of inaccurate target object positioning caused by large touchscreen partitions, improving the accuracy of touchscreen touch response, and preventing misoperation.

Description

Technical Field

[0001] This invention relates to the field of smart terminal technology, specifically to a control method for a smart terminal and a smart terminal. Background Technology

[0002] In today's era of rapid development in information technology, smart terminal devices have been deeply integrated into people's daily life and work, becoming indispensable tools. As the core link for smart terminals to realize their functions and services, the convenience, accuracy, and intelligence of human-computer interaction directly affect the user experience and the application efficiency of the device.

[0003] Currently, the touch operation method commonly used in smart terminals relies on the direct contact between the user's finger and the screen to trigger commands. However, this interaction method has many limitations. In some special scenarios, such as when the user's hands are occupied or their finger dexterity is limited, it is difficult to complete touch operations smoothly. At the same time, for devices with large screen sizes, single-handed operation often cannot reach the entire screen area, requiring frequent adjustments to hand posture, increasing the user's operational burden and reducing efficiency. In addition, traditional touch interaction lacks the ability to predict the user's operational intentions, executing commands solely based on the touch location, which is insufficient to meet users' needs for efficient and natural interaction, and is prone to errors in calculating the operation object, leading to misoperations. To address these issues, we propose a control method and a smart terminal. Summary of the Invention

[0004] To solve the above-mentioned technical problems, this technical solution provides a control method for a smart terminal and a smart terminal in general.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] The control method and steps for the smart terminal are as follows:

[0007] S1. Obtain the user's eye gaze position based on the camera, and convert the gaze direction into specific position coordinates on the screen;

[0008] S2. Detect the location of the user's touch on the screen based on the touch screen sensor and obtain the coordinates of the touch location;

[0009] S3. Compare the obtained gaze position coordinates with the touch position coordinates to determine whether the gaze position is within the range defined by the touch position, establish a judgment range threshold, and perform the comparison.

[0010] S4. If the gaze position matches the touch position, then respond to the user's touch and execute the operation control.

[0011] S5. When the gaze position and the touch position are inconsistent, the touch operation is not responded to, the screen gives a vibration prompt to remind the user, the user performs the touch operation again, and the control operation data is stored.

[0012] S6. When determining whether the gaze position and the touch position are consistent, the range threshold of the judgment is adaptively adjusted based on user habits and operation scenario factors.

[0013] Preferably, the position coordinate determination step in step S1 is as follows:

[0014] The system acquires facial images using a camera, and extracts eye features from these images using an eye detection algorithm.

[0015] An eye model is established to describe the geometric structure and movement of the eye. Based on the position of the eye feature points and the eye model, the vector from the center of the eye to the gaze point, i.e. the gaze vector, is calculated, which represents the user's gaze direction.

[0016] By displaying calibration points on the screen and allowing the user to focus on those points, the eye features captured by the camera and their corresponding screen coordinates are recorded. A mapping relationship between the gaze direction and screen coordinates is established, and based on this mapping relationship, the calculated gaze direction is converted into specific position coordinates on the screen.

[0017] Preferably, the eyeball feature extraction steps are as follows:

[0018] After preprocessing the images captured by the camera, the region where the face is located is found in the preprocessed image, the face bounding box is determined, and the face is separated from the whole image.

[0019] Within the already located face region, a deep learning-based eye detection model determines the specific location of the eyes within the face region, obtains the bounding box of the eyes, and narrows down the processing range.

[0020] The image of the eye region is binarized to divide it into foreground and background, highlighting the eyeball from the eye region. An edge detection algorithm is used to find the edge contour of the eyeball, obtain edge information, and analyze the edge information to determine the key feature points of the eyeball.

[0021] Preferably, the steps for determining the gaze direction are as follows:

[0022] A composite model is used to simulate the structure of a real eye, the model parameters are defined, and the initial position and orientation of the model in three-dimensional space are determined.

[0023] Based on the coordinates of the extracted eye feature points in the two-dimensional image, the eye feature points on the two-dimensional image are mapped to three-dimensional space through the inverse transformation of perspective projection to obtain the three-dimensional coordinates of the feature points in the world coordinate system.

[0024] Based on the established geometric model of the eye and the determined three-dimensional coordinates of the eye feature points, the center position of the eye in three-dimensional space is determined through geometric calculation.

[0025] Set a target point on the screen as the gaze point, determine the actual gaze target position, and obtain the coordinates of the gaze point in three-dimensional space;

[0026] Based on vector operations, a three-dimensional gaze vector is calculated starting from the center of the eye and ending at the gaze point. The user's gaze direction is determined by analyzing the gaze vector direction.

[0027] Preferably, the step of obtaining the coordinates of the touch position in step S2 is as follows:

[0028] When a user touches the touchscreen, the signal acquisition circuit inside the touchscreen acquires the electrical signals generated by the sensor and converts them into digital signals, with the upper left corner of the screen as the origin (0,0), the horizontal axis as the X-axis, and the vertical axis as the Y-axis.

[0029] The system processes and calibrates the digital signal when the user touches the screen, calculates the coordinates of the touch position, determines the touch point based on the position and intensity of the capacitance change, obtains the X and Y coordinates of the touch point in the touch screen coordinate system, and acquires the coordinate information of the touch position.

[0030] Preferably, the comparison step in step S3 is as follows:

[0031] Obtain the coordinates of the gaze position and the touch position;

[0032] Based on experimental analysis, a judgment range threshold is set. The range threshold is the maximum distance that the gaze position is allowed to deviate from, centered on the touch position.

[0033] Calculate the Euclidean distance between the gaze position and the touch position;

[0034] The calculated Euclidean distance is compared with a set range threshold. If the distance is less than or equal to the threshold, the gaze position is determined to be within the range defined by the touch position; otherwise, it is determined to be outside the range.

[0035] Preferably, in step S4, when a match is determined, the corresponding control logic is executed based on the operation corresponding to the touch position.

[0036] Preferably, the adaptive adjustment of the judgment range threshold step in step S6 is as follows:

[0037] Continuously collect data on the user's gaze position, touch position, and corresponding operation results;

[0038] Record the current operation scenario and the operation status under different task types;

[0039] During system initialization, a default judgment range threshold is set for each scene category;

[0040] After each operation, evaluate the success rate. If the operation is successful, the current threshold is appropriate. If the operation fails, the threshold needs to be adjusted.

[0041] The threshold is dynamically adjusted based on the success rate of the operation and user feedback. If the success rate of the operation is low, the threshold is increased.

[0042] A smooth adjustment strategy is adopted, with each adjustment based on a proportional adjustment threshold;

[0043] Regularly mine and analyze the collected user operation data.

[0044] Preferably, a certain number of recent operations are selected to calculate the success rate. The formula for calculating the success rate is: number of successful operations / total number of operations.

[0045] The intelligent terminal, and the control method applied to any of the above-mentioned intelligent terminals, includes: a camera, a touch screen, a vibration module, a storage unit, and a sensor, wherein the camera is located directly above the touch screen, and the vibration module, storage unit, and sensor are all located inside the intelligent terminal.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0047] In terms of interaction accuracy, this invention integrates the gaze position coordinates obtained by the camera with the touch position coordinates of the touch screen to avoid misoperation. In terms of convenience, the system automatically responds to operations when the two coordinates match, simplifying the process. The system can also adaptively adjust the judgment threshold based on user habits and scenarios to meet personalized needs. When the gaze and touch positions are inconsistent, the system guides the user to correct the position through vibration prompts and stores operation data for analysis and optimization. This solves the problem of inaccurate positioning of target objects caused by large touch screen partitions, improves the accuracy of touch screen touch response, and prevents misoperation. Attached Figure Description

[0048] Figure 1 This is a flowchart of the control steps of the present invention;

[0049] Figure 2 This is a schematic diagram of the smart terminal of the present invention. Detailed Implementation

[0050] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0051] Reference Figure 1 As shown, the control method for the smart terminal includes the following control steps:

[0052] S1. Obtain the user's eye gaze position based on the camera, and convert the gaze direction into specific position coordinates on the screen;

[0053] S2. Detect the location of the user's touch on the screen based on the touch screen sensor and obtain the coordinates of the touch location;

[0054] S3. Compare the obtained gaze position coordinates with the touch position coordinates to determine whether the gaze position is within the range defined by the touch position, establish a judgment range threshold, and perform the comparison.

[0055] S4. If the gaze position matches the touch position, then respond to the user's touch and execute the operation control.

[0056] S5. When the gaze position and the touch position are inconsistent, the touch operation is not responded to, the screen gives a vibration prompt to remind the user, the user performs the touch operation again, and the control operation data is stored.

[0057] S6. When determining whether the gaze position and the touch position are consistent, the range threshold of the judgment is adaptively adjusted based on user habits and operation scenario factors.

[0058] This application achieves a comprehensive improvement in interactive performance. In terms of interactive accuracy, it integrates the gaze position coordinates obtained by the camera with the touch position coordinates of the touch screen to avoid misoperation. In terms of convenience, it automatically responds to operations when the two are consistent, simplifying the process. The system can also adaptively adjust the judgment threshold based on user habits and scenarios to meet personalized needs, such as providing different precision interactive experiences in game and browsing scenarios. When the gaze and touch positions are inconsistent, it guides the user to correct through vibration prompts and stores operation data for analysis and optimization. In addition, this method breaks through the limitations of traditional interaction, helping to expand the application of smart terminals in emerging fields such as VR and AR. At the same time, it enhances the system's adaptive capabilities, dynamically optimizing interaction parameters according to user behavior and scene changes to ensure stable and efficient interactive services under various usage conditions.

[0059] The steps for determining the position coordinates in step S1 are as follows:

[0060] The system acquires facial images using a camera, and extracts eye features from these images using an eye detection algorithm.

[0061] An eye model is established to describe the geometric structure and movement of the eye. Based on the position of the eye feature points and the eye model, the vector from the center of the eye to the gaze point, i.e. the gaze vector, is calculated, which represents the user's gaze direction.

[0062] By displaying calibration points on the screen and allowing the user to focus on those points, the eye features captured by the camera and their corresponding screen coordinates are recorded. A mapping relationship between the gaze direction and screen coordinates is established, and based on this mapping relationship, the calculated gaze direction is converted into specific position coordinates on the screen.

[0063] This application utilizes an eye detection algorithm to extract eye features from facial images, accurately capturing subtle changes in the eyes. Combined with the established eye model, it calculates the gaze vector, quantifying eye movements into analyzable gaze direction data from a physiological structural perspective. Based on this, by calibrating and establishing a mapping relationship between the gaze direction and screen coordinates, the user's gaze direction can be accurately converted into screen position coordinates. This effectively avoids the accidental touch problem caused by operation errors in traditional interaction methods, greatly improving the accuracy of interaction and meeting the needs of scenarios with extremely high positioning accuracy requirements, such as drawing design and precision operation.

[0064] The steps for extracting eyeball features are as follows:

[0065] After preprocessing the images captured by the camera, the region where the face is located is found in the preprocessed image, the face bounding box is determined, and the face is separated from the whole image.

[0066] Within the already located face region, a deep learning-based eye detection model determines the specific location of the eyes within the face region, obtains the bounding box of the eyes, and narrows down the processing range.

[0067] The image of the eye region is binarized to divide it into foreground and background, highlighting the eyeball from the eye region. An edge detection algorithm is used to find the edge contour of the eyeball, obtain edge information, and analyze the edge information to determine the key feature points of the eyeball.

[0068] Eye location is determined using a deep learning-based eye detection model: the input is a face region image. Eye detection model based on deep learning After processing, output the bounding box coordinates of the eyes within the face region. , is represented as:

[0069]

[0070] in These are the coordinates of the top-left corner of the bounding box. These are the coordinates of the bottom right corner of the bounding box; the eye region image can be extracted using this bounding box. :

[0071]

[0072] Binarization of eye region images: For eye region images Let the binarization threshold be... Using a binary function The image is processed to divide it into foreground and background (values ​​are...). ) and background (value is ), to obtain a binarized image :

[0073]

[0074] in Represents the pixel coordinates in the image.

[0075] Edge detection algorithms are used to find the contours of the eyeball's edge: The Canny edge detection algorithm is used to... For binarized images Processing is performed to obtain the edge image. :

[0076]

[0077] In edge image In the middle, the value is Pixels with a value of 0 represent edge points, and pixels with a value of 0 represent non-edge points;

[0078] Analyzing edge information to determine key eye feature points: [Analyzing edge images] For analysis, let the feature point detection function be... By analyzing edge information (such as the curvature and direction of the edges), the set of key feature points of the eyeball is determined. , is represented as:

[0079]

[0080] in Indicates the first The coordinates of the key feature points.

[0081] The steps for determining the gaze direction are as follows:

[0082] A composite model is used to simulate the structure of a real eye, the model parameters are defined, and the initial position and orientation of the model in three-dimensional space are determined.

[0083] Based on the coordinates of the extracted eye feature points in the two-dimensional image, the eye feature points on the two-dimensional image are mapped to three-dimensional space through the inverse transformation of perspective projection to obtain the three-dimensional coordinates of the feature points in the world coordinate system.

[0084] Based on the established geometric model of the eye and the determined three-dimensional coordinates of the eye feature points, the center position of the eye in three-dimensional space is determined through geometric calculation.

[0085] Set a target point on the screen as the gaze point, determine the actual gaze target position, and obtain the coordinates of the gaze point in three-dimensional space;

[0086] Based on vector operations, a three-dimensional gaze vector is calculated starting from the center of the eye and ending at the gaze point. The user's gaze direction is determined by analyzing the gaze vector direction.

[0087] A composite model is used to simulate the structure of a real eye and determine its initial position and pose: Let the composite eye model be... The model parameters are The initial position of the model in three-dimensional space is The initial attitude is achieved using a rotation matrix. It is represented as:

[0088]

[0089] in It is an initialization function used to determine the model and its initial state in three-dimensional space based on the model parameters;

[0090] Mapping eye feature points from a 2D image to 3D space: Let the set of coordinates of the extracted 2D eye feature points be... ,in Indicates the first The coordinates of each feature point in the 2D image, and the camera's intrinsic parameter matrix are... The extrinsic parameter matrix is and (Transformation from world coordinate system to camera coordinate system) Through the inverse transformation of perspective projection, two-dimensional feature points are mapped to three-dimensional space, resulting in the set of three-dimensional coordinates of the feature points in the world coordinate system. ,in The inverse transform formula is as follows:

[0091]

[0092] By solving the above equations (which usually requires solving a system of multiple feature points simultaneously), we obtain... :

[0093]

[0094] in It is the inverse projection function;

[0095] Determine the center position of the eye in three-dimensional space: Establish the geometric model of the eye. The geometric relation function is Based on the determined three-dimensional coordinates of eye feature points The center position of the eye in three-dimensional space is determined through geometric calculations. :

[0096]

[0097] Obtain the coordinates of the gaze point in 3D space: The coordinates of the target point (gaze point) set on the screen in the screen coordinate system are... Similarly, through the camera intrinsic parameter matrix extrinsic parameter matrix and And related spatial transformations, converting it to three-dimensional space to obtain the coordinates of the gaze point in three-dimensional space: :

[0098]

[0099] in: It is a function that transforms screen coordinates to three-dimensional space;

[0100] Calculate the 3D gaze vector and determine the user's gaze direction: based on the eye center position. Starting from the position of the gaze point Calculate the 3D gaze vector with the endpoint as the endpoint. :

[0101]

[0102] By observing the gaze vector Directional analysis determines the user's gaze direction.

[0103] The steps for obtaining the coordinates of the touch position in step S2 are as follows:

[0104] When a user touches the touchscreen, the signal acquisition circuit inside the touchscreen acquires the electrical signals generated by the sensor and converts them into digital signals, with the upper left corner of the screen as the origin (0,0), the horizontal axis as the X-axis, and the vertical axis as the Y-axis.

[0105] The system processes and calibrates the digital signal when the user touches the screen, calculates the coordinates of the touch position, determines the touch point based on the position and intensity of the capacitance change, obtains the X and Y coordinates of the touch point in the touch screen coordinate system, and acquires the coordinate information of the touch position.

[0106] This application accurately acquires sensor electrical signals through the internal signal acquisition circuit of the touch screen and converts them into digital signals. By combining the position and intensity of capacitance changes, the coordinates of the touch point are determined, achieving sub-millimeter-level positioning accuracy. This high-precision positioning allows the system to accurately respond to commands when users perform fine operations, such as handwriting input, drawing design, and clicking small icons, effectively reducing accidental touches and significantly improving the accuracy of operation and interactive experience.

[0107] The general method of establishing a coordinate system with the top left corner of the screen as the origin, and the touch detection principle based on capacitance changes, make this coordinate acquisition method applicable to various types of touch screen devices, including smartphones, tablets, smart vehicle central control screens, and touch all-in-one machines.

[0108] The comparison step in step S3 is as follows:

[0109] Obtain the coordinates of the gaze position and the touch position;

[0110] Based on experimental analysis, a judgment range threshold is set. The range threshold is the maximum distance that the gaze position is allowed to deviate from, centered on the touch position.

[0111] Calculate the Euclidean distance between the gaze position and the touch position;

[0112] The calculated Euclidean distance is compared with a set range threshold. If the distance is less than or equal to the threshold, the gaze position is determined to be within the range defined by the touch position; otherwise, it is determined to be outside the range.

[0113] This application accurately calculates the Euclidean distance between the gaze position and the touch position and compares it with a set range threshold. This allows for accurate judgment of the user's true operational intent. The reasonably set range threshold provides a certain margin of error for the user's operation. When the user makes a slight gaze or touch deviation during the operation, as long as the distance is within the threshold range, the system can still recognize the operation as valid, avoiding operation failure due to minor errors and reducing the user's frustration.

[0114] Once the S3 step determines that the gaze position and the touch position are consistent, the system enters the critical control logic execution stage. The system will quickly retrieve the pre-established "touch position-operation mapping table". This mapping table divides the screen into different areas according to the functional layout of the smart terminal and the user's operating habits, and associates specific operations with each area. In the main interface, the upper left corner of the screen corresponds to the "return to the previous menu" operation, and the bottom middle area corresponds to the "open frequently used application list" operation.

[0115] After retrieving the operation command corresponding to the touch position, the system parses and verifies the command. The parsing process converts the abstract operation command into specific code or command parameters that the computer can execute. The verification process checks the legality and validity of the command to ensure that the operation can be executed in the current system state. For example, when playing a video, if the touch position corresponds to the "pause video" operation, but the video is currently paused, the system will ignore the duplicate command to avoid invalid operation.

[0116] The adaptive adjustment of the judgment range threshold in step S6 is as follows:

[0117] Continuously collect data on the user's gaze position, touch position, and corresponding operation results;

[0118] Record the current operation scenario and the operation status under different task types;

[0119] During system initialization, a default judgment range threshold is set for each scene category;

[0120] After each operation, evaluate the success rate. If the operation is successful, the current threshold is appropriate. If the operation fails, the threshold needs to be adjusted.

[0121] The threshold is dynamically adjusted based on the success rate of the operation and user feedback. If the success rate of the operation is low, the threshold is increased.

[0122] A smooth adjustment strategy is adopted, with each adjustment based on a proportional adjustment threshold;

[0123] Regularly mine and analyze the collected user operation data.

[0124] This application continuously collects user gaze position, touch position, and operation result data. Combined with operation scenario analysis, it can accurately locate the adaptation relationship between user operation habits and thresholds in different scenarios. By recording user feedback and analyzing operation success rate, the system can perceive each user's unique operation habits and preferences, set default thresholds for different scenario categories, and dynamically adjust them based on real-time data, giving the system strong adaptability.

[0125] The success rate is calculated by selecting a certain number of recent operations. The formula for calculating the success rate is: number of successful operations / total number of operations.

[0126] The intelligent terminal, and the control method applied to any of the above-mentioned intelligent terminals, includes: a camera, a touch screen, a vibration module, a storage unit, and a sensor, wherein the camera is located directly above the touch screen, and the vibration module, storage unit, and sensor are all located inside the intelligent terminal.

[0127] The camera in this application is positioned directly above the touchscreen, which can accurately capture the user's eye information and provide clear image data for determining the coordinates of the gaze position. Combined with the touchscreen, it realizes dual interactive verification based on gaze and touch, reducing misoperation. The integrated vibration module can provide the user with intuitive feedback through vibration when the gaze position and touch position are inconsistent, guiding the user to reoperate.

[0128] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A control method for a smart terminal, characterized in that, The control steps are as follows: S1. Obtain the user's eye gaze position based on the camera, and convert the gaze direction into specific position coordinates on the screen; S2. Detect the location of the user's touch on the screen based on the touch screen sensor and obtain the coordinates of the touch location; S3. Compare the obtained gaze position coordinates with the touch position coordinates to determine whether the gaze position is within the range defined by the touch position, establish a judgment range threshold, and perform the comparison. S4. If the gaze position matches the touch position, then respond to the user's touch and execute the operation control. S5. When the gaze position and the touch position are inconsistent, the touch operation is not responded to, the screen gives a vibration prompt to remind the user, the user performs the touch operation again, and the control operation data is stored. S6. When determining whether the gaze position and the touch position are consistent, the range threshold of the judgment is adaptively adjusted based on user habits and operation scenario factors. The adaptive adjustment of the judgment range threshold in step S6 is as follows: Continuously collect data on the user's gaze position, touch position, and corresponding operation results; Record the current operation scenario and the operation status under different task types; During system initialization, a default judgment range threshold is set for each scene category; After each operation, evaluate the success rate. If the operation is successful, the current threshold is appropriate. If the operation fails, the threshold needs to be adjusted. The threshold is dynamically adjusted based on the success rate of the operation and user feedback. If the success rate of the operation is low, the threshold is increased. A smooth adjustment strategy is adopted, with each adjustment based on a proportional adjustment threshold; Regularly mine and analyze the collected user operation data; The success rate is calculated by selecting a certain number of recent operations. The formula for calculating the success rate is: number of successful operations / total number of operations.

2. The control method for a smart terminal according to claim 1, characterized in that, The steps for determining the position coordinates in step S1 are as follows: The system acquires facial images using a camera, and extracts eye features from these images using an eye detection algorithm. An eye model is established to describe the geometric structure and movement of the eye. Based on the position of the eye feature points and the eye model, the vector from the center of the eye to the gaze point, i.e. the gaze vector, is calculated, which represents the user's gaze direction. By displaying calibration points on the screen and allowing the user to focus on those points, the eye features captured by the camera and their corresponding screen coordinates are recorded. A mapping relationship between the gaze direction and screen coordinates is established, and based on this mapping relationship, the calculated gaze direction is converted into specific position coordinates on the screen.

3. The control method for a smart terminal according to claim 2, characterized in that, The steps for extracting eyeball features are as follows: After preprocessing the images captured by the camera, the region where the face is located is found in the preprocessed image, the face bounding box is determined, and the face is separated from the whole image. Within the already located face region, a deep learning-based eye detection model determines the specific location of the eyes within the face region, obtains the bounding box of the eyes, and narrows down the processing range. The image of the eye region is binarized to divide it into foreground and background, highlighting the eyeball from the eye region. An edge detection algorithm is used to find the edge contour of the eyeball, obtain edge information, and analyze the edge information to determine the key feature points of the eyeball.

4. The control method for a smart terminal according to claim 2, characterized in that, The steps for determining the gaze direction are as follows: A composite model is used to simulate the structure of a real eye, the model parameters are defined, and the initial position and orientation of the model in three-dimensional space are determined. Based on the coordinates of the extracted eye feature points in the two-dimensional image, the eye feature points on the two-dimensional image are mapped to three-dimensional space through the inverse transformation of perspective projection to obtain the three-dimensional coordinates of the feature points in the world coordinate system. Based on the established eye geometric model and the determined three-dimensional coordinates of eye feature points, the center position of the eye in three-dimensional space is determined through geometric calculation. Set a target point on the screen as the gaze point, determine the actual gaze target position, and obtain the coordinates of the gaze point in three-dimensional space; Based on vector operations, a three-dimensional gaze vector is calculated starting from the center of the eye and ending at the gaze point. The user's gaze direction is determined by analyzing the gaze vector direction.

5. The control method for a smart terminal according to claim 1, characterized in that, The steps for obtaining the coordinates of the touch position in step S2 are as follows: When a user touches the touchscreen, the signal acquisition circuit inside the touchscreen acquires the electrical signals generated by the sensor and converts them into digital signals, with the upper left corner of the screen as the origin (0,0), the horizontal axis as the X-axis, and the vertical axis as the Y-axis. The system processes and calibrates the digital signal when the user touches the screen, calculates the coordinates of the touch position, determines the touch point based on the position and intensity of the capacitance change, obtains the X and Y coordinates of the touch point in the touch screen coordinate system, and acquires the coordinate information of the touch position.

6. The control method for a smart terminal according to claim 1, characterized in that, The comparison step in step S3 is as follows: Obtain the coordinates of the gaze position and the touch position; Based on experimental analysis, a judgment range threshold is set. The range threshold is the maximum distance that the gaze position is allowed to deviate from, centered on the touch position. Calculate the Euclidean distance between the gaze position and the touch position; The calculated Euclidean distance is compared with a set range threshold. If the distance is less than or equal to the threshold, the gaze position is determined to be within the range defined by the touch position. Conversely, it is determined that it is not within the range.

7. The control method for a smart terminal according to claim 1, characterized in that, In step S4, when a match is found, the corresponding control logic is executed based on the operation corresponding to the touch position.

8. A smart terminal, and a control method for the smart terminal according to any one of claims 1-7, characterized in that, include: The device includes a camera, a touchscreen, a vibration module, a storage unit, and sensors. The camera is located directly above the touchscreen, while the vibration module, storage unit, and sensors are all located inside the smart terminal.

Images