User interface control method and related device

By using cameras and infrared sensors on the TV terminal and using human movement to control the user interface, the inconvenience problem of existing TV control methods for the elderly and hands is solved, and a convenient and intelligent interactive experience is achieved.

CN116095407BActive Publication Date: 2024-11-01SHENZHEN SKYWORTH RGB ELECTRONICS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211657830.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-22
Publication Date
2024-11-01
Estimated Expiration
2042-12-22

AI Technical Summary

Technical Problem

The existing TV control methods (remote control or touch screen) are inconvenient to the elderly, disabled and users who are inconvenient with their hands, and cannot meet the needs of intelligent interaction.

Method used

By installing cameras and infrared sensors on the TV terminal, using human body movements (such as the movement of human faces or eyes) to control the user interface, and implementing air-distance control, including judging the existence of the human body, obtaining body parts movements, calculating action parameters, and based on these Actions are performed accordingly.

Benefits of technology

It provides a convenient user interface control method, which improves the intelligent interactive experience between users and TV terminals, especially suitable for the elderly and users with hand inconvenience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116095407B_ABST
    Figure CN116095407B_ABST
Patent Text Reader

Abstract

The present application is applicable to the field of television technology, and provides a control method and related device for a user interface of a television terminal, aiming to realize a new technical solution for intelligent interaction between a user and a television. The present application method mainly includes: determining whether there is a human body in a target area of ​​the space where the television terminal is located; if there is a human body in the target area, obtaining the action of the body part of the human body; and controlling the corresponding operation of the user interface of the television terminal according to the action.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the technical field of televisions, and in particular, relates to a control method and related devices for a user interface applied to a television terminal. Background Art

[0002] With people's growing demand for intelligence, major manufacturers are gradually increasing the multifunctional attributes of televisions. However, in terms of how to control the television, users are still limited to controlling the television by using a remote control or touching the TV screen. This is not convenient for the elderly, the disabled, or users in scenarios where it is inconvenient for their hands to touch objects. For example, when a user is eating food, their hands are covered with oil, and it is not convenient to directly pick up the remote control or touch the TV screen. The above two control methods undoubtedly limit the interaction between users and the TV. Moreover, from the perspective of intelligent interaction, the method of using auxiliary tools to control the TV can no longer meet people's expectations for intelligent interaction. Therefore, it is necessary to find a new way to facilitate the intelligent interaction between users and the TV. Summary of the invention

[0003] The purpose of this application is to provide a control method and related devices for a user interface of a television terminal, aiming to realize a new technical solution for intelligent interaction between a user and a television.

[0004] In a first aspect, the present application provides a method for controlling a user interface, which is applied to a television terminal, wherein the television terminal is equipped with a camera, and the method comprises:

[0005] Determine whether there is a human body in the target area of ​​the space where the television terminal is located;

[0006] If the human body exists in the target area, obtaining the motion of the body part of the human body;

[0007] Control corresponding operations of the user interface of the television terminal according to the action.

[0008] Optionally, the body part is a human face, and the action of obtaining the body part of the human body includes:

[0009] Focusing the camera on the face;

[0010] Acquire a second rotation angle of the camera;

[0011] Calculating a second distance between the camera and the face;

[0012] Calculating the second spatial position coordinates of the face according to the second rotation angle and the second distance;

[0013] Determine a line connecting the second spatial position coordinates and the center point of the screen of the television terminal;

[0014] Calculating the angle between the connecting line and the central axis of the screen of the television terminal;

[0015] The angle is considered as the motion.

[0016] Optionally, the controlling a corresponding operation of the user interface of the television terminal according to the movement of the body part includes:

[0017] Determining the included angle as a first rotation angle of the user interface;

[0018] The user interface is rotated by the first rotation angle.

[0019] Optionally, the method further includes:

[0020] displaying a virtual image of the user interface;

[0021] The corresponding operation of controlling the user interface of the television terminal according to the movement of the body part includes:

[0022] Determining the included angle as a second rotation angle of the virtual image;

[0023] The virtual image is rotated by the second rotation angle.

[0024] Optionally, the television terminal is equipped with an infrared sensor, the body part is a human eye, and the action of obtaining the body part of the human body includes:

[0025] Focusing the camera on the human eye;

[0026] emitting an infrared beam toward the human eye;

[0027] Receiving a target highlight point formed by the cornea of ​​the human eye reflecting the infrared light beam;

[0028] Align the optical axis of the camera with the target highlight point;

[0029] Determine a first rotation angle of the camera aimed at the target highlight point;

[0030] Calculating a first distance between the camera and the target highlight point;

[0031] Calculate the first target spatial position coordinates of the target highlight point according to the first rotation angle and the first distance;

[0032] Determine the target projection coordinates of the first target space position coordinates on the screen of the television terminal;

[0033] Receiving a first target light spot formed by the retina of the human eye reflecting the infrared light beam;

[0034] Determine a first target offset corresponding to the shape of the first target light spot in a preset comparison table, wherein the comparison table records the correspondence between different light spot shapes and offsets, the shape of the first target light spot is one of the different light spot shapes, and the first target offset is one of the offsets;

[0035] Determine a first screen position of the television terminal viewed by the human eye according to the target projection coordinates and the first target offset;

[0036] Receiving a second target light spot formed by the retina of the human eye reflecting the infrared light beam;

[0037] Determine in the comparison table a second target offset corresponding to the shape of the second target light spot, the shape of the second target light spot being one of the shapes of the different light spots, and the second target offset being one of the offsets;

[0038] Determine a second screen position of the television terminal viewed by the human eye according to the target projection coordinates and the second target offset;

[0039] A position change between the second screen position and the first screen position is regarded as the action.

[0040] Optionally, the corresponding operation of controlling the user interface of the television terminal according to the movement of the body part includes:

[0041] A cursor on the user interface is moved according to the position change.

[0042] Optionally, after the human body exists and before obtaining the action of the body part of the human body, the method further includes:

[0043] Determine the number X of the human bodies, where X is a positive integer greater than 0;

[0044] Establishing X number of user accounts according to the facial image of the human body;

[0045] receiving a selection of a target user account, wherein the target user account is one of the X number of user accounts;

[0046] Determine the human body corresponding to the target user account as the controller of the television terminal;

[0047] The step of obtaining the body part of the human body is triggered.

[0048] In a second aspect, the present application provides a control system for a user interface, the system being applied to a television terminal, the television terminal being equipped with a camera, the system comprising:

[0049] A judging unit, used for judging whether there is a human body in a target area of ​​the space where the television terminal is located;

[0050] an acquisition unit, configured to acquire a motion of a body part of the human body if the human body exists in the target area;

[0051] A control unit is used to control the corresponding operation of the user interface of the television terminal according to the action.

[0052] Optionally, the body part is a face, and the acquiring unit is further used for:

[0053] Focusing the camera on the face;

[0054] Acquire a second rotation angle of the camera;

[0055] Calculating a second distance between the camera and the face;

[0056] Calculating the second spatial position coordinates of the face according to the second rotation angle and the second distance;

[0057] Determine a line connecting the second spatial position coordinates and the center point of the screen of the television terminal;

[0058] Calculating the angle between the connecting line and the central axis of the screen of the television terminal;

[0059] The angle is considered as the motion.

[0060] Optionally, the control unit is further used for:

[0061] Determining the included angle as a first rotation angle of the user interface;

[0062] The user interface is rotated by the first rotation angle.

[0063] Optionally, the system further includes:

[0064] A display unit, used to display a virtual image of the user interface;

[0065] The control unit is also used for:

[0066] Determining the included angle as a second rotation angle of the virtual image;

[0067] The virtual image is rotated by the second rotation angle.

[0068] Optionally, the television terminal is equipped with an infrared sensor, the body part is a human eye, and the acquisition unit is further used for:

[0069] Focusing the camera on the human eye;

[0070] emitting an infrared beam toward the human eye;

[0071] Receiving a target highlight point formed by the cornea of ​​the human eye reflecting the infrared light beam;

[0072] Align the optical axis of the camera with the target highlight point;

[0073] Determine a first rotation angle of the camera aimed at the target highlight point;

[0074] Calculating a first distance between the camera and the target highlight point;

[0075] Calculate the first target spatial position coordinates of the target highlight point according to the first rotation angle and the first distance;

[0076] Determine the target projection coordinates of the first target space position coordinates on the screen of the television terminal;

[0077] Receiving a first target light spot formed by the retina of the human eye reflecting the infrared light beam;

[0078] Determine a first target offset corresponding to the shape of the first target light spot in a preset comparison table, wherein the comparison table records the correspondence between different light spot shapes and offsets, the shape of the first target light spot is one of the different light spot shapes, and the first target offset is one of the offsets;

[0079] Determine a first screen position of the television terminal viewed by the human eye according to the target projection coordinates and the first target offset;

[0080] Receiving a second target light spot formed by the retina of the human eye reflecting the infrared light beam;

[0081] Determine in the comparison table a second target offset corresponding to the shape of the second target light spot, the shape of the second target light spot being one of the shapes of the different light spots, and the second target offset being one of the offsets;

[0082] Determine a second screen position of the television terminal viewed by the human eye according to the target projection coordinates and the second target offset;

[0083] A position change between the second screen position and the first screen position is regarded as the action.

[0084] Optionally, the control unit is further used for:

[0085] A cursor on the user interface is moved according to the position change.

[0086] Optionally, the system further includes:

[0087] A first determining unit, configured to determine the number X of the human bodies, where X is a positive integer greater than 0;

[0088] An establishing unit, used for establishing X number of user accounts according to the facial image of the human body;

[0089] A receiving unit, configured to receive a selection of a target user account, wherein the target user account is one of the X number of user accounts;

[0090] A second determining unit, configured to determine the human body corresponding to the target user account as the controller of the television terminal;

[0091] A trigger unit is used to trigger the execution of an action step of obtaining the body part of the human body.

[0092] In a third aspect, the present application provides a computer device, including:

[0093] Processor, memory, bus, input and output interface, network interface;

[0094] The processor is connected to the memory, the input / output interface, and the network interface via a bus;

[0095] The memory stores a program;

[0096] When the processor executes the program stored in the memory, the user interface control method as described in any one of the first aspects above is implemented.

[0097] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the user interface control method as described in any one of the first aspects above.

[0098] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0099] The user interface control method provided in the embodiment of the present application is different from the prior art method of controlling the TV terminal by using a remote control or pressing a button on the TV terminal, or touching the screen of the TV terminal. The embodiment of the present application provides an air control method, that is, a method of controlling the TV terminal without touching the TV terminal and its remote control. First, it is necessary to determine whether there is a human body in the target area of ​​the space where the TV terminal is located. If there is a human body in the target area, the movement of the body parts of the human body is obtained, and the corresponding operation of the user interface of the TV terminal is controlled according to the movement of the body parts, thereby providing users with a method for conveniently controlling the TV terminal and enhancing the intelligent interactive experience between the user and the TV terminal. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Figure 1 A flowchart of an embodiment of a method for controlling a user interface of the present application;

[0101] Figure 2 The principle diagram of the control method of the user interface of this application is shown as follows Figure 1 ;

[0102] Figure 3 The principle diagram of the control method of the user interface of this application is shown as follows Figure 2 ;

[0103] Figure 4 The principle diagram of the control method of the user interface of this application is shown as follows Figure 3 ;

[0104] Figure 5 The principle diagram of the control method of the user interface of this application is shown as follows Figure 4 ;

[0105] Figure 6 A schematic diagram of the structure of an embodiment of a control system of a user interface of the present application;

[0106] Figure 7 This is a schematic diagram of the structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0107] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0108] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0109] In order to provide users with a new and convenient way to control a TV terminal, this application proposes a method for controlling a user interface. Figure 1 This is an embodiment of a method for controlling a user interface of the present application, and the present application is applied to a television terminal.

[0110] User Interface (UI) is the medium for interaction and information exchange between the system and the user. It realizes the conversion between the internal form of information and the form acceptable to humans. The user interface is the software designed between the user and the hardware to enable the user to operate the hardware conveniently and efficiently to achieve two-way interaction and complete the work they want to do with the help of the hardware.

[0111] The television terminal of the present application is equipped with a camera. The camera can be a camera that is externally mounted on the television terminal and is separately equipped, or it can be a camera component that is built into the frame of the television terminal. The orientation of the camera in the television terminal can be determined according to actual design requirements, and the present application does not impose any restrictions on this. The camera can be a wide-angle camera or a non-wide-angle camera. Since the non-wide-angle camera has a smaller field of view, the range of the shooting area can be expanded by rotating it. The control method of the user interface includes: steps 101-103.

[0112] 101. Determine whether there is a human body in a target area of ​​the space where the television terminal is located.

[0113] In order to enable the user interface of the television terminal to perform corresponding operations according to the movements of the human body parts, the present embodiment needs to first use the camera to capture whether there is a human body in the audience area, and thus uses the shooting area of ​​the camera as the target area. Specifically, if the camera is a wide-angle camera, the capture field of view of the wide-angle camera is used as the target area; if it is a non-wide-angle camera, a rotation angle range is set for it, and the field of view area that can be captured within the rotation angle range is used as the target area. The rotation angle range may depend on the size of the television terminal, and the present embodiment does not impose any restrictions on this.

[0114] The TV terminal is equipped with a neural network model chip. The neural network model has been trained to be able to recognize human feature data. It can perform image recognition on the pictures taken by the camera. If the object in the target area meets the human contour feature data or facial feature data, it is determined that a human body exists. Alternatively, the TV terminal is equipped with a human infrared sensing sensor. When a human body enters the detection range, it is determined that a human body exists in the target area.

[0115] Before entering step 101, this embodiment may also include initializing the relevant programs and related devices used in the method described in this application. The initialization content may include starting to load and run the relevant programs of the camera, restoring various data to the default values, turning the camera to the default state, focusing the camera, adjusting the focal length to the default focal length, etc. Through initialization to perform preparatory work, precise control of the camera can be achieved, and it can be ensured that the data used when the program is running is accurate.

[0116] 102. If the human body exists in the target area, obtain the motion of the body part of the human body.

[0117] After the judgment in step 101, if there is a human body in the target area, the movement of the body parts of the human body is further obtained. The body parts can be human limbs, hands, face, eyes, and the movements obtained can be: human body walking, limbs in a certain posture; waving arms, raising hands, putting arms in a certain shape, etc.; changing from clenching a fist to spreading five fingers, making a scissors hand gesture, etc.; making a certain expression on the face; rotating the eyes, blinking three times in a row, etc. The above-mentioned movement of body parts can be a certain form, such as the above-mentioned scissors hand gesture; the movement can also be a continuous movement of some forms, such as the above-mentioned change from clenching a fist to spreading five fingers; the movement can also be displacement, angular deflection or deformation relative to some set position or form, such as the human body walking to a certain position, and the position is displaced relative to the default position set by the system, and displacement is regarded as movement.

[0118] 103. Control corresponding operations of the user interface of the television terminal according to the action.

[0119] The TV terminal may store preset body part motion contents. Different body part motions correspond to different user interface operation contents. When the tracked human body makes a certain motion, the user interface of the TV terminal is controlled to perform corresponding operations according to the motion. The user interface operation contents may be moving the cursor in the user interface, rotating the user interface relative to the human body, switching between different user interfaces, rotating or shaking the application icons in the user interface, etc. The operation contents may be for practicality or entertainment purposes to enhance the user experience.

[0120] The TV terminal can also provide users with a personalized customization function. Users can match preset body part movements with the operation of a certain user interface, and can even use the learning ability of the neural network model to set certain movements by themselves, so that the neural network model can recognize and learn the movement, and set the corresponding user interface operation content for the movement. When the tracked human body makes the movement, the user interface of the TV terminal performs the corresponding operation. In addition, in order not to trigger the corresponding operation of the user interface at will, the set body part movement cannot be some movements that the human body often makes. Therefore, when the user performs personalized customization, the TV terminal can prompt that a certain movement cannot be set as the action content that triggers the execution of step 103.

[0121] The method for controlling the user interface of the embodiment of the present application is different from the prior art method of controlling the TV terminal by using a remote control or pressing a button on the TV terminal, or touching the screen of the TV terminal. The embodiment of the present application provides a remote control method, that is, a method of controlling the TV terminal without touching the TV terminal and its remote control. First, it is necessary to determine whether there is a human body in the target area of ​​the space where the TV terminal is located. If there is a human body in the target area, the movement of the body parts of the human body is obtained, and the corresponding operation of the user interface of the TV terminal is controlled according to the movement of the body parts, thereby providing users with a method for conveniently controlling the TV terminal and enhancing the intelligent interactive experience between the user and the TV terminal.

[0122] To facilitate the use of this method in practical applications, this application provides three more detailed specific implementation methods for the above-mentioned steps 102 and 103. These three implementation methods are described in detail below.

[0123] In the first implementation, the purpose is to control the movement of the cursor on the user interface of the TV terminal by the movement of the human eye. Here we first introduce the principle on which this implementation is based: when a light beam is irradiated to the human eye, the human eye will reflect the light beam and present a light spot at the pupil of the human eye. If the human eye looks directly at the camera at this time, the shape of the light spot obtained by the camera is circular, and if the human eye looks in other directions, the shape obtained is elliptical, which is similar to the perspective principle in art, such as Figure 2As shown in the figure, when the camera is aimed at the human eye to shoot, if the human eye is rotating, the shape of the light spot at the pupil captured by the camera will continue to deform. Different deformation amounts correspond to different angles of the human eye. At different distances from the human eye to the camera, the deformation amount of the light spot corresponds to different angles of the human eye. Therefore, based on the distance from the human eye to the camera and the obtained deformation amount of the light spot, the change in the position of the human eye on the screen can be calculated.

[0124] Also, since the human eye is within the shooting range of the camera, no matter where the human eye is in space, when the human eye is staring at the origin (the origin is the location of the camera), the light spot at the pupil captured by the camera is a circle, and the only difference is that the radius of the circle captured at different distances is different. Therefore, this feature can also be used to directly determine where the human eye is staring at the screen. Specifically, multiple comparison tables are established, and different distance values ​​of the human eye from the camera correspond to a comparison table. The comparison table records the offsets corresponding to the shapes of different light spots. The offset refers to the offset of the position coordinates of the human eye staring at the screen relative to the position coordinates when the human eye is looking directly at the camera. The offset is a vector, that is, it has positive and negative signs, and the positive and negative signs mean different offset directions. The following step description will explain in detail how this embodiment uses this principle to implement human eye control of the cursor.

[0125] The TV terminal is equipped with an infrared sensor, which is mainly used to emit infrared beams to the human eye. Since light waves in other bands will produce strong stimulation to the human eye compared to infrared beams, this embodiment uses infrared beams to illuminate the human eye. The infrared sensor can be an infrared camera integrated with the above-mentioned camera, that is, the infrared sensor rotates with the camera. The infrared sensor can also be a separate device that can emit infrared beams, which can be located outside the TV terminal or encapsulated in the frame of the TV terminal. The action of obtaining the body part of the human body in step 102 can further include steps 201-step 215.

[0126] 201. Focus the camera on the human eye.

[0127] Since the size of the human eye is relatively small relative to the default shooting field of view of the camera, in order to capture the movement of the human eye, it is necessary to make the camera further accurately aim at the human eye for shooting. After determining that there is a human body in the target area through step 101, the camera is focused on the face area. If it is found through image recognition that there is no face in the target area, for example, the human body is facing away from the TV terminal at this time, there is no need to further search for the human eye. If it is found through image recognition that there is a face in the target area, the camera is aimed at the eye position of the identified face for focusing and shooting. In the process of focusing, if it is a wide-angle camera, there is no need to rotate the camera. If it is a non-wide-angle camera, the camera needs to be rotated to aim at the human eye for shooting. Since the non-wide-angle camera involves the action of rotating shooting, the calculation is more complicated than that of the wide-angle camera. However, because it can be rotated, the optical axis of the camera can be adjusted to accurately adjust the shooting angle, so that the data information obtained subsequently is more accurate, which is conducive to improving the accuracy of the calculation results. Therefore, this application takes the non-wide-angle camera as an example to explain this embodiment in detail.

[0128] Please note that Figure 3 The camera of this embodiment has two rotation directions, namely, longitude direction and latitude direction. When the camera is at the default origin position, it is in a state of shooting forward perpendicular to the screen direction of the TV terminal. At this time, the rotation angle in the longitude direction and the rotation angle in the latitude direction are both zero. When the camera focuses on a certain point, the TV terminal can obtain the angle value of the camera rotation and record it. The TV terminal can calculate the spatial position coordinates of the human eye according to the rotation angle of the camera and the focusing principle of the camera. For example, the camera focuses on the human eye by adjusting the focal length, so the value of the camera focal length during focusing is a known quantity. Different focal lengths have different object distance ratios, which can be found by looking up a table or calculated by an existing optical formula. During the process of focusing the camera, the distance value can also be directly obtained. Therefore, the object distance value can be calculated by multiplying the distance value and the object distance ratio. The object distance is the distance from the human eye to the camera. Substituting the object distance value and the rotation angle value of the camera into the trigonometric function formula can calculate the spatial position coordinates of the human eye.

[0129] 202. Send infrared beams to human eyes.

[0130] In this embodiment, in order to obtain the shape of the reflected light spot of the pupil of the human eye, it is necessary to emit an infrared beam to the human eye, and if it is to obtain a coordinate value closer to the pupil, it is necessary to perform steps 203 to 207. It should be noted that if the camera is an infrared camera, since the camera is aimed at the human eye, the infrared beam can be directly emitted to the human eye. If the infrared beam is a separate infrared sensor device, a rotation command is issued to it so that the infrared beam is aimed at the human eye for emission.

[0131] 203. The target highlight point is formed by the reflection of the infrared light beam by the cornea of ​​the receiving human eye.

[0132] When an infrared beam shines on the human eye, a highlight is formed on the cornea, such as Figure 4 shown.

[0133] 204. Align the optical axis of the camera with the target highlight point.

[0134] In order to accurately find the target highlight point so that the camera can be aimed at the target highlight point, the TV terminal can also perform image processing on the acquired picture, such as filtering, denoising, and reducing the brightness of the acquired infrared reflection image. This embodiment does not limit this.

[0135] 205. Determine a first rotation angle of the camera aimed at the target highlight point.

[0136] The first rotation angle can be directly obtained when the camera is rotated to align with the target highlight point. The first rotation angle refers to the angle value when the camera turns from the default origin position to the target highlight point. The angle value has a positive and negative sign, and the direction of rotation can be known according to the positive and negative signs. The angle value includes the rotation angle in the longitude direction and the rotation angle in the latitude direction.

[0137] 206. Calculate a first distance between the camera and the target highlight point.

[0138] According to the calculation principle mentioned in the above step 201, the first distance between the camera and the target highlight point can be calculated through the focusing principle.

[0139] 207. Calculate first target space position coordinates of the target highlight point according to the first rotation angle and the first distance.

[0140] According to the calculation principle mentioned in the above step 201, the first target space position coordinates of the target highlight point can be calculated by using a trigonometric function formula.

[0141] 208. Determine the target projection coordinates of the first target space position coordinates on the screen of the television terminal.

[0142] See also Figure 5 In the figure, point A is the position of the camera, point G is the position of the human eye, point C is the target projection coordinate of the human eye on the TV terminal display screen, and point I is the position where the human eye is currently staring at the screen. Since the first target space position coordinates are known to be three-dimensional coordinates (X, Y, Z) through the calculation of step 207, it can be known that the target projection coordinates of the target highlight point on the TV terminal screen are (X, Y).

[0143] It should be noted that steps 203 to 207 are for accurately obtaining the spatial position coordinates of the pupil of the human eye. Therefore, it is necessary to first calculate the first target spatial position coordinates of the target highlight point, and then regard the first target spatial position coordinates as the spatial position coordinates of the pupil of the human eye. If you do not want to calculate the spatial position coordinates of the pupil too accurately, you do not need to execute steps 203 to 207. That is, in step 201, according to the rotation angle of the camera when focusing on the human eye, and the calculated distance value between the human eye and the camera, a spatial position coordinate can also be calculated, and it is also possible to use the spatial position coordinate as the pupil position coordinate.

[0144] 209. Receive the first target light spot formed by the reflection of the infrared light beam by the retina of the human eye.

[0145] According to the principle described at the beginning of this embodiment, in order to control the cursor movement of the user interface through the movement of the human eye, the shape of the light spot at the pupil must be obtained. The reason behind this practice is: because the reflected light spot at the pupil is actually a shaped light spot formed by the infrared light beam entering the human eye, reflected by the retina of the human eye, and the reflected light beam passing through the pupil of the human eye. Since the pupil of the human eye is approximately circular, the shape of the first target light spot captured by the camera is circular or an elliptical shape after the circular deformation.

[0146] 210. Determine a first target offset corresponding to the shape of the first target light spot in a preset comparison table, wherein the comparison table records the correspondence between different light spot shapes and offsets, the shape of the first target light spot is one of the different light spot shapes, and the first target offset is one of the offsets.

[0147] According to the principle described at the beginning of this embodiment, a preset comparison table is stored in the TV terminal in this embodiment, and the preset comparison table records the correspondence between different light spot shapes and offsets, and the comparison tables corresponding to different distance values ​​from the human eye to the camera are different. Therefore, in practical applications, if the distance value from the human eye to the camera is known, the comparison table corresponding to the distance value should be found, and according to the shape of the first target light spot obtained by the camera, the offset corresponding to the light spot shape should be found in the comparison table, that is, the first target offset. The first target offset is the difference between the position coordinates of the human eye on the screen and the origin, and the difference has a positive and negative sign to distinguish the direction.

[0148] 211. Determine a first screen position of a television terminal viewed by human eyes according to the target projection coordinates and the first target offset.

[0149] The target projection coordinates have been determined in step 208, and the first target offset has been found in step 210, so the coordinates of the first screen position of the current human eye looking at the TV terminal can be calculated by subtracting the first target offset from the target projection coordinates.

[0150] 212. A second target light spot is formed by the reflection of the infrared light beam by the retina of the receiving human eye.

[0151] Similar to the above step 209, step 212 here is to obtain a second target light spot formed at the pupil when the human eye looks at another part of the screen and the retina of the human eye reflects the infrared light beam.

[0152] 213. Determine a second target offset corresponding to the shape of the second target light spot in the comparison table, the shape of the second target light spot is one of the shapes of different light spots, and the second target offset is one of the offsets.

[0153] Similar to the above step 210, step 213 here is to find out the second target offset in the comparison table according to the shape of the second target light spot obtained when the human eye looks at another part of the screen.

[0154] 214. Determine a second screen position of the television terminal viewed by human eyes according to the target projection coordinates and the second target offset.

[0155] Similar to the above step 211, step 214 here is to determine the position coordinates of the second screen position when the human eye looks at another part of the screen.

[0156] 215. Consider a position change between the second screen position and the first screen position as an action.

[0157] Since the position of the human eye changes when it stares at the screen of the television terminal twice, this position change can be regarded as the movement of the human eye.

[0158] After the movement of the human eye is determined through the above steps 201 to 215, the TV terminal will respond to the movement of the human eye, that is, control the corresponding operation of the user interface of the TV terminal according to the movement of the body part, step 216: move the cursor on the user interface according to the position change.

[0159] The cursor may be in the form of a design effect that highlights an application icon or an option, or in the form of a small arrow cursor similar to a mouse on a computer display, or in the form of a flashing vertical line cursor in a document, etc. After the above steps 211 and 214 are performed to calculate the first screen position and the second screen position of the human eye looking at the TV terminal, this step can directly respond to the movement state of the cursor at the two positions. The movement method may be to move the distance of one icon each time, and the cursor gradually moves from the icon at the first screen position to the icon at the second screen position, or there may be no transition stage, and the cursor directly jumps from the icon at the first screen position to the icon at the second screen position.

[0160] It should be noted that since the cursor on the TV terminal is often a square box that highlights the icon of an application, which is regarded as selecting the application, and the square box is different from the small arrow cursor represented by the mouse on the computer display and the flashing vertical line cursor in the document, its shape is relatively larger, that is, when moving on different application icons, there is no need to strictly require whether the calculated position of the human eye looking at the screen is completely accurate. As long as the TV terminal can move the cursor on the user interface according to the rotation of the human eye, that is, the accuracy of identifying the movement distance on the screen caused by the rotation of the human eye and the movement distance of the cursor can be determined according to actual needs, and this accuracy can be achieved by adjusting the above-mentioned comparison table. That is, if you want to improve the accuracy, you should make a more detailed division of the deformation of the light spot shape, and the offset value obtained by looking up the table will be more accurate, so that the final calculated position coordinates of the human eye looking at the screen will also be more accurate.

[0161] In this embodiment, the actions of human body parts can be further combined to execute commands such as confirm, cancel, return, pause / start human eye operation, etc. The human body parts can be human eyes, face, gestures, limbs, etc. For example, blinking three times in a row to confirm entering the application selected by the cursor.

[0162] In order to prevent the problem of not knowing who will be the controller when there is more than one viewer in the target area, or the current controller wants to give the control of the TV terminal to other viewers, after determining that there is a human body in the target area and before obtaining the movement of the body part of the human body, the user interface control method further includes steps 217 to 221:

[0163] 217. Determine the number X of the human bodies, where X is a positive integer greater than 0.

[0164] That is, there is more than one viewer in the target area, and multiple people are watching the TV terminal together.

[0165] 218. Create X number of user accounts according to the facial image of the human body.

[0166] The user account uses the facial image of the viewer in the target area as the account information. There is no need to create a user account by entering a name. Instead, the focus is directly placed on the face of each viewer and the captured facial image is captured and recorded. Each facial image corresponds to a user account.

[0167] 219. Receive a selection of a target user account, where the target user account is one of the X number of user accounts.

[0168] The target user account is the user account corresponding to the viewer who has the control right to control the TV terminal. In this step, the method of receiving the selection of the target user account is divided into two situations. The first one is: if the user account has not been established before, or the viewer of the established user account is not in the target area, it is the first selection situation. In this case, the TV terminal needs to select a user account by default as the target user account. Specifically, it can be to select the user account of the first viewer who enters the target area, or to select the user account of the viewer facing the TV terminal screen, or to select randomly. The second one is: if the user account of an existing viewer is selected as the target user account, the viewer can move the cursor by rotating the human eye to move the cursor to the avatar of the target viewer.

[0169] 220. Determine the human body corresponding to the target user account as the controller of the television terminal.

[0170] If it is the first selection as mentioned above, the TV terminal will automatically select and confirm the target user account, and the human body corresponding to the target user account will be the controller of the TV terminal; if it is not the first selection as mentioned above, the TV terminal will further receive a confirmation command from the target user account, such as executing the confirmation command by blinking or other body movements.

[0171] 221. Trigger execution of an action step of obtaining a body part of the human body.

[0172] This step is a follow-up step, and step 201 is executed continuously.

[0173] In the second embodiment, the identified body part is a human face, and the TV terminal responds to the rotation of the user interface according to the change in the position of the human face. The purpose of this embodiment is to make the user interface of the TV terminal present a three-dimensional effect, rotate the user interface according to the position of the human face, and present the user with the effect of the user interface following the face, so as to achieve an intelligent interactive effect that the user interface is displayed facing the face no matter where the user watches the TV terminal screen. Specifically, the action of obtaining the body part of the human body in step 102 further includes steps 301-307:

[0174] 301. Focus the camera on the face.

[0175] After determining that there is a human body in the target area in step 101, this step 301 further focuses the camera on the human face, that is, it is necessary to ensure that the human face is at the viewing angle of the TV terminal. Only when the human face is viewing the TV terminal can the human face be recognized in the target area and the human face is focused, so as to further implement the following

[0176] It should be noted that, in this embodiment, the infrared 5 sensor may not be used, so the camera may or may not be an infrared camera, and this embodiment does not limit this.

[0177] 302. Obtain a second rotation angle of the camera.

[0178] In step 301, the camera is turned toward the face, so that the TV terminal can directly obtain the second rotation angle of the camera, which refers to the angle when the camera is turned from the default origin position to the face.

[0179] Angle value, which has a positive or negative sign. The direction of rotation can be known according to the positive or negative sign. This angle value includes the rotation angle in the longitude direction and the rotation angle in the latitude direction. When the camera is in the default origin position, it is in a state of shooting forward perpendicular to the screen direction of the TV terminal. At this time, the rotation angle in the longitude direction and the rotation angle in the latitude direction are both zero.

[0180] 303. Calculate a second distance between the camera and the face.

[0181] The second distance between the camera and the face can be calculated based on the focusing principle of the camera. The detailed principle can be referred to the description of the above step 201.

[0182] 304. Calculate second spatial position coordinates of the face according to the second rotation angle and the second distance.

[0183] After obtaining the second rotation angle and the second distance, the spatial position coordinates of the face in the target area, ie, the second spatial position coordinates, can be calculated according to the trigonometric formula. The detailed principle can be referred to the description of step 201 above.

[0184] 305. Determine a line connecting the second spatial position coordinates and the center point of the screen of the television terminal.

[0185] The second spatial position coordinates have been calculated by the above step 304, and the position coordinates of the center point of the TV terminal screen are also stored in the TV terminal, so the expression of the line connecting the two can be determined.

[0186] 306. Calculate the angle between the connecting line and the central axis of the screen of the television terminal.

[0187] The coordinates of the center of the screen are known, so the expression of the center axis of the screen can also be obtained. Calculating the angle between the connecting line and the center axis of the screen of the TV terminal is to solve the angle between the expressions of the two spatial straight lines. The solution method can refer to the calculation method mentioned in higher mathematics. There are many calculation methods, and this embodiment does not limit this.

[0188] 307. Consider the angle as an action.

[0189] That is to say, if the line connecting the face and the center point of the screen is at an angle with the center axis of the screen, it is considered that there is an action, which serves as a trigger condition for step 308. The reason for calculating the angle between the line and the center axis of the screen of the TV terminal is that when the face is on the center axis of the screen to watch the TV terminal, the user interface is displayed directly to the face, and the viewing effect is best at this time. If the face is not on the center axis of the screen, the subsequent step of triggering the rotation of the user interface needs to be determined, so the angle needs to be measured first.

[0190] After obtaining the angle, the corresponding operation of the user interface of the television terminal can be controlled according to the action, that is, the entire user interface is rotated according to the angle, which specifically includes steps 308 and 309:

[0191] 308. Determine the included angle as a first rotation angle of the user interface.

[0192] In order to display the user interface at an angle facing the face, the center point of the screen can be used as a fixed base point, and the user interface can be rotated around this base point, or a vertical line passing through the center point of the screen can be used as a rotation axis, and the user interface can be rotated around this rotation axis. Therefore, the angle can be directly used as the first rotation angle of the user interface. In other embodiments, other points or lines can be selected as base points or rotation axes, and the value of the first rotation angle can be determined according to the situation, and its value can be determined based on the angle.

[0193] 309. Rotate the user interface by the first rotation angle.

[0194] This step is an execution step, and the user interface is rotated by the first rotation angle determined in step 308. Of course, in order to create the effect that the user interface is displayed directly on the face of the person, the icons and fonts in the user interface may also be tilted accordingly.

[0195] In the third embodiment, the purpose is to enable the virtual character in the user interface to rotate according to the movement of the human face, so as to reflect an effect of intelligent interaction between the virtual character and the user, for example, to create an interesting effect that the face of the virtual character is always looking at the human face. In this embodiment, the method of determining the action can refer to steps 301 to 307, and the corresponding operation of the user interface of the TV terminal is controlled according to the action, specifically including steps 310 to 312:

[0196] 310. Display a virtual image of the user interface.

[0197] The virtual image can be displayed when the TV terminal is turned on, or the display of the virtual image can be triggered according to user instructions, such as by calling up the virtual image through voice wake-up to display the virtual image on the user interface. The virtual image can be a cartoon character, etc., which is not limited in this embodiment.

[0198] 311. Determine the angle as a second rotation angle of the virtual image.

[0199] When rotating the face or body of the avatar, the vertical center axis of the avatar model can be used as the rotation axis, and the above-mentioned angle can be directly used as the second rotation angle, or in other embodiments, a multiple of the angle can be used as the second rotation angle.

[0200] 312. Rotate the virtual image to a second rotation angle.

[0201] This step 312 is an execution step, which is to rotate the body or face of the virtual character around the vertical center axis by a second rotation angle.

[0202] The present application specifically introduces the above three implementation methods, namely, controlling the movement of the cursor in the user interface according to the rotation of the human eye, rotating the entire user interface according to the rotation angle of the human face, rotating the body or face of the virtual character in the user interface according to the rotation angle of the human face, and combining the image recognition of the neural network to learn the human body movements to trigger the execution of confirm, cancel, return and other commands. The present application covers many aspects of user interface control, and can realize intelligent interaction between users and TV terminals through air, thereby improving user experience.

[0203] The above embodiment describes the control method based on the user interface of the present application. The following describes the control system of the user interface of the present application. Figure 6 , an embodiment of a control system of a user interface includes:

[0204] A judging unit 401 is used to judge whether there is a human body in a target area of ​​the space where the television terminal is located;

[0205] An acquisition unit 402 is configured to acquire a motion of a body part of the human body if the human body exists in the target area;

[0206] The control unit 403 is used to control the corresponding operation of the user interface of the television terminal according to the action.

[0207] Optionally, the body part is a face, and the acquiring unit is further used for:

[0208] Focusing the camera on the face;

[0209] Acquire a second rotation angle of the camera;

[0210] Calculating a second distance between the camera and the face;

[0211] Calculating the second spatial position coordinates of the face according to the second rotation angle and the second distance;

[0212] Determine a line connecting the second spatial position coordinates and the center point of the screen of the television terminal;

[0213] Calculating the angle between the connecting line and the central axis of the screen of the television terminal;

[0214] The angle is considered as the motion.

[0215] Optionally, the control unit is further used for:

[0216] 0 determining the included angle as a first rotation angle of the user interface;

[0217] The user interface is rotated by the first rotation angle.

[0218] Optionally, the system further includes:

[0219] A display unit, used to display a virtual image of the user interface;

[0220] The control unit is further used to: 5 determine the angle as a second rotation angle of the virtual image;

[0221] The virtual image is rotated by the second rotation angle.

[0222] Optionally, the television terminal is equipped with an infrared sensor, the body part is a human eye, and the acquisition unit is further used for:

[0223] Focusing the camera on the human eye;

[0224] 0 emitting an infrared beam toward the human eye;

[0225] Receiving a target highlight point formed by the cornea of ​​the human eye reflecting the infrared light beam;

[0226] Align the optical axis of the camera with the target highlight point;

[0227] Determine a first rotation angle of the camera aimed at the target highlight point;

[0228] Calculating a first distance between the camera and the target highlight point;

[0229] 5. Calculate the first target space of the target highlight point according to the first rotation angle and the first distance.

[0230] Inter-location coordinates;

[0231] Determine the target projection coordinates of the first target space position coordinates on the screen of the television terminal;

[0232] Receiving a first target light spot formed by the retina of the human eye reflecting the infrared light beam;

[0233] Determine a first target offset corresponding to the shape of the first target light spot in a preset comparison table, wherein the comparison table records the correspondence between different light spot shapes and offsets, the shape of the first target light spot is one of the different light spot shapes, and the first target offset is one of the offsets;

[0234] Determine a first screen position of the television terminal viewed by the human eye according to the target projection coordinates and the first target offset;

[0235] Receiving a second target light spot formed by the retina of the human eye reflecting the infrared light beam;

[0236] Determine in the comparison table a second target offset corresponding to the shape of the second target light spot, the shape of the second target light spot being one of the shapes of the different light spots, and the second target offset being one of the offsets;

[0237] Determine a second screen position of the television terminal viewed by the human eye according to the target projection coordinates and the second target offset;

[0238] A position change between the second screen position and the first screen position is regarded as the action.

[0239] Optionally, the control unit is further used for:

[0240] A cursor on the user interface is moved according to the position change.

[0241] Optionally, the system further includes:

[0242] A first determining unit, configured to determine the number X of the human bodies, where X is a positive integer greater than 0;

[0243] An establishing unit, used for establishing X number of user accounts according to the facial image of the human body;

[0244] A receiving unit, configured to receive a selection of a target user account, wherein the target user account is one of the X number of user accounts;

[0245] A second determining unit, configured to determine the human body corresponding to the target user account as the controller of the television terminal;

[0246] A trigger unit is used to trigger the execution of an action step of obtaining the body part of the human body.

[0247] The control system of the user interface of the embodiment of the present application performs the same operations as the aforementioned Figure 1 The operations performed in the embodiment are similar and will not be described in detail here.

[0248] The following describes the computer device of the embodiment of the present application. Figure 7 , an embodiment of the computer device in the embodiment of the present application includes:

[0249] The computer device 700 may include one or more processors (central processing units, CPU) 701 and a memory 702, in which one or more application programs or data are stored. The memory 702 is a volatile storage or a persistent storage. The program stored in the memory 702 may include one or more modules, and each module may include a series of instruction operations in the computer device. Furthermore, the processor 701 may be configured to communicate with the memory 702 to execute a series of instruction operations in the memory 702 on the computer device 700. The computer device 700 may also include one or more network interfaces 703, one or more input and output interfaces 704, and / or one or more operating systems, such as Windows Server, Mac OS, Unix, Linux, FreeBSD, etc. The processor 701 may execute the aforementioned Figure 1 The operations performed in the illustrated embodiment will not be described in detail here.

[0250] The present application also provides a computer storage medium for storing computer software instructions used for the above-mentioned user interface control method, which includes a program designed for executing the user interface control method. The method can be as described above. Figure 1 A method for controlling a user interface is described.

[0251] In several embodiments provided in the embodiments of the present application, those skilled in the art should understand that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0252] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

[0253] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for controlling a user interface, characterized in that: Applied to a television terminal, the television terminal is equipped with a camera, and the method comprises: Determine whether there is a human body in the target area of ​​the space where the television terminal is located; If the human body exists in the target area, obtaining the motion of the body part of the human body; Controlling corresponding operations of the user interface of the television terminal according to the action; The television terminal is equipped with an infrared sensor, the body part is a human eye, and the action of obtaining the body part of the human body includes: Focusing the camera on the human eye; emitting an infrared beam toward the human eye; Receiving a target highlight point formed by the cornea of ​​the human eye reflecting the infrared light beam; Align the optical axis of the camera with the target highlight point; Determine a first rotation angle of the camera aimed at the target highlight point; Calculating a first distance between the camera and the target highlight point; Calculating first target spatial position coordinates of the target highlight point according to the first rotation angle of the target highlight point and the first distance; Determine the target projection coordinates of the first target space position coordinates on the screen of the television terminal; Receiving a first target light spot formed by the retina of the human eye reflecting the infrared light beam; Determine a first target offset corresponding to the shape of the first target light spot in a preset comparison table, wherein the comparison table records the correspondence between different light spot shapes and offsets, the shape of the first target light spot is one of the different light spot shapes, and the first target offset is one of the offsets; Determine a first screen position of the television terminal viewed by the human eye according to the target projection coordinates and the first target offset; Receiving a second target light spot formed by the retina of the human eye reflecting the infrared light beam; Determine in the comparison table a second target offset corresponding to the shape of the second target light spot, the shape of the second target light spot being one of the shapes of the different light spots, and the second target offset being one of the offsets; Determine a second screen position of the television terminal viewed by the human eye according to the target projection coordinates and the second target offset; A position change between the second screen position and the first screen position is regarded as the action.

2. The method for controlling a user interface according to claim 1, characterized in that: The body part is a human face, and the action of obtaining the body part of the human body includes: Focusing the camera on the face; Acquire a second rotation angle of the camera; Calculating a second distance between the camera and the face; Calculating the second spatial position coordinates of the face according to the second rotation angle and the second distance; Determine a line connecting the second spatial position coordinates and the center point of the screen of the television terminal; Calculating the angle between the connecting line and the central axis of the screen of the television terminal; The angle is considered as the motion.

3. The method for controlling a user interface according to claim 2, characterized in that: The corresponding operation of controlling the user interface of the television terminal according to the movement of the body part includes: Determining the included angle as a first rotation angle of the user interface; The user interface is rotated by a first rotation angle of the user interface.

4. The method for controlling a user interface according to claim 2, characterized in that: The method further comprises: displaying a virtual image of the user interface; The corresponding operation of controlling the user interface of the television terminal according to the movement of the body part includes: Determining the included angle as a second rotation angle of the virtual image; The virtual image is rotated by the second rotation angle.

5. The method for controlling a user interface according to claim 1, characterized in that: The corresponding operation of controlling the user interface of the television terminal according to the movement of the body part includes: A cursor on the user interface is moved according to the position change.

6. The method for controlling a user interface according to claim 1, characterized in that: After the human body exists and before obtaining the motion of the body part of the human body, the method further includes: Determine the number X of the human bodies, where X is a positive integer greater than 0; Establishing X number of user accounts according to the facial image of the human body; receiving a selection of a target user account, wherein the target user account is one of the X number of user accounts; Determine the human body corresponding to the target user account as the controller of the television terminal; The step of obtaining the body part of the human body is triggered.

7. A control system for a user interface, characterized in that, Applied to a television terminal, the system comprises: A judging unit, used for judging whether there is a human body in a target area of ​​the space where the television terminal is located; an acquisition unit, configured to acquire a motion of a body part of the human body if the human body exists in the target area; A control unit, used for controlling corresponding operations of the user interface of the television terminal according to the action; The television terminal is equipped with an infrared sensor, the body part is a human eye, and the acquisition unit is further used for: focusing a camera on the human eye; emitting an infrared beam toward the human eye; Receiving a target highlight point formed by the cornea of ​​the human eye reflecting the infrared light beam; Align the optical axis of the camera with the target highlight point; Determine a first rotation angle of the camera aimed at the target highlight point; Calculating a first distance between the camera and the target highlight point; Calculating first target spatial position coordinates of the target highlight point according to the first rotation angle of the target highlight point and the first distance; Determine the target projection coordinates of the first target space position coordinates on the screen of the television terminal; Receiving a first target light spot formed by the retina of the human eye reflecting the infrared light beam; Determine a first target offset corresponding to the shape of the first target light spot in a preset comparison table, wherein the comparison table records the correspondence between different light spot shapes and offsets, the shape of the first target light spot is one of the different light spot shapes, and the first target offset is one of the offsets; Determine a first screen position of the television terminal viewed by the human eye according to the target projection coordinates and the first target offset; Receiving a second target light spot formed by the retina of the human eye reflecting the infrared light beam; Determine in the comparison table a second target offset corresponding to the shape of the second target light spot, the shape of the second target light spot being one of the shapes of the different light spots, and the second target offset being one of the offsets; Determine a second screen position of the television terminal viewed by the human eye according to the target projection coordinates and the second target offset; A position change between the second screen position and the first screen position is regarded as the action.

8. A computer device, characterized in that: include: Processor, memory, bus, input and output interface, network interface; The processor is connected to the memory, the input / output interface, and the network interface via a bus; The memory stores a program; When the processor executes the program stored in the memory, the method for controlling the user interface as claimed in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the user interface control method according to any one of claims 1 to 6.