Tracking method and device

Abstract

The present application discloses a tracking method and apparatus, the tracking method comprising: obtaining position information of a first object relative to an area to be controlled and line of sight parameters of the first object, and inputting the position information and line of sight parameters into a target tracking model to determine the gaze position of the first object in the area to be controlled. The area to be controlled includes at least a display area of ​​an electronic device for outputting display content, the target tracking model includes a first tracking model obtained by processing an original tracking model based on change information between a sensor module and the display area, and the sensor module is used to collect the position information and / or line of sight parameters.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a tracking method and device. Background Art

[0002] Eye tracking is a machine vision technology that uses a device to capture images of a user's eyes, analyzes them using algorithms, and ultimately determines the location of the user's gaze on the display screen. This technology has high application value. Eye tracking technology first requires determining the relative position of the user's eyes and the display screen. However, this is difficult to determine when the user is in a large area, especially when the user's eyes are far away from the display screen.

[0003] To achieve this, eye tracking currently requires fixing the eye tracking device at a fixed relative position to the display screen of a specific device model. However, for touch-sensitive display systems, since the display screen and the eye tracking device have a fixed relative position, and the size and position of the display screen are fixed, the user can only use the eye tracking system when they are relatively close to the display screen and within a specific angular range between the user and the display screen. Furthermore, since the eye tracking device must be fixed relative to the corresponding display screen, the user must use the eye tracking system at a very close distance, which imposes many restrictions and is relatively inconvenient. Summary of the Invention

[0004] The embodiments of the present application provide a tracking method and device.

[0005] According to a first aspect of an embodiment of the present application, a tracking method is provided, the method comprising: obtaining position information of a first object relative to an area to be controlled and line of sight parameters of the first object, the area to be controlled including at least a display area of ​​an electronic device for outputting display content; inputting the position information and the line of sight parameters into a target tracking model to determine a gaze position of the first object in the area to be controlled; wherein the target tracking model comprises a first tracking model obtained by processing an original tracking model based on change information of a sensor module and the display area, and the sensor module is used to collect the position information and / or the line of sight parameters.

[0006] According to one embodiment of the present application, inputting the first position information and the line of sight parameters into a target tracking model includes: obtaining change information between the sensor module and the display area; if the change information meets the adjustment conditions, inputting the position information and the line of sight parameters into the first tracking model; and / or, if the change information does not meet the adjustment conditions, inputting the position information and the line of sight parameters into the original tracking model, wherein the original tracking model is a model trained based on the first object or a second object different from the first object.

[0007] According to one embodiment of the present application, if the change in at least one of the display area, the sensor module, or the relative position relationship between the sensor module and the display area is not less than a first set value, it is determined that the change information meets the adjustment condition; and / or, if any one of the display area, the sensor module, and the relative position relationship between the sensor module and the display area has not changed or the change in at least one of them is less than a second set value, it is determined that the change information does not meet the adjustment condition.

[0008] According to one embodiment of the present application, the training process of the original tracking model includes: obtaining the area information of the area to be controlled, the area information including at least the size and / or shape of the area to be controlled; obtaining the distance between the calibration object and the area to be controlled, and the line of sight parameters of the calibration object looking at multiple specified positions in the area to be controlled, the line of sight parameters including the line of sight angle and the line of sight height; constructing a three-dimensional space coordinate system based on the area information, the distance and the line of sight parameters as the original tracking model; wherein the calibration object is the first object or a second object different from the first object.

[0009] According to one embodiment of the present application, determining the gaze position of the first object in the area to be controlled includes: determining the vertical distance of the first object's eyeball relative to the display area; determining the angle between the first object's current line of sight and the display area and the first object's current line of sight height; and determining the intersection of the line of sight on the display area as the gaze position based on the angle, line of sight height and vertical distance.

[0010] According to one embodiment of the present application, the original tracking model is processed based on the change information of the sensor module and the display area, including: determining the position change information of the first object relative to the area to be controlled and the line of sight change parameters of the first object based on the change information; adjusting the parameters of the original tracking model based on the position change information and the line of sight change parameters to obtain the first tracking model.

[0011] According to one embodiment of the present application, the sensor module includes a first module and a second module; accordingly, obtaining the position information of the first object relative to the area to be controlled and the line of sight parameters of the first object includes: if the first object exceeds the acquisition range of the first module, executing: outputting a prompt message; and / or controlling the second module to obtain the position information and the line of sight parameters.

[0012] According to one embodiment of the present application, the method also includes: obtaining position information of multiple objects relative to the area to be controlled and line of sight parameters of the multiple objects, the multiple objects including a first object; determining the gaze position of the second object in the area to be controlled based on the gaze position of the first object in the area to be controlled, the positional relationship between the first object and the second object among the multiple objects, and the line of sight parameters of the second object.

[0013] According to one embodiment of the present application, the method also includes: obtaining first position information of the first object relative to the area to be controlled at the current moment and first line of sight parameters of the first object; obtaining motion information and line of sight change information of the first object; predicting the second line of sight parameters of the first object at the target moment and its second position information relative to the area to be controlled based on the motion information and the line of sight change information; and determining the gaze position of the first object in the area to be controlled based on the second line of sight parameters and the second position information.

[0014] According to the second aspect of the present application, a tracking device is also provided, including: an acquisition module for obtaining position information of a first object relative to an area to be controlled and line of sight parameters of the first object, wherein the area to be controlled includes at least a display area of ​​an electronic device for outputting display content; a tracking module for inputting the position information and the line of sight parameters into a target tracking model to determine the gaze position of the first object in the area to be controlled; wherein the target tracking model includes a first tracking model obtained by processing an original tracking model based on change information of a sensor module and the display area, and the sensor module is used to collect the position information and / or the line of sight parameters.

[0015] According to the third aspect of the present application, the device is further provided, comprising at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; and the processor is used to call program instructions in the memory to execute the above-mentioned tracking method.

[0016] The tracking method and apparatus of the present application embodiment obtains position information of a first object relative to a to-be-controlled area and the sight line parameters of the first object, and inputs the position information and sight line parameters into a target tracking model to determine the gaze position of the first object within the to-be-controlled area. The to-be-controlled area includes at least a display area of ​​an electronic device for outputting display content, and the target tracking model includes a first tracking model obtained by processing an original tracking model based on information about changes between a sensor module and the display area, wherein the sensor module is used to collect the position information and / or sight line parameters. Thus, a target tracking model obtained by processing the original tracking model based on information about changes between the sensor module and the display area can be used to track the gaze position of the first object. This effectively avoids the need for complex joint debugging of the sensor module and the electronic device when the position or model specifications of the sensor module or the display area of ​​the electronic device change. The sensor module can be deployed independently and does not need to be fixed to a specific position on the screen. This allows users to use the tracking function of the electronic device no longer limited to a distance close to the display area, effectively broadening the user's use position and viewing angle of the tracking function of the electronic device.

[0017] It should be understood that the teachings of this application do not necessarily achieve all of the beneficial effects described above, but that specific technical solutions can achieve specific technical effects, and other embodiments of this application can also achieve beneficial effects not mentioned above. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown in an illustrative and non-limiting manner, in which:

[0019] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0020] Figure 1 A schematic diagram of an application scenario of the tracking method according to an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram of the implementation process of the tracking method according to an embodiment of the present application is shown;

[0022] Figure 3 The spatial correspondence between the display area and the object of the tracking method according to the embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram of the structure of the tracking device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present application, and are not intended to limit the scope of the present application in any way. Rather, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0025] The technical solution of the present application is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Figure 1 A schematic diagram of an application scenario of the tracking method according to an embodiment of the present application is shown.

[0027] refer to Figure 1 The tracking system tracks the user's line of sight. The tracking system includes at least a sensor module. When the user stands at any position facing the display area and within the working range of the sensor module, the sensor module can collect the user's line of sight parameters and the user's position information relative to the display area. The user can be a real person, a robot, or an animal that needs to track the line of sight. Therefore, the solution is described below with reference to the first object and the calibration object. The sensor module may include an image acquisition device, etc. There are four pre-set gaze positions in the display area, and the display area here is the area to be controlled.

[0028] It should be noted that the above application scenarios are only provided to better illustrate the specific implementation details of the tracking method of the embodiment of the present application, and are not used to limit the implementation of the tracking method of the present application.

[0029] Figure 2 A schematic diagram of the implementation flow of the tracking method according to an embodiment of the present application is shown.

[0030] refer to Figure 2 The tracking method of the embodiment of the present application includes at least the following operation process: operation 201, obtaining the position information of the first object relative to the area to be controlled and the line of sight parameters of the first object, where the area to be controlled includes at least the display area of ​​the electronic device for outputting display content; operation 202, inputting the position information and line of sight parameters into a target tracking model to determine the gaze position of the first object in the area to be controlled; wherein the target tracking model includes a first tracking model obtained by processing the original tracking model based on the change information of the sensor module and the display area, and the sensor module is used to collect the position information and / or line of sight parameters.

[0031] In operation 201 , position information of a first object relative to an area to be controlled and a sight line parameter of the first object are obtained. The area to be controlled at least includes a display area of ​​the electronic device for outputting display content.

[0032] In this embodiment of the present application, the area to be controlled may be the entire area of ​​the display screen of the electronic device, or may include only the display area of ​​the display screen currently used to output display content, or may be the projection area projected by the electronic device.

[0033] In this embodiment of the present application, the position information may include: the vertical relative distance of the first object relative to the area to be controlled, the relative height ratio of the first object's eye height to the display area with the horizontal ground as the reference plane, the movement trajectory or movement trend of the first object, etc. The sight line parameters may include: the sight line height of the first object with the horizontal ground as the reference plane, the sight line angle determined by the first object's eyeball when looking at the area to be controlled, and the length of time the first object's eyeball remains stationary.

[0034] In this embodiment of the present application, the position information of the first object relative to the area to be controlled and the line of sight parameters of the first object can be obtained through the sensor module. The sensor module may include a first module and a second module. Accordingly, in the process of obtaining the position information of the first object relative to the area to be controlled and the line of sight parameters of the first object, if the first object exceeds the acquisition range of the first module, a prompt message can be output. For example: a reminder to use the second module for line of sight tracking, such as "Please wear AR glasses" or "Please look at the screen through the desktop sensor at the information collection table". The second module can also be directly controlled to obtain the position information and line of sight parameters. Similarly, the second module can be controlled to obtain the position information and line of sight parameters while outputting the prompt message.

[0035] For example, the first module can be a fixed image acquisition device. The second module can be a mobile pair of glasses or a desktop sensor equipped with an eye-tracking sensor. The second data acquisition range of the second module is larger than the first data acquisition range of the first module. Therefore, for some extreme cases where the first object is located far away from the display area, for example, the distance between the first object and the display area exceeds ten meters, the accuracy of eye tracking can be effectively increased by configuring peripherals such as glasses or desktop sensors equipped with eye-tracking sensors, thereby further improving the tracking precision.

[0036] In other embodiments, the first module and the second module may also be camera modules with different configurations, such as the first module being a fixed-focus or short-focus camera or camera array, and the second module being a zoom or long-focus camera or camera array, or even the second module may be a movable camera or camera array to distinguish it from the first module.

[0037] In operation 202, the position information and the line of sight parameters are input into a target tracking model to determine the gaze position of the first object in the area to be controlled, wherein the target tracking model includes a first tracking model obtained by processing the original tracking model based on the change information of the sensor module and the display area, and the sensor module is used to collect the position information and / or line of sight parameters.

[0038] In this embodiment of the present application, the training process of the original tracking model may include the following operations: obtaining regional information of the area to be controlled, the regional information including at least the size and / or shape of the area to be controlled, and obtaining the distance between the calibration object and the area to be controlled, as well as the line of sight parameters of the calibration object looking at multiple specified positions in the area to be controlled, the line of sight parameters including the line of sight angle and the line of sight height, thereby constructing a three-dimensional spatial coordinate system as the original tracking model based on the regional information, distance, and line of sight parameters. The calibration object may be a first object or a second object different from the first object.

[0039] For example, when a user first uses the tracking system, they can perform calibration operations on the control area at any location within the sensor module's operating range to build and train the initial tracking model. Initial use refers to the first time the tracking system is used. Any location within the sensor module's operating range refers to the range within which the sensor module's eye tracking sensors and image acquisition device can detect.

[0040] Combine Figure 1 As shown, the tracking system can display the positioning coordinates in the four corners of the display area in sequence, for example: Figure 1 The first object can look at each positioning coordinate in turn according to the display order of the positioning coordinates. The first object or the calibration image Figure 1 and the following text Figure 3 An icon with a circle on top and a rounded rectangle on the bottom is shown. When a first subject gazes at any positioning coordinate at a fixed position within the sensor module's operating range, the sensor module in the tracking system can obtain regional information about the area to be controlled, including at least the size and / or shape of the area to be controlled, the distance between the calibration subject and the area to be controlled, and line-of-sight parameters of the calibration subject's gaze at multiple designated locations within the area to be controlled, including line-of-sight angle and line-of-sight height.

[0041] Specifically, the area information of the area to be controlled can be obtained by the image acquisition device in the sensor module. The area information may include the relative position relationship and / or size and / or shape between multiple designated positions in the area to be controlled. Figure 1The area to be controlled is shown as a rectangle in the figure, where the area to be controlled is the same position and size as the display area. In actual application, the area to be controlled may be a rectangle, and the corresponding size of the area to be controlled may include the length and width of the rectangle. The area to be controlled may also be a circle, and the corresponding size of the area to be controlled may include the center position and radius of the circle. Similarly, the area to be controlled may also be an ellipse, a semicircle, a sector, or a combination of two or more graphics. Moreover, the entire area of ​​a partial area of ​​the screen of the electronic device may be determined as the area to be controlled, and line of sight tracking may be performed, while only the area on the right side of the rectangular screen area is configured as the display area. Figure 1 As a reference, it is assumed that the calibration object Figure 1 As shown in the fixed position gaze positioning coordinate 1, the distance sensor module between the calibration object and the area to be controlled can collect the first distance between the calibration object at the fixed position and the sensor module, and the second distance between the sensor module and the display area. If the sensor module and the calibration object are on the same side of the plane where the display area is located, the distance between the calibration object and the area to be controlled is the sum of the first distance and the second distance. If the sensor module and the calibration object are on different sides of the plane where the display area is located, the distance between the calibration object and the area to be controlled is the first distance minus the second distance. The line of sight parameters of the calibration object gazing at the positioning coordinate 1 may include the line of sight angle and the line of sight height. The line of sight angle is also called the eye deviation angle, which can be directly obtained by the eye sensor in the sensor module. The line of sight height refers to the vertical distance between the eyeball and the ground when the calibration object is gazing at the positioning coordinate 1 at the fixed position. It can also be directly obtained using the module sensor.

[0042] Furthermore, the same method can be used to obtain the corresponding line of sight parameters when the calibration object is fixed at the positioning coordinates 2, 3, and 4. Thus, a three-dimensional spatial coordinate system is constructed based on the distance between the calibration object and the area to be controlled, and the line of sight parameters of the calibration object at multiple specified locations within the area to be controlled, which can be used as the original tracking model.

[0043] Figure 3 The spatial correspondence between the display area and the object of the tracking method of the embodiment of the present application is shown. Figure 3, the three-dimensional space coordinate system is explained. The three-dimensional space coordinate system can show the three-dimensional coordinate position (x, y, z) of the calibration object's eyeball when the calibration object is gazing at the area to be controlled at any position, as well as the mapping between the gaze position of the calibration object at the position with the coordinates (x, y, z) and the eye angle of the calibration object. Therefore, when the relative position of the sensor module and the area to be controlled does not change, as long as the three-dimensional coordinate position (x1, y1, z1) of the first object's eyeball and the current eye angle of the first object can be obtained, the gaze position of the area to be controlled currently being gazed at by the first object can be determined based on the mapping between the gaze position of the calibration object at the position with the coordinates (x1, y1, z1) and the eye angle of the calibration object in the three-dimensional space coordinate system representing the original tracking model.

[0044] It should be noted that the line of sight parameters of the first object may also include the gaze time during which the first object maintains gaze at the current position. The duration of the gaze time can be determined according to actual needs, for example, it can be set to 1s, 3s or 3.5s, etc. The gaze time is set to ensure the stability of the tracking method, avoid the first object's brief stay in the process of scanning the area to be controlled to determine the gaze position, and effectively reduce the computational complexity of the tracking method.

[0045] In this embodiment of the present application, the first position information and line of sight parameters are input into the target tracking model. During this process, change information between the sensor module and the display area can be first obtained. If the change information meets the adjustment conditions, the position information and line of sight parameters are input into the first tracking model. If the change information does not meet the adjustment conditions, the position information and line of sight parameters are input into the original tracking model, which is a model trained based on the first object or a second object different from the first object.

[0046] In this embodiment of the present application, if the change in at least one of the display area, the sensor module, or the relative position relationship between the sensor module and the display area is not less than a first set value, it is determined that the change information meets the adjustment condition; and / or, if any one of the display area, the sensor module, and the relative position relationship between the sensor module and the display area has not changed or the change in at least one of them is less than a second set value, it is determined that the change information does not meet the adjustment condition.

[0047] It should be noted that the first set value of the change amount here can be 0, that is, as long as there is a change, it is determined that the change information meets the adjustment condition.

[0048] Specifically, when the display area changes and the magnitude of the change exceeds a corresponding set threshold, it can be determined that the change information meets the adjustment condition, for example, the size or shape of the display area changes.

[0049] It is also possible to determine that the change information meets the adjustment conditions when the relative position between the sensor module and the display area changes and the amplitude of the change exceeds the corresponding set threshold, for example: the angle or relative position between the display area and the sensor module or the vertical distance between the sensor module and the display area changes.

[0050] It is also possible to determine that the change information meets the adjustment conditions when the sensor module itself changes, such as: replacing the sensor module type, upgrading the firmware, upgrading the configuration, etc.

[0051] On the contrary, if all configurations such as the position and firmware of the display area and sensor module have not changed, the adjustment conditions are not met.

[0052] In this embodiment of the present application, it can also be set that when the relative position between the display area and the sensor module does not change or changes within a preset range, it is determined that the adjustment condition is not met.

[0053] In this embodiment of the present application, the following operations can be used to process the original tracking model based on the change information of the sensor module and the display area: based on the change information, the position change information of the first object relative to the area to be controlled and the line of sight change parameters of the first object are determined, and the original tracking model is adjusted based on the position change information and the line of sight change parameters to obtain a first tracking model.

[0054] For example, combining Figure 3 If the vertical distance between the sensor module and the display area remains unchanged, the height of the sensor module from the ground remains unchanged, and the relative position of the sensor module relative to the display area changes only in the X-axis direction, then the Y- and Z-axis information can remain unchanged. If the sensor module moves x1 along the positive X-axis direction, then x1 can be subtracted from the X-axis coordinates of all positions represented by 3D coordinates in the original 3D spatial coordinate system. If the position of the sensor module relative to the display area changes in the Y or Z axis, the corresponding principles can be applied to adjust the original tracking model based on the position change information and the line of sight change parameters to obtain a first tracking model.

[0055] In this embodiment of the present application, the following operations can be used to determine the gaze position of the first object in the area to be controlled: determine the vertical distance of the first object's eyeball relative to the display area; determine the angle between the first object's current line of sight and the display area and the first object's current line of sight height; based on the angle, line of sight height and vertical distance, determine the intersection of the line of sight on the display area as the gaze position.

[0056] For example, again combined Figure 3, based on the vertical distance of the first object's eyeball relative to the display area, the Y-axis information of the first object's eyeball in the three-dimensional space coordinate system can be determined. Moreover, in the process of determining the vertical distance of the first object's eyeball relative to the display area, the vertical connection line from the first object's eyeball to the display area can be determined, and the X-axis information of the first object's eyeball in the three-dimensional space can be further determined. The Z-axis information of the first object's eyeball in the three-dimensional space coordinate system can be determined based on the first object's current line of sight height. The angle between the first object's current line of sight and the display area can be determined based on the data of the eyeball offset angle sensor in the sensor module. In this way, the problem of determining the gaze position of the first object in the area to be controlled can be converted into a pure mathematical trigonometric function problem for solution.

[0057] In this embodiment of the present application, the position information of multiple objects relative to the area to be controlled and the line of sight parameters of multiple objects can also be obtained, and the gaze position of the second object in the area to be controlled can be determined based on the gaze position of the first object in the area to be controlled, the positional relationship between the first object and the second object among the multiple objects, and the line of sight parameters of the second object, wherein the multiple objects include the first object.

[0058] For example, if there are multiple objects looking at the same or different positions in the same area to be controlled at the same time, it is necessary to determine the gaze position of each object. At this time, the sight lines of multiple objects can be tracked in time according to the order and gaze time of the objects looking at the area to be controlled. It is also possible to track the sight lines of multiple objects synchronously in real time using multiple threads. In this process, the positional relationship between the objects can be determined based on the identifications of multiple objects. In the process of tracking the sight lines of objects in time, the sight parameters and gaze position of the first object at the previous moment, as well as the positional relationship between the first object and the second object and the gaze parameters of the current second object can be used to determine the gaze position of the current second object in the area to be controlled.

[0059] In this embodiment of the present application, the first position information of the first object relative to the area to be controlled at the current moment and the first line of sight parameter of the first object are also obtained, and the motion information and line of sight change information of the first object are obtained. Thus, the second line of sight parameter of the first object at the target moment and its second position information relative to the area to be controlled are predicted based on the motion information and line of sight change information. Further, the gaze position of the first object in the area to be controlled is determined based on the second line of sight parameter and the second position information.

[0060] For example, if the first object's gaze parameters remain unchanged, but the first object moves only in the positive X-axis direction relative to the display area, the first object's gaze position will also move in the positive X-axis direction based on the first object's movement information relative to the display area. It should be noted that this is merely an example and does not limit the tracking method in actual applications.

[0061] The tracking method and device of the present application embodiment obtains position information of a first object relative to a to-be-controlled area and the first object's line of sight parameters, and inputs the position information and line of sight parameters into a target tracking model to determine the first object's gaze position within the to-be-controlled area. The to-be-controlled area includes at least the display area of ​​the electronic device used to output display content, and the target tracking model includes a first tracking model obtained by processing an original tracking model based on information about changes in the sensor module and the display area. The sensor module is used to collect the position information and / or line of sight parameters. Thus, the target tracking model obtained by processing the original tracking model based on information about changes in the sensor module and the display area can be used to track the gaze position of the first object. This effectively avoids the need for complex joint debugging of the sensor module and the electronic device when the position or model specifications of the sensor module or the display area of ​​the electronic device change. The sensor module can be deployed independently and does not need to be fixed to a specific position on the screen. This allows users to use the tracking function of the electronic device no longer limited to a distance close to the display area, effectively broadening the user's use position and viewing angle of the tracking function of the electronic device.

[0062] Similarly, based on the above tracking method, an embodiment of the present application also provides a computer-readable storage medium, which stores a program. When the program is executed by a processor, the processor performs at least the following operation steps: operation 201, obtaining the position information of the first object relative to the area to be controlled and the line of sight parameters of the first object, and the area to be controlled includes at least the display area of ​​the electronic device for outputting display content; operation 202, inputting the position information and line of sight parameters into a target tracking model to determine the gaze position of the first object in the area to be controlled; wherein the target tracking model includes a first tracking model obtained by processing the original tracking model based on the change information of the sensor module and the display area, and the sensor module is used to collect position information and / or line of sight parameters.

[0063] Furthermore, based on the above tracking method, the present application also provides a tracking device, such as Figure 4The tracking device 40 includes: an acquisition module 401, used to obtain position information of the first object relative to the area to be controlled and the line of sight parameters of the first object, where the area to be controlled at least includes a display area of ​​the electronic device for outputting display content; a tracking module 402, used to input the position information and line of sight parameters into a target tracking model to determine the gaze position of the first object in the area to be controlled; wherein the target tracking model includes a first tracking model obtained by processing the original tracking model based on change information of the sensor module and the display area, and the sensor module is used to collect position information and / or line of sight parameters.

[0064] In this embodiment of the present application, the tracking module 402 includes: a change acquisition submodule, used to obtain change information of the sensor module and the display area; an input submodule, used to input the position information and line of sight parameters into the first tracking model if the change information meets the adjustment conditions; and / or, if the change information does not meet the adjustment conditions, input the position information and line of sight parameters into the original tracking model, the original tracking model being a model trained based on the first object or a second object different from the first object.

[0065] In this embodiment of the present application, the input submodule determines whether the change information meets the adjustment conditions based on the following operations: if the change in at least one of the display area, the sensor module, or the relative position relationship between the sensor module and the display area is not less than a first set value, it is determined that the change information meets the adjustment conditions; and / or, if any one of the display area, the sensor module, and the relative position relationship between the sensor module and the display area has not changed or the change in at least one of them is less than a second set value, it is determined that the change information does not meet the adjustment conditions.

[0066] In this embodiment of the present application, the training process of the original tracking model includes: obtaining regional information of the area to be controlled, the regional information at least including the size and / or shape of the area to be controlled; obtaining the distance between the calibration object and the area to be controlled, and the line of sight parameters of the calibration object looking at multiple specified positions in the area to be controlled, the line of sight parameters including line of sight angle and line of sight height; constructing a three-dimensional space coordinate system based on the regional information, distance and line of sight parameters as the original tracking model; wherein the calibration object is a first object or a second object different from the first object.

[0067] In this embodiment of the present application, the tracking module 402 includes: a distance determination submodule, which is used to determine the vertical distance of the eyeball of the first object relative to the display area; a height determination submodule, which is used to determine the angle between the current line of sight of the first object and the display area and the current line of sight height of the first object; and a gaze position determination submodule, which is used to determine the intersection of the line of sight on the display area as the gaze position based on the angle, line of sight height and vertical distance.

[0068] In this embodiment of the present application, the original tracking model is processed based on the change information of the sensor module and the display area, including: determining the position change information of the first object relative to the area to be controlled and the line of sight change parameters of the first object based on the change information; adjusting the parameters of the original tracking model based on the position change information and the line of sight change parameters to obtain a first tracking model.

[0069] In this embodiment of the present application, the sensor module includes a first module and a second module; accordingly, the acquisition module 401 includes: a first acquisition sub-module, which is used to execute: output prompt information; and / or control the second module to obtain position information and line of sight parameters if the first object exceeds the acquisition range of the first module.

[0070] In this embodiment of the present application, the tracking device 40 also includes: a multi-object acquisition module, used to obtain position information of multiple objects relative to the area to be controlled and line of sight parameters of multiple objects, the multiple objects including the first object; a relative tracking module, used to determine the gaze position of the second object in the area to be controlled based on the gaze position of the first object in the area to be controlled, the positional relationship between the first object and the second object among the multiple objects, and the line of sight parameters of the second object.

[0071] In this embodiment of the present application, the tracking device 40 also includes: a real-time acquisition module, used to obtain the first position information of the first object relative to the area to be controlled at the current moment and the first line of sight parameter of the first object; a motion acquisition module, used to obtain the motion information and line of sight change information of the first object; a position determination module, used to predict the second line of sight parameter of the first object at the target moment and its second position information relative to the area to be controlled based on the motion information and line of sight change information; a motion tracking module, used to determine the gaze position of the first object in the area to be controlled based on the second line of sight parameter and the second position information.

[0072] It should be noted that the above description of the tracking device embodiment is different from the above description of the tracking device embodiment. Figures 1 to 3 The description of the method embodiment shown is similar, with the same Figures 1 to 3 The method embodiment shown has similar beneficial effects, so it will not be described in detail. For technical details not disclosed in the embodiment of the tracking device of this application, please refer to the aforementioned Figures 1 to 3 The description of the method embodiment shown in the figure is understood, and in order to save space, it is not repeated here.

[0073] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0074] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0075] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0076] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0077] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, and other media that can store program codes.

[0078] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0079] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A tracking method comprising: Obtaining position information of a first object relative to an area to be controlled and a sight line parameter of the first object, wherein the area to be controlled includes at least a display area of ​​the electronic device for outputting display content; Inputting the position information and the sight line parameter into a target tracking model to determine the gaze position of the first object in the area to be controlled; The target tracking model includes a first tracking model obtained by processing an original tracking model based on change information of a sensor module and the display area, and the sensor module is used to collect the position information and / or the line of sight parameters; The step of inputting the position information and the sight line parameters into a target tracking model includes: Obtaining change information of the sensor module and the display area; If the change information meets the adjustment condition, input the position information and the sight line parameter into the first tracking model; and / or, If the change information does not meet the adjustment condition, the position information and the sight line parameter are input into the original tracking model, where the original tracking model is a model trained based on the first object or a second object different from the first object.

2. The method according to claim 1, wherein If the change in at least one of the display area, the sensor module, or the relative positional relationship between the sensor module and the display area is not less than a first set value, it is determined that the change information meets the adjustment condition; and / or, If any one of the display area, the sensor module, and the relative positional relationship between the sensor module and the display area does not change or the change amount of at least one of them is less than a second set value, it is determined that the change information does not meet the adjustment condition.

3. The method according to claim 1, wherein the training process of the original tracking model comprises: Acquiring area information of the area to be controlled, where the area information at least includes a size and / or a shape of the area to be controlled; Obtaining the distance between the calibration object and the area to be controlled, and sight line parameters of the calibration object looking at multiple designated positions in the area to be controlled, the sight line parameters including sight line angle and sight line height; constructing a three-dimensional space coordinate system based on the area information, the distance, and the line of sight parameters as the original tracking model; The calibration object is the first object or a second object different from the first object.

4. The method according to claim 1 or 2, wherein: Determining the gaze position of the first object in the area to be controlled includes: determining a vertical distance of an eyeball of a first subject relative to the display area; Determining an angle between a current line of sight of the first object and the display area and a current line of sight height of the first object; According to the included angle, the sight line height and the vertical distance, an intersection point of the sight lines on the display area is determined as the gaze position.

5. A tracking method comprising: Obtaining position information of a first object relative to an area to be controlled and a sight line parameter of the first object, wherein the area to be controlled includes at least a display area of ​​the electronic device for outputting display content; Inputting the position information and the sight line parameter into a target tracking model to determine the gaze position of the first object in the area to be controlled; The target tracking model includes a first tracking model obtained by processing an original tracking model based on change information of a sensor module and the display area, and the sensor module is used to collect the position information and / or the line of sight parameters; The processing of the original tracking model based on the change information of the sensor module and the display area includes: Determining, based on the change information, position change information of the first object relative to the area to be controlled and a sight line change parameter of the first object; The original tracking model is adjusted based on the position change information and the sight line change parameter to obtain the first tracking model.

6. The method according to claim 1 or 5, The sensor module includes a first module and a second module; accordingly, The obtaining of the position information of the first object relative to the area to be controlled and the sight line parameter of the first object includes: If the first object is beyond the acquisition range of the first module, execute: Output prompt information; and / or, The second module is controlled to obtain the position information and the sight line parameter.

7. The method according to claim 1 or 5, further comprising: Obtaining position information of a plurality of objects relative to the area to be controlled and sight line parameters of the plurality of objects, the plurality of objects including a first object; The gaze position of the second object in the area to be controlled is determined based on the gaze position of the first object in the area to be controlled, the positional relationship between the first object and a second object among the multiple objects, and the sight line parameter of the second object.

8. A tracking method comprising: Obtaining position information of a first object relative to an area to be controlled and a sight line parameter of the first object, wherein the area to be controlled includes at least a display area of ​​the electronic device for outputting display content; Inputting the position information and the sight line parameter into a target tracking model to determine the gaze position of the first object in the area to be controlled; The target tracking model includes a first tracking model obtained by processing an original tracking model based on change information of a sensor module and the display area, and the sensor module is used to collect the position information and / or the line of sight parameters; The method further comprises: Obtaining first position information of a first object relative to the area to be controlled at a current moment and first sight line parameters of the first object; Obtaining motion information and sight line change information of the first object; Predicting, based on the motion information and the line of sight change information, a second line of sight parameter of the first object at a target time and second position information thereof relative to the area to be controlled; The gaze position of the first object in the area to be controlled is determined based on the second sight line parameter and the second position information.

9. A tracking device comprising: an acquisition module, configured to obtain position information of a first object relative to an area to be controlled and a sight line parameter of the first object, wherein the area to be controlled includes at least a display area of ​​the electronic device for outputting display content; a tracking module, configured to input the position information and the sight line parameter into a target tracking model to determine a gaze position of the first object in the area to be controlled; The target tracking model includes a first tracking model obtained by processing an original tracking model based on change information of a sensor module and the display area, and the sensor module is used to collect the position information and / or the line of sight parameters; The step of inputting the position information and the sight line parameters into a target tracking model includes: Obtaining change information of the sensor module and the display area; If the change information meets the adjustment condition, input the position information and the sight line parameter into the first tracking model; and / or, If the change information does not meet the adjustment condition, the position information and the sight line parameter are input into the original tracking model, where the original tracking model is a model trained based on the first object or a second object different from the first object.

10. A tracking device comprising: an acquisition module, configured to obtain position information of a first object relative to an area to be controlled and a sight line parameter of the first object, wherein the area to be controlled includes at least a display area of ​​the electronic device for outputting display content; a tracking module, configured to input the position information and the sight line parameter into a target tracking model to determine a gaze position of the first object in the area to be controlled; The target tracking model includes a first tracking model obtained by processing an original tracking model based on change information of a sensor module and the display area, and the sensor module is used to collect the position information and / or the line of sight parameters; The processing of the original tracking model based on the change information of the sensor module and the display area includes: Determining, based on the change information, position change information of the first object relative to the area to be controlled and a sight line change parameter of the first object; The original tracking model is adjusted based on the position change information and the sight line change parameter to obtain the first tracking model.

11. A tracking device comprising: an acquisition module, configured to obtain position information of a first object relative to an area to be controlled and a sight line parameter of the first object, wherein the area to be controlled includes at least a display area of ​​the electronic device for outputting display content; a tracking module, configured to input the position information and the sight line parameter into a target tracking model to determine a gaze position of the first object in the area to be controlled; The target tracking model includes a first tracking model obtained by processing an original tracking model based on change information of a sensor module and the display area, and the sensor module is used to collect the position information and / or the line of sight parameters; Obtaining first position information of a first object relative to the area to be controlled at a current moment and first sight line parameters of the first object; Obtaining motion information and sight line change information of the first object; Predicting, based on the motion information and the line of sight change information, a second line of sight parameter of the first object at a target time and second position information thereof relative to the area to be controlled; The gaze position of the first object in the area to be controlled is determined based on the second sight line parameter and the second position information.

Images