Electronic device and method of operating the same

Abstract

The operation method of the electronic device can include obtaining position change information of a target site corresponding to a reference viewpoint, predicting a future speed of the target site, obtaining speed change information of the target site corresponding to the reference viewpoint, predicting a future acceleration of the target site, predicting future positions of both eyes corresponding to a target viewpoint, and outputting an image based on the future positions of both eyes corresponding to the target viewpoint.

Description

Technical Field

[0001] This disclosure relates to an electronic device for outputting an image based on the result of predicting the movement of a viewer, and a method of operating the electronic device. Background Technology

[0002] Special types of lenses, such as lenticular lenses, can be used to create three-dimensional (3D) effects, optical effects, or other visual effects in a display area. By utilizing the properties of lenticular lenses, display devices can output multiple images, allowing different images to be observed depending on the viewer's viewing position. For example, a display device can output a first image to pixels visible from a first viewing position, and simultaneously output a second image to pixels visible from a second viewing position. In this case, the viewer can observe the first image from the first viewing position and the second image from the second viewing position.

[0003] By utilizing the properties of lenticular lenses, display devices can provide 3D images to viewers, functioning as glasses-free 3D display devices. Due to the properties of lenticular lenses, the viewer's left and right eyes can perceive different images. For example, the display device can output a first image to the pixels visible to the viewer's left eye and simultaneously output a second image to the pixels visible to the viewer's right eye. In this case, due to the difference between the first image observed by the viewer's left eye and the second image observed by the viewer's right eye, the viewer may perceive that he or she is viewing a 3D image.

[0004] The above information is presented as background information only to aid in understanding this disclosure. No determination or assertion is made regarding whether any of the above content can be used as prior art with respect to this disclosure. Summary of the Invention

[0005] Solution to the problem

[0006] One aspect of this disclosure is to provide a method, system, apparatus, or computer program stored in a computer-readable storage medium.

[0007] Other aspects will be set forth in part in the description which follows, and in part will be apparent from the description, or may be learned by practice of the embodiments presented.

[0008] In embodiments of this disclosure, the operation method of the electronic device may include: obtaining position information of a target part corresponding to a past time point and position information of a target part corresponding to a reference time point from an image including a viewer's facial area and input via a camera. In embodiments of this disclosure, the operation method of the electronic device may include: obtaining position change information of a target part corresponding to a reference time point based on the position information of the target part corresponding to a past time point and the position information of the target part corresponding to a reference time point. In embodiments of this disclosure, the operation method of the electronic device may include: predicting the future velocity of a target part based on the position change information of the target part corresponding to a reference time point and the position change information of the target part corresponding to a past time point. In embodiments of this disclosure, the operation method of the electronic device may include: obtaining velocity change information of a target part corresponding to a reference time point based on the position change information of the target part corresponding to a reference time point and the position change information of the target part corresponding to a past time point. In embodiments of this disclosure, the operation method of the electronic device may include: predicting the future acceleration of a target part based on the velocity change information of the target part corresponding to a reference time point and the velocity change information of the target part corresponding to a past time point. In embodiments of this disclosure, the operation method of the electronic device may include: predicting the future positions of both eyes corresponding to a target time point based on future velocity and future acceleration. In embodiments of this disclosure, the operation method of the electronic device may include: outputting an image based on the future positions of the eyes corresponding to the target time point.

[0009] In embodiments of this disclosure, one or more computer-readable storage media are provided storing one or more computer programs including computer-executable instructions that, when executed individually or jointly by one or more processors of an electronic device, cause the electronic device to perform operations. In embodiments of this disclosure, operations may include the electronic device obtaining position information of a target part corresponding to a past time point and position information of a target part corresponding to a reference time point from an image including a viewer's facial region and input via a camera. In embodiments of this disclosure, operations may include the electronic device obtaining position change information of a target part corresponding to a reference time point based on the position information of the target part corresponding to a past time point and the position information of the target part corresponding to a reference time point. In embodiments of this disclosure, operations may include the electronic device predicting the future velocity of a target part based on the position change information of the target part corresponding to a reference time point and the position change information of the target part corresponding to a past time point. In embodiments of this disclosure, operations may include the electronic device obtaining velocity change information of a target part corresponding to a reference time point based on the position change information of the target part corresponding to a reference time point and the position change information of the target part corresponding to a past time point. In embodiments of this disclosure, the operation may include having an electronic device predict the future acceleration of a target region based on velocity change information of the target region corresponding to a reference time point and velocity change information of the target region corresponding to a past time point. In embodiments of this disclosure, the operation may include having an electronic device predict the future position of both eyes corresponding to a target time point based on future velocity and future acceleration. In embodiments of this disclosure, the operation may include having an electronic device output an image based on the future position of the eyes corresponding to a target time point.

[0010] In embodiments of this disclosure, an electronic device may include a memory storing one or more computer programs and one or more processors communicatively coupled to the memory. In embodiments of this disclosure, the one or more computer programs may include computer-executable instructions that, when executed individually or jointly by the one or more processors, cause the electronic device to obtain position information of a target part corresponding to a past time point and position information of a target part corresponding to a reference time point from an image that may include a viewer's facial region and is input via a camera. In embodiments of this disclosure, the one or more computer programs may include computer-executable instructions that, when executed individually or jointly by the one or more processors, cause the electronic device to obtain position change information of a target part corresponding to a reference time point based on the position information of the target part corresponding to a past time point and the position information of the target part corresponding to a reference time point. In embodiments of this disclosure, the one or more computer programs may include computer-executable instructions that, when executed individually or jointly by the one or more processors, cause the electronic device to predict the future velocity of a target part based on the position change information of the target part corresponding to a reference time point and the position change information of the target part corresponding to a past time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to obtain velocity change information of a target part corresponding to a reference time point based on position change information of the target part corresponding to a reference time point and position change information of the target part corresponding to a past time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to predict future acceleration of a target part based on velocity change information of the target part corresponding to a reference time point and velocity change information of the target part corresponding to a past time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause the electronic device to predict the future position of both eyes corresponding to a target time point based on future velocity and future acceleration. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to output an image based on the future position of the eyes corresponding to a target time point.

[0011] Other aspects, advantages, and salient features of this disclosure will become apparent to those skilled in the art from the following detailed description of various embodiments disclosed in conjunction with the accompanying drawings. Attached Figure Description

[0012] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0013] Figure 1 This is a diagram illustrating the output of an image based on the viewer's viewing position according to an embodiment of the present disclosure;

[0014] Figure 2 This is a diagram illustrating a method performed by an electronic device according to an embodiment of the present disclosure for displaying an image by predicting the future viewing position of a viewer;

[0015] Figure 3 This is a diagram illustrating a method performed by an electronic device to predict a viewer's future viewing position according to an embodiment of the present disclosure;

[0016] Figure 4 This is a diagram illustrating a method performed by an electronic device, according to an embodiment of the present disclosure, for filtering noise from position and velocity information of target parts included in a viewer's face;

[0017] Figure 5 This is a diagram illustrating a method for predicting the future velocity of a target location performed by an electronic device according to an embodiment of the present disclosure;

[0018] Figure 6 This is a diagram illustrating a method performed by an electronic device according to an embodiment of the present disclosure for predicting the future acceleration of a target part and predicting the future position of the target part;

[0019] Figure 7 This is a diagram illustrating a method performed by an electronic device to perform independent correction on predicted future location information according to an embodiment of the present disclosure;

[0020] Figure 8 This is a diagram illustrating a method performed by an electronic device to perform dependency correction on predicted future location information according to an embodiment of the present disclosure;

[0021] Figure 9 This is a diagram illustrating a method performed by an electronic device to predict a viewer's future viewing position according to an embodiment of the present disclosure;

[0022] Figure 10 This is a diagram illustrating a method performed by an electronic device to predict a viewer's future viewing position according to an embodiment of the present disclosure;

[0023] Figure 11 This is a diagram illustrating a method performed by an electronic device to predict a viewer's future viewing position according to an embodiment of the present disclosure;

[0024] Figure 12 This is a diagram illustrating a method performed by an electronic device to predict a viewer's future viewing position according to an embodiment of the present disclosure;

[0025] Figure 13 This is a diagram illustrating an electronic device according to an embodiment of the present disclosure that further filters noise from the viewer's future viewing position information;

[0026] Figure 14 This is a diagram illustrating an operation method of an electronic device according to an embodiment of the present disclosure; and

[0027] Figure 15 This is a diagram illustrating an example of an electronic device according to an embodiment of the present disclosure.

[0028] Throughout the accompanying drawings, it should be noted that the same reference numerals are used to depict the same or similar elements, features, and structures. Detailed Implementation

[0029] The following description, with reference to the accompanying drawings, is provided to aid in a full understanding of the various embodiments of this disclosure as defined by the claims and their equivalents. It includes various specific details to aid understanding, but these are to be considered exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the various embodiments described in detail herein without departing from the scope and spirit of this disclosure. Additionally, for clarity and brevity, descriptions of well-known functions and constructions may be omitted.

[0030] The terms and words used in the following description and claims are not limited to their literal meaning, but are used by the inventors only to enable a clear and consistent understanding of this disclosure. Therefore, those skilled in the art should understand that the following description of various embodiments of this disclosure is for illustrative purposes only and is not intended to limit the disclosure as defined by the appended claims and their equivalents.

[0031] It will be understood that, unless the context clearly specifies otherwise, the singular forms “a,” “an,” and “the” include plural indicators. Thus, for example, a reference to “component surface” includes a reference to one or more such surfaces.

[0032] In describing embodiments of this disclosure, detailed descriptions of related technologies will be omitted where it is deemed that such descriptions might unnecessarily obscure the gist of the disclosure. Furthermore, ordinal numbers (e.g., "first" or "second") used in the description of embodiments of this disclosure are identifier codes used to distinguish one component from another.

[0033] In the following description, embodiments of the present disclosure will be illustrated with reference to the accompanying drawings to allow those skilled in the art to readily implement the embodiments. However, the present disclosure may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein. Before describing the present disclosure in detail, the terminology used herein may be defined or understood as follows.

[0034] In this disclosure, it will be understood that when components are “connected” or “coupled” to each other, unless otherwise stated, components may be directly connected or coupled to each other, but alternatively, they may be connected or coupled to each other through components therebetween. Furthermore, “connection” can include wireless or wired connections.

[0035] Additionally, as used herein, components expressed as, for example, '...device', '...unit', '...module', etc., can represent units in which two or more components are combined into one component or where one component is divided into two or more components. Furthermore, in addition to their primary functions, each component described below can additionally perform some or all of the functions that other components are responsible for, and some of the primary functions of each component can be specifically performed by other components.

[0036] As used herein, the expression "at least one of a, b, or c" can indicate only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or variations thereof. In this disclosure, the expression "a or b" can refer to "a", "b", "a and b", or variations thereof. In this disclosure, the expression "a (or b or c)" or the expression "a, b, or c" can refer to "a", "b", "c", "a and b", "a and c", "b and c", "all of a, b, and c", or variations thereof.

[0037] In embodiments of this disclosure, a "time point" may include a time point in units of frames of the input or output image.

[0038] In embodiments of this disclosure, "position" may include a relative position within an input image. For example, "position" may include the coordinate information of a corresponding pixel among a plurality of pixels included in the input image. For example, the position of the left eye may include the coordinate information of the pixel containing the left eye among a plurality of pixels included in the input image. For example, the position of the left eye corresponding to a specific point in time may include the coordinate information of the pixel containing the left eye among a plurality of pixels included in a frame of the input image corresponding to that point in time.

[0039] In embodiments of this disclosure, "independent correction" may include correction that does not take into account information predicted for other parts (e.g., future location information). For example, "independent correction" may include correction that is independent of information predicted for other parts (e.g., future location information). For example, independent correction may include monocular correction.

[0040] In embodiments of this disclosure, "dependency correction" may include correction that takes into account information predicted for other sites (e.g., future location information). For example, "dependency correction" may include correction that depends on information predicted for other sites (e.g., future location information). For example, "dependency correction" may include correction that uses information predicted for other sites (e.g., future location information). For example, "dependency correction" may include correction based on information predicted for other sites (e.g., future location information). For example, dependency correction may include binocular correction.

[0041] In embodiments of this disclosure, a "target time point" may include a time point corresponding to the information (or data) to be predicted. In embodiments of this disclosure, a "reference time point" may include the most recent or closest time point among one or more time points associated with the information used to predict the target time point. For example, a "reference time point" may include the current time point. In embodiments of this disclosure, a "past time point" may include a time point before the reference time point. In embodiments of this disclosure, a "future time point" may include a time point after the reference time point.

[0042] In embodiments of this disclosure, the value of "center between the eyes" may include a value predicted with the center between the eyes as the target location, or a value calculated (or determined) based on the values ​​of the left and right eyes (e.g., predicted values).

[0043] In embodiments of this disclosure, interpupillary distance (IPD) information may include information about the distance between the viewer's eyes specific to the viewer, or information about the common distance between the eyes. For example, IPD information may be preset or pre-stored. For example, IPD information may be updated in real time or periodically. For example, IPD information may be obtained from an image containing the user's face.

[0044] It will be understood that the boxes in each flowchart and the combination of flowcharts can be executed by one or more computer programs that include computer-executable instructions. The entirety of one or more computer programs can be stored in a single memory device, or one or more computer programs can be divided into different parts stored in multiple different memory devices.

[0045] Any function or operation described herein can be processed by a processor or a combination of processors. A processor or combination of processors is circuitry that performs processing and includes, for example, an application processor (AP, such as a central processing unit (CPU)), a communication processor (CP, such as a modem), a graphics processing unit (GPU), a neural processing unit (NPU) (e.g., an artificial intelligence (AI) chip), a Wi-Fi chip, Bluetooth, etc. TM Circuits including chips, GPS chips, NFC chips, connectivity chips, sensor controllers, touch controllers, fingerprint sensor controllers, display driver integrated circuits (ICs), audio codec chips, Universal Serial Bus (USB) controllers, camera controllers, image processing ICs, microprocessor units (MPUs), system-on-a-chip (SoCs), and other ICs.

[0046] Figure 1 This is a diagram illustrating an image output based on the viewer's viewing position according to an embodiment of the present disclosure.

[0047] Even when electronic devices (e.g., display devices) render and output the same image (e.g., a light field image) in the same manner, the image actually perceived by a viewer may differ depending on the viewer's viewing position (e.g., viewpoint). For example, when the output image of an electronic device passes through a special lens (e.g., a lenticular lens or parallax barrier) to reach the viewer's eye, the viewer may visually perceive an image distorted or warped by the lens, and the image perceived by the viewer may vary depending on the viewer's viewing position. Therefore, electronic devices can render the output image based on the viewer's viewing position and output the rendered output image through an image output device (or image output module) so that the viewer visually perceives the image to be perceived by the viewer (e.g., the target image), i.e., the appropriate image.

[0048] When an electronic device outputs an image through an image output device (or image output module), the viewer's viewing position may change due to the viewer's movement. In embodiments of this disclosure, the electronic device can identify (or detect or obtain) the changed viewing position due to the viewer's movement, render the output image based on the changed viewing position, and output the rendered output image through the image output device (or image output module). For example, the electronic device can track the viewer's viewing position in real time and render and output the output image in real time based on the viewing position, which can change in real time.

[0049] In embodiments of this disclosure, in order to detect or track a viewer's viewing position (or a change in the viewer's viewing position), the electronic device can receive an image including the viewer. For example, the electronic device can identify (or acquire or receive) an image including the viewer input via an image input device (or image input module). For example, the electronic device can receive (or acquire or identify) an image including the viewer from an image input device (or image input module). For example, in order to detect or track a viewer's viewing position (or a change in the viewer's viewing position) in real time, the electronic device can identify (or acquire or receive) a real-time image including the viewer.

[0050] refer to Figure 1 An electronic device (e.g., a display device) may include a display 102 and a lenticular lens 104. The electronic device may output an image 106 to be perceived by the left eye to the left-eye-perceived pixels among a plurality of pixels included in the display 102, and an image 108 to be perceived by the right eye to the right-eye-perceived pixels. Among the plurality of pixels included in the display 102, the left-eye-perceived pixels and the right-eye-perceived pixels may vary depending on the viewer's viewing position.

[0051] In order to Figure 1 At a first time point 110, an image suitable for a viewer at a first viewing position 112 is provided. The electronic device can render (or generate) an output image such that data of a specific region 114 of image 106 is output to a first pixel 118 of display 102, and data of a specific region 116 of image 108 is output to a second pixel 120 of display 102. When the electronic device outputs the rendered output image on display 102, due to the optical properties of the lenticular lens (e.g., refractive properties), the region of the first pixel 118 of the output image can be perceived by the left eye of the viewer at the first viewing position 112, and the region of the second pixel 120 of the output image can be perceived by the right eye of the viewer at the first viewing position 112.

[0052] When an electronic device outputs an image rendered in the same manner as at the first time point 110 on display 102 (even if the viewer's viewing position changes (or moves), a viewer at a different viewing position (e.g., a second viewing position 134) may visually perceive a distorted image (or an inappropriate or warped image). Therefore, the electronic device needs to render the output image differently from the first time point 110 based on the change in the viewer's viewing position and output it on display 102. For example, to provide an image suitable for the viewer even when the viewer's viewing position changes at the second time point 130, the electronic device can detect the changed viewing position, i.e., the second viewing position 134, and render (or generate) an output image such that data from a specific region 114 of image 106 is output to the first pixel 118 of display 102, and data from a specific region 116 of image 108 is output to the third pixel 132 of display 102.

[0053] Because of the processing time required for electronic devices to detect the viewer's viewing position in real time from the input image and re-render and output the output image based on the detected viewing position, there may be a time delay in outputting an image suitable for the previous viewing position, even if the viewer's viewing position has changed. For example, due to the processing time of the electronic devices, there may be a delay between the point in time when the viewer's viewing position changes and the point in time when the rendered image is output according to the changed viewing position. During the delay, the electronic devices may output an image rendered based on the previously detected viewer's viewing position, and the viewer may perceive an image that differs from the appropriate image to be perceived (e.g., an inappropriate image, a distorted image, a warped image, an unnatural or broken image). To improve the degradation of the output image quality due to the delay, the processing rate of image input (e.g., image sensing), viewing position detection (or tracking), image rendering, and / or image output (e.g., display) can be increased, and the required time can be reduced; however, there may be limitations to increasing the processing rate and reducing the required time.

[0054] In embodiments of this disclosure, a method or electronic device may be provided for predicting a viewer's viewing position at a future point in time and outputting a rendered image based on the predicted viewing position. For example, refer to... Figure 1 The electronic device can predict the viewer's viewing position at the second time point 130 as a second viewing position 134 before reaching the second time point 130, and output an output image rendered for the second viewing position 134 on the display 102 at the second time point 130 (or the frame time corresponding to the second time point 130). Therefore, the electronic device can reduce latency by rendering an image according to the predicted viewing position and outputting the rendered image on the display 102.

[0055] Figure 2 This is a diagram illustrating a method performed by an electronic device according to an embodiment of the present disclosure for displaying an image by predicting the viewer's future viewing position.

[0056] In description Figure 2 In this case, the above references can be omitted. Figure 1 Redundant descriptions provided.

[0057] refer to Figure 2 In embodiments of this disclosure, a method 200 performed by an electronic device to predict a viewer's viewing position and display an image may include operations 210 to 260. In embodiments of this disclosure, the electronic device performing method 200 may include a display device. In embodiments of this disclosure, operations 210 to 260 of method 200 may be performed by at least one processor included in the electronic device. In embodiments of this disclosure, the electronic device may perform method 200 in real time. In embodiments of this disclosure, the electronic device may repeatedly perform method 200.

[0058] In embodiments of this disclosure, operations 210 to 260 of method 200 can be performed by multiple electronic devices. For example, some operations of method 200 can be performed by a first electronic device, while others can be performed by a second electronic device. Method 200 is not limited to... Figure 2 The method shown, and in one or more embodiments of this disclosure, method 200 may further include Figure 2 Operations not shown in the diagram, or some operations can be omitted.

[0059] In operation 210, the electronic device can detect a viewer by using an image sensor. For example, the electronic device may include an image input device (or module) (e.g., a camera) that includes a lens and an image sensor. The image sensor can convert light entering the electronic device through the camera lens into an electrical image signal. For example, the image sensor of the electronic device can detect a viewer.

[0060] In embodiments of this disclosure, an electronic device can acquire (or identify) an image (hereinafter referred to as the "input image") that includes a viewer and is captured using an image input device (or module). The image input device that captures the image including the viewer can be positioned (or arranged) to capture the space where the viewer is positioned to use the electronic device. For example, the electronic device can acquire (or identify) an image including the viewer by (or using) an image input device embedded in or included therein. For example, the electronic device can receive an image including the viewer via the image input device. In embodiments of this disclosure, the electronic device can receive an image including the viewer from an image input device connected to it or capable of communicating with it via wired or wireless means.

[0061] In operation 220, the electronic device can track (or identify or detect) the location of a viewer's facial region in the input image. For example, the tracker (or tracking module) of the electronic device can track the location of a viewer's facial region in the input image. For example, the electronic device can track the location of target areas (e.g., the left eye, right eye, area between the eyebrows, or the center area between the eyes) included in the viewer's face in the input image.

[0062] In operation 230, the electronic device can predict the viewer's viewing position at a future point in time (e.g., the position of the left eye, the position of the right eye). For example, the electronic device can analyze the viewer's movement and predict the viewer's viewing position using the results of the movement analysis. For example, the electronic device can use the viewer's cumulative viewing position information to predict the viewer's viewing position at a future point in time (hereinafter referred to as the "viewer's future viewing position"). For example, the electronic device can predict the viewer's future viewing position by using information about the viewer's viewing position corresponding to the current point in time or the viewer's viewing position corresponding to a past point in time.

[0063] refer to Figure 2 Operation 230 may include operations 232 to 240. In embodiments of this disclosure, the electronic device may perform operations 232 to 240 on a target site. In the following description of operations 232 to 240, operations 232 to 240 are described as being performed by the electronic device on an eye (e.g., the right or left eye) as an example of a target site, but this disclosure is not limited thereto. For example, the electronic device may perform at least one of operations 232 to 240 on another target site.

[0064] In operation 232, the electronic device can obtain the position and velocity information of one eye. For example, the electronic device can obtain the position and velocity information of one eye from an input image in real time. For example, the electronic device can obtain the position information of one of the viewer's eyes by tracking (or detecting) the position of the eye in the input image. For example, the electronic device can obtain the position information of one eye corresponding to a reference time point. For example, the electronic device can filter out noise from the obtained position information of one eye.

[0065] For example, an electronic device can obtain the velocity information of an eye by using the eye's position information. For example, an electronic device can obtain the change in the position of an eye as the eye's velocity information. For example, an electronic device can use the position information of an eye to calculate or determine the velocity of the eye. For example, an electronic device can obtain the velocity information of an eye corresponding to a reference time point. For example, an electronic device can filter out noise from the obtained position information of an eye.

[0066] In operation 234, the electronic device can predict (or calculate or determine) the future velocity of an eye. For example, the electronic device can obtain information about the future velocity of an eye. For example, the electronic device can predict the velocity of an eye corresponding to a future point in time. For example, the electronic device can obtain information about the predicted velocity of an eye corresponding to a future point in time. For example, the electronic device can use the velocity information of an eye to predict the future velocity of the eye.

[0067] In operation 236, the electronic device can obtain acceleration information of an eye. For example, the electronic device can obtain acceleration information of an eye by using the eye's velocity information. For example, the electronic device can obtain the change in the eye's velocity as acceleration information of the eye. For example, the electronic device can use the eye's velocity information to calculate or determine the eye's acceleration. For example, the electronic device can obtain acceleration information of an eye corresponding to a reference time point.

[0068] In operation 238, the electronic device can predict (or calculate or determine) the future acceleration of an eye. For example, the electronic device can obtain future acceleration information of an eye. For example, the electronic device can predict the acceleration of an eye corresponding to a future point in time. For example, the electronic device can obtain predicted acceleration information of an eye corresponding to a future point in time. For example, the electronic device can use the acceleration information of an eye to predict the future acceleration of the eye.

[0069] In operation 240, the electronic device can predict (or calculate or determine) the future position of an eye. For example, the electronic device can obtain information about the future position of an eye. For example, the electronic device can predict the position of an eye corresponding to a future point in time. For example, the electronic device can obtain predicted position information of an eye corresponding to a future point in time. For example, the electronic device can predict the future position of an eye based on the eye's future acceleration or future velocity. For example, the electronic device can use predicted acceleration information or predicted velocity information of an eye to predict the future position of the eye.

[0070] The future time point in operation 234 and the future time point in operation 238 can be different or the same. The future time point in operation 234 and the future time point in operation 240 can be different or the same. The future time point in operation 240 and the future time point in operation 238 can be different or the same.

[0071] In embodiments of this disclosure, the electronic device can predict a viewer's viewing position at a future point in time based on the future position of one eye. For example, the electronic device can predict (or calculate or determine) the position of the viewer's eyes at a future point in time based on the future position of one eye.

[0072] In operation 250, the electronic device can render (or generate) an output image by using the prediction result. For example, the electronic device can render the output image based on the viewer's viewing position at a future point in time.

[0073] In operation 260, the electronic device can display a rendered (or generated) output image. For example, the electronic device can display a rendered (or generated) output image on a display. For example, the electronic device can output an output image via an image output device (or module) connected to it or capable of communicating with it via wired or wireless means. For example, the electronic device can output an output image via an image output device (or module) embedded therein or included therein.

[0074] although Figure 2 The electronic device is shown performing operations 232 to 240 sequentially, but this disclosure is not limited thereto. For example, the electronic device may perform operation 232 and operations 234 and 236 in parallel. For example, the electronic device may perform operation 238 after performing operation 236, regardless of whether operation 234 is performed.

[0075] The following can be used as a reference. Figures 3 to 14 Describe in detail the operation of an electronic device that predicts the viewer's future viewing position.

[0076] Figure 3 This is a diagram illustrating a method performed by an electronic device to predict a viewer's future viewing position according to an embodiment of the present disclosure.

[0077] In description Figure 3 In this case, the above references can be omitted. Figure 1 or Figure 2 Redundant descriptions provided.

[0078] refer to Figure 3 In embodiments of this disclosure, the method 300 for predicting a viewer's viewing position, performed by an electronic device, may include operations 310 to 320. In embodiments of this disclosure, the electronic device performing method 300 may include a display device. In embodiments of this disclosure, operations 310 to 320 of method 300 may be performed by at least one processor included in the electronic device. In embodiments of this disclosure, the electronic device may perform method 300 in real time. In embodiments of this disclosure, the electronic device may repeatedly perform method 300.

[0079] In embodiments of this disclosure, operations 310 to 320 of method 300 can be performed by multiple electronic devices. For example, some operations of method 300 can be performed by a first electronic device, while others can be performed by a second electronic device. Method 300 is not limited to... Figure 3 The method shown, and in one or more embodiments of this disclosure, method 300 may further include Figure 3 Operations not shown in the diagram, or some operations can be omitted.

[0080] In embodiments of this disclosure, the electronic device can predict a viewer's future viewing position based on the position and velocity information of one eye. (See reference...) Figure 3 Electronic devices can perform the same operation on both the left and right eyes. Therefore, refer to the following... Figure 3 The description given for the left eye can be applied to the right eye; therefore, the description for the right eye can be omitted.

[0081] In operation 310, the electronic device can obtain the position information of the left eye, filter out noise from the position information of the left eye, obtain the velocity information of the left eye, and filter out noise from the velocity information of the left eye. In embodiments of this disclosure, the electronic device can determine the degree (or intensity) of filtering based on the degree of movement of the viewer (e.g., movement of the left eye) corresponding to a reference time point. For example, based on the identification of a large viewer movement, the electronic device can filter out noise from the position information or velocity information of the left eye with a weak filtering intensity.

[0082] In operation 312, the electronic device can predict the future velocity of the left eye. For example, the electronic device can use the velocity information of the left eye to predict (or calculate or determine) the future velocity of the left eye. For example, the electronic device can use the velocity information of the left eye after noise has been filtered out to predict the future velocity of the left eye.

[0083] In embodiments of this disclosure, the electronic device can use noise-filtered velocity information of the left eye to predict the future velocity of the left eye by weighted summing of the velocity of the left eye corresponding to a reference time point and the velocity of the left eye corresponding to past time points. For example, the electronic device can determine the weights based on the degree of viewer movement corresponding to the reference time point. For example, based on identifying a large viewer movement, the electronic device can increase the weight of the velocity corresponding to the reference time point and decrease the weight of the velocity corresponding to past time points. For example, based on identifying a small viewer movement, the electronic device can decrease the weight of the velocity corresponding to the reference time point and increase the weight of the velocity corresponding to past time points.

[0084] In operation 314, the electronic device can obtain acceleration information of the left eye, predict the future acceleration of the left eye, and filter out noise from the future acceleration of the left eye. For example, the electronic device can obtain acceleration information of the left eye based on the velocity information of the left eye. For example, the electronic device can obtain acceleration information of the left eye by using the velocity information of the left eye that has been filtered out of noise.

[0085] In embodiments of this disclosure, the electronic device can obtain nonlinear predicted acceleration information of the left eye corresponding to a future time point by using acceleration information of the left eye. The electronic device can predict (or calculate or determine) the future acceleration of the left eye by filtering noise from the nonlinear predicted acceleration information of the left eye. In embodiments of this disclosure, the electronic device can determine a decrease in the acceleration of the left eye based on the viewer's movement state or movement speed. For example, when the viewer is moving or the viewer's speed is greater than or equal to (or greater than) a certain speed, the electronic device can predict (or calculate or determine) the future acceleration of the left eye, where the acceleration has a small decrease. For example, when the viewer is stationary or the viewer's speed is less than (or less than or equal to) a certain speed, the electronic device can predict (or calculate or determine) the future acceleration of the left eye, where the acceleration has a large decrease.

[0086] In operation 316, the electronic device can predict the first future position of the left eye (e.g., the future position before correction) based on the predicted future acceleration and future velocity of the left eye. For example, the electronic device can predict the first future position of the left eye based on the future acceleration of the left eye after noise has been filtered out. For example, the electronic device can predict the future position of the left eye linearly based on the future velocity of the left eye. For example, the electronic device can predict the future position of the left eye non-linearly based on the future acceleration of the left eye.

[0087] In operation 318, the electronic device can perform monocular correction on the first future position of the left eye. For example, the electronic device can obtain a second future position of the left eye (e.g., a monocularly corrected future position) by correcting the first future position of the left eye. The first and second future positions on the left eye can be positions corresponding to the same target time point.

[0088] In embodiments of this disclosure, the electronic device can correct the future position of the left eye corresponding to the target time point based on the future position of the left eye corresponding to a time point prior to the target time point. In embodiments of this disclosure, the electronic device can adjust the correction intensity based on the time difference between a reference time point and a future time point (e.g., the target time point). For example, when predicting a relatively near future, i.e., when the difference between the reference time point and the target time point is small, the electronic device can correct the future position of the left eye with a relatively low correction intensity. For example, when predicting a relatively distant future, i.e., when the difference between the reference time point and the target time point is large, the electronic device can correct the future position of the left eye with a relatively high correction intensity.

[0089] In operation 320, the electronic device can perform binocular correction based on the second future position of the left eye and the second future position of the right eye. For example, the electronic device can predict (or calculate or determine) the third future position of the left eye by correcting the second future position of the left eye based on the future position of the binocular centers. For example, the future position of the center between the two eyes can be determined based on the future positions of the left and right eyes. For example, the electronic device can correct the second future position of the right eye based on the second future position of the left eye, and correct the second future position of the left eye based on the second future position of the right eye. For example, the electronic device can correct the second future position information of the right eye and the second future position information of the left eye based on the viewer's IPD information or predefined IPD information.

[0090] In embodiments of this disclosure, the electronic device can correct the future position of the center between the two eyes corresponding to the target time point or the second future position of the left eye corresponding to the target time point based on the future position of the center between the two eyes corresponding to a time point prior to the target time point. In embodiments of this disclosure, the electronic device can determine the correction intensity based on the viewer's movement state. For example, based on identifying the viewer's movement as slow, the electronic device can strongly filter the IPD information and use it for binocular correction. For example, based on identifying the viewer's movement as fast, the electronic device can weakly filter the IPD information and use it for binocular correction.

[0091] although Figure 3 The future positions of each in the left and right eyes are shown as the final output of method 300, but this disclosure is not limited thereto. For example, an electronic device may predict (or calculate or determine) the future position of the center between the eyes as the final output of method 300.

[0092] although Figure 3 The illustration shows an electronic device performing binocular correction after monocular correction, but this disclosure is not limited thereto. For example, the electronic device may perform monocular correction after performing binocular correction.

[0093] The following can be used as a reference. Figures 4 to 8 Description in Figure 3 The detailed operations performed in each operation.

[0094] Figure 4 This is a diagram illustrating a method performed by an electronic device, according to an embodiment of the present disclosure, for filtering noise from position and velocity information of a target part included in a viewer's face.

[0095] In description Figure 4 The above reference can be omitted. Figures 1 to 3 Any redundant description provided by any of them.

[0096] In operation 410, the electronic device can obtain the position information of the target part and filter out noise from the obtained position information of the target part. For example, the electronic device can filter out noise from the position information of the target part using the following Equation 1.

[0097] x t =α x t +(1-α)A t-1 Equation 1

[0098] In equation 1, x t It can represent the location of the target part corresponding to time point t (e.g., a reference time point), and A t-1 This can represent the location of the target part corresponding to time point t-1 (e.g., a past time point). In Equation 1, x on the right side... t This can represent the location of the target part before noise filtering (e.g., original location information), and the x-axis on the left... t This can indicate the location of the target area where noise has been filtered out. The left side, A... t-1 This can indicate the location of the target area where noise has been filtered out. For example, the value of α can be determined based on the filtering intensity. The value of α can be determined or set experimentally or empirically.

[0099] For example, electronic devices can filter out noise from the location information of a target part using Equation 2 below.

[0100] …Equation 2

[0101] In equation 2, It can represent the location of the target part corresponding to time point t (e.g., reference time point). It can represent the location of the target part corresponding to time point t-1 (e.g., a past time point), and This can represent the velocity of the target location corresponding to time point t-1. In Equation 2, the right side... This can indicate the location of the target area before noise is filtered out, on the left side. This can indicate the location of the target area where noise has been filtered out. (Left side) It can represent the location of the target area where noise has been filtered out. For example, the value of α can be determined based on the filtering intensity. For example, the value of α can be determined or set through experiments or experience.

[0102] In operation 420, the electronic device can calculate the velocity of the target part based on the position information of the target part. For example, the electronic device can calculate the velocity of the target part using Equation 3 below.

[0103] Equation 3

[0104] In equation 3, It can represent the location of the target part corresponding to time point t, and It can represent the location of the target part corresponding to time point t-1. It can represent the velocity of the target part corresponding to time point t.

[0105] In operation 430, the electronic device can determine whether noise exists in the speed information. For example, the electronic device can determine whether noise exists in the speed information based on the degree of movement of the viewer. For example, when the degree of movement of the viewer is large, the calculated speed value can be considered to be caused by the movement of the viewer, and therefore, it can be determined that there is no noise or very little noise in the speed information. Conversely, when the degree of movement of the viewer is small, the calculated speed value can be considered to be caused by noise, and therefore, it can be determined that there is a lot of noise in the speed information.

[0106] In embodiments of this disclosure, the degree of a viewer's movement can be determined based on the speed corresponding to a reference time point and the speed corresponding to past time points. For example, it can be based on the magnitude of the speed corresponding to the reference time point. And the magnitude of the velocity corresponding to a past point in time. The difference between the two (e.g., the amount of change in speed) is used to determine the degree of movement of the viewer. For example, when the difference is less than (or less than or equal to) a threshold, it can be determined that the degree of movement is small, i.e., there is noise (or a lot of noise). For example, when the difference is greater than or equal to (or greater than) a threshold, it can be determined that the degree of movement is large, i.e., there is no noise (or very little noise).

[0107] In embodiments of this disclosure, the electronic device can filter noise from velocity information of a target location based on determined results. For example, the electronic device can determine the values ​​of parameters used for velocity filtering based on the determined results. The values ​​of the parameters used in velocity filtering can be used to determine the weight of the velocity corresponding to a reference time point (e.g., time point t) when filtering out velocity noise.

[0108] refer to Figure 4 Based on the determination that no noise exists, in operation 432, the electronic device can determine the value of parameter s as a first value α. Based on the determination that noise exists, in operation 434, the electronic device can determine the value of parameter s as a second value β. In operation 440, the electronic device can filter out noise from the velocity information of the target part based on the value of the parameter determined according to the determination result.

[0109] For example, electronic devices can filter out noise from the velocity information of the target part by using Equation 4 below.

[0110] …Equation 4

[0111] In equation 4, It can represent the velocity of the target part corresponding to time point t. This can represent the velocity of the target location corresponding to time point t-1. s can represent the parameter used to filter noise from the velocity information. In Equation 4, the right side... This can represent the velocity of the target location before noise is filtered out (e.g., raw velocity information), and the left side... This can represent the velocity of the target area where noise has been filtered out. For example, the one on the left. It can represent the speed of the target part where noise has been filtered out.

[0112] For example, electronic devices can filter out noise from the velocity information of the target part by using Equation 5 below.

[0113] …Equation 5

[0114] In equation 5, It can represent the velocity of the target part corresponding to time point t. It can represent the velocity of the target part corresponding to time point t-1, and This can represent the acceleration of the target location corresponding to time point t-1. s can represent a parameter used to filter noise from the velocity information. In Equation 5, the right side... This can represent the velocity of the target location before noise is filtered out, and the left side... This can represent the velocity of the target area where noise has been filtered out. For example, the one on the left. It can represent the speed of the target part where noise has been filtered out.

[0115] For example, based on the determination that noise exists, the electronic device can determine the value of s as β. For example, based on the determination that noise does not exist, the electronic device can determine the value of s as α. α and β can be values ​​determined or set experimentally or empirically. For example, α can be a value greater than β. Therefore, when it is determined that there is no noise or very little noise, a larger weight can be applied to the speed corresponding to the reference time point, and when it is determined that there is noise or a large amount of noise, a larger weight can be applied to the speed corresponding to past time points.

[0116] refer to Figure 4 The presence of noise may include: whether the speed value corresponding to the reference time point is caused by noise, whether the viewer's movement is small, whether a high degree of filtering is applied to the speed corresponding to the reference time point, or whether the value of the parameter used for speed filtering is determined to be a second value.

[0117] Figure 5 This is a diagram illustrating a method performed by an electronic device to predict the future speed of a target location according to an embodiment of the present disclosure.

[0118] Reference Figure 5 The above references can be omitted. Figures 1 to 4 Any redundant description provided by any of them.

[0119] In embodiments of this disclosure, the electronic device can predict the future velocity of a target part based on the velocity of the target part corresponding to a reference time point (e.g., current velocity) and the velocity of the target part corresponding to a past time point (e.g., past velocity information). For example, the electronic device can use the current velocity and past velocity of the target part and / or weights to predict the future velocity of the target part. Figure 5 An example is shown of an electronic device according to an embodiment of the present disclosure that determines the weight of the current speed of a target part and predicts the future speed based on the determined weight.

[0120] In operation 510, the electronic device can calculate, determine, or obtain the weight of the current velocity of the target part (or the velocity of the target part corresponding to a reference time point). For example, the electronic device can calculate the weight of the current velocity using Equation 6 below.

[0121] …Equation 6

[0122] In equation 6, It can represent the velocity corresponding to time point t-1 (e.g., a past time point). b can represent the speed corresponding to a time point t (e.g., a reference time point or the current time point), and b can represent the weight of the current speed. β and β are parameter values ​​used to calculate the weights, and can be predetermined or preset. For example, The value of β can be determined or set through experimentation or experience.

[0123] In operation 520, the electronic device can determine whether the weight of the current speed is greater than an upper limit value (or a maximum value or threshold). For example, the upper limit value can be a predetermined or preset value. For example, the upper limit value can be determined or set through experimentation or experience. Based on the determination that the weight is greater than the upper limit value, in operation 522, the electronic device can determine the weight of the current speed as the upper limit value.

[0124] In operation 530, the electronic device can predict the future speed of the target part by using the weights of the current speed. For example, the electronic device can calculate or determine the future speed of the target part by weighted summation of the current speed and past speed of the target part. For example, based on the determination that the weights are not greater than an upper limit value, the electronic device can predict the future speed of the target part by using the weights calculated in operation 510. For example, based on the determination that the weights are greater than an upper limit value, the electronic device can predict the future speed of the target part by using the weights of the current speed (i.e., the upper limit value) determined in operation 522.

[0125] For example, electronic devices can predict the future speed of a target part by using Equation 7 below.

[0126] …Equation 7

[0127] In equation 7, It can represent the velocity of a target part corresponding to a time point t (e.g., a reference time point). b can represent the velocity of the target part corresponding to time point t-1 (e.g., a past time point), and b can represent the weight of the velocity of the target part corresponding to time point t (e.g., the weight of the current velocity). It can represent the future velocity of a target area. For example, It can represent the future velocity of a target part corresponding to a future time point (e.g., time point t+1).

[0128] Figure 6 This is a diagram illustrating a method performed by an electronic device according to an embodiment of the present disclosure for predicting the future acceleration of a target part and predicting the future position of the target part.

[0129] Reference Figure 6 The above references can be omitted. Figures 1 to 5 Any redundant description provided by any of them.

[0130] In operation 610, the electronic device can calculate (or obtain) the average acceleration of the target part from a specific time point to a reference time point (e.g., the current time point). For example, the electronic device can calculate or determine the average acceleration of the target part based on the acceleration of the target part corresponding to the reference time point (e.g., time point t) and the acceleration of the target part corresponding to past time points (e.g., time points t-1 or t-2). For example, the electronic device can calculate the average acceleration of the target part by using Equation 8 below.

[0131] …Equation 8

[0132] In equation 8, It can represent the acceleration of the target part corresponding to time point t. It can represent the acceleration of the target part corresponding to time point t-1, and This can represent the acceleration of the target part corresponding to time point t-2. m can represent the average acceleration of the target part.

[0133] In operation 620, the electronic device can determine whether the viewer's movement state corresponding to a reference time point (e.g., time point t or the current time point) is movement. In embodiments of this disclosure, the electronic device can determine whether the viewer's movement state corresponding to the reference time point is movement by determining whether the viewer's eyes move in the same direction at the reference time point and / or whether the eyes move at a speed greater than or equal to (or greater than) a specific speed. For example, based on determining that the viewer's eyes move in the same direction at the reference time point and that the viewer's eyes move at a specific speed or greater at the reference time point, the electronic device can determine the viewer's movement state corresponding to the reference time point as movement. For example, when the directions of the left and right eye velocities corresponding to the reference time point are the same, and the magnitudes of the left and right eye velocities corresponding to the reference time point are greater than or equal to (or greater than) a threshold, the electronic device can determine the viewer's movement state corresponding to the reference time point as movement.

[0134] Based on the determination that the viewer's movement state corresponding to the reference time point is not moving, in operation 630, the electronic device can determine whether the viewer's movement state corresponding to the reference time point is stationary. In embodiments of this disclosure, the electronic device can determine whether the viewer's movement state corresponding to the reference time point is stationary by determining whether the viewer's eyes move in the same direction at the reference time point and / or whether the eyes move at a speed less than (or less than or equal to) a specific speed. For example, based on the determination that the viewer's eyes move at a speed less than a specific speed at the reference time point, the electronic device can determine the viewer's movement state as stationary. For example, when the magnitudes of the speeds of the left and right eyes corresponding to the reference time point are less than (or less than or equal to) a threshold, the electronic device can determine the viewer's movement state corresponding to the reference time point as stationary.

[0135] Based on the determination that the viewer's movement state corresponding to the reference time point is not stationary, in operation 640, the electronic device can determine whether the viewer's movement state corresponding to a past time point (e.g., time point t-1) is moving.

[0136] In embodiments of this disclosure, the electronic device can predict the future acceleration of a target location based on parameter values ​​determined according to the viewer's movement state and the average acceleration calculated in operation 610. For example, the electronic device can predict a nonlinear predicted acceleration of the target location corresponding to a target time point by using parameter values ​​determined based on the viewer's movement state and the average acceleration of the target location. For example, the electronic device can predict the future acceleration of the target location corresponding to a target time point by filtering out noise from the nonlinear predicted acceleration of the target location corresponding to the target time point. For example, the electronic device can predict the future acceleration of the target location by using Equation 9 below.

[0137]

[0138] …Equation 9

[0139] In Equation 9, m can represent the average acceleration, r can represent a parameter whose value is determined based on the viewer's motion state, and It can represent the nonlinear predicted acceleration of the target location corresponding to time point t+n (e.g., the target time point), with time point t as a reference time point. γ can represent a parameter indicating the degree of filtering of acceleration noise. For example, γ can be determined or set experimentally or empirically. This can represent the future acceleration of the target location corresponding to time point t-1+n, predicted with time point t-1 as a reference time point. For example, It can represent the future acceleration of the target part from which noise in the acceleration is filtered out. It can represent the future acceleration of the target part corresponding to time point t+n, predicted with time point t as a reference time point.

[0140] When the viewer's movement state corresponding to the reference time point is determined to be moving in operation 620, or when the viewer's movement state corresponding to a past time point is determined to be moving in operation 640, in operation 650, the electronic device can predict the future acceleration of the target area by using parameter values ​​corresponding to the movement. For example, when the viewer's movement state corresponding to the reference time point is determined to be moving in operation 620, or when the viewer's movement state corresponding to a past time point is determined to be moving in operation 640, the future acceleration of the target area (e.g., The direction can be related to the future velocity of the target part (e.g., () are in the same direction.

[0141] When it is determined in operation 630 that the viewer's movement state corresponding to the reference time point is stationary, or when it is determined in operation 640 that the viewer's movement state corresponding to a past time point is stationary, in operation 660, the electronic device can predict future acceleration by using parameter values ​​corresponding to stationary states. For example, when the viewer's movement state corresponding to the reference time point is determined to be stationary in operation 630, or when the viewer's movement state corresponding to a past time point is determined to be stationary in operation 640, the future acceleration of the target area (e.g., The direction of the target part can be different from its future velocity (e.g., (The direction of)

[0142] For example, the parameter value corresponding to movement can be greater than the parameter value corresponding to stillness. For example, the parameter value corresponding to movement or the parameter value corresponding to stillness can be determined or set through experimentation or experience.

[0143] In embodiments of this disclosure, the electronic device can predict the future position of the target part based on its future acceleration. For example, the electronic device can obtain, determine, or predict the future position of the target part by using its future acceleration and future velocity.

[0144] refer to Figure 6 In operation 650, the electronic device can predict future acceleration, and then in operation 680, predict the future position of the target part based on the future acceleration and future velocity of the target part. For example, the electronic device can predict the future position of the target part by using Equation 10 below.

[0145] …Equation 10

[0146] In equation 10, This can represent the location information of the target part corresponding to time point t (e.g., a reference time point) after noise has been filtered out, and n can represent the difference between the target time point (time point t+n) and time point t. It can represent the future velocity (e.g., predicted velocity) of a target part corresponding to a future point in time (e.g., time t+1). It can represent the future acceleration (e.g., predicted acceleration) predicted at time t+n, with time t as a reference time point, and It can represent the future position of the target part corresponding to time point t+n, predicted with time point t as a reference time point.

[0147] refer to Figure 6 In operation 660, the electronic device can predict the future acceleration of the target part, and in operation 670, it determines whether the predicted acceleration distance of the target part is greater than the predicted velocity distance. The predicted velocity distance can refer to the distance of movement (e.g., positional movement distance) predicted by the predicted velocity of the target part (e.g., the future velocity of the target part), and the predicted acceleration distance can refer to the distance of movement (e.g., positional movement distance) predicted by the predicted acceleration of the target part (e.g., the future acceleration of the target part). For example, the predicted velocity distance can correspond to... The value, and the predicted acceleration distance can correspond to The value of .

[0148] Based on the premise that the predicted acceleration distance is no greater than the predicted velocity distance, in operation 680, the electronic device can predict the future position of the target part. For example, based on the premise that the predicted acceleration distance is no greater than the predicted velocity distance, the electronic device can predict the future position of the target part by using Equation 10 above.

[0149] Based on the determination that the user's movement state is stationary and the predicted acceleration distance is greater than the predicted velocity distance, in operation 690, the electronic device can predict the position of the target part corresponding to the reference time point as the future position of the target part. For example, the electronic device can determine the position of the target part corresponding to the reference time point, after filtering out noise in the position information, as the future position of the target part corresponding to the target time point. For example, when the predicted velocity distance of the future velocity in the positive (+) direction corresponds to n pixels relative to the position of the target part corresponding to the reference time point (hereinafter referred to as the "reference position"), and the predicted acceleration distance of the future acceleration in the negative (-) direction corresponds to n+1 pixels or more, the future position of the target part predicted by using Equation 10 above is in the negative (-) direction from the reference position. Therefore, the direction of the future velocity relative to the reference position (i.e., the positive (+) direction) and the direction of the future position (i.e., the negative (-) direction) can be different from each other. Therefore, when the user's movement state corresponds to being stationary and the predicted acceleration distance is greater than the predicted velocity distance, the electronic device can determine the position of the target part corresponding to the reference time point as the future position of the target part.

[0150] although Figure 6 The illustration shows an electronic device determining whether a viewer is moving and then whether the viewer is stationary, but this disclosure is not limited thereto. For example, the electronic device could determine whether the viewer is stationary and then determine whether the viewer is moving.

[0151] Figure 7 This is a diagram illustrating a method performed by an electronic device to perform independent correction on predicted future location information according to an embodiment of the present disclosure.

[0152] Reference Figure 7 The above references can be omitted. Figures 1 to 6 Any redundant description provided by any of them.

[0153] In operation 710, the electronic device can identify (or obtain or determine) weights for independently correcting the future position information of the target location. The weights for independent correction can refer to the degree or intensity of independent correction. In embodiments of this disclosure, the values ​​of the weights for independent correction of the target location corresponding to a target time point can be predetermined or preset. In embodiments of this disclosure, based on the target time point, the electronic device can calculate, determine, or obtain the weights for independent correction of the target location corresponding to the target time point. For example, the electronic device can calculate the weights for independent correction of the target location corresponding to the target time point using Equation 11 below.

[0154] w = min(0.1n, 0.5) ... Equation 11

[0155] In Equation 11, n can represent the difference between the target time point (e.g., time point t+n) and the reference time point (e.g., the reference time point or time point t), and w can represent the weights used for independent correction of the target part corresponding to the target time point.

[0156] In operation 720, the electronic device can perform independent correction on the future position of the target part using the weights of independent corrections of the target part corresponding to the target time point. For example, the electronic device can perform independent correction on the future position of the target part using Equation 12 below.

[0157] …Equation 12

[0158] In Equation 12, w can represent the weight used for independent correction of the target part corresponding to the target time point. It can represent the future position predicted from time point t+n, using time point t as a reference time point, and This can represent the future position predicted from time point t-1+n, using time point t-1 as a reference time point. In Equation 12, the right side... It can indicate the future position of the target area before independent correction, on the left side. It can indicate the future position of the target part after independent correction.

[0159] Figure 8 This is a diagram illustrating a method performed by an electronic device to perform dependency correction on predicted future location information according to an embodiment of the present disclosure.

[0160] Reference Figure 8 The above references can be omitted. Figures 1 to 7 Any redundant description provided by any of them.

[0161] In operation 810, the electronic device can calculate (or determine or obtain) the center position between the future position of the target site and the future position of another site. For example, the electronic device can calculate, determine or obtain the center position (e.g., the center between the eyes) between the future position of the target site (e.g., the left eye) corresponding to the target time point and the future position of another site (e.g., the right eye) corresponding to the target time point.

[0162] In operation 820, the electronic device can determine whether the viewer's movement meets the movement conditions. For example, the electronic device can determine whether the movement of a target part and another part meets the movement conditions. For example, based on whether the direction of the velocity of the target part is the same as the direction of the velocity of another part, and / or whether the magnitude of the velocity of the target part and the magnitude of the velocity of the other part are less than (or less than or equal to) a threshold, the electronic device can determine whether the viewer's movement meets the movement conditions. For example, when the direction of the velocity of the target part is the same as the direction of the velocity of another part, and the magnitude of the velocity of the target part and the magnitude of the velocity of the other part are less than the threshold, the electronic device can determine that the viewer's movement meets the movement conditions. For example, when the direction of the velocity of the target part is different from the direction of the velocity of another part, or when the magnitude of the velocity of the target part or the magnitude of the velocity of the other part is greater than or equal to the threshold, the electronic device can determine that the viewer's movement does not meet the movement conditions.

[0163] In embodiments of this disclosure, based on determining whether a viewer's movement meets a movement condition, the electronic device can calculate or determine a weight k for dependency correction. The weight k for dependency correction can refer to the degree or strength of dependency correction.

[0164] refer to Figure 8 When the viewer's movement does not meet the movement condition, in operation 850, the electronic device can determine the weight k used for dependency correction as a first value α. When the viewer's movement meets the movement condition, in operation 840, the electronic device can determine the weight k used for dependency correction as a second value β. For example, the first value can be greater than the second value.

[0165] In operation 860, the electronic device can dependently correct the future position of the target site based on information about the distance between the target site and another site (e.g., information about the distance between the eyes) and a weight k for dependent correction. In embodiments of this disclosure, the electronic device can correct the center position between the future position of the target site and the future position of the other site by using the weight for dependent correction. The electronic device can obtain the dependent corrected future position of the target site and the dependent corrected future position of the other site based on information about the distance between the target site and the other site and the corrected center position. For example, the electronic device can dependently correct the future position of the target site and the future position of the other site by using Equation 13 below.

[0166]

[0167] …Equation 13

[0168] In equation 13, This can represent the center position between the future position of a target part corresponding to time point t+n, predicted with time point t as a reference time point, and the future position of another part. It can represent the center position of the future position of the target part corresponding to the future position of another part, with time point t-1 as the reference time point. d can represent information about the distance between the target part and the other part, and k can represent the weight used for dependency correction. It can represent the future position of the target body part (e.g., the left eye) corresponding to time point t+n, with time point t as a reference time point; that is, the future position dependent on correction. It can represent the future position of another part (e.g., the right eye) corresponding to time point t+n, with time point t as a reference time point, i.e., the future position dependent on correction.

[0169] For example, electronic devices can use Equation 14 below to dependently correct the future position of a target part and the future position of another part.

[0170]

[0171] …Equation 14

[0172] In equation 14, It can represent the center position between the future positions of the target parts corresponding to time point t+n, with time point t as a reference time point, and d can represent information about the distance between the target parts and another part. It can represent the future position of the target body part (e.g., the left eye) corresponding to time point t+n, with time point t as a reference time point; that is, the future position dependent on correction. It can represent the future position of another part (e.g., the right eye) corresponding to time point t+n, with time point t as a reference time point, i.e., the future position dependent on correction.

[0173] Reference above Figures 4 to 8 The equations described are examples of operations performed in embodiments of this disclosure, and this disclosure is not limited to the above references. Figures 4 to 8 The equations described. Furthermore, this disclosure is not limited to... Figures 4 to 8 The flowchart shown illustrates the sequence of operations.

[0174] Figure 9 This is a diagram illustrating a method performed by an electronic device to predict a viewer's future viewing position according to an embodiment of the present disclosure.

[0175] Reference Figure 9 The above references can be omitted. Figures 1 to 8 Any redundant description provided by any of them.

[0176] For example, Figure 9 Operation 910 can correspond to Figure 3 Operation 310, Figure 9 Operation 912 can correspond to Figure 3 Operation 312, Figure 9 Operation 916 can correspond to Figure 3 Operation 314, Figure 9 Operation 918 can correspond to Figure 3 Operation 316, Figure 9 Operation 920 can correspond to Figure 3 Operation 318, and Figure 9 Operation 922 can correspond to Figure 3 Operation 230. Therefore, in the description Figure 9 When operating on steps 910, 912, 916, 918, 920, and 922, the above references can be omitted. Figure 3 The provided description is redundant. Additionally, see the following reference: Figure 3 The description given for the left eye can be applied to the right eye; therefore, the description for the right eye can be omitted.

[0177] In embodiments of this disclosure, when a positional change greater than a threshold occurs, the electronic device can scale the future velocity of the target part. For example, when a positional change greater than the threshold occurs at a reference time point, the electronic device can scale the future velocity of the target part based on cumulative positional information of the target part (or the viewer's viewing position information). For example, based on determining that the viewer's viewing position (e.g., the position of the target part or the center position between the eyes) corresponding to the reference time point is outside the reference area, the electronic device can determine the scaling of the future velocity. For example, the reference area can be determined based on the viewer's cumulative viewing position information (e.g., positional information of the target part or the center position information between the eyes) and the distance between the eyes. For example, the reference area can be determined according to Equation 15 below.

[0178] A t =A t-1 (K-1) + (center position between the eyes) / K

[0179] d=A t ±C (Distance between the eyes)

[0180] …Equation 15

[0181] In equation 15, A t-1It can represent the center position of the reference region corresponding to time point t-1, K can represent a predetermined or preset parameter value, and the center position between the two eyes can represent the center position between the left and right eye positions corresponding to time point t, and A t It can represent the center position of the reference region corresponding to time point t. In Equation 15, C can represent a predetermined or preset parameter value, the distance between the two eyes can represent preset or pre-stored IPD information or the difference between the positions of the left and right eyes corresponding to time point t, and d can represent the distance from the center of the reference region to the boundary position of the reference region.

[0182] In embodiments of this disclosure, the electronic device can obtain (or determine or calculate) a scaled value of the future velocity of a target location. For example, referring to... Figure 9 In operation 914, the electronic device can obtain a scaled value of the future velocity of the left eye. In embodiments of this disclosure, when the direction of the future velocity of the target area (e.g., the left eye) is toward the outside of the reference area (or outward), the electronic device can determine a value less than or equal to 1 as the scaled value of the future velocity of the target area. For example, the electronic device can determine a scaled value of the future velocity less than or equal to 1 by using Equation 16 below.

[0183] Scaling value = (A t ±C (Distance between the eyes) / Center position between the eyes

[0184] …Equation 16

[0185] In Equation 16, the position of the center between the two eyes can be represented by the center position between the left and right eye positions corresponding to time point t, A t It can represent the center position of the reference area corresponding to time point t, C can represent a predetermined or preset parameter value, and the distance between the two eyes can represent preset or pre-stored IPD information or the difference between the left and right eye positions corresponding to time point t.

[0186] In embodiments of this disclosure, when the direction of the future velocity of a target area (e.g., the left eye) is toward the center of a reference area, the electronic device can determine a value greater than or equal to 1 as a scaling value of the future velocity. For example, the electronic device can determine a scaling value of the future velocity greater than or equal to 1 by using Equation 17 below.

[0187] Scaling value = Center position between eyes / (A) t ±C (Distance between the eyes)

[0188] …Equation 17

[0189] In Equation 17, the position of the center between the two eyes can be represented by the center position between the left and right eye positions corresponding to time point t, A t It can represent the center position of the reference area corresponding to time point t, C can represent a predetermined or preset parameter value, and the distance between the two eyes can represent preset or pre-stored IPD information or the difference between the left and right eye positions corresponding to time point t.

[0190] In embodiments of this disclosure, the electronic device can predict the future acceleration of a target part based on the viewer's movement state. The viewer's movement state, used as the basis for predicting the future acceleration of the target part, can be associated with a scaling value of the target part's future velocity. For example, a scaling value of a future velocity less than or equal to (or less than) 1 may correspond to being stationary, and a scaling value of a future velocity greater than or equal to (or greater than) 1 may correspond to movement. For instance, when the direction of the target part's future velocity is towards the outside (or outward) of a reference area, the viewer's movement state can be determined to be stationary, and when the direction of the target part's future velocity is towards the center of the reference area, the viewer's movement state can be determined to be moving.

[0191] In operation 916, the electronic device can predict the future acceleration of the left eye based on the viewer's movement state. For example, when the scaling value of the future velocity obtained in operation 914 is less than or equal to (or less than) 1, the electronic device can determine that the viewer's movement state is stationary and predict (or determine or obtain) the future acceleration of the left eye. For example, when the scaling value of the future velocity obtained in operation 914 is greater than or equal to (or greater than) 1, the electronic device can determine that the viewer's movement state is moving and predict (or determine) the future acceleration of the left eye.

[0192] In Operation 918, the electronic device can predict the future position of the left eye based on a scaling value of the future velocity, the future velocity of the left eye, and the future acceleration of the left eye. For example, the electronic device can predict the future position of the left eye using Equation 18 below.

[0193] …Equation 18

[0194] In equation 18, This can represent the location information of the target part corresponding to time point t (e.g., reference time point) after noise has been filtered out, and n can represent the difference between the target time point (e.g., time point t+n) and the reference time point (e.g., time point t). A scaling function that can represent the future velocity of the target part. It can represent the future velocity (e.g., predicted velocity) of a target location corresponding to a future time point (e.g., time point t+1), and It can represent the future acceleration (e.g., predicted acceleration) of the target part corresponding to time point t+n, with time point t as a reference time point.

[0195] although Figure 9 An example is shown of an electronic device obtaining the future position of the center between the eyes by performing binocular correction, but this disclosure is not limited thereto. For example, as Figure 3 As shown, an electronic device can obtain the future positions of the left and right eyes by performing binocular correction. For example, an electronic device can obtain the future position of the center between the two eyes by performing binocular correction, and obtain (or determine or calculate) the future positions of the left and / or right eyes by using the future position of the center between the two eyes and IPD information.

[0196] Figure 10 This is a diagram illustrating a method performed by an electronic device to predict a viewer's future viewing position according to an embodiment of the present disclosure.

[0197] Figure 10 An example can be shown of an electronic device according to an embodiment of the present disclosure that predicts a viewer’s future viewing position by using a prediction result corresponding to time point t+n and a prediction result corresponding to time point t+n+1.

[0198] Reference Figure 10 The above references can be omitted. Figures 1 to 9 Any redundant description provided by any of them. For example, Figure 10 Operation 1010 can correspond to Figure 9 Operation 910, Figure 10 Operation 1012 can correspond to Figure 9 Operation 912, Figure 10 Operation 1014 can correspond to Figure 9 Operation 914, Figure 10 Operation 1016 can correspond to Figure 9 Operation 916, Figure 10 Operation 1018 can correspond to Figure 9 Operations 918 and 920, and Figure 10 Operation 1020 can correspond to Figure 9 Operation 922. Therefore, in the description Figure 10 When performing operations 1010 to 1022, the above references can be omitted. Figure 3 or Figure 9 Redundant descriptions provided.

[0199] In embodiments of this disclosure, the electronic device can perform monocular correction on the future position of the target region, which depends on the correction. (See reference...) Figure 10In operation 1022, the electronic device can perform auxiliary monocular correction on the future position of the binocular correction of the left eye. For example, the electronic device can perform auxiliary monocular correction on the future position of the binocular correction of the left eye by using the same or different weights as the primary monocular correction in operation 1018.

[0200] In embodiments of this disclosure, to improve the accuracy of predicting a viewer's future viewing position relative to a target time point (or at the target time point), the electronic device can use a prediction result 1030 corresponding to the target time point and a prediction result 1040 corresponding to a time point after the target time point (e.g., the next time point). For example, the electronic device can predict the viewer's future viewing position (e.g., the future position of both eyes) relative to the target time point based on the future position of the target part corresponding to the target time point and the future position of the target part corresponding to a time point after the target time point. For example, the electronic device can also sequentially, in parallel, or independently perform operations that have already been performed for the target time point to obtain the prediction result 1030 corresponding to the target time point to obtain the prediction result 1040 corresponding to the time point after the target time point.

[0201] The prediction result 1030 corresponding to the target time point may include the future location of the target part corresponding to the target time point. The prediction result 1040 corresponding to a time point after the target time point may include the future location of the target part corresponding to a time point after the target time point. For example... Figure 10 As shown, the prediction result 1030 corresponding to the target time point (time point t+n) may include the future position of the left eye and the future position of the right eye corresponding to the target time point. Similarly, the prediction result 1040 corresponding to the time point t+n+1 after the target time point may include the future position of the left eye and the future position of the right eye corresponding to the time point after the target time point.

[0202] refer to Figure 10The electronic device can predict the future positions of the left and right eyes using time point t+n as the first target time point, and predict the future positions of the left and right eyes using time point t+n+1 as the second target time point. The electronic device can calculate (or determine or obtain) the viewer's future viewing position relative to the first target time point based on the future positions of the left and right eyes corresponding to the first target time point and the future positions of the left and right eyes corresponding to the second target time point. For example, the electronic device can calculate the average (or weighted average) position of the future positions of the left and right eyes corresponding to the first target time point and the future positions of the left and right eyes corresponding to the second target time point as the future position of the center between the eyes relative to the first target time point (1050). The electronic device can determine the position of the eyes relative to the first target time point as the viewer's future viewing position based on the position of the center between the eyes relative to the first target time point and IPD information (e.g., the viewer's IPD information).

[0203] Figure 11 This is a diagram illustrating a method performed by an electronic device to predict a viewer's future viewing position according to an embodiment of the present disclosure.

[0204] Figure 11 This is a diagram illustrating an electronic device according to an embodiment of the present disclosure predicting a viewer's future viewing position using prediction results corresponding to time point t+n, prediction results corresponding to time point t+n+0.5, and prediction results corresponding to time point t+n+1.

[0205] Reference Figure 11 The above references can be omitted. Figures 1 to 10 Any redundant description provided by any of them. For example, Figure 11 1110 can correspond to Figure 10 1030, Figure 11 1120 can correspond to Figure 10 1040, and Figure 11 1140 can correspond to Figure 10 1050.

[0206] In embodiments of this disclosure, to improve the accuracy of predicting the viewer's future viewing position relative to a target time point (or at the target time point), the electronic device can use a prediction result 1110 corresponding to the target time point, a prediction result 1120 corresponding to a time point after the target time point (e.g., the next time point), and a prediction result 1130 corresponding to an intermediate time point between the target time point and the time point after the target time point. For example, the electronic device can predict the viewer's future viewing position (e.g., the future position of both eyes) relative to the target time point based on the future position of the target part corresponding to the target time point, the future position of the target part corresponding to a time point after the target time point, and the future position of the target part corresponding to an intermediate time point.

[0207] The prediction result 1110 corresponding to the target time point may include the future location of the target part corresponding to the target time point. The prediction result 1120 corresponding to a time point after the target time point may include the future location of the target part corresponding to the time point after the target time point. The prediction result 1130 corresponding to an intermediate time point may include the future location of the target part corresponding to the intermediate time point. For example... Figure 11 As shown, the prediction result 1110 corresponding to the target time point (time point t+n) can include the future position of the left eye and the future position of the right eye corresponding to the target time point. Similarly, the prediction result 1120 corresponding to the time point t+n+1 after the target time point can include the future position of the left eye and the future position of the right eye corresponding to the time point after the target time point. Similarly, the prediction result 1130 corresponding to the intermediate time point t+n+0.5 can include the future position of the left eye and the future position of the right eye corresponding to the intermediate time point.

[0208] In embodiments of this disclosure, the electronic device can obtain a prediction result 1130 corresponding to an intermediate time point by sequentially, in parallel, or independently performing at least some of the operations already performed for a target time point. In embodiments of this disclosure, the electronic device can obtain a prediction result 1130 corresponding to an intermediate time point by performing operations at the intermediate time point that are partially different from those already performed for the target time point. For example, to obtain a prediction result 1130 corresponding to an intermediate time point, the electronic device can obtain (or calculate) the future acceleration of the target site (e.g., the left and right eyes) corresponding to the intermediate time point using Equation 19 below.

[0209]

[0210] …Equation 19

[0211] In Equation 19, m can represent the average acceleration, r can represent a parameter whose value is determined based on the viewer's motion state, and γ can represent the nonlinear predicted acceleration of the target location corresponding to time point t+n+0.5 (e.g., an intermediate time point), with time point t as a reference time point. γ can represent the degree of filtering of acceleration noise. For example, γ can be determined or set experimentally or empirically. This can represent the future acceleration of the target location predicted using time point t-1 as a reference time point, corresponding to time point t-1+n+0.5. For example, It can represent the future acceleration of the target part from which noise in the acceleration is filtered out. It can represent the future acceleration of the target part corresponding to time point t+n+0.5, with time point t as the reference time point. This acceleration is obtained by filtering out noise from the nonlinear predicted acceleration of the target part corresponding to time point t+n+0.5, with time point t as the reference time point. It can represent an interpolation function.

[0212] refer to Figure 11 The electronic device can predict the future positions of the left and right eyes using time point t+n as the first target time point, time point t+n+1 as the second target time point, and time point t+n+0.5 as the third target time point. Based on the future positions of the left and right eyes corresponding to the first, second, and third target time points, the electronic device can calculate (or determine or obtain) the viewer's future viewing position relative to the first target time point. For example, the electronic device can calculate the average (or weighted average) position of the future positions of the left and right eyes corresponding to the first, second, and third target time points, as the future position of the center between the eyes relative to the first target time point (1140). Electronic devices can determine the viewer's future viewing position based on the position of the eyes relative to the first target time point and IPD information. The position of the eyes relative to the first target time point is obtained based on the position of the center between the eyes relative to the first target time point.

[0213] Figure 12 This is a diagram illustrating a method performed by an electronic device to predict a viewer's future viewing position according to an embodiment of the present disclosure.

[0214] Reference Figure 12 The above references can be omitted. Figures 1 to 11 Any redundant description provided by any of them.

[0215] Figure 12 An example is shown that, according to an embodiment of the present disclosure, the future position of the center between the eyes is predicted by using the center between the eyes as the target location, and the future viewing position of the viewer is predicted based on the predicted future position of the center between the eyes. Figure 12 Operations 1210 to 1220 can correspond to Figure 10 Operations 1010 to 1022. For example, electronic devices in the above... Figure 3 , 9 Or, as described in 10, at least some of the operations performed on the left eye can be performed by an electronic device. Figure 12 Operations 1210 to 1220 are performed targeting the center between the eyes. For example, as... Figure 12 As shown, unlike the left or right eye, electronic devices may not perform center-dependent correction (e.g., binocular correction) between the two eyes, but this disclosure is not limited thereto.

[0216] In embodiments of this disclosure, to improve the accuracy of predicting the viewer's future viewing position relative to a target time point (or at the target time point), the electronic device can use a prediction result 1230 corresponding to the target time point, a prediction result 1240 corresponding to a time point after the target time point (e.g., the next time point), and a prediction result 1250 corresponding to an intermediate time point between the target time point and the time point after the target time point. For example, the electronic device can predict the viewer's future viewing position (e.g., the future position of both eyes) relative to the target time point based on the future position of the target part corresponding to the target time point, the future position of the target part corresponding to a time point after the target time point, and the future position of the target part corresponding to an intermediate time point.

[0217] The prediction result 1230 corresponding to the target time point may include the future location of the target part corresponding to the target time point. The prediction result 1240 corresponding to a time point after the target time point may include the future location of the target part corresponding to that time point. The prediction result 1250 corresponding to an intermediate time point may include the future location of the target part corresponding to that intermediate time point. For example... Figure 12As shown, the prediction result 1230 corresponding to the target time point may include the future position of the left eye, the future position of the right eye, and the future position of the center between the two eyes corresponding to the target time point. The prediction result 1240 corresponding to a time point after the target time point may include the future position of the left eye, the future position of the right eye, and the future position of the center between the two eyes corresponding to the time point after the target time point. The prediction result 1250 corresponding to an intermediate time point may include the future position of the left eye, the future position of the right eye, and the future position of the center between the two eyes corresponding to the intermediate time point.

[0218] refer to Figure 12 The electronic device can predict the future position of each of the left eye, right eye, and the center between both eyes by using time point t+n as the first target time point, using time point t+n+1 as the second target time point, and using time point t+n+0.5 as the third target time point. The electronic device can calculate (or determine or obtain) the viewer's future viewing position relative to the first target time point based on the future positions of the left eye, right eye, and the center between both eyes corresponding to the first target time point, the second target time point, the third target time point, and the fourth target time point. For example, the electronic device can calculate the average (or weighted average) position of the future position of the left eye corresponding to the first target time point, the future position of the right eye corresponding to the first target time point, the future position of the center between the eyes corresponding to the first target time point, the future position of the left eye corresponding to the second target time point, the future position of the right eye corresponding to the second target time point, the future position of the center between the eyes corresponding to the second target time point, the future position of the left eye corresponding to the third target time point, the future position of the right eye corresponding to the third target time point, and the future position of the center between the eyes corresponding to the third target time point, as the future position of the center between the eyes relative to the first target time point (1260). The electronic device can determine the position of the eyes relative to the first target time point and the IPD information as the viewer's future viewing position, whereby the position of the eyes relative to the first target time point is obtained based on the position of the center between the eyes relative to the first target time point.

[0219] Figure 13This is a diagram illustrating an electronic device according to an embodiment of the present disclosure that further filters noise from the viewer's future viewing position information.

[0220] Reference Figure 13 The above references can be omitted. Figures 1 to 12 Any redundant description provided by any of them.

[0221] In embodiments of this disclosure, the electronic device may additionally filter noise from the viewer's future viewing position information predicted relative to a target time point before rendering (250) the output image. For example, to remove noise that may occur when predicting the distant future (e.g., more than n frames later, where n is a natural number), the electronic device may additionally filter noise before rendering the output image. For example, the electronic device may additionally filter noise from the viewer's future viewing position information predicted relative to the target time point. Figures 2 to 12 Noise can be filtered out from the final prediction information (e.g., the viewer's future viewing position) obtained from one or more embodiments described herein. For example, the electronic device may additionally filter out noise from the future position of the target part corresponding to (or relative to) the target time point, the future position of the center between the eyes corresponding to (or relative to) the target time point, or the future position of the eyes corresponding to (or relative to) the target time point.

[0222] For example, electronic devices can further filter out noise from the viewer's predicted future viewing location information relative to the target time point by using Equation 20 below.

[0223] s t =αx t +(1-α)(s t-1 +b t-1 )

[0224] b t =β(s) t -s t-1 )+(1-β)(b t-1 +k t-1 )

[0225] kt=γ(b t -b t-1 )+(1-γ)k t-1

[0226] …Equation 20

[0227] In equation 20, x tThis can represent the viewer's future viewing position predicted relative to the target time point (e.g., the future position of both eyes or the future position of the center between the eyes). The values ​​of s0, b0, and k0 can be predetermined or preset. The values ​​of α, β, and γ can be predetermined or preset.

[0228] refer to Figure 13 In operation 1310, the electronic device can predict the viewer's future viewing position relative to (or at) a target time point, and in operation 1320, the electronic device can additionally filter noise from the prediction result. For example, the prediction result from which noise is filtered in operation 1320 may include the prediction result as described above relative to... Figures 2 to 12 At least one of the embodiments described herein predicts the viewer’s future viewing position, the future position of the left eye corresponding to (or relative to) the target time point, the future position of the right eye corresponding to (or relative to) the target time point, the future position of the center of both eyes corresponding to (or relative to) the target time point, or the future position of the target part corresponding to (or relative to) the target time point.

[0229] Figure 14 This is a diagram illustrating an operation method of an electronic device according to an embodiment of the present disclosure.

[0230] Reference Figure 14 The above references can be omitted. Figures 1 to 13 Any redundant description provided by any of them.

[0231] Figure 14 An example of an operation method 1400 of an electronic device may be shown. In embodiments of this disclosure, the electronic device performing method 1400 may include a display device. (See reference...) Figure 14 The method 1400 according to embodiments of the present disclosure may include operations 1410 to 1470. In embodiments of the present disclosure, operations 1410 to 1470 of method 1400 may be executed by at least one processor included in an electronic device. Method 1400 is not limited to... Figure 14 The method shown, and in one or more embodiments of this disclosure, method 1400 may further include Figure 14 Operations not shown in the diagram, or some operations can be omitted.

[0232] In operation 1410, the electronic device can obtain position information of a target part corresponding to a past time point and position information of a target part corresponding to a reference time point from an image including the viewer's facial region and input via a camera. In embodiments of this disclosure, the electronic device can obtain the original position information of the target part corresponding to the reference time point from an image including the viewer's facial region, and obtain the position information of the target part corresponding to the reference time point based on the position information of the target part corresponding to the past time point and the original position information of the target part.

[0233] In operation 1420, the electronic device can obtain position change information of the target part corresponding to the reference time point based on the position information of the target part corresponding to the past time point and the position information of the target part corresponding to the reference time point. In embodiments of this disclosure, the electronic device can obtain raw information about the position change of the target part corresponding to the reference time point based on the position information of the target part corresponding to the past time point and the position information of the target part corresponding to the reference time point, and obtain the position change information of the target part corresponding to the reference time point based on the filtering intensity determined according to the degree of movement of the target part, the position change information of the target part corresponding to the past time point, and the raw information about the position change of the target part.

[0234] In operation 1430, the electronic device can predict the future speed of the target part based on the position change information of the target part corresponding to the reference time point and the position change information of the target part corresponding to past time points.

[0235] In operation 1440, the electronic device can obtain the velocity change information of the target part corresponding to the reference time point based on the position change information of the target part corresponding to the reference time point and the position change information of the target part corresponding to the past time point.

[0236] In operation 1450, the electronic device can predict the future acceleration of the target part based on velocity change information of the target part corresponding to a reference time point and velocity change information of the target part corresponding to a past time point. In embodiments of this disclosure, the electronic device can obtain nonlinear predicted acceleration information of the target part corresponding to a target time point based on velocity change information of the target part corresponding to a reference time point, velocity change information of the target part corresponding to a past time point, and parameter values ​​determined according to the viewer's movement state, and predict the future acceleration of the target part based on the nonlinear predicted acceleration information and the acceleration of the target part predicted for a first time point prior to the target time point.

[0237] In operation 1460, the electronic device can predict the future position of both eyes corresponding to a target time point based on future velocity and future acceleration. In embodiments of this disclosure, the electronic device can predict a first future position of a target part corresponding to a target time point based on future velocity and future acceleration, and obtain a second future position of the target part corresponding to the target time point based on the first future position of the target part corresponding to the target time point and the future position of the target part corresponding to a first time point prior to the target time point. The electronic device can predict the future position of both eyes corresponding to the target time point based on the second future position of the target part. In embodiments of this disclosure, the electronic device can predict a first future position of the center between the eyes corresponding to a target time point based on future velocity and future acceleration, and predict the future position of both eyes corresponding to the target time point based on the first future position of the center between the eyes corresponding to the target time point and IPD information.

[0238] In embodiments of this disclosure, the electronic device can obtain a scaling value of the future velocity based on the direction of the future velocity, and predict the future position of the eyes corresponding to the target time point based on the future velocity, future acceleration, and the scaling value of the future velocity.

[0239] In operation 1470, the electronic device can output an image based on the future position of both eyes corresponding to the target time point.

[0240] Figure 15 This is a diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0241] refer to Figure 15 Electronic device 1500 may include outdoor monitors, game monitors, televisions, electronic photo frames, laptops, desktop computers, wearable devices, etc., as display devices. Figure 15 The electronic device 1500 shown is an electronic device capable of generating an output image and providing it to a display device, or providing a prediction result to a display device, and may include a set-top box, a console device, a video player device, etc. In embodiments of this disclosure, the electronic device 1500 may include, but is not limited to, at least one processor 1510 and a memory 1520.

[0242] Processor 1510 may be electrically connected to components included in electronic device 1500 to perform computations or data processing for control and / or communication with components included in electronic device 1500. In embodiments of this disclosure, processor 1510 may load requests, commands, or data received from at least one of other components into memory, process the requests, commands, or data, and store the processing result data in memory. According to one or more embodiments of this disclosure, processor 1510 may include at least one of a general-purpose processor (such as a central processing unit (CPU), application processor (AP), or digital signal processor (DSP)), a dedicated graphics processor (such as a graphics processing unit (GPU) or vision processing unit (VPU)), or a dedicated artificial intelligence processor (such as a neural processing unit (NPU)).

[0243] Processor 1510 can process input data or control other components to process input data based on data, operating rules, algorithms, methods, or models stored in memory 1520. Processor 1510 can execute operations based on predefined operating rules, algorithms, methods, or models stored in memory 1520 using input data.

[0244] Memory 1520 may be electrically connected to processor 1510 and may store one or more modules, algorithms, operating rules, models, programs, instructions, or data associated with the operation of components included in electronic device 1500. For example, memory 1520 may store one or more modules, algorithms, operating rules, models, programs, instructions, or data for processor 1510 to perform processing and control. Memory 1520 may include, but is not limited to, at least one of flash memory, hard disk storage media, multimedia card microstorage media, card-type memory (e.g., SD or XD memory), random access memory (RAM), static RAM (SRAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), programmable ROM (PROM), magnetic storage, magnetic disk, or optical disk.

[0245] In embodiments of this disclosure, memory 1520 may store data and / or information identified, obtained, generated, or determined by electronic device 1500. For example, memory 1520 may store data and / or information identified, obtained, generated, or determined by electronic device 1500 in compressed form.

[0246] In embodiments of this disclosure, electronic device 1500 may include modules configured to perform (or be used to perform) at least one operation. Some modules performing at least one operation of electronic device 1500 may include multiple sub-modules or may constitute a single module.

[0247] Some modules performing at least one operation of electronic device 1500 can be implemented as hardware modules, software modules, and / or combinations thereof. Software modules included in electronic device 1500 may be included in memory 1520. In embodiments of this disclosure, modules included in memory 1520 may be executed by processor 1510 to perform operations. For example, modules included in memory 1520 (i.e., software modules) may be executed under the control of processor 1510 or according to commands from processor 1510, and may include programs, models, or algorithms configured to perform operations that derive output data from input data.

[0248] Electronic device 1500 may include more than Figure 15 The components shown are further components. In embodiments of this disclosure, the electronic device 1500 may also include a communication interface (or communication module) for communicating with external devices. In embodiments of this disclosure, the electronic device 1500 may also include input devices, output devices, and / or input / output interfaces. For example, the input device of the electronic device 1500 may include a camera as an image input module. For example, the output device of the electronic device 1500 may include a display as an image output module. For example, the output device of the electronic device 1500 may include a cylindrical lens.

[0249] The above references may have been omitted in this disclosure. Figures 1 to 15 The provided redundant description, and the above reference Figures 1 to 15 One or more embodiments described can be applied / implemented in combination with each other. In this disclosure, operations described as being performed by a module can be run / executed by a device that includes or stores the module, or can be run / executed under the control of at least one processor of a device that includes the module. In this disclosure, operations described as being performed by a device can be run / executed by a module included or stored in the device, or can be executed under the control of at least one processor of the device by using a module included or stored in the device.

[0250] In embodiments of this disclosure, the operation method of the electronic device may include: obtaining position information of a target part corresponding to a past time point and position information of a target part corresponding to a reference time point from an image including a viewer's facial area and input via a camera. In embodiments of this disclosure, the operation method of the electronic device may include: obtaining position change information of a target part corresponding to a reference time point based on the position information of the target part corresponding to a past time point and the position information of the target part corresponding to a reference time point. In embodiments of this disclosure, the operation method of the electronic device may include: predicting the future velocity of a target part based on the position change information of the target part corresponding to a reference time point and the position change information of the target part corresponding to a past time point. In embodiments of this disclosure, the operation method of the electronic device may include: obtaining velocity change information of a target part corresponding to a reference time point based on the position change information of the target part corresponding to a reference time point and the position change information of the target part corresponding to a past time point. In embodiments of this disclosure, the operation method of the electronic device may include: predicting the future acceleration of a target part based on the velocity change information of the target part corresponding to a reference time point and the velocity change information of the target part corresponding to a past time point. In embodiments of this disclosure, the operation method of the electronic device may include: predicting the future positions of both eyes corresponding to a target time point based on future velocity and future acceleration. In embodiments of this disclosure, the operation method of the electronic device may include: outputting an image based on the future positions of the eyes corresponding to the target time point.

[0251] In embodiments of this disclosure, predicting the future position of the eye corresponding to a target time point based on future velocity and future acceleration may include: predicting a first future position of a target body part corresponding to the target time point based on future velocity and future acceleration. In embodiments of this disclosure, predicting the future position of the eye corresponding to a target time point based on future velocity and future acceleration may include: obtaining a second future position of the target body part corresponding to the target time point based on the first future position of the target body part corresponding to the target time point and the future position of the target body part corresponding to a first time point prior to the target time point. In embodiments of this disclosure, predicting the future position of the eye corresponding to a target time point based on future velocity and future acceleration may include predicting the future position of the eye corresponding to the target time point based on the second future position of the target body part.

[0252] In embodiments of this disclosure, predicting the future position of the eye corresponding to a target time point based on future velocity and future acceleration may include: predicting a first future position of the center between the two eyes corresponding to the target time point based on future velocity and future acceleration. In embodiments of this disclosure, predicting the future position of the eye corresponding to a target time point based on future velocity and future acceleration may include predicting the future position of the eye corresponding to the target time point based on IPD information and the first future position of the center between the two eyes corresponding to the target time point.

[0253] In embodiments of this disclosure, obtaining location information of a target part corresponding to a past time point and location information of a target part corresponding to a reference time point from an image that may include a viewer's facial region and is input through a camera may include obtaining the original location information of the target part corresponding to the reference time point from the image including the viewer's facial region. In embodiments of this disclosure, obtaining location information of a target part corresponding to a past time point and location information of a target part corresponding to a reference time point from an image that may include a viewer's facial region and is input through a camera may include: obtaining the location information of the target part corresponding to the reference time point based on the location information of the target part corresponding to the past time point and the original location information of the target part.

[0254] In embodiments of this disclosure, obtaining position change information of a target part corresponding to a reference time point based on the position information of the target part corresponding to a past time point and the position information of the target part corresponding to a reference time point may include: obtaining original information about the position change of the target part corresponding to the reference time point based on the position information of the target part corresponding to the past time point and the position information of the target part corresponding to the reference time point. In embodiments of this disclosure, obtaining position change information of a target part corresponding to a reference time point based on the position information of the target part corresponding to a past time point and the position information of the target part corresponding to a reference time point may include: obtaining position change information of the target part corresponding to the reference time point based on a filtering intensity determined according to the degree of motion of the target part, the position change information of the target part corresponding to the past time point, and the original information about the position change of the target part.

[0255] In embodiments of this disclosure, predicting the future acceleration of a target part based on velocity change information of the target part corresponding to a reference time point and velocity change information of the target part corresponding to a past time point may include: obtaining nonlinear predicted acceleration information of the target part corresponding to a target time point based on the velocity change information of the target part corresponding to the reference time point, the velocity change information of the target part corresponding to a past time point, and parameter values ​​determined according to the viewer's movement state. In embodiments of this disclosure, predicting the future acceleration of a target part based on velocity change information of the target part corresponding to a reference time point and velocity change information of the target part corresponding to a past time point may include: predicting the future acceleration of the target part based on nonlinear predicted acceleration information and the acceleration of the target part predicted relative to a first time point before the target time point. In embodiments of this disclosure, the future acceleration of the target part may correspond to the target time point.

[0256] In embodiments of this disclosure, the method may include obtaining a scaling value of the future velocity based on the direction of the future velocity. In embodiments of this disclosure, predicting the future position of the eye corresponding to a target time point based on future velocity and future acceleration may include: predicting the future position of the eye corresponding to a target time point based on the future velocity, future acceleration, and the scaling value of the future velocity.

[0257] In embodiments of this disclosure, the method may include obtaining the future position of the eye corresponding to a second time point after the target time point. In embodiments of this disclosure, outputting an image based on the future position of the eye corresponding to the target time point may include: predicting the future position of the eye relative to the target time point based on the future position of the eye corresponding to the target time point and the future position of the eye corresponding to the second time point. In embodiments of this disclosure, outputting an image based on the future position of the eye corresponding to the target time point may include outputting an image based on the future position of the eye relative to the target time point.

[0258] In embodiments of this disclosure, the method may include obtaining the future position of the eye corresponding to a third time point between a target time point and a second time point. In embodiments of this disclosure, predicting the future position of the eye relative to a target time point based on the future position of the eye corresponding to the target time point and the future position of the eye corresponding to the second time point may include: predicting the future position of the eye relative to the target time point based on the future position of the eye corresponding to the target time point, the future position of the eye corresponding to the second time point, and the future position of the eye corresponding to the third time point.

[0259] In embodiments of this disclosure, the method may include: predicting the future position of the center between the eyes corresponding to a target time point based on the future acceleration and future velocity of the center between the eyes. In embodiments of this disclosure, the target region may include at least one of the left or right eyes. In embodiments of this disclosure, outputting an image based on the future position of the eye corresponding to the target time point may include: predicting the future position of the eye relative to the target time point based on the future position of the eye corresponding to the target time point and the future position of the center between the eyes corresponding to the target time point. In embodiments of this disclosure, outputting an image based on the future position of the eye corresponding to the target time point may include outputting an image based on the future position of the eye relative to the target time point.

[0260] In embodiments of this disclosure, predicting the future position of the eye relative to a target time point based on the future position of the eye corresponding to the target time point and the future position of the center between the two eyes corresponding to the target time point may include: predicting the future position of the center between the two eyes relative to the target time point based on the future position of the eye corresponding to the target time point and the future position of the center between the two eyes corresponding to the target time point. In embodiments of this disclosure, predicting the future position of the eye relative to a target time point based on the future position of the eye corresponding to the target time point and the future position of the center between the two eyes corresponding to the target time point may include filtering out noise from the future position of the center between the two eyes relative to the target time point. In embodiments of this disclosure, predicting the future position of the eye relative to a target time point based on the future position of the eye corresponding to the target time point and the future position of the center between the two eyes corresponding to the target time point may include: predicting the future position of the eye relative to the target time point based on the noise-filtered future position of the center between the two eyes.

[0261] In embodiments of this disclosure, one or more computer-readable storage media may be provided storing one or more computer programs including computer-executable instructions that, when executed individually or jointly by one or more processors of an electronic device, cause the electronic device to perform operations. In embodiments of this disclosure, operations may include the electronic device obtaining position information of a target part corresponding to a past time point and position information of a target part corresponding to a reference time point from an image including a viewer's facial region input via a camera. In embodiments of this disclosure, operations may include the electronic device obtaining position change information of a target part corresponding to a reference time point based on the position information of the target part corresponding to a past time point and the position information of the target part corresponding to a reference time point. In embodiments of this disclosure, operations may include the electronic device predicting the future velocity of a target part based on the position change information of the target part corresponding to a reference time point and the position change information of the target part corresponding to a past time point. In embodiments of this disclosure, operations may include the electronic device obtaining velocity change information of a target part corresponding to a reference time point based on the position change information of the target part corresponding to a reference time point and the position change information of the target part corresponding to a past time point. In embodiments of this disclosure, operation may include having an electronic device predict future acceleration of a target region based on velocity change information of the target region corresponding to a reference time point and velocity change information of the target region corresponding to a past time point. In embodiments of this disclosure, operation may include having an electronic device predict future eye positions corresponding to a target time point based on future velocity and future acceleration. In embodiments of this disclosure, operation may include having an electronic device output an image based on the future eye positions corresponding to a target time point. In embodiments of this disclosure, the electronic device may include a memory storing one or more computer programs and one or more processors communicatively coupled to the memory. In embodiments of this disclosure, the one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause the electronic device to obtain position information of a target region corresponding to a past time point and position information of a target region corresponding to a reference time point from an image that may include a viewer's facial region and is input via a camera. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to: obtain position change information of a target part corresponding to a reference time point, based on position information of the target part corresponding to a past time point and position information of the target part corresponding to a reference time point.In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to predict the future velocity of a target part based on position change information of the target part corresponding to a reference time point and position change information of the target part corresponding to a past time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause the electronic device to: obtain velocity change information of the target part corresponding to a reference time point based on position change information of the target part corresponding to a reference time point and position change information of the target part corresponding to a past time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause the electronic device to predict the future acceleration of the target part based on velocity change information of the target part corresponding to a reference time point and velocity change information of the target part corresponding to a past time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause the electronic device to: predict the future position of both eyes corresponding to a target time point based on future velocity and future acceleration. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause the electronic device to output an image based on the future position of the eyes corresponding to a target time point.

[0262] In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to predict a first future position of a target body part corresponding to a target time point based on future velocity and future acceleration. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause the electronic device to: obtain a second future position of a target body part corresponding to a target time point based on the first future position of the target body part corresponding to the target time point and the future position of the target body part corresponding to a first time point prior to the target time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause the electronic device to predict the future position of an eye corresponding to a target time point based on the second future position of the target body part.

[0263] In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to predict a first future position of the center between the eyes corresponding to a target time point based on future velocity and future acceleration. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to predict the future position of the eyes corresponding to a target time point based on IPD information and a first future position of the center between the eyes corresponding to a target time point.

[0264] In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to obtain, from an image including a region of a viewer's face, original location information of a target body part corresponding to a reference time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause the electronic device to: obtain location information of a target body part corresponding to a reference time point based on location information of the target body part corresponding to a past time point and the original location information of the target body part.

[0265] In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to obtain raw information about the position change of a target part corresponding to a reference time point, based on position information of a target part corresponding to a past time point and position information of a target part corresponding to a reference time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to obtain position change information of a target part corresponding to a reference time point, based on a filtering strength determined according to the degree of movement of the target part, position change information of the target part corresponding to a past time point, and raw information about the position change of the target part.

[0266] In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or collectively by one or more processors, cause an electronic device to obtain nonlinear predicted acceleration information of a target part corresponding to a target time point, based on velocity change information of the target part corresponding to a reference time point, velocity change information of the target part corresponding to a past time point, and parameter values ​​determined according to the viewer's movement state. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or collectively by one or more processors, cause an electronic device to predict the future acceleration of a target part based on the nonlinear predicted acceleration information and the acceleration of the target part predicted relative to a first time point prior to the target time point. In embodiments of this disclosure, the future acceleration of the target part may correspond to the target time point.

[0267] In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to obtain a scaling value of a future velocity based on the direction of the future velocity. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to predict the future position of an eye corresponding to a target time point based on future velocity, future acceleration, and the scaling value of the future velocity.

[0268] In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to obtain the future position of the eye corresponding to a second time point after a target time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause the electronic device to: predict the future position of the eye relative to a target time point based on the future position of the eye corresponding to the target time point and the future position of the eye corresponding to the second time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause the electronic device to output an image based on the future position of the eye relative to a target time point.

[0269] In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to obtain the future position of the eye corresponding to a third time point between a target time point and a second time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause the electronic device to: predict the future position of the eye relative to a target time point based on the future position of the eye corresponding to the target time point, the future position of the eye corresponding to the second time point, and the future position of the eye corresponding to the third time point.

[0270] In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to: predict the future position of the center between the eyes corresponding to a target time point based on the future acceleration and future velocity of the center between the eyes. In embodiments of this disclosure, the target region may include at least one of the left or right eyes. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to: predict the future position of the eye relative to a target time point based on the future position of the eye and the future position of the center between the eyes corresponding to the target time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or jointly by one or more processors, cause an electronic device to output an image based on the future position of the eye relative to a target time point.

[0271] In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or collectively by one or more processors, cause an electronic device to: predict the future position of the center between the eyes relative to a target time point based on the future position of the center between the eyes corresponding to a target time point and the future position of the center between the eyes corresponding to a target time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or collectively by one or more processors, cause an electronic device to filter out noise from the future position of the center between the eyes relative to a target time point. In embodiments of this disclosure, one or more computer programs may include computer-executable instructions that, when executed individually or collectively by one or more processors, cause an electronic device to: predict the future position of the eyes relative to a target time point based on the future position of the center between the eyes after noise has been filtered out.

[0272] Machine-readable storage media may be provided in the form of non-transitory storage media. Here, the term "non-transitory storage media" refers to a tangible device and does not include signals (e.g., electromagnetic waves), and the term "non-transitory storage media" does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is temporarily stored. For example, "non-transitory storage media" may include buffers for temporarily storing data.

[0273] According to embodiments of this disclosure, methods according to one or more embodiments disclosed herein can be included in a computer program product and then provided. The computer program product can be traded as a commodity between a seller and a buyer. The computer program product can be distributed in the form of a machine-readable storage medium (e.g., a compact ROM (CD-ROM)), or distributed online through an app store (e.g., download or upload) or directly between two viewer devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable application) can be temporarily stored in a machine-readable storage medium, such as the memory of a manufacturer's server, an app store's server, or a relay server.

[0274] It will be understood that the various embodiments of this disclosure described in the claims and specification can be implemented in hardware, software, or a combination of hardware and software.

[0275] Any such software may be stored in a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores one or more computer programs (software modules) that include computer-executable instructions, which, when executed by one or more processors of an electronic device, cause the electronic device to perform the methods of this disclosure.

[0276] Any such software may be stored in the form of volatile or non-volatile memory, such as a storage device like read-only memory (ROM), whether erasable or rewritable, or in the form of memory such as random access memory (RAM), memory chips, devices, or integrated circuits, or stored on an optically or magnetically readable medium such as, for example, a compact disc (CD), a digital versatile disc (DVD), a magnetic disk, or magnetic tape. It will be understood that storage devices and storage media are various embodiments of non-transitory machine-readable storage means suitable for storing one or more computer programs including instructions that, when executed, implement various embodiments of this disclosure. Therefore, various embodiments provide programs and non-transitory machine-readable storage means for storing such programs, which include code for implementing means or methods claimed as in any of the claims of this specification.

[0277] While this disclosure has been shown and described with reference to various embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of this disclosure as defined by the appended claims and their equivalents.

Claims

1. A method performed by an electronic device, the method comprising: The electronic device obtains the position information of the target part corresponding to a past time point and the position information of the target part corresponding to a reference time point from an image including the viewer's facial area and input through a camera; The electronic device obtains the position change information of the target part corresponding to the reference time point based on the position information of the target part corresponding to the past time point and the position information of the target part corresponding to the reference time point; The electronic device predicts the future speed of the target part based on the position change information of the target part corresponding to the reference time point and the position change information of the target part corresponding to the past time point; The electronic device obtains the velocity change information of the target part corresponding to the reference time point based on the position change information of the target part corresponding to the reference time point and the position change information of the target part corresponding to the past time point; The electronic device predicts the future acceleration of the target part based on the velocity change information of the target part corresponding to the reference time point and the velocity change information of the target part corresponding to the past time point; The electronic device predicts the future position of the eyes corresponding to the target time point based on the future velocity and the future acceleration; as well as The electronic device outputs an image based on the future position of the eye corresponding to the target time point.

2. The method according to claim 1, wherein, Predicting the future position of the eye corresponding to the target time point based on the future velocity and the future acceleration includes: Based on the future velocity and the future acceleration, predict the first future position of the target part corresponding to the target time point; Based on the first future position of the target location corresponding to the target time point and the future position of the target location corresponding to a first time point before the target time point, a second future position of the target location corresponding to the target time point is obtained; and Based on the second future position of the target location, the future position of the eye corresponding to the target time point is predicted.

3. The method according to claim 1 or 2, wherein, Predicting the future position of the eye corresponding to the target time point based on the future velocity and the future acceleration includes: Based on the future velocity and the future acceleration, predict the first future position of the center between the eyes corresponding to the target time point; and Based on interpupillary distance (IPD) information and the first future position of the center between the two eyes corresponding to the target time point, the future position of the eyes corresponding to the target time point is predicted.

4. The method according to any one of claims 1 to 3, wherein, Obtaining location information of the target part corresponding to the past time point and location information of the target part corresponding to the reference time point from the image input through the camera, including the facial region of the viewer, includes: Obtain the original location information of the target area corresponding to the reference time point from the image including the viewer's facial region; and Based on the location information of the target part corresponding to the past time point and the original location information of the target part, the location information of the target part corresponding to the reference time point is obtained.

5. The method according to any one of claims 1 to 4, wherein, Based on the location information of the target part corresponding to the past time point and the location information of the target part corresponding to the reference time point, obtaining the location change information of the target part corresponding to the reference time point includes: Based on the location information of the target part corresponding to the past time point and the location information of the target part corresponding to the reference time point, original information regarding the change in the location of the target part corresponding to the reference time point is obtained; and Based on the filtering intensity determined according to the degree of movement of the target part, the position change information of the target part corresponding to the past time point, and the original information about the position change of the target part, the position change information of the target part corresponding to the reference time point is obtained.

6. The method according to any one of claims 1 to 5, wherein, Predicting the future acceleration of the target part based on the velocity change information of the target part corresponding to the reference time point and the velocity change information of the target part corresponding to the past time point includes: Based on the velocity change information of the target part corresponding to the reference time point, the velocity change information of the target part corresponding to past time points, and parameter values ​​determined according to the viewer's movement state, nonlinear predicted acceleration information of the target part corresponding to the target time point is obtained; and The future acceleration of the target part is predicted based on the nonlinear predicted acceleration information and the acceleration of the target part predicted relative to a first time point before the target time point. Wherein, the future acceleration of the target part corresponds to the target time point.

7. The method according to any one of claims 1 to 6, further comprising: The scaling value of the future velocity is obtained based on the direction of the future velocity. The method of predicting the future position of the eye corresponding to the target time point based on the future velocity and the future acceleration includes: predicting the future position of the eye corresponding to the target time point based on the future velocity, the future acceleration, and a scaling value of the future velocity.

8. The method according to any one of claims 1 to 7, further comprising: Obtain the future position of the eye corresponding to a second time point after the target time point. The process of outputting the image based on the future position of the eye corresponding to the target time point includes: Based on the future position of the eye corresponding to the target time point and the future position of the eye corresponding to the second time point, predict the future position of the eye relative to the target time point; and The image is output based on the future position of the eye relative to the target time point.

9. The method according to claim 8, further comprising: Obtain the future position of the eye corresponding to a third time point between the target time point and the second time point. The method of predicting the future position of the eye relative to the target time point based on the future position of the eye corresponding to the target time point and the future position of the eye corresponding to the second time point includes: predicting the future position of the eye relative to the target time point based on the future position of the eye corresponding to the target time point, the future position of the eye corresponding to the second time point, and the future position of the eye corresponding to the third time point.

10. The method according to any one of claims 1 to 9, further comprising: Based on the future acceleration and future velocity of the center between the eyes, predict the future position of the center between the eyes corresponding to the target time point. The target area includes at least one of the left or right eyes, and The output of the image based on the future position of the eye corresponding to the target time point includes: Based on the future position of the eye corresponding to the target time point and the future position of the center between the two eyes corresponding to the target time point, predict the future position of the eye relative to the target time point, and The image is output based on the future position of the eye relative to the target time point.

11. The method according to claim 10, wherein, Predicting the future position of the eye relative to the target time point, based on the future position of the eye corresponding to the target time point and the future position of the center between the eyes corresponding to the target time point, includes: Based on the future position of the eye corresponding to the target time point and the future position of the center between the two eyes corresponding to the target time point, predict the future position of the center between the two eyes relative to the target time point; Filter out noise from the future position of the center between the two eyes relative to the target time point; and Based on the future position of the center between the two eyes after noise has been filtered out, the future position of the eyes relative to the target time point is predicted.

12. One or more computer-readable storage media storing one or more computer programs comprising computer-executable instructions, which, when executed individually or jointly by one or more processors of an electronic device, cause the electronic device to perform operations, said operations including: The electronic device obtains the position information of the target part corresponding to a past time point and the position information of the target part corresponding to a reference time point from an image including the viewer's facial area and input through a camera; The electronic device obtains the position change information of the target part corresponding to the reference time point based on the position information of the target part corresponding to the past time point and the position information of the target part corresponding to the reference time point; The electronic device predicts the future speed of the target part based on the position change information of the target part corresponding to the reference time point and the position change information of the target part corresponding to the past time point; The electronic device obtains the velocity change information of the target part corresponding to the reference time point based on the position change information of the target part corresponding to the reference time point and the position change information of the target part corresponding to the past time point; The electronic device predicts the future acceleration of the target part based on the velocity change information of the target part corresponding to the reference time point and the velocity change information of the target part corresponding to the past time point; The electronic device predicts the future position of the eyes corresponding to the target time point based on the future velocity and the future acceleration; as well as The electronic device outputs an image based on the future position of the eye corresponding to the target time point.

13. An electronic device, comprising: Memory (1520) that stores one or more computer programs. as well as One or more processors (1510) are communicatively coupled to the memory (1520). The one or more computer programs include computer-executable instructions that, when executed individually or jointly by the one or more processors (1510), cause the electronic device to: From images including the viewer's facial area and input via a camera, obtain positional information of the target body parts corresponding to past time points and positional information of the target body parts corresponding to reference time points. Based on the location information of the target part corresponding to the past time point and the location information of the target part corresponding to the reference time point, the location change information of the target part corresponding to the reference time point is obtained. The future velocity of the target part is predicted based on the position change information of the target part corresponding to the reference time point and the position change information of the target part corresponding to the past time point. Based on the position change information of the target part corresponding to the reference time point and the position change information of the target part corresponding to the past time point, the velocity change information of the target part corresponding to the reference time point is obtained. Based on the velocity change information of the target part corresponding to the reference time point and the velocity change information of the target part corresponding to past time points, the future acceleration of the target part is predicted. Based on the future velocity and the future acceleration, predict the future position of both eyes corresponding to the target time point, and An image is output based on the future position of the eye corresponding to the target time point.

14. The electronic device according to claim 13, wherein, The one or more computer programs further include computer-executable instructions that, when executed individually or jointly by the one or more processors (1510), cause the electronic device to: The first future position of the target location corresponding to the target time point is predicted based on the future velocity and the future acceleration. Based on the first future position of the target part corresponding to the target time point and the future position of the target part corresponding to a first time point before the target time point, a second future position of the target part corresponding to the target time point is obtained, and Based on the second future position of the target location, the future position of the eye corresponding to the target time point is predicted.

15. The electronic device according to claim 13 or 14, wherein, The one or more computer programs further include computer-executable instructions that, when executed individually or jointly by the one or more processors (1510), cause the electronic device to: Based on the future velocity and the future acceleration, a first future position of the center between the eyes corresponding to the target time point is predicted, and Based on interpupillary distance (IPD) information and a first future position of the center between the two eyes corresponding to the target time point, the future position of the eyes corresponding to the target time point is predicted.

Images