Electronic device and operating method thereof

Abstract

This operating method of an electronic device may comprise the steps of: obtaining position change information of a target portion corresponding to a reference viewpoint; predicting a future speed of the target portion; obtaining speed change information of the target portion corresponding to the reference viewpoint; predicting a future acceleration of the target portion; predicting future positions of both eyes corresponding to a target viewpoint; and outputting an image on the basis of the future positions of the both eyes corresponding to the target viewpoint.

Description

Electronic devices and their operating methods

[0001] The present disclosure relates to an electronic device for outputting an image based on a result of predicting a viewer's movement, and an operating method thereof.

[0002] A special type of lens, such as a lenticular lens, may be used to create 3D effects, optical effects, or other visual effects in the display area. By utilizing the characteristics of a lenticular lens, a display device can output multiple images so that different images are viewed depending on the viewer's viewing position. For example, the display device can output a first image to pixels viewable from a first viewing position and simultaneously output a second image to pixels viewable from a second viewing position. In this case, a viewer at the first viewing position can view the first image, and a viewer at the second viewing position can view the second image.

[0003] By utilizing the characteristics of a lenticular lens, a display device can provide a 3D image to a viewer as a glasses-free 3D (dimension) display device. Due to the characteristics of the lenticular lens, different images can be observed by the viewer's left and right eyes. For example, the display device can output a first image to pixels observable by the viewer's left eye, and simultaneously output a second image to pixels observable by the viewer's right eye. In this case, the viewer can perceive that he or she is observing a 3D image due to the difference between the first image observed by the viewer's left eye and the second image observed by the viewer's right eye.

[0004] The above information is provided solely as background information to aid in understanding the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above information constitutes prior art in connection with the present disclosure.

[0005] One aspect of the present disclosure is to provide a method, system, device or computer program stored on a computer-readable storage medium.

[0006] Additional aspects are presented in part in the following description, and in part are obvious from the description or can be learned by practice of the embodiments presented.

[0007] In one embodiment of the present disclosure, a method of operating an electronic device may include a step of obtaining, by the electronic device, position information of a target region corresponding to a past time point and position information of the target region corresponding to a reference time point from an image including a viewer's face region input through a camera. In one embodiment of the present disclosure, the method of operating an electronic device may include a step of obtaining, by the electronic device, position change information of the target region corresponding to the reference time point based on the position information of the target region corresponding to the past time point and the position information of the target region corresponding to the reference time point. In one embodiment of the present disclosure, the method of operating an electronic device may include a step of predicting, by the electronic device, a future velocity of the target region based on the position change information of the target region corresponding to the reference time point and the position change information of the target region corresponding to the past time point. In one embodiment of the present disclosure, the method of operating an electronic device may include a step of obtaining, by the electronic device, velocity change information of the target region corresponding to the reference time point based on the position change information of the target region corresponding to the reference time point and the position change information of the target region corresponding to the past time point. In one embodiment of the present disclosure, a method of operating an electronic device may include a step of predicting, by the electronic device, a future acceleration of the target portion based on velocity change information of the target portion corresponding to the reference time point and velocity change information of the target portion corresponding to the past time point. In one embodiment of the present disclosure, a method of operating an electronic device may include a step of predicting, by the electronic device, a future position of both eyes corresponding to the target time point based on the future velocity and the future acceleration. In one embodiment of the present disclosure, a method of operating an electronic device may include a step of outputting, by the electronic device, an image based on the future positions of both eyes corresponding to the target time point.

[0008] In one embodiment of the present disclosure, one or more computer-readable storage media may be provided that store one or more computer programs including computer-executable instructions that, when individually or collectively executed by one or more processors of an electronic device, cause the electronic device to perform an operation. In one embodiment of the present disclosure, the operation may include obtaining, by the electronic device, from an image including a viewer's face region input through a camera, position information of a target region corresponding to a past time point and position information of the target region corresponding to a reference time point. In one embodiment of the present disclosure, the operation may include obtaining, by the electronic device, position change information of the target region corresponding to the reference time point based on the position information of the target region corresponding to the past time point and the position information of the target region corresponding to the reference time point. In one embodiment of the present disclosure, the operation may include predicting, by the electronic device, a future velocity of the target region based on the position change information of the target region corresponding to the reference time point and the position change information of the target region corresponding to the past time point. In one embodiment of the present disclosure, the operation may include obtaining, by the electronic device, velocity change information of the target portion corresponding to the reference point in time based on position change information of the target portion corresponding to the reference point in time and position change information of the target portion corresponding to the past point in time. In one embodiment of the present disclosure, the operation may include predicting, by the electronic device, future acceleration of the target portion based on velocity change information of the target portion corresponding to the reference point in time and velocity change information of the target portion corresponding to the past point in time.In one embodiment of the present disclosure, the operation may include predicting, by the electronic device, the future positions of the two eyes corresponding to the target time point based on the future velocity and the future acceleration. In one embodiment of the present disclosure, the operation may include outputting, by the electronic device, an image based on the future positions of the two eyes corresponding to the target time point.

[0009] In one embodiment of the present disclosure, an electronic device may include a memory storing one or more computer programs and one or more processors communicatively connected to the memory. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain, from an image including a viewer's face area input through a camera, location information of a target region corresponding to a past time point and location information of the target region corresponding to a reference time point. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain location change information of the target region corresponding to the reference time point based on the location information of the target region corresponding to the past time point and the location information of the target region corresponding to the reference time point. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict a future speed of the target portion based on position change information of the target portion corresponding to the reference time point and position change information of the target portion corresponding to the past time point. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain speed change information of the target portion corresponding to the reference time point based on position change information of the target portion corresponding to the reference time point and position change information of the target portion corresponding to the past time point.In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that cause the electronic device, when individually or collectively executed by the one or more processors, to predict future acceleration of the target portion based on velocity change information of the target portion corresponding to the reference time point and velocity change information of the target portion corresponding to the past time point. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that cause the electronic device, when individually or collectively executed by the one or more processors, to predict future positions of both eyes corresponding to the target time point based on the future velocity and the future acceleration. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that cause the electronic device, when individually or collectively executed by the one or more processors, to output an image based on the future positions of both eyes corresponding to the target time point.

[0010] Other aspects, advantages and key features of the present disclosure will become apparent to those skilled in the art from the following detailed description of various embodiments of the present disclosure taken in conjunction with the accompanying drawings.

[0011] The above and other aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0012] FIG. 1 is a diagram illustrating outputting an image based on a viewer's visual position according to one embodiment of the present disclosure;

[0013] FIG. 2 is a diagram illustrating a method for an electronic device to display an image by predicting a future visual position of a viewer according to an embodiment of the present disclosure;

[0014] FIG. 3 is a diagram illustrating a method for an electronic device to predict a future visual position of a viewer according to one embodiment of the present disclosure;

[0015] FIG. 4 is a diagram illustrating a method for an electronic device to filter noise from position information and velocity information of a target area included in a viewer's face, according to one embodiment of the present disclosure;

[0016] FIG. 5 is a diagram illustrating a method for an electronic device to predict a future velocity of a target area according to one embodiment of the present disclosure;

[0017] FIG. 6 is a diagram illustrating a method for an electronic device to predict future acceleration of a target area and predict future position according to one embodiment of the present disclosure;

[0018] FIG. 7 is a diagram illustrating a method for an electronic device to perform independent correction on predicted future position information according to one embodiment of the present disclosure;

[0019] FIG. 8 is a diagram illustrating a method for an electronic device to perform dependent correction on predicted future position information, according to one embodiment of the present disclosure;

[0020] FIG. 9 is a diagram illustrating a method for an electronic device to predict a future visual position of a viewer according to one embodiment of the present disclosure;

[0021] FIG. 10 is a diagram illustrating a method for an electronic device to predict a future visual position of a viewer according to one embodiment of the present disclosure;

[0022] FIG. 11 is a diagram illustrating a method for an electronic device to predict a future visual position of a viewer according to one embodiment of the present disclosure;

[0023] FIG. 12 is a diagram illustrating a method for an electronic device to predict a future visual position of a viewer according to one embodiment of the present disclosure;

[0024] FIG. 13 is a diagram illustrating an electronic device additionally filtering noise from future visual position information of a viewer according to one embodiment of the present disclosure;

[0025] FIG. 14 is a diagram illustrating an operation method of an electronic device according to one embodiment of the present disclosure; and

[0026] FIG. 15 is a drawing showing an example of an electronic device according to one embodiment of the present disclosure.

[0027] It should be noted that throughout the drawings, identical reference numbers are used to describe identical or similar elements, features and structures.

[0028] The following description, with reference to the attached drawings, is provided to facilitate a comprehensive understanding of various embodiments of the present disclosure as defined by the claims and their equivalents. While numerous specific details are included to aid understanding, they are to be considered merely exemplary. Accordingly, those skilled in the art will recognize that various changes and modifications to the various embodiments detailed herein may be made without departing from the scope and spirit of the present disclosure. Furthermore, descriptions of known functions and configurations may be omitted for clarity and brevity.

[0029] The terms and words used in the following description and claims are not intended to be limited by their bibliographic meanings, but rather have been used by the inventors solely to facilitate a clear and consistent understanding of the present disclosure. Therefore, it will be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only and is not intended to limit the present disclosure, which is defined by the appended claims and their equivalents.

[0030] The singular forms "a," "an," and "the" should be understood to include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a surface of a part" includes reference to one or more such surfaces.

[0031] In describing embodiments of the present disclosure, detailed descriptions of related known technologies will be omitted if they are deemed to unnecessarily obscure the main point. Furthermore, numbers (e.g., "first," "second," etc.) used in the description of embodiments are merely identifiers used to distinguish one component from another.

[0032] Below, with reference to the attached drawings, embodiments of the present disclosure are described so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. Before proceeding with a detailed description of the invention, the terms used in this specification are defined or understood as follows.

[0033] When a component is referred to as being "connected" or "connected" to another component in this disclosure, it should be understood that the component may be directly connected to or connected to the other component, but may also be connected or connected via another component in between, unless otherwise specifically stated. Furthermore, "connection" may include a wireless connection or a wired connection.

[0034] In addition, in this specification, components expressed as "unit", "module", etc. may be two or more components combined into one component, or one component may be divided into two or more components with more detailed functions. In addition, each component described below may additionally perform some or all of the functions performed by other components in addition to its own main function, and of course, some of the main functions performed by each component may be exclusively performed by other components.

[0035] In this specification, the expression "at least one of a, b, or c" can refer to "a", "b", "c", "a and b", "a and c", "b and c", "all of a, b, and c", or variations thereof. In this disclosure, the expression "a or b" can refer to "a", "b", "a and b", or variations thereof. In this disclosure, the expression "a (or, b, c)" or the expression "a, b, or c" can refer to "a", "b", "c", "a and b", "a and c", "b and c", "all of a, b, and c", or variations thereof.

[0036] In one embodiment of the present disclosure, a “viewpoint” may include a viewpoint in units of frames of an input image or an output image.

[0037] In one embodiment of the present disclosure, "position" may include a relative position within an input image. For example, "position" may include coordinate information of a corresponding pixel among a plurality of pixels included in the input image. For example, the position of the left eye may include coordinate information of a pixel where the left eye is located among a plurality of pixels included in the input image. For example, the position of the left eye corresponding to a specific point in time may include coordinate information of a pixel where the left eye is located among a plurality of pixels included in a frame image corresponding to a specific point in time of the input image.

[0038] In one embodiment of the present disclosure, "independent correction" may include correction that does not take into account predicted information (e.g., future position information) for other parts. For example, "independent correction" may include correction that is independent of predicted information (e.g., future position information) for other parts. For example, independent correction may include monocular correction.

[0039] In one embodiment of the present disclosure, "dependent correction" may include corrections that take into account predicted information (e.g., future position information) for other parts. For example, "dependent correction" may include corrections that depend on predicted information (e.g., future position information) for other parts. For example, "dependent correction" may include corrections that utilize predicted information (e.g., future position information) for other parts. For example, "dependent correction" may include corrections based on predicted information (e.g., future position information) for other parts. For example, dependent correction may include binocular correction.

[0040] In one embodiment of the present disclosure, a "target point in time" may include a point in time corresponding to information (or data) to be predicted. In one embodiment of the present disclosure, a "reference point in time" may include the most recent point in time or the point in time closest to the target point in time among one or more points in time associated with information used for prediction of the target point in time. For example, a "reference point in time" may include the current point in time. In one embodiment of the present disclosure, a "past point in time" may include a point in time prior to the reference point in time. In one embodiment of the present disclosure, a "future point in time" may include a point in time after the reference point in time.

[0041] In one embodiment of the present disclosure, the value for “binocular center” may include a value predicted with the binocular center as the target region or a value calculated (or determined) based on values ​​(e.g., predicted values) for the left and right eyes.

[0042] In one embodiment of the present disclosure, interpupillary distance (IPD) information may include viewer-specific binocular distance information or common binocular distance information. For example, the IPD information may be preset or stored. For example, the IPD information may be updated in real time or periodically. For example, the IPD information may be obtained from an image containing the user's face.

[0043] It should be understood that the blocks in each flowchart and combination of flowcharts can be performed by one or more computer programs containing computer-executable instructions. The entirety of one or more computer programs may be stored in a single memory device, or the one or more computer programs may be divided into different portions stored in multiple different memory devices.

[0044] The functions or operations described herein may be processed by a single processor or a combination of processors. A single processor or a combination of processors is a circuit that performs processing, and may include an application processor (AP) (e.g., a central processing unit (CPU)), a communication processor (CP) (e.g., a modem), a graphical processing unit (GPU), a neural processing unit (NPU) (e.g., an artificial intelligence (AI) chip), a wireless-fidelity (Wi-Fi) chip, or a Bluetooth chip. TMIt includes circuits such as chips, global positioning system (GPS) chips, near field communication (NFC) chips, connection chips, sensor controllers, touch controllers, fingerprint sensor controllers, display driver integrated circuits (ICs), audio codec chips, universal serial bus (USB) controllers, camera controllers, image processing ICs, microprocessor units (MPUs), system on chip (SoCs), or ICs.

[0045] FIG. 1 is a diagram illustrating outputting an image based on a viewer's visual position according to one embodiment of the present disclosure.

[0046] Even if an electronic device (e.g., a display device) renders and outputs the same image (e.g., a light field image) in the same manner, the image actually visually perceived by the viewer may differ depending on the viewer's visual position (e.g., viewpoint). For example, if the output image of the electronic device passes through a special lens (e.g., a lenticular lens, a parallax barrier), the viewer may visually perceive an image that is deformed or distorted by the lens, and the image visually perceived by the viewer may vary depending on the viewer's visual position. Therefore, the electronic device can render the output image based on the viewer's visual position and output it through the image output device (or image output module) so that the image that should be perceived by the viewer (e.g., the target image), i.e., the appropriate image, can be visually perceived by the viewer.

[0047] While an electronic device outputs an output image through an image output device (or an image output module), the viewer's visual position may change depending on the viewer's movement. In one embodiment of the present disclosure, the electronic device may identify (or detect, acquire) the visual position changed by the viewer's movement, render an output image based on the changed visual position, and output the rendered output image through the image output device (or the image output module). For example, the electronic device may track the viewer's visual position in real time, render and output an output image in real time based on the real-time changeable visual position.

[0048] In one embodiment of the present disclosure, an electronic device may receive an image including a viewer in order to detect or track the viewer's visual position (or a change in the viewer's visual position). For example, the electronic device may identify (or acquire, receive) an image including a viewer input through an image input device (or an image input module). For example, the electronic device may receive (or acquire, identify) an image including a viewer from the image input device (or an image input module). For example, the electronic device may identify (or acquire, receive) a real-time image including a viewer in order to detect or track the viewer's visual position (or a change in the viewer's visual position) in real time.

[0049] Referring to FIG. 1, an electronic device (e.g., a display device) may include a display (102) and a lenticular lens (104). The electronic device may output an image (106) to be perceived by the left eye to pixels perceived by the left eye among a plurality of pixels included in the display (102), and may output an image (108) to be perceived by the right eye to pixels perceived by the right eye. Among the plurality of pixels included in the display (102), the pixels perceived by the left eye and the pixels perceived by the right eye may vary depending on the viewer's visual position.

[0050] In order to provide an image suitable for a viewer at a first viewing position (112) at a first time point (110) of FIG. 1, the electronic device may render (or generate) an output image such that data of a specific area (114) of the image (106) is output to a first pixel (118) of the display (102), and data of a specific area (116) of the image (108) is output to a second pixel (120) of the display (102). When the electronic device outputs the rendered output image on the display (102), the first pixel (118) area of ​​the output image may be perceived by the left eye of the viewer at the first viewing position (112), and the second pixel (120) area of ​​the output image may be perceived by the right eye of the viewer at the first viewing position (112), due to the optical characteristics (e.g., refractive characteristics) of the lenticular lens.

[0051] If the electronic device outputs an output image rendered in the same manner as at the first point of view (110) on the display (102) even though the viewer's visual position changes (or moves), a distorted image (or an inappropriate image, a deformed image) may be visually perceived by a viewer at a different visual position (e.g., a second visual position (134)) than the first visual position (112). Therefore, the electronic device needs to render the output image in a different manner than at the first point of view (110) and output it on the display (102) according to the change in the viewer's visual position. For example, in order to provide an image suitable for the viewer even when the viewer's visual position changes at the second point in time (130), the electronic device can detect the changed visual position, i.e., the second visual position (134), and render (or generate) the output image so that data of a specific area (114) of the image (106) is output to the first pixel (118) of the display (102), and data of a specific area (116) of the image (108) is output to the third pixel (132) of the display (102).

[0052] Due to the processing time required for an electronic device to detect a viewer's visual position in real time from an input video and to render and output a new output video based on the detected visual position, there may be a time period during which an image appropriate for the previous visual position is output even though the viewer's visual position has changed. For example, due to the processing time of the electronic device, a delay time may occur between the time when the viewer's visual position changes and the time when the rendered image based on the changed visual position is output. During the delay time, the electronic device may output an image rendered based on the previously detected viewer's visual position, and an image different from the appropriate image that the viewer should perceive (e.g., an inappropriate image, a distorted image, a deformed image, an unnatural or broken image) may be perceived by the viewer. In order to improve the deterioration of the output video quality due to the delay time, the processing speed for image input (e.g., image sensing), visual position detection (or tracking), image rendering, and / or image output (e.g., display) may be improved and the required time may be reduced; however, there may be limitations to the improvement of the processing speed and the reduction of the required time.

[0053] In one embodiment of the present disclosure, a method or electronic device may be provided that predicts a viewer's visual position at a future point in time and outputs a rendered image according to the predicted visual position. For example, referring to FIG. 1, the electronic device may predict the viewer's visual position at a second point in time (130) as a second visual position (134) before reaching the second point in time (130), and output an output image rendered for the second visual position (134) on a display (102) at the second point in time (130) (or a frame time corresponding to the second point in time (130). Accordingly, the electronic device may reduce delay time by rendering an image according to a previously predicted visual position and outputting the image on the display (102).

[0054] FIG. 2 is a diagram illustrating a method for an electronic device according to one embodiment of the present disclosure to display an image by predicting a future visual position of a viewer.

[0055] In explaining Fig. 2, any explanation that overlaps with the explanation given above in Fig. 1 may be omitted.

[0056] Referring to FIG. 2, in one embodiment of the present disclosure, a method (200) for an electronic device to predict a viewer's visual position and display an image may include steps 210 to 260. In one embodiment of the present disclosure, the electronic device performing the method (200) may include a display device. In one embodiment of the present disclosure, steps 210 to 260 of the method (200) may be executed by at least one processor included in the electronic device. In one embodiment of the present disclosure, the electronic device may perform the method (200) in real time. In one embodiment of the present disclosure, the electronic device may repeatedly perform the method (200).

[0057] In one embodiment of the present disclosure, steps 210 to 260 of method (200) may be performed by multiple electronic devices. For example, some steps of method (200) may be performed by a first electronic device, and other steps may be performed by a second electronic device. Method (200) is not limited to that illustrated in FIG. 2 , and in one or more embodiments, steps not illustrated in FIG. 2 may be further included, or some steps may be omitted.

[0058] In step 210, the electronic device may detect a viewer using an image sensor. For example, the electronic device may include an image input device (or module) (e.g., a camera) that includes a lens and an image sensor. The image sensor may convert light entering the electronic device through the camera lens into an electrical image signal. For example, the image sensor of the electronic device may detect a viewer.

[0059] In one embodiment of the present disclosure, an electronic device can acquire (or identify) an image including a viewer (hereinafter, “input image”) captured using an image input device (or module). The image input device that captures the image including the viewer can be positioned (or arranged) so as to capture a space where the viewer is positioned to use the electronic device. For example, the electronic device can acquire (or identify) an image including the viewer through (or using) an image input device built into or included in the electronic device. For example, the electronic device can receive an image including the viewer through the image input device. In one embodiment of the present disclosure, the electronic device can receive an image including the viewer from an image input device that is connected or communicated with by wire or wirelessly.

[0060] In step 220, the electronic device can track (or identify, detect) the location of the viewer's face area in the input image. For example, a tracker (or tracking module) of the electronic device can track the location of the viewer's face area in the input image. For example, the electronic device can track the location of target areas (e.g., the left eye, the right eye, the area between the eyebrows, the center of both eyes, etc.) included in the viewer's face in the input image.

[0061] In step 230, the electronic device can predict the viewer's visual position (e.g., left eye position, right eye position) at a future point in time. For example, the electronic device can analyze the viewer's movements and use the movement analysis results to predict the viewer's visual position. For example, the electronic device can predict the viewer's visual position at a future point in time (hereinafter, "the viewer's future visual position") using accumulated viewer's visual position information. For example, the electronic device can predict the viewer's future visual position using the viewer's visual position information corresponding to the current point in time or the viewer's visual position information corresponding to a past point in time.

[0062] Referring to FIG. 2, step 230 may include steps 232 to 240. In one embodiment of the present disclosure, the electronic device may perform the operations of steps 232 to 240 for a target region. In the description of steps 232 to 240 below, it is described that the electronic device performs the operations of steps 232 to 240 for a single eye (e.g., the right eye or the left eye) as an example of the target region, but this is not limited thereto. For example, the electronic device may perform the operations of at least one of steps 232 to 240 for another target region.

[0063] In step 232, the electronic device can obtain position information and velocity information of the monocular. For example, the electronic device can obtain position information and velocity information of the monocular in real time from an input image. For example, the electronic device can obtain position information of the monocular by tracking (or detecting) the position of the viewer's monocular in the input image. For example, the electronic device can obtain position information of the monocular corresponding to a reference point in time. For example, the electronic device can filter out noise from the obtained position information of the monocular.

[0064] For example, an electronic device can obtain velocity information of a monocular using monocular position information. For example, an electronic device can obtain changes in the position of a monocular as velocity information of the monocular. For example, an electronic device can calculate or determine the velocity of a monocular using monocular position information. For example, an electronic device can obtain velocity information of a monocular corresponding to a reference point in time. For example, an electronic device can filter out noise from the obtained monocular position information.

[0065] In step 234, the electronic device can predict (or calculate, determine) the future velocity of the monocular. For example, the electronic device can obtain information about the future velocity of the monocular. For example, the electronic device can predict the velocity of the monocular corresponding to a future point in time. For example, the electronic device can obtain predicted velocity information of the monocular corresponding to a future point in time. For example, the electronic device can predict the future velocity of the monocular using the velocity information of the monocular.

[0066] In step 236, the electronic device may acquire acceleration information of the monocular. For example, the electronic device may acquire acceleration information of the monocular using velocity information of the monocular. For example, the electronic device may acquire changes in the velocity of the monocular as acceleration information of the monocular. For example, the electronic device may calculate or determine acceleration of the monocular using velocity information of the monocular. For example, the electronic device may acquire acceleration information of the monocular corresponding to a reference point in time.

[0067] In step 238, the electronic device can predict (or calculate, determine) the future acceleration of the monocular. For example, the electronic device can obtain information about the future acceleration of the monocular. For example, the electronic device can predict the acceleration of the monocular corresponding to a future point in time. For example, the electronic device can obtain predicted acceleration information of the monocular corresponding to a future point in time. For example, the electronic device can predict the future acceleration of the monocular using the acceleration information of the monocular.

[0068] In step 240, the electronic device can predict (or calculate, determine) the future position of the monocular. For example, the electronic device can obtain information on the future position of the monocular. For example, the electronic device can predict the position of the monocular corresponding to a future point in time. For example, the electronic device can obtain information on the predicted position of the monocular corresponding to a future point in time. For example, the electronic device can predict the future position of the monocular based on the future acceleration or future velocity of the monocular. For example, the electronic device can predict the future position of the monocular using the predicted acceleration information or predicted velocity information of the monocular.

[0069] The future time points in step 234 and step 238 may be different or the same time points. The future time points in step 234 and step 240 may be different or the same time points. The future time points in step 240 and step 238 may be different or the same time points.

[0070] In one embodiment of the present disclosure, an electronic device can predict the visual position of a viewer at a future point in time based on the future position of the monocular eye. For example, the electronic device can predict (or calculate, determine) the positions of the viewer's binocular eyes at a future point in time based on the future position of the monocular eye.

[0071] In step 250, the electronic device can render (or generate) an output image using the predicted result. For example, the electronic device can render the output image based on the viewer's visual position at a future point in time. For example, the electronic device can generate the output image by rendering the output image based on the viewer's future visual position.

[0072] In step 260, the electronic device may display the rendered (or generated) output image. For example, the electronic device may output the rendered (or generated) output image on a display. For example, the electronic device may output the output image through an image output device (or module) that is connected or communicated with wirelessly. For example, the electronic device may output the output image through an image output device (or module) that is built-in or included.

[0073] Although FIG. 2 illustrates that the electronic device sequentially performs steps 232 to 240, this is not a limitation. For example, the electronic device may perform step 232 and then perform steps 234 and 236 in parallel. For example, the electronic device may perform step 236 and then step 238 regardless of whether step 234 is performed.

[0074] The specific operation of the electronic device to predict the future visual position of the viewer can be described with reference to FIGS. 3 to 14 below.

[0075] FIG. 3 is a diagram illustrating a method for an electronic device to predict a future visual position of a viewer according to one embodiment of the present disclosure.

[0076] In explaining Fig. 3, any explanation that overlaps with the explanation given above in Fig. 1 or 2 may be omitted.

[0077] Referring to FIG. 3, in one embodiment of the present disclosure, a method (300) for an electronic device to predict a viewer's visual position may include steps 310 to 320. In one embodiment of the present disclosure, the electronic device performing the method (300) may include a display device. In one embodiment of the present disclosure, steps 310 to 320 of the method (300) may be executed by at least one processor included in the electronic device. In one embodiment of the present disclosure, the electronic device may perform the method (300) in real time. In one embodiment of the present disclosure, the electronic device may repeatedly perform the method (300).

[0078] In one embodiment of the present disclosure, steps 310 to 320 of method (300) may be performed by multiple electronic devices. For example, some steps of method (300) may be performed by a first electronic device, and other steps may be performed by a second electronic device. Method (300) is not limited to that illustrated in FIG. 3 , and in one or more embodiments, steps not illustrated in FIG. 3 may be further included, or some steps may be omitted.

[0079] In one embodiment of the present disclosure, an electronic device can predict a viewer's future visual position based on monocular position and velocity information. Referring to FIG. 3 , the electronic device can perform identical operations for both the left and right eyes. Therefore, the description given below for the left eye in FIG. 3 can also be applied to the right eye, and thus, a description of the right eye may be omitted.

[0080] In step 310, the electronic device may obtain left eye position information, filter out noise from the left eye position information, obtain left eye velocity information, and filter out noise from the left eye velocity information. In one embodiment of the present disclosure, the electronic device may determine the degree (or strength) of filtering based on the degree of viewer movement (e.g., left eye movement) corresponding to a reference point in time. For example, the electronic device may filter out noise from the left eye position information or left eye velocity information with a weak filtering strength based on the identification of a large viewer movement.

[0081] In step 312, the electronic device can predict the future velocity of the left eye. For example, the electronic device can predict (or calculate, determine) the future velocity of the left eye using the velocity information of the left eye. For example, the electronic device can predict the future velocity of the left eye using the noise-filtered velocity information of the left eye.

[0082] In one embodiment of the present disclosure, an electronic device can predict the future velocity of the left eye by weighting the velocity of the left eye corresponding to a reference time point and the velocity of the left eye corresponding to a past time point using the velocity information of the left eye from which noise has been filtered. For example, the electronic device can determine the weight based on the degree of movement of the viewer corresponding to the reference time point. For example, based on identifying that the viewer's movement is large, the electronic device can increase the weight for the velocity corresponding to the reference time point and decrease the weight for the velocity corresponding to the past time point. For example, based on identifying that the viewer's movement is small, the electronic device can decrease the weight for the velocity corresponding to the reference time point and increase the weight for the velocity corresponding to the past time point.

[0083] In step 314, the electronic device may acquire acceleration information of the left eye, predict future acceleration of the left eye, and filter out noise from the future acceleration of the left eye. For example, the electronic device may acquire acceleration information of the left eye based on velocity information of the left eye. For example, the electronic device may acquire acceleration information of the left eye using velocity information of the left eye from which noise has been filtered.

[0084] In one embodiment of the present disclosure, an electronic device can obtain nonlinear predicted acceleration information of the left eye corresponding to a future time using acceleration information of the left eye. The electronic device can predict (or calculate, determine) the future acceleration of the left eye by filtering out noise from the nonlinear predicted acceleration information of the left eye. In one embodiment of the present disclosure, the electronic device can determine the deceleration amount of the acceleration of the left eye based on the movement state or movement speed of the viewer. For example, when the movement state of the viewer is a moving state or the movement speed is equal to or greater than a certain speed, the electronic device can predict (or calculate, determine) the future acceleration of the left eye with a deceleration amount of a low acceleration. For example, when the movement state of the viewer is a stationary state or the movement speed is less than (or below) a certain speed, the electronic device can predict (or calculate, determine) the future acceleration of the left eye with a deceleration amount of a high acceleration.

[0085] In step 316, the electronic device can predict a first future position of the left eye (e.g., a future position before correction) based on the predicted future acceleration and future velocity of the left eye. For example, the electronic device can predict the first future position of the left eye based on the noise-filtered future acceleration of the left eye. For example, the electronic device can linearly predict the future position of the left eye based on the future velocity of the left eye. For example, the electronic device can nonlinearly predict the future position of the left eye based on the future acceleration of the left eye.

[0086] In step 318, the electronic device may perform monocular correction for the first future position of the left eye. For example, the electronic device may correct the first future position of the left eye to obtain a second future position of the left eye (e.g., a monocularly corrected future position). The first and second future positions of the left eye may correspond to the same target viewpoint.

[0087] In one embodiment of the present disclosure, an electronic device can correct the future position of the left eye corresponding to the target time point based on the future position of the left eye corresponding to a time point prior to the target time point. In one embodiment of the present disclosure, the electronic device can adjust the correction strength according to the temporal difference between the reference time point and the future time point (e.g., the target time point). For example, when the electronic device predicts a relatively near future, i.e., when the difference between the reference time point and the target time point is small, the electronic device can correct the future position of the left eye with a relatively low correction strength. For example, when the electronic device predicts a relatively distant future, i.e., when the difference between the reference time point and the target time point is large, the electronic device can correct the future position of the left eye with a relatively high correction strength.

[0088] In step 320, the electronic device may perform binocular correction based on the second future position of the left eye and the second future position of the right eye. For example, the electronic device may correct the second future position of the left eye based on the future position of the center of the binoculars, thereby predicting (or calculating, determining) the third future position of the left eye. For example, the future position of the center of the binoculars may be determined based on the future position of the left eye and the future position of the right eye. For example, the electronic device may correct the second future position of the right eye based on the second future position of the left eye, and correct the second future position of the left eye based on the second future position of the right eye. For example, the electronic device may correct the second future position information of the right eye and the second future position information of the left eye based on the interpupillary distance (IPD) information of the viewer or predefined IPD information.

[0089] In one embodiment of the present disclosure, the electronic device can correct the future position of the binocular center corresponding to the target time point or the second future position of the left eye corresponding to the target time point based on the future position of the binocular center corresponding to the time point before the target time point. In one embodiment of the present disclosure, the electronic device can determine the correction strength based on the movement state of the viewer. For example, the electronic device can strongly filter the IPD information and use it for binocular correction based on identifying the movement of the viewer as slow movement. For example, the electronic device can weakly filter the IPD information and use it for binocular correction based on identifying the movement of the viewer as fast movement.

[0090] Although FIG. 3 illustrates the future positions of the left and right eyes as the final outputs of the method (300), the present invention is not limited thereto. For example, the electronic device may predict (or, calculate, determine) the future positions of the centers of both eyes as the final outputs of the method (300).

[0091] Although FIG. 3 illustrates that the electronic device performs binocular correction after monocular correction, this is not a limitation. For example, the electronic device may perform monocular correction after binocular correction.

[0092] The specific operations performed in each step of Fig. 3 can be described with reference to Figs. 4 to 8 below.

[0093] FIG. 4 is a diagram illustrating a method for an electronic device to filter noise from position information and velocity information of a target area included in a viewer's face, according to one embodiment of the present disclosure.

[0094] In explaining Fig. 4, any explanation that overlaps with the explanation given above in any one of Figs. 1 to 3 may be omitted.

[0095] In step 410, the electronic device may obtain location information of the target area and filter out noise from the obtained location information. For example, the electronic device may filter out noise from the location information of the target area using the following mathematical expression 1.

[0096] x t = α x t + (1-α) x t-1 ...수학식 1

[0097] In the above mathematical expression 1, x t represents the position of the target region corresponding to time t (e.g., reference time), and x t-1 can represent the position of the target region corresponding to time t-1 (e.g., past time). x on the right side of mathematical expression 1 t represents the location of the target region before filtering noise (i.e. raw location information), and x on the left t can represent the location of the target area where the noise is filtered. The x on the left side t-1can represent the location of the target region where the noise is filtered. For example, the value of α can be determined based on the filtering strength. For example, the value of α can be determined or set experimentally or empirically.

[0098] For example, an electronic device can filter out noise from the location information of a target area using the mathematical expression 2 below.

[0099] ... Mathematical formula 2

[0100] In the above mathematical expression 2, represents the position of the target area corresponding to time point t (e.g., reference time point), represents the location of the target region corresponding to time t-1 (e.g., past time), can represent the velocity of the target area corresponding to time t-1. The right side of mathematical expression 2 represents the location of the target area before filtering noise, and the left side can represent the location of the target area where the noise is filtered. The left side can represent the location of the target region where the noise is filtered. For example, the value of α can be determined based on the filtering strength. For example, the value of α can be determined or set experimentally or empirically.

[0101] In step 420, the electronic device can calculate the velocity of the target area based on the location information of the target area. For example, the electronic device can calculate the velocity of the target area using the following mathematical expression 3.

[0102] ... Mathematical formula 3

[0103] In the above mathematical formula 3, represents the location of the target area corresponding to time point t, can represent the location of the target area corresponding to time t-1. can represent the velocity of the target area corresponding to time point t.

[0104] In step 430, the electronic device can determine whether the velocity information contains noise. For example, the electronic device can determine whether the velocity information contains noise based on the viewer's degree of movement. For example, if the viewer's degree of movement is large, the calculated velocity value may be considered to be due to the viewer's movement, and thus the velocity information may be determined to have little or no noise. Conversely, if the viewer's degree of movement is small, the calculated velocity value may be considered to be due to noise, and thus the velocity information may be determined to contain a lot of noise.

[0105] In one embodiment of the present disclosure, the degree of movement of the viewer can be determined based on the speed corresponding to a reference point in time and the speed corresponding to a past point in time. For example, the degree of movement of the viewer can be determined based on the magnitude of the speed corresponding to a reference point in time ( ) and the magnitude of the velocity corresponding to the past point in time ( ) can be determined based on the difference (e.g., the amount of change in speed). For example, if the difference is less than (or below) a threshold value, it can be determined that the degree of movement is small, that is, noise exists (or there is a lot of noise). For example, if the difference is greater than (or exceeds) a threshold value, it can be determined that the degree of movement is large, that is, no noise exists (or there is little noise).

[0106] In one embodiment of the present disclosure, an electronic device may filter out noise from velocity information of a target region based on the determination result. For example, the electronic device may determine the value of a parameter used for velocity filtering based on the determination result. The value of the parameter used for velocity filtering may be used to determine a weight for the velocity corresponding to a reference time point (e.g., time point t) in filtering out velocity noise.

[0107] Referring to FIG. 4, based on determining that noise does not exist, the electronic device may determine the value of the parameter (s) as the first value (α) in step 432. Based on determining that noise exists, the electronic device may determine the value of the parameter (s) as the second value (β) in step 434. In step 440, the electronic device may filter out noise from the velocity information of the target area based on the value of the parameter determined based on the determination result.

[0108] For example, an electronic device can filter out noise from velocity information of a target area using the mathematical expression 4 below.

[0109] ... Mathematical formula 4

[0110] In the above mathematical formula 4, can represent the velocity of the target area corresponding to time point t. can represent the velocity of the target area corresponding to time t-1. s can be a parameter used to filter noise from velocity information. The right side of mathematical expression 4 represents the velocity of the target region before filtering noise (i.e., raw velocity information), and the left side can represent the velocity of the target region with the noise filtered out. For example, the left side can represent the velocity of the target area with noise filtered out.

[0111] For example, an electronic device can filter out noise from velocity information of a target area using the mathematical expression 5 below.

[0112] ... Mathematical Formula 5

[0113] In the above mathematical expression 5, can represent the velocity of the target area corresponding to time point t. represents the velocity of the target area corresponding to time t-1, can represent the acceleration of the target area corresponding to time t-1. s can be a parameter used to filter noise from velocity information. The right side of mathematical expression 5 represents the velocity of the target area before filtering noise, and the left side can represent the velocity of the target region with the noise filtered out. For example, the left side can represent the velocity of the target area with noise filtered out.

[0114] For example, if noise is determined to be present, the electronic device may determine the value of s as β. For example, if noise is determined to be absent, the electronic device may determine the value of s as α. α and β may be experimentally or empirically determined or established values. For example, α may be a greater value than β. Accordingly, if noise is determined to be small or absent, a greater weight may be applied to the velocity corresponding to a reference point in time, and if noise is determined to be present or large, a greater weight may be applied to the velocity corresponding to a past point in time.

[0115] Referring to FIG. 4, the presence or absence of noise may include meanings such as whether the value of the velocity corresponding to the reference point is due to noise, whether the viewer's movement is small, whether a high degree of filtering is applied to the velocity corresponding to the reference point, or whether the value of the parameter for velocity filtering is determined as a second value.

[0116] FIG. 5 is a diagram illustrating a method for an electronic device to predict a future velocity of a target area according to one embodiment of the present disclosure.

[0117] In referring to FIG. 5, any description that overlaps with the description described above in any one of FIGS. 1 to 4 may be omitted.

[0118] In one embodiment of the present disclosure, an electronic device may predict a future velocity of a target portion based on the velocity of the target portion corresponding to a reference point in time (e.g., current velocity) and the velocity of the target portion corresponding to a past point in time (e.g., past velocity information). For example, the electronic device may predict the future velocity of the target portion using the current velocity, past velocity, and / or weights of the target portion. FIG. 5 illustrates an example in which an electronic device determines a weight for the current velocity of the target portion and predicts the future velocity based on the determined weights, according to one embodiment of the present disclosure.

[0119] In step 510, the electronic device may calculate, determine, or obtain a weight for the current velocity of the target area (or the velocity of the target area corresponding to a reference point in time). For example, the electronic device may calculate a weight for the current velocity using Equation 6 below.

[0120] ... Mathematical formula 6

[0121] In the above mathematical expression 6, represents the velocity corresponding to time t-1 (i.e., a past time), represents the velocity corresponding to time t (e.g., reference time, current time), and b can represent a weight for the current velocity. and β are parameter values ​​used to calculate the weights, which can be predetermined or set. For example, Alternatively, the value of β may be determined or set experimentally or empirically.

[0122] In step 520, the electronic device may determine whether the weight for the current speed exceeds an upper limit (or maximum value, threshold). For example, the upper limit may be a predetermined or set value. For example, the upper limit may be determined or set experimentally or empirically. Based on determining that the weight exceeds the upper limit, in step 522, the electronic device may determine the weight for the current speed as the upper limit.

[0123] In step 530, the electronic device can predict the future velocity of the target portion using a weighted value for the current velocity. For example, the electronic device can calculate or determine the future velocity of the target portion by weighting the current velocity and the past velocity of the target portion. For example, the electronic device can predict the future velocity of the target portion using the weight calculated in step 510 based on determining that the weight does not exceed the upper limit. For example, the electronic device can predict the future velocity of the target portion using the weight (i.e., the upper limit value) for the current velocity determined in step 522 based on determining that the weight exceeds the upper limit.

[0124] For example, an electronic device can predict the future velocity of a target area using the mathematical expression 7 below.

[0125] ... Mathematical formula 7

[0126] In the above mathematical formula 7, represents the velocity of the target area corresponding to time point t (e.g., reference time point), represents the velocity of the target region corresponding to time t-1 (e.g., past time), and b can represent a weight for the velocity of the target region corresponding to time t (e.g., weight for the current velocity). can represent the future velocity of the target area. For example, can represent the future velocity of the target region corresponding to a future time point (e.g., time point t+1).

[0127] FIG. 6 is a diagram illustrating a method for an electronic device to predict future acceleration of a target area and predict future position according to one embodiment of the present disclosure.

[0128] In referring to Fig. 6, any description that overlaps with the description described above in any one of Figs. 1 to 5 may be omitted.

[0129] In step 610, the electronic device can calculate (or obtain) the average acceleration of the target portion from a specific point in time to a reference point in time (e.g., the current point in time). For example, the electronic device can calculate or determine the average acceleration of the target portion based on the acceleration of the target portion corresponding to the reference point in time (e.g., point in time t) and the acceleration of the target portion corresponding to past points in time (e.g., point in time t-1, point in time t-2). For example, the electronic device can calculate the average acceleration of the target portion using the following mathematical expression 8.

[0130] ... Mathematical formula 8

[0131] In the above mathematical expression 8, represents the acceleration of the target area corresponding to time t, represents the acceleration of the target area corresponding to time t-1, can represent the acceleration of the target area corresponding to time t-2. m can represent the average acceleration of the target area.

[0132] In step 620, the electronic device may determine whether the viewer's movement state corresponding to a reference time point (e.g., time point t, current time point) is a moving state. In one embodiment of the present disclosure, the electronic device may determine whether the viewer's movement state corresponding to the reference time point is a moving state by determining whether both eyes of the viewer move in the same direction with respect to the reference time point and / or whether both eyes move at a speed equal to or greater than a certain speed. For example, the electronic device may determine the viewer's movement state corresponding to the reference time point as a moving state based on determining that both eyes of the viewer move in the same direction with respect to the reference time point and that both eyes of the viewer move at a speed equal to or greater than a certain speed with respect to the reference time point. For example, if the velocity direction of the left eye corresponding to the reference time point matches the velocity direction of the right eye, and the velocity magnitude of the left eye corresponding to the reference time point and the velocity magnitude of the right eye are equal to or greater than a threshold value, the electronic device may determine the viewer's movement state corresponding to the reference time point as a moving state.

[0133] Based on determining that the viewer's motion state corresponding to the reference time point is not a moving state, in step 630, the electronic device can determine whether the viewer's motion state corresponding to the reference time point is a stationary state. In one embodiment of the present disclosure, the electronic device can determine whether the viewer's motion state corresponding to the reference time point is a stationary state by determining whether both eyes of the viewer move in the same direction with respect to the reference time point and / or whether both eyes move at a speed less than (or below) a certain speed. For example, the electronic device can determine the viewer's motion state as a stationary state based on determining that both eyes of the viewer move at a speed less than a certain speed with respect to the reference time point. For example, if the velocity magnitude of the left eye corresponding to the reference time point and the velocity magnitude of the right eye corresponding to the reference time point are less than (or below) a threshold value, the electronic device can determine the viewer's motion state corresponding to the reference time point to be a stationary state.

[0134] Based on the determination that the viewer's movement state corresponding to the reference point in time is not a stationary state, in step 640, the electronic device can determine whether the viewer's movement state corresponding to a past point in time (e.g., point in time t-1) is a moving state.

[0135] In one embodiment of the present disclosure, the electronic device can predict the future acceleration of the target portion based on the parameter value determined based on the movement state of the viewer and the average acceleration calculated in step 610. For example, the electronic device can predict the nonlinear predicted acceleration of the target portion corresponding to the target time point using the parameter value determined based on the movement state of the viewer and the average acceleration of the target portion. For example, the electronic device can predict the future acceleration of the target portion corresponding to the target time point by filtering out noise from the nonlinear predicted acceleration of the target portion corresponding to the target time point. For example, the electronic device can predict the future acceleration of the target portion using the following mathematical expression 9.

[0136]

[0137] ... Mathematical Formula 9

[0138] In the above mathematical expression 9, m represents the average acceleration, r represents a parameter whose value is determined based on the viewer's movement status, can represent the nonlinear predicted acceleration of the target region corresponding to the time point t+n (e.g., target time point) predicted with time point t as the reference time point. γ can represent a parameter indicating the degree of filtering for acceleration noise. For example, γ can be determined or set experimentally or empirically. can represent the future acceleration of the target area corresponding to time t-1+n predicted based on time t-1 as a reference point. For example, can represent the future acceleration of the target area, with the noise filtered out for acceleration. can represent the future acceleration of the target area corresponding to time t+n predicted based on time t as a reference time.

[0139] In step 620, if the movement state of the viewer corresponding to the reference time is determined to be a movement state, or in step 640, if the movement state of the viewer corresponding to the past time is determined to be a movement state, in step 650, the electronic device can predict the future acceleration of the target part using the parameter value corresponding to the movement state. For example, in step 620, if the movement state of the viewer corresponding to the reference time is determined to be a movement state, or in step 640, if the movement state of the viewer corresponding to the past time is determined to be a movement state, the future acceleration of the target part (e.g., ) is the future velocity of the target area (e.g. ) may be the same as the direction of the

[0140] In step 630, if the viewer's movement state corresponding to the reference time is determined to be stationary, or in step 640, if the viewer's movement state corresponding to the past time is determined to be stationary, in step 660, the electronic device can predict future acceleration using the parameter value corresponding to the stationary state. For example, in step 630, if the viewer's movement state corresponding to the reference time is determined to be stationary, or in step 640, if the viewer's movement state corresponding to the past time is determined to be stationary, the future acceleration of the target part (e.g., ) is the future velocity of the target area (e.g. ) may differ from the direction of the

[0141] For example, a parameter value corresponding to a moving state may be greater than a parameter value corresponding to a stationary state. For example, a parameter value corresponding to a moving state or a parameter value corresponding to a stationary state may be experimentally or empirically determined or set.

[0142] In one embodiment of the present disclosure, an electronic device can predict the future position of a target portion based on the future acceleration of the target portion. For example, the electronic device can obtain, determine, or predict the future position of the target portion using the future acceleration and future velocity of the target portion.

[0143] Referring to FIG. 6, the electronic device can predict the future position of the target portion based on the future acceleration and future velocity of the target portion at step 680 after predicting the future acceleration at step 650. For example, the electronic device can predict the future position of the target portion using the following mathematical expression 10.

[0144] ... Mathematical expression 10

[0145] In the above mathematical expression 10, represents the location information of the target area corresponding to time point t (e.g., reference time point) with noise filtered, and n represents the difference between the target time point (time point t+n) and time point t. represents the future velocity (e.g. predicted velocity) of the target region corresponding to a future time point (e.g. time point t+1), represents the future acceleration (e.g. predicted acceleration) corresponding to time t+n predicted based on time t, can represent the future position of the target area corresponding to time t+n predicted based on time t as a reference time.

[0146] Referring to FIG. 6, the electronic device can predict the future acceleration of the target portion in step 660, and determine whether the predicted acceleration distance of the target portion exceeds the predicted velocity distance in step 670. The predicted velocity distance may represent a movement distance (e.g., a positional movement distance) predicted by the predicted velocity of the target portion (e.g., a future velocity of the target portion), and the predicted acceleration distance may represent a movement distance (e.g., a positional movement distance) predicted by the predicted acceleration of the target portion (e.g., a future acceleration of the target portion). For example, the predicted velocity distance may be corresponds to the value of , and the predicted acceleration distance is can correspond to the value of .

[0147] The electronic device can predict the future position of the target portion at step 680 based on determining that the predicted acceleration distance does not exceed the predicted velocity distance. For example, the electronic device can predict the future position of the target portion using Equation 10 above based on determining that the predicted acceleration distance does not exceed the predicted velocity distance.

[0148] Based on determining that the user's movement state is stationary and the predicted acceleration distance exceeds the predicted velocity distance, the electronic device may predict the position of the target part corresponding to the reference time point as the future position of the target part in step 690. For example, the electronic device may determine the position of the target part corresponding to the reference time point, with noise in the position information filtered out, as the future position of the target part corresponding to the target time point. For example, if the predicted velocity distance by the future velocity in the + direction corresponds to n pixels with respect to the position of the target part corresponding to the reference time point (hereinafter, "reference position") and the predicted acceleration distance by the future acceleration in the - direction corresponds to n+1 pixels or more, the future position of the target part predicted using the above mathematical expression 10 is in the - direction from the reference position, and therefore, the direction of the future velocity (i.e., the + direction) and the direction of the future position (i.e., the - direction) with respect to the reference position may be different from each other. Therefore, the electronic device may determine the position of the target part corresponding to the reference time point as the future position of the target part when the user's movement state corresponds to stationary and the predicted acceleration distance exceeds the predicted velocity distance.

[0149] Although Figure 6 illustrates determining whether an electronic device is in a moving state and then determining whether it is stationary, this is not a limitation. For example, the electronic device may determine whether it is in a moving state after determining whether it is stationary.

[0150] FIG. 7 is a diagram illustrating a method for an electronic device to perform independent correction on predicted future position information according to one embodiment of the present disclosure.

[0151] In referring to Fig. 7, any description that overlaps with the description described above in any one of Figs. 1 to 6 may be omitted.

[0152] In step 710, the electronic device may identify (or obtain, determine) weights for independently correcting future position information of the target region. The weights for independent correction may represent the degree of independent correction or the strength of independent correction. In one embodiment of the present disclosure, the values ​​of the weights for independent correction of the target region corresponding to the target time point may be predetermined or set. In one embodiment of the present disclosure, the electronic device may calculate, determine, or obtain weights for independent correction of the target region corresponding to the target time point based on the target time point. For example, the electronic device may calculate weights for independent correction of the target region corresponding to the target time point using the following mathematical expression 11.

[0153] w = min(0.1n,0.5) ... Mathematical Formula 11

[0154] In the above mathematical expression 11, n represents the difference between the target time point (e.g., time point t+n) and the reference time point (e.g., reference time point, time t), and w may represent a weight for independent correction of the target portion corresponding to the target time point.

[0155] In step 720, the electronic device may perform independent correction for the future position of the target portion using weights for independent correction of the target portion corresponding to the target point in time. For example, the electronic device may perform independent correction for the future position of the target portion using the following mathematical expression 12.

[0156] ... Mathematical expression 12

[0157] In the above mathematical expression 12, w represents a weight for independent correction of the target area corresponding to the target time point, represents the future position corresponding to time t+n, predicted based on time t as the reference point, can represent the future position corresponding to time t-1+n, predicted based on time t-1 as the reference time. In the above mathematical expression 12, the right side represents the future position of the target area before independent correction, and the left side can represent the future position of the target area after independent correction.

[0158] FIG. 8 is a diagram illustrating a method for an electronic device to perform dependent correction on predicted future position information according to one embodiment of the present disclosure.

[0159] In referring to Fig. 8, any description that overlaps with the description described above in any one of Figs. 1 to 7 may be omitted.

[0160] In step 810, the electronic device may calculate (or determine, obtain) the center positions of the future positions of the target portion and the future positions of other portions. For example, the electronic device may calculate, determine, or obtain the center positions (e.g., binocular centers) of the future positions of the target portion (e.g., the left eye) corresponding to the target viewpoint and the future positions of other portions (e.g., the right eye) corresponding to the target viewpoint.

[0161] In step 820, the electronic device may determine whether the viewer's movement satisfies the movement condition. For example, the electronic device may determine whether the movement of the target portion and the other portion satisfies the movement condition. For example, the electronic device may determine whether the viewer's movement satisfies the movement condition based on whether the direction of the velocity of the target portion is the same as the direction of the velocity of the other portion and / or whether the magnitude of the velocity of the target portion and the magnitude of the velocity of the other portion are less than (or below) a threshold. For example, if the direction of the velocity of the target portion is the same as the direction of the velocity of the other portion and the magnitude of the velocity of the target portion and the magnitude of the velocity of the other portion are less than the threshold, the electronic device may determine that the viewer's movement satisfies the movement condition. For example, if the direction of the velocity of the target portion is different from the direction of the velocity of the other portion, or the magnitude of the velocity of the target portion or the magnitude of the velocity of the other portion is greater than or equal to the threshold, the electronic device may determine that the viewer's movement does not satisfy the movement condition.

[0162] In one embodiment of the present disclosure, based on determining whether the viewer's movement satisfies a movement condition, the electronic device may calculate or determine a weight (k) for dependent compensation. The weight (k) for dependent compensation may represent a degree of dependent compensation or a strength of dependent compensation.

[0163] Referring to FIG. 8, if the viewer's movement does not satisfy the movement condition, in step 850, the electronic device may determine the weight (k) for dependent compensation as a first value (α). If the viewer's movement satisfies the movement condition, in step 840, the electronic device may determine the weight (k) for dependent compensation as a second value (β). For example, the first value may be greater than the second value.

[0164] In step 860, the electronic device can perform dependent correction on the future position of the target portion based on distance information (e.g., binocular distance information) between the target portion and another portion and a weight (k) for dependent correction. In one embodiment of the present disclosure, the electronic device can use the weight for dependent correction to correct the future position of the target portion and the center position of the future position of the other portion. Based on the distance information between the target portion and another portion and the corrected center position, the electronic device can obtain the future position of the dependently corrected target position and the future position of the dependently corrected other portion. For example, the electronic device can perform dependent correction on the future position of the target portion and the future position of the other portion using the following mathematical expression 13.

[0165]

[0166] ... Mathematical expression 13

[0167] In the above mathematical expression 13, represents the center position of the future position of the target part corresponding to time point t+n predicted based on time point t and the future position of other parts, represents the center position of the future position of the target region corresponding to time point t-1+n predicted based on time point t-1 and the future position of other regions, d represents distance information between the target region and other regions, and k can represent a weight for dependent correction. is the future position of the target area (e.g., left eye) corresponding to time point t+n predicted based on time point t, and represents the dependently corrected future position. is the future position of another part (e.g., right eye) corresponding to time point t+n predicted based on time point t as a reference point, and can represent a dependently corrected future position.

[0168] For example, an electronic device can use the following mathematical expression 14 to dependently compensate the future position of a target portion and the future position of another portion.

[0169]

[0170] ... Mathematical formula 14

[0171] In the above mathematical expression 14, represents the center position of the future position of the target part corresponding to time point t+n predicted based on time point t and the future position of other parts, and d can represent distance information between the target part and other parts. is the future position of the target area (e.g., left eye) corresponding to time point t+n predicted based on time point t, and represents the dependently corrected future position. is the future position of another part (e.g., right eye) corresponding to time point t+n predicted based on time point t as a reference point, and can represent a dependently corrected future position.

[0172] The mathematical formulas described in FIGS. 4 to 8 are examples of operations performed in one embodiment of the present disclosure, and the present disclosure is not limited to the mathematical formulas described in FIGS. 4 to 8. Furthermore, the present disclosure is not limited to the operational sequence of the flowcharts depicted in FIGS. 4 to 8.

[0173] FIG. 9 is a diagram illustrating a method for an electronic device to predict a future visual position of a viewer according to one embodiment of the present disclosure.

[0174] In referring to FIG. 9, any description that overlaps with the description described above in any one of FIGS. 1 to 8 may be omitted.

[0175] For example, step 910 of FIG. 9 may correspond to step 310 of FIG. 3, step 912 of FIG. 9 may correspond to step 312 of FIG. 3, step 916 of FIG. 9 may correspond to step 314 of FIG. 3, step 918 of FIG. 9 may correspond to step 316 of FIG. 3, step 920 of FIG. 9 may correspond to step 318 of FIG. 3, and step 922 of FIG. 9 may correspond to step 230 of FIG. 3. Accordingly, descriptions of steps 910, 912, 916, 918, 920, and 922 of FIG. 9 that overlap with those of FIG. 3 may be omitted. In addition, since the description given below for the left eye in FIG. 9 may be applied to the right eye, descriptions for the right eye may be omitted.

[0176] In one embodiment of the present disclosure, an electronic device may scale a future velocity of a target region when a position change exceeding a threshold occurs. For example, the electronic device may scale the future velocity of the target region when a position change exceeding a threshold occurs at a reference point in time based on accumulated position information of the target region (or, viewer viewing position information). For example, the electronic device may determine scaling for the future velocity based on determining that the viewer's viewing position corresponding to the reference point in time (e.g., the position of the target region, the position of the center of the binoculars) is outside a reference area. For example, the reference area may be determined based on accumulated viewer viewing position information (e.g., the position information of the target region, the position information of the center of the binoculars) and the binocular distance. For example, the reference area may be determined according to the following mathematical expression (15).

[0177] A t = A t-1 * (K-1) + (center position of both eyes) / K

[0178] d = A t ± C* (binocular distance)

[0179] ... Mathematical expression 15

[0180] In the above mathematical expression 15, A t-1 represents the center position of the reference area corresponding to time t-1, K represents a predetermined or set parameter value, the binocular center position represents the center position of the left eye position and the right eye position corresponding to time t, and A t can represent the center position of the reference area corresponding to time point t. In the above mathematical expression 15, C represents a predetermined or set parameter value, the binocular distance represents the difference between the position of the left eye and the position of the right eye corresponding to time point t or the IPD information that is set or stored, and d can represent the distance from the center of the reference area to the boundary position of the reference area.

[0181] In one embodiment of the present disclosure, the electronic device may obtain (or determine, calculate) a scaling value for the future velocity of the target region. For example, referring to FIG. 9 , in step 914, the electronic device may obtain a scaling value for the future velocity of the left eye. In one embodiment of the present disclosure, if the direction of the future velocity of the target region (e.g., the left eye) is outside (or in the outer direction) of the reference region, the electronic device may determine a value less than or equal to 1 as the scaling value for the future velocity of the target region. For example, the electronic device may determine a scaling value for the future velocity less than or equal to 1 using the following mathematical expression 16.

[0182] Scaling value = (A t ± C*(binocular distance)) / binocular center position

[0183] ... Mathematical expression 16

[0184] In the above mathematical expression 16, the center position of both eyes represents the center position of the left eye and the right eye corresponding to the time point t, and A trepresents the center position of the reference area corresponding to time point t, C represents a predetermined or set parameter value, and the binocular distance may represent pre-set or stored IPD information or the difference between the position of the left eye and the position of the right eye corresponding to time point t.

[0185] In one embodiment of the present disclosure, if the direction of the future velocity of the target region (e.g., the left eye) is toward the center of the reference region, the electronic device may determine a value greater than or equal to 1 (or exceeding) as the scaling value for the future velocity. For example, the electronic device may determine a scaling value for the future velocity greater than or equal to 1 using the following mathematical expression 17.

[0186] Scaling value = binocular center position / (A t ± C*(binocular distance))

[0187] ... Mathematical expression 17

[0188] In the above mathematical expression 17, the center position of both eyes represents the center position of the left eye and the right eye corresponding to the time point t, and A t represents the center position of the reference area corresponding to time point t, C represents a predetermined or set parameter value, and the binocular distance may represent pre-set or stored IPD information or the difference between the position of the left eye and the position of the right eye corresponding to time point t.

[0189] In one embodiment of the present disclosure, an electronic device can predict future acceleration of a target portion based on a motion state of a viewer. The motion state of the viewer, which serves as a basis for predicting the future acceleration of the target portion, may be associated with a scaling value for the future velocity of the target portion. For example, a scaling value for a future velocity that is less than or equal to 1 may correspond to a stationary state, and a scaling value for a future velocity that is greater than or equal to 1 may correspond to a moving state. For example, if the direction of the future velocity of the target portion is outward (or in the outer direction) of a reference area, the motion state of the viewer may be determined to be stationary, and if the direction of the future velocity of the target portion is toward the center of the reference area, the motion state of the viewer may be determined to be moving.

[0190] In step 916, the electronic device can predict the future acceleration of the left eye based on the viewer's movement state. For example, if the scaling value for the future velocity acquired in step 914 is less than or equal to 1 (or less than), the electronic device can determine the viewer's movement state as a stationary state and predict (or determine, acquire) the future acceleration of the left eye. For example, if the scaling value for the future velocity acquired in step 914 is greater than or equal to 1 (or greater than), the electronic device can determine the viewer's movement state as a moving state and predict (or determine) the future acceleration of the left eye.

[0191] At step 918, the electronic device can predict the future position of the left eye based on the scaling value for the future velocity, the future velocity of the left eye, and the future acceleration of the left eye. For example, the electronic device can predict the future position of the left eye using the following mathematical expression (18).

[0192] ... Mathematical expression 18

[0193] In the above mathematical expression 18, represents the location information of the target area corresponding to the time point t (e.g., the reference time point) with the noise filtered out, and n represents the difference between the target time point (e.g., time point t+n) and the reference time point (e.g., time point t). represents a scaling function for the future velocity of the target region, represents the future velocity (e.g. predicted velocity) of the target region corresponding to a future time point (e.g. time point t+1), can represent the future acceleration (e.g., predicted acceleration) of the target area corresponding to time t+n predicted based on time t as a reference time.

[0194] Although FIG. 9 illustrates an example of an electronic device obtaining the future positions of the centers of the two eyes by performing binocular correction, the present invention is not limited thereto. For example, as illustrated in FIG. 3, the electronic device may obtain the future positions of the left and right eyes by performing binocular correction. For example, the electronic device may obtain the future positions of the centers of the two eyes by performing binocular correction, and may obtain (or determine, calculate) the future positions of the left and / or right eyes using the future positions of the centers of the two eyes and IPD information.

[0195] FIG. 10 is a diagram illustrating a method for an electronic device to predict a future visual position of a viewer according to one embodiment of the present disclosure.

[0196] FIG. 10 illustrates an example in which an electronic device predicts a future visual position of a viewer using a prediction result corresponding to time t+n and a prediction result corresponding to time t+n+1 in one embodiment of the present disclosure.

[0197] In referring to Fig. 10, descriptions that overlap with the descriptions given above in any one of Figs. 1 to 9 may be omitted. For example, step 1010 of Fig. 10 may correspond to step 910 of Fig. 9, step 1012 of Fig. 10 may correspond to step 912 of Fig. 9, step 1014 of Fig. 10 may correspond to step 914 of Fig. 9, step 1016 of Fig. 10 may correspond to step 916 of Fig. 9, step 1018 of Fig. 10 may correspond to steps 918 and 920 of Fig. 9, and step 1020 of Fig. 10 may correspond to step 922 of Fig. 9. Accordingly, descriptions that overlap with those in Fig. 3 or Fig. 9 in steps 1010 to 1022 of Fig. 10 may be omitted.

[0198] In one embodiment of the present disclosure, the electronic device may perform monocular correction for the future position of the subordinately corrected target region. Referring to FIG. 10 , in step 1022, the electronic device may perform secondary monocular correction for the future position of the binocularly corrected left eye. For example, the electronic device may perform secondary monocular correction for the future position of the binocularly corrected left eye using weights that are the same as or different from the primary monocular correction of step 1018.

[0199] In one embodiment of the present disclosure, the electronic device may use a prediction result (1030) corresponding to the target time point and a prediction result (1040) corresponding to a time point subsequent to the target time point (e.g., a next time point) to improve the prediction accuracy of the viewer's future visual position for (or at) the target time point. For example, the electronic device may predict the viewer's future visual position (e.g., the future positions of both eyes) for the target time point based on the future position of the target part corresponding to the target time point and the future position of the target part corresponding to a time point subsequent to the target time point. For example, the electronic device may sequentially, parallelly, or independently perform the operations performed for the target time point to obtain the prediction result (1030) corresponding to the target time point, thereby obtaining the prediction result (1040) corresponding to a time point subsequent to the target time point.

[0200] The prediction result (1030) corresponding to the target time point may include the future position of the target part corresponding to the target time point. The prediction result (1040) corresponding to a time point after the target time point may include the future position of the target part corresponding to a time point after the target time point. As illustrated in FIG. 10, the prediction result (1030) corresponding to the target time point (time point t+n) may include the future position of the left eye corresponding to the target time point and the future position of the right eye corresponding to the target time point. Similarly, the prediction result (1040) corresponding to a time point after the target time point (t+n+1) may include the future position of the left eye corresponding to a time point after the target time point and the future position of the right eye corresponding to a time point after the target time point.

[0201] Referring to FIG. 10, the electronic device can predict the future positions of the left and right eyes at a time point t+n as a first target time point, and can predict the future positions of the left and right eyes at a time point t+n+1 as a second target time point. The electronic device can calculate (or determine, obtain) the future viewing positions of the viewer for the first target time point based on the future positions of the left eye and the right eye corresponding to the first target time point and the future positions of the left eye and the right eye corresponding to the second target time point. For example, the electronic device can calculate an average (or weighted average) position of the future positions of the left eye and the right eye corresponding to the first target time point and the future positions of the left eye and the right eye corresponding to the second target time point as the future positions of the centers of the two eyes for the first target time point (1050). The electronic device can determine the positions of the two eyes for the first target time point as the future viewing positions of the viewer based on the calculated centers of the two eyes for the first target time point and IPD information (e.g., the viewer's IPD information).

[0202] FIG. 11 is a diagram illustrating a method for an electronic device to predict a future visual position of a viewer according to one embodiment of the present disclosure.

[0203] FIG. 11 is a diagram showing an electronic device predicting a future visual position of a viewer using a prediction result corresponding to time point t+n, a prediction result corresponding to time point t+n+0.5, and a prediction result corresponding to time point t+n+1 in one embodiment of the present disclosure.

[0204] In referring to Fig. 11, any description that overlaps with the description described above in any one of Figs. 1 to 10 may be omitted. For example, 1110 of Fig. 11 may correspond to 1030 of Fig. 10, 1120 of Fig. 11 may correspond to 1040 of Fig. 10, and 1140 of Fig. 11 may correspond to 1050 of Fig. 10.

[0205] In one embodiment of the present disclosure, the electronic device may use a prediction result (1110) corresponding to the target time point, a prediction result (1120) corresponding to a time point after the target time point (e.g., the next time point), and a prediction result (1130) corresponding to an intermediate time point between the target time point and the time point after the target time point to improve the prediction accuracy of the future visual position of the viewer with respect to (or at) the target time point. For example, the electronic device may predict the future visual position of the viewer with respect to the target time point (e.g., the future positions of both eyes) based on the future position of the target part corresponding to the target time point, the future position of the target part corresponding to a time point after the target time point, and the future position of the target part corresponding to an intermediate time point.

[0206] The prediction result (1110) corresponding to the target time point may include the future position of the target part corresponding to the target time point. The prediction result (1120) corresponding to the time point after the target time point may include the future position of the target part corresponding to the time point after the target time point. The prediction result (1130) corresponding to the intermediate time point may include the future position of the target part corresponding to the intermediate time point. As illustrated in FIG. 11, the prediction result (1110) corresponding to the target time point (time point t+n) may include the future position of the left eye corresponding to the target time point and the future position of the right eye corresponding to the target time point. Similarly, the prediction result (1120) corresponding to the time point after the target time point (t+n+1) may include the future position of the left eye corresponding to the time point after the target time point and the future position of the right eye corresponding to the time point after the target time point. Similarly, the prediction result (1130) corresponding to the intermediate time point (t+n+0.5) may include the future position of the left eye corresponding to the intermediate time point and the future position of the right eye corresponding to the intermediate time point.

[0207] In one embodiment of the present disclosure, the electronic device may obtain a prediction result (1130) corresponding to the intermediate time point by performing at least some of the operations performed for the target time point sequentially, in parallel, or independently, also for the intermediate time point. In one embodiment of the present disclosure, the electronic device may obtain a prediction result (1130) corresponding to the intermediate time point by performing some different operations for the intermediate time point from the operations performed for the target time point. For example, in order to obtain the prediction result (1130) corresponding to the intermediate time point, the electronic device may obtain (or calculate) the future acceleration of the target part (e.g., the left eye and the right eye) corresponding to the intermediate time point by using the following mathematical expression 19.

[0208]

[0209] ... Mathematical Formula 19

[0210] In the above mathematical expression 19, m represents the average acceleration, r is a parameter whose value is determined based on the viewer's movement status, can represent the nonlinear predicted acceleration of the target region corresponding to the time point t+n+0.5 (e.g., the intermediate time point) predicted with time point t as the reference time point. γ can represent the degree of filtering for acceleration noise. For example, γ can be determined or set experimentally or empirically. can represent the future acceleration of the target area corresponding to the time point t-1+n+0.5 predicted based on the time point t-1. For example, can represent the future acceleration of the target area, with the noise filtered out for acceleration. It can represent the future acceleration of the target part corresponding to time t+n+0.5 predicted with time t as the reference time, obtained by filtering out the noise of acceleration from the nonlinear predicted acceleration of the target part corresponding to time t+n+0.5 predicted with time t as the reference time. can represent an interpolation function.

[0211] Referring to FIG. 11, the electronic device can predict the future positions of the left and right eyes at a time point t+n as a first target time point, predict the future positions of the left and right eyes at a time point t+n+1 as a second target time point, and predict the future positions of the left and right eyes at a time point t+n+0.5 as a third target time point. The electronic device can calculate (or determine, obtain) the future viewing position of the viewer for the first target time point based on the future positions of the left eye and the right eye corresponding to the first target time point, the future positions of the left eye and the right eye corresponding to the second target time point, and the future positions of the left eye and the right eye corresponding to the third target time point. For example, the electronic device may calculate an average (or weighted average) position of the future positions of the left eye and the right eye corresponding to the first target viewpoint, the future positions of the left eye and the right eye corresponding to the second target viewpoint, and the future positions of the left eye and the right eye corresponding to the third target viewpoint as the future position of the center of the two eyes for the first target viewpoint (1140). The electronic device may determine the positions of the two eyes for the first target viewpoint, which are obtained based on the calculated center positions of the two eyes for the first target viewpoint and the IPD information, as the viewer's future viewing positions.

[0212] FIG. 12 is a diagram illustrating a method for an electronic device to predict a future visual position of a viewer according to one embodiment of the present disclosure.

[0213] In referring to Fig. 12, any description that overlaps with the description described above in any one of Figs. 1 to 11 may be omitted.

[0214] FIG. 12 illustrates an example of predicting the future position of the binocular centers as target regions according to one embodiment of the present disclosure, and predicting the future visual position of a viewer based on the predicted future position of the binocular centers. Steps 1210 to 1220 of FIG. 12 may correspond to steps 1010 to 1022 of FIG. 10. For example, in steps 1210 to 1220 of FIG. 12, the electronic device may perform at least some of the operations performed by the electronic device for the left eye in the above-described descriptions of FIG. 3, FIG. 9, or FIG. 10, for the binocular centers. For example, as illustrated in FIG. 12, the electronic device may not perform dependent correction (e.g., binocular correction) for the binocular centers, unlike the left eye or the right eye, but is not limited thereto.

[0215] In one embodiment of the present disclosure, the electronic device may use a prediction result (1230) corresponding to the target time point, a prediction result (1240) corresponding to a time point after the target time point (e.g., the next time point), and a prediction result (1250) corresponding to an intermediate time point between the target time point and the time point after the target time point to improve the prediction accuracy of the future visual position of the viewer with respect to (or at) the target time point. For example, the electronic device may predict the future visual position of the viewer with respect to the target time point (e.g., the future positions of both eyes) based on the future position of the target part corresponding to the target time point, the future position of the target part corresponding to a time point after the target time point, and the future position of the target part corresponding to an intermediate time point.

[0216] The prediction result (1230) corresponding to the target time point may include the future position of the target part corresponding to the target time point. The prediction result (1240) corresponding to a time point after the target time point may include the future position of the target part corresponding to a time point after the target time point. The prediction result (1250) corresponding to the intermediate time point may include the future position of the target part corresponding to the intermediate time point. As illustrated in FIG. 12, the prediction result (1230) corresponding to the target time point may include the future position of the left eye, the future position of the right eye, and the future position of the center of both eyes corresponding to the target time point. The prediction result (1240) corresponding to a time point after the target time point may include the future position of the left eye, the future position of the right eye, and the future position of the center of both eyes corresponding to a time point after the target time point. The prediction result (1250) corresponding to the intermediate time point may include the future position of the left eye, the future position of the right eye, and the future position of the center of both eyes corresponding to the intermediate time point.

[0217] Referring to FIG. 12, the electronic device can predict the future positions of the left eye, the right eye, and the binocular center, respectively, at a time point t+n as a first target time point, predict the future positions of the left eye, the right eye, and the binocular center, respectively, at a time point t+n+1 as a second target time point, and predict the future positions of the left eye, the right eye, and the binocular center, respectively, at a time point t+n+0.5 as a third target time point. The electronic device can calculate (or determine, obtain) the future viewing position of the viewer for the first target time point based on the future position of the left eye corresponding to the first target time point, the future position of the right eye corresponding to the first target time point, the future position of the binocular center corresponding to the first target time point, the future position of the left eye corresponding to the second target time point, the future position of the binocular center corresponding to the second target time point, the future position of the left eye corresponding to the third target time point, the future position of the right eye corresponding to the third target time point, and the future position of the binocular center corresponding to the third target time point. For example, the electronic device may calculate an average (or weighted average) position of the future position of the left eye corresponding to the first target viewpoint, the future position of the right eye corresponding to the first target viewpoint, the future position of the binocular center corresponding to the first target viewpoint, the future position of the left eye corresponding to the second target viewpoint, the future position of the right eye corresponding to the second target viewpoint, the future position of the binocular center corresponding to the second target viewpoint, the future position of the left eye corresponding to the third target viewpoint, the future position of the right eye corresponding to the third target viewpoint, and the future position of the binocular center corresponding to the third target viewpoint as the future position of the binocular center for the first target viewpoint (1260). The electronic device may determine the positions of the binocular centers for the first target viewpoint, which are obtained based on the calculated positions of the binocular centers for the first target viewpoint and the IPD information, as the viewer's future viewing positions.

[0218] FIG. 13 is a diagram illustrating an electronic device additionally filtering noise from future visual position information of a viewer, according to one embodiment of the present disclosure.

[0219] In referring to Fig. 13, any description that overlaps with the description described above in any one of Figs. 1 to 12 may be omitted.

[0220] In one embodiment of the present disclosure, the electronic device may additionally filter noise from the viewer's future visual position information before rendering (250) an output image using the viewer's future visual position information predicted for the target viewpoint. For example, the electronic device may additionally filter noise before rendering the output image to remove noise that may occur when predicting a distant future (e.g., more than n frames in the future, where n is a natural number). For example, the electronic device may additionally filter noise from the final prediction information (e.g., the viewer's future visual position) obtained by one or more of the embodiments described above in any one of FIGS. 2 to 12. For example, the electronic device may additionally filter noise from the future position of the target part corresponding to (or with respect to) the target viewpoint, the future position of the binocular center corresponding to (or with respect to) the target viewpoint, or the future position of the binoculars corresponding to (or with respect to) the target viewpoint.

[0221] For example, the electronic device can additionally filter out noise from the predicted future visual position information of the viewer for the target time point using the mathematical expression 20 below.

[0222] s t = αx t +(1-α)(s t-1 + b t-1 )

[0223] b t = β(s t -s t-1 )+(1- β)(bt-1 + k t-1 )

[0224] kt=γ(b t -b t-1 )+(1-γ)k t-1

[0225] ... Mathematical expression 20

[0226] In the above mathematical expression 20, x t can represent the predicted future visual positions of the viewer with respect to the target time point (e.g., future positions of the binoculars, future positions of the binocular centers). The values ​​of s0, b0, and k0 can be predetermined or set. The values ​​of α, β, and γ can be predetermined or set.

[0227] Referring to FIG. 13, in step 1310, the electronic device may predict a future visual position of a viewer with respect to (or at) a target viewpoint, and in step 1320, the electronic device may additionally filter out noise from the prediction result. For example, the prediction result from which noise is filtered in step 1320 may include at least one of a future visual position of a viewer predicted by one or more embodiments described above in at least one of FIGS. 2 to 12, a future position of a left eye corresponding to (or at, with respect to) the target viewpoint, a future position of a right eye corresponding to (or at, with respect to) the target viewpoint, a future position of the center of both eyes corresponding to (or at, with respect to) the target viewpoint, or a future position of a target part corresponding to (or at, with respect to) the target viewpoint.

[0228] FIG. 14 is a diagram showing an operation method of an electronic device according to one embodiment of the present disclosure.

[0229] In referring to Fig. 14, any description that overlaps with the description described above in any one of Figs. 1 to 13 may be omitted.

[0230] FIG. 14 may illustrate an example of an operating method (1400) of an electronic device. In one embodiment of the present disclosure, an electronic device performing the method (1400) may include a display device. Referring to FIG. 14 , the method (1400) according to one embodiment of the present disclosure may include steps 1410 to 1470. In one embodiment of the present disclosure, steps 1410 to 1470 of the method (1400) may be executed by at least one processor included in the electronic device. The method (1400) is not limited to that illustrated in FIG. 14 , and in one or more embodiments, steps not illustrated in FIG. 14 may be further included, or some steps may be omitted.

[0231] In step 1410, the electronic device can obtain, from an image including a viewer's face area input through a camera, location information of a target area corresponding to a past time point and location information of a target area corresponding to a reference time point. In one embodiment of the present disclosure, the electronic device can obtain, from an image including a viewer's face area, raw location information of a target area corresponding to a reference time point, and, based on the location information of the target area corresponding to the past time point and the raw location information of the target area, obtain location information of the target area corresponding to the reference time point.

[0232] In step 1420, the electronic device may obtain position change information of the target portion corresponding to the reference point in time based on the position information of the target portion corresponding to the past point in time and the position information of the target portion corresponding to the reference point in time. In one embodiment of the present disclosure, the electronic device may obtain raw information on position change of the target portion corresponding to the reference point in time based on the position information of the target portion corresponding to the past point in time and the position information of the target portion corresponding to the reference point in time, and may obtain position change information of the target portion corresponding to the reference point in time based on a filtering strength determined based on the degree of movement of the target portion, the position change information of the target portion corresponding to the past point in time, and the raw information on the position change of the target portion.

[0233] In step 1430, the electronic device can predict the future speed of the target portion based on the position change information of the target portion corresponding to the reference point in time and the position change information of the target portion corresponding to the past point in time.

[0234] In step 1440, the electronic device can obtain information on change in speed of the target portion corresponding to the reference point in time based on information on change in position of the target portion corresponding to the reference point in time and information on change in position of the target portion corresponding to the past point in time.

[0235] In step 1450, the electronic device can predict future acceleration of the target portion based on velocity change information of the target portion corresponding to the reference time point and velocity change information of the target portion corresponding to the past time point. In one embodiment of the present disclosure, the electronic device can obtain nonlinear predicted acceleration information of the target portion corresponding to the target time point based on the velocity change information of the target portion corresponding to the reference time point, the velocity change information of the target portion corresponding to the past time point, and a parameter value determined based on the viewer's movement state, and can predict future acceleration of the target portion based on the nonlinear predicted acceleration information and the acceleration of the target portion predicted for a first time point prior to the target time point.

[0236] In step 1460, the electronic device can predict the future positions of both eyes corresponding to the target time point based on the future velocity and the future acceleration. In one embodiment of the present disclosure, the electronic device can predict the first future position of the target part corresponding to the target time point based on the future velocity and the future acceleration, and can obtain the second future position of the target part corresponding to the target time point based on the first future position of the target part corresponding to the target time point and the future position of the target part corresponding to a first time point before the target time point. The electronic device can predict the future positions of both eyes corresponding to the target time point based on the second future position of the target part. In one embodiment of the present disclosure, the electronic device can predict the first future position of the center of both eyes corresponding to the target time point based on the future velocity and the future acceleration, and can predict the future positions of both eyes corresponding to the target time point based on the first future position of the center of both eyes corresponding to the target time point and IPD information.

[0237] In one embodiment of the present disclosure, the electronic device can obtain a scaling value for the future velocity based on a direction of the future velocity, and predict a future position of the binoculars corresponding to a target time point based on the future velocity, future acceleration, and the scaling value for the future velocity.

[0238] At step 1470, the electronic device can output an image based on the future positions of the two eyes corresponding to the target point in time.

[0239] FIG. 15 is a drawing showing an electronic device according to one embodiment of the present disclosure.

[0240] Referring to FIG. 15, the electronic device (1500) may include, as a display device, an outdoor monitor, a gaming monitor, a television, an electronic picture frame, a laptop, a desktop computer, a wearable device, etc. The electronic device (1500) illustrated in FIG. 15 may include, as an electronic device that generates an output image and provides it to a display device or provides a prediction result to a display device, a set-top box, a console device, a video player device, etc. In one embodiment of the present disclosure, the electronic device (1500) may include, but is not limited to, at least one processor (1510) and a memory (1520).

[0241] The processor (1510) is electrically connected to components included in the electronic device (1500) and can execute operations or data processing related to control and / or communication of the components included in the electronic device (1500). In one embodiment of the present disclosure, the processor (1510) can load and process requests, commands, or data received from at least one of the other components into a memory and store the processing result data in the memory. According to one or more embodiments, the processor (1510) may include at least one of a general-purpose processor such as a central processing unit (CPU), an application processor (AP), a digital signal processor (DSP), a graphics-only processor such as a graphics processing unit (GPU), a vision processing unit (VPU), or an artificial intelligence-only processor such as a neural processing unit (NPU).

[0242] The processor (1510) may process input data or control other components to process the input data according to data, operation rules, algorithms, methods, or models stored in the memory (1520). The processor (1510) may perform operations of predefined operation rules, algorithms, methods, or models stored in the memory (1520) using the input data.

[0243] The memory (1520) is electrically connected to the processor (1510) and can store one or more modules, algorithms, operation rules, models, programs, commands, or data related to the operations of the components included in the electronic device (1500). For example, the memory (1520) can store one or more modules, algorithms, operation rules, models, programs, commands, or data for processing and controlling the processor (1510). The memory (1520) can include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk, but is not limited thereto.

[0244] In one embodiment of the present disclosure, the memory (1520) may store data and / or information identified, acquired, generated, or determined by the electronic device (1500). For example, the memory (1520) may store data and / or information identified, acquired, generated, or determined by the electronic device (1500) in a compressed form.

[0245] In one embodiment of the present disclosure, an electronic device (1500) may include a module that performs (or is used to perform) at least one operation. Some modules of the electronic device (1500) that perform at least one operation may be composed of multiple sub-modules or may constitute a single module.

[0246] Some modules that perform at least one operation of the electronic device (1500) may be implemented as hardware modules, software modules, and / or a combination thereof. The software modules included in the electronic device (1500) may be included in the memory (1520). In one embodiment of the present disclosure, the modules included in the memory (1520) may be executed by the processor (1510) to perform operations. For example, the modules (i.e., software modules) included in the memory (1520) may include programs, models, or algorithms that are executed according to the control or instructions of the processor (1510) and are configured to perform operations that derive output data for input data.

[0247] The electronic device (1500) may include more components than those illustrated in FIG. 15. In one embodiment of the present disclosure, the electronic device (1500) may further include a communication interface (or communication module) for communicating with an external device. In one embodiment of the present disclosure, the electronic device (1500) may further include an input device, an output device, and / or an input / output interface. For example, the input device of the electronic device (1500) may include a camera as an image input module. For example, the output device of the electronic device (1500) may include a display as an image output module. For example, the output device of the electronic device (1500) may include a lenticular lens.

[0248] In the present disclosure, overlapping descriptions in FIGS. 1 to 15 may be omitted, and one or more embodiments described above in FIGS. 1 to 15 may be applied / implemented in combination with each other. In the present disclosure, an operation described as being performed by a module may be executed / performed by a device in which the module is included or stored, or may be executed / performed by the control of at least one processor of the device in which the module is included. In the present disclosure, an operation described as being performed by a device may be executed / performed by a module included or stored in the device, or may be performed by the control of at least one processor of the device using a module included or stored in the device.

[0249] In one embodiment of the present disclosure, a method of operating an electronic device may include a step of obtaining, by the electronic device, position information of a target region corresponding to a past time point and position information of the target region corresponding to a reference time point from an image including a viewer's face region input through a camera. In one embodiment of the present disclosure, the method of operating an electronic device may include a step of obtaining, by the electronic device, position change information of the target region corresponding to the reference time point based on the position information of the target region corresponding to the past time point and the position information of the target region corresponding to the reference time point. In one embodiment of the present disclosure, the method of operating an electronic device may include a step of predicting, by the electronic device, a future velocity of the target region based on the position change information of the target region corresponding to the reference time point and the position change information of the target region corresponding to the past time point. In one embodiment of the present disclosure, the method of operating an electronic device may include a step of obtaining, by the electronic device, velocity change information of the target region corresponding to the reference time point based on the position change information of the target region corresponding to the reference time point and the position change information of the target region corresponding to the past time point. In one embodiment of the present disclosure, a method of operating an electronic device may include a step of predicting, by the electronic device, a future acceleration of the target portion based on velocity change information of the target portion corresponding to the reference time point and velocity change information of the target portion corresponding to the past time point. In one embodiment of the present disclosure, a method of operating an electronic device may include a step of predicting, by the electronic device, a future position of both eyes corresponding to the target time point based on the future velocity and the future acceleration. In one embodiment of the present disclosure, a method of operating an electronic device may include a step of outputting, by the electronic device, an image based on the future positions of both eyes corresponding to the target time point.

[0250] In one embodiment of the present disclosure, the step of predicting future positions of both eyes corresponding to the target time point based on the future velocity and the future acceleration may include a step of predicting a first future position of the target portion corresponding to the target time point based on the future velocity and the future acceleration. In one embodiment of the present disclosure, the step of predicting future positions of both eyes corresponding to the target time point based on the future velocity and the future acceleration may include a step of obtaining a second future position of the target portion corresponding to the target time point based on the first future position of the target portion corresponding to the target time point and the future position of the target portion corresponding to a first time point prior to the target time point. In one embodiment of the present disclosure, the step of predicting future positions of both eyes corresponding to the target time point based on the future velocity and the future acceleration may include a step of predicting future positions of both eyes corresponding to the target time point based on the second future position of the target portion.

[0251] In one embodiment of the present disclosure, the step of predicting the future positions of both eyes corresponding to the target time point based on the future velocity and the future acceleration may include the step of predicting the first future positions of the centers of both eyes corresponding to the target time point based on the future velocity and the future acceleration. In one embodiment of the present disclosure, the step of predicting the future positions of both eyes corresponding to the target time point based on the future velocity and the future acceleration may include the step of predicting the future positions of both eyes corresponding to the target time point based on the first future positions of the centers of both eyes corresponding to the target time point and inter-pupillary distance (IPD) information.

[0252] In one embodiment of the present disclosure, the step of obtaining, from an image including a viewer face region input through the camera, position information of a target region corresponding to a past time point and position information of the target region corresponding to a reference time point may include a step of obtaining, from an image including the viewer face region, raw position information of the target region corresponding to the reference time point. In one embodiment of the present disclosure, the step of obtaining, from an image including a viewer face region input through the camera, position information of the target region corresponding to a past time point and position information of the target region corresponding to the reference time point may include a step of obtaining position information of the target region corresponding to the reference time point based on the position information of the target region corresponding to the past time point and the raw position information of the target region.

[0253] In one embodiment of the present disclosure, the step of obtaining position change information of the target portion corresponding to the reference point in time based on the position information of the target portion corresponding to the past point in time and the position information of the target portion corresponding to the reference point in time may include the step of obtaining raw information about the position change of the target portion corresponding to the reference point in time based on the position information of the target portion corresponding to the past point in time and the position information of the target portion corresponding to the reference point in time. In one embodiment of the present disclosure, the step of obtaining position change information of the target portion corresponding to the reference point in time based on the position information of the target portion corresponding to the past point in time and the position information of the target portion corresponding to the reference point in time may include the step of obtaining position change information of the target portion corresponding to the reference point in time based on a filtering strength determined based on a degree of movement of the target portion, the position change information of the target portion corresponding to the past point in time, and the raw information about the position change of the target portion.

[0254] In one embodiment of the present disclosure, the step of predicting future acceleration of the target portion based on the velocity change information of the target portion corresponding to the reference time point and the velocity change information of the target portion corresponding to the past time point may include the step of obtaining nonlinear predicted acceleration information of the target portion corresponding to the target time point based on the velocity change information of the target portion corresponding to the reference time point, the velocity change information of the target portion corresponding to the past time point, and a parameter value determined based on the viewer's movement state. In one embodiment of the present disclosure, the step of predicting future acceleration of the target portion based on the velocity change information of the target portion corresponding to the reference time point and the velocity change information of the target portion corresponding to the past time point may include the step of predicting future acceleration of the target portion based on the nonlinear predicted acceleration information and the acceleration of the target portion predicted for a first time point prior to the target time point. In one embodiment of the present disclosure, the future acceleration of the target portion may correspond to the target time point.

[0255] In one embodiment of the present disclosure, the method may include a step of obtaining a scaling value for the future velocity based on the direction of the future velocity. In one embodiment of the present disclosure, the step of predicting the future positions of the two eyes corresponding to the target time point based on the future velocity and the future acceleration may include a step of predicting the future positions of the two eyes corresponding to the target time point based on the future velocity, the future acceleration, and the scaling value for the future velocity.

[0256] In one embodiment of the present disclosure, the method may include a step of acquiring future positions of both eyes corresponding to a second time point after the target time point. In one embodiment of the present disclosure, the step of outputting an image based on the future positions of both eyes corresponding to the target time point may include a step of predicting future positions of both eyes with respect to the target time point based on the future positions of both eyes corresponding to the target time point and the future positions of both eyes corresponding to the second time point. In one embodiment of the present disclosure, the step of outputting an image based on the future positions of both eyes corresponding to the target time point may include a step of outputting the image based on the future positions of both eyes with respect to the target time point.

[0257] In one embodiment of the present disclosure, the method may include a step of acquiring future positions of both eyes corresponding to a third time point between the target time point and the second time point. In one embodiment of the present disclosure, the step of predicting future positions of both eyes for the target time point based on the future positions of both eyes corresponding to the target time point and the future positions of both eyes corresponding to the second time point may include a step of predicting future positions of both eyes for the target time point based on the future positions of both eyes corresponding to the target time point, the future positions of both eyes corresponding to the second time point, and the future positions of both eyes corresponding to the third time point.

[0258] In one embodiment of the present disclosure, the method may include a step of predicting a future position of the binocular center corresponding to the target viewpoint based on a future acceleration of the binocular center and a future velocity of the binocular center. In one embodiment of the present disclosure, the target region may include at least one of the left eye or the right eye. In one embodiment of the present disclosure, the step of outputting an image based on the future positions of the binocular center corresponding to the target viewpoint may include a step of predicting a future position of the binocular center for the target viewpoint based on the future positions of the binocular center corresponding to the target viewpoint and the future positions of the binocular center corresponding to the target viewpoint. In one embodiment of the present disclosure, the step of outputting the image may include a step of outputting the image based on the future positions of the binocular center for the target viewpoint.

[0259] In one embodiment of the present disclosure, the step of predicting the future positions of both eyes for the target viewpoint based on the future positions of both eyes corresponding to the target viewpoint and the future positions of the centers of both eyes corresponding to the target viewpoint may include the step of predicting the future positions of the centers of both eyes for the target viewpoint based on the future positions of both eyes corresponding to the target viewpoint and the future positions of the centers of both eyes corresponding to the target viewpoint. In one embodiment of the present disclosure, the step of predicting the future positions of both eyes for the target viewpoint based on the future positions of both eyes corresponding to the target viewpoint and the future positions of the centers of both eyes corresponding to the target viewpoint may include the step of filtering out noise from the future positions of the centers of both eyes for the target viewpoint. In one embodiment of the present disclosure, the step of predicting the future positions of both eyes for the target viewpoint based on the future positions of both eyes corresponding to the target viewpoint and the future positions of the centers of both eyes corresponding to the target viewpoint may include the step of predicting the future positions of both eyes for the target viewpoint based on the future positions of the centers of both eyes from which the noise is filtered.

[0260] In one embodiment of the present disclosure, one or more computer-readable storage media may be provided that store one or more computer programs including computer-executable instructions that, when individually or collectively executed by one or more processors of an electronic device, cause the electronic device to perform an operation. In one embodiment of the present disclosure, the operation may include obtaining, by the electronic device, from an image including a viewer's face region input through a camera, position information of a target region corresponding to a past time point and position information of the target region corresponding to a reference time point. In one embodiment of the present disclosure, the operation may include obtaining, by the electronic device, position change information of the target region corresponding to the reference time point based on the position information of the target region corresponding to the past time point and the position information of the target region corresponding to the reference time point. In one embodiment of the present disclosure, the operation may include predicting, by the electronic device, a future velocity of the target region based on the position change information of the target region corresponding to the reference time point and the position change information of the target region corresponding to the past time point. In one embodiment of the present disclosure, the operation may include obtaining, by the electronic device, velocity change information of the target portion corresponding to the reference point in time based on position change information of the target portion corresponding to the reference point in time and position change information of the target portion corresponding to the past point in time. In one embodiment of the present disclosure, the operation may include predicting, by the electronic device, future acceleration of the target portion based on velocity change information of the target portion corresponding to the reference point in time and velocity change information of the target portion corresponding to the past point in time.In one embodiment of the present disclosure, the operation may include predicting, by the electronic device, future positions of both eyes corresponding to a target time point based on the future velocity and the future acceleration. In one embodiment of the present disclosure, the operation may include outputting, by the electronic device, an image based on the future positions of both eyes corresponding to the target time point. In one embodiment of the present disclosure, the electronic device may include a memory storing one or more computer programs and one or more processors communicatively connected to the memory. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain, from an image including a viewer's face area input through a camera, position information of a target region corresponding to a past time point and position information of the target region corresponding to a reference time point. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain position change information of the target portion corresponding to the reference point in time based on position information of the target portion corresponding to the past point in time and position information of the target portion corresponding to the reference point in time. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict a future speed of the target portion based on position change information of the target portion corresponding to the reference point in time and position change information of the target portion corresponding to the past point in time.In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that cause the electronic device, when individually or collectively executed by the one or more processors, to obtain velocity change information of the target portion corresponding to the reference time point based on position change information of the target portion corresponding to the reference time point and position change information of the target portion corresponding to the past time point. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that cause the electronic device, when individually or collectively executed by the one or more processors, to predict future acceleration of the target portion based on velocity change information of the target portion corresponding to the reference time point and velocity change information of the target portion corresponding to the past time point. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that cause the electronic device, when individually or collectively executed by the one or more processors, to predict future positions of both eyes corresponding to the target time point based on the future velocity and the future acceleration. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to output an image based on the future positions of the two eyes corresponding to the target time point.

[0261] In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict a first future position of the target portion corresponding to the target time point based on the future velocity and the future acceleration. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain a second future position of the target portion corresponding to the target time point based on the first future position of the target portion corresponding to the target time point and the future position of the target portion corresponding to a first time point prior to the target time point. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict a future position of both eyes corresponding to the target time point based on the second future position of the target portion.

[0262] In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict a first future position of the center of the binoculars corresponding to the target time point based on the future velocity and the future acceleration. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict a future position of the binoculars corresponding to the target time point based on the first future position of the center of the binoculars corresponding to the target time point and inter-pupillary distance (IPD) information.

[0263] In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain raw position information of the target region corresponding to the reference time point from an image including the viewer's face area. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain position information of the target region corresponding to the reference time point based on the position information of the target region corresponding to the past time point and the raw position information of the target region.

[0264] In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain raw information about a change in the position of the target portion corresponding to the reference point in time based on the position information of the target portion corresponding to the past point in time and the position information of the target portion corresponding to the reference point in time. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain position change information of the target portion corresponding to the reference point in time based on a filtering strength determined based on a degree of movement of the target portion, position change information of the target portion corresponding to the past point in time, and raw information about the position change of the target portion.

[0265] In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain nonlinear predicted acceleration information of the target portion corresponding to the target time point based on velocity change information of the target portion corresponding to the reference time point, velocity change information of the target portion corresponding to the past time point, and parameter values ​​determined based on the viewer's movement state. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict future acceleration of the target portion based on the nonlinear predicted acceleration information and the acceleration of the target portion predicted for a first time point prior to the target time point. In one embodiment of the present disclosure, the future acceleration of the target portion may correspond to the target time point.

[0266] In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain a scaling value for the future velocity based on a direction of the future velocity. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict a future position of both eyes corresponding to the target time point based on the future velocity, the future acceleration, and the scaling value for the future velocity.

[0267] In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain future positions of both eyes corresponding to a second time point after the target time point. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict future positions of both eyes for the target time point based on the future positions of both eyes corresponding to the target time point and the future positions of both eyes corresponding to the second time point. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to output the image based on the future positions of both eyes for the target time point.

[0268] In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to obtain future positions of both eyes corresponding to a third time point between the target time point and the second time point. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict future positions of both eyes for the target time point based on the future positions of both eyes corresponding to the target time point, the future positions of both eyes corresponding to the second time point, and the future positions of both eyes corresponding to the third time point.

[0269] In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict future positions of the binocular centers corresponding to the target viewpoint based on future acceleration of the binocular centers and future velocity of the binocular centers. In one embodiment of the present disclosure, the target region may include at least one of the left eye or the right eye. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict future positions of the binocular centers for the target viewpoint based on future positions of the binocular centers corresponding to the target viewpoint and future positions of the binocular centers corresponding to the target viewpoint. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to output the image based on future positions of the binocular centers for the target viewpoint.

[0270] In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict future positions of the binocular centers for the target viewpoint based on the future positions of the binocular centers corresponding to the target viewpoint and the future positions of the binocular centers corresponding to the target viewpoint. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to filter noise from the future positions of the binocular centers for the target viewpoint. In one embodiment of the present disclosure, the one or more computer programs may include computer-executable instructions that, when individually or collectively executed by the one or more processors, cause the electronic device to predict future positions of the binocular centers for the target viewpoint based on the future positions of the binocular centers from which the noise has been filtered.

[0271] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0272] According to one embodiment of the present disclosure, a method according to one or more embodiments disclosed herein may be provided as a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two viewer devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0273] It will be appreciated that the various embodiments of the present disclosure, in accordance with the claims and description of this specification, may be implemented in the form of hardware, software, or a combination of hardware and software.

[0274] Such software may be stored on a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores one or more computer programs (software modules), and the one or more computer programs include computer-executable instructions that, when executed by one or more processors of the electronic device, cause the electronic device to perform the methods of the present disclosure.

[0275] Such software may be stored in the form of volatile or non-volatile storage, such as, for example, a storage device such as read only memory (ROM), whether erasable or rewritable, or in the form of memory such as, for example, a random access memory (RAM), a memory chip, device, or integrated circuit, or in an optical or magnetically readable medium such as, for example, a compact disk (CD), a digital versatile disc (DVD), a magnetic disk, or a magnetic tape. It will be appreciated that the storage device and the storage medium are various embodiments of non-transitory machine-readable storage suitable for storing a computer program or computer programs that, when executed, include instructions for implementing various embodiments of the present disclosure. Accordingly, various embodiments provide a program comprising code for implementing an apparatus or method as claimed in any of the claims of this specification, and a non-transitory machine-readable storage storing such a program.

[0276] While the present disclosure has been illustrated and described with reference to various embodiments, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. In a method performed by an electronic device, A step of obtaining, by the electronic device, position information of a target part corresponding to a past point in time and position information of the target part corresponding to a reference point in time from an image including a viewer's face area input through a camera; A step of obtaining location change information of the target portion corresponding to the reference point in time based on location information of the target portion corresponding to the past point in time and location information of the target portion corresponding to the reference point in time by the electronic device; A step of predicting the future speed of the target portion based on position change information of the target portion corresponding to the reference point in time and position change information of the target portion corresponding to the past point in time by the electronic device; A step of obtaining, by the electronic device, information on change in position of the target portion corresponding to the reference point in time and information on change in position of the target portion corresponding to the past point in time, information on change in speed of the target portion corresponding to the reference point in time; A step of predicting future acceleration of the target portion based on velocity change information of the target portion corresponding to the reference point in time and velocity change information of the target portion corresponding to the past point in time by the electronic device; A step of predicting the future position of both eyes corresponding to the target time based on the future speed and the future acceleration by the electronic device; and A method comprising the step of outputting an image based on the future positions of both eyes corresponding to the target time point by the electronic device.

2. In paragraph 1, The step of predicting the future positions of both eyes corresponding to the target time based on the above future speed and the above future acceleration is: A step of predicting a first future position of the target portion corresponding to the target time based on the future velocity and the future acceleration; A step of obtaining a second future position of the target portion corresponding to the target point in time based on a first future position of the target portion corresponding to the target point in time and a future position of the target portion corresponding to a first point in time prior to the target point in time; and A method comprising a step of predicting future positions of both eyes corresponding to the target time point based on a second future position of the target portion.

3. In paragraph 1 or 2, The step of predicting the future positions of both eyes corresponding to the target time based on the above future speed and the above future acceleration is: A step of predicting a first future position of the center of the binocular vision corresponding to the target time based on the future speed and the future acceleration; and A method comprising a step of predicting future positions of both eyes corresponding to the target time point based on first future positions of the centers of both eyes corresponding to the target time point and IPD (interpupillary distance) information.

4. In any one of paragraphs 1 to 3, The step of obtaining location information of a target area corresponding to a past point in time and location information of the target area corresponding to a reference point in time from an image including a viewer face area input through the above camera is as follows. A step of obtaining raw position information of the target area corresponding to the reference point in time from an image including the viewer face area; and A method comprising the step of obtaining location information of the target portion corresponding to the reference point in time based on location information of the target portion corresponding to the past point in time and raw location information of the target portion.

5. In any one of paragraphs 1 to 4, The step of obtaining location change information of the target part corresponding to the reference point in time based on the location information of the target part corresponding to the past point in time and the location information of the target part corresponding to the reference point in time is as follows. A step of obtaining raw information on the change in the position of the target portion corresponding to the reference point in time based on the position information of the target portion corresponding to the past point in time and the position information of the target portion corresponding to the reference point in time; and A method comprising the step of obtaining position change information of the target portion corresponding to the reference point in time based on a filtering strength determined based on the degree of movement of the target portion, position change information of the target portion corresponding to the past point in time, and raw information about the position change of the target portion.

6. In any one of paragraphs 1 to 5, The step of predicting the future acceleration of the target portion based on the velocity change information of the target portion corresponding to the reference point in time and the velocity change information of the target portion corresponding to the past point in time is as follows. A step of obtaining nonlinear predicted acceleration information of the target portion corresponding to the target point in time based on the velocity change information of the target portion corresponding to the reference point in time, the velocity change information of the target portion corresponding to the past point in time, and the parameter value determined based on the viewer's movement status; and A step of predicting future acceleration of the target portion based on the nonlinear predicted acceleration information and the acceleration of the target portion predicted for a first time point prior to the target time point, A method in which the future acceleration of the above target portion corresponds to the above target time point.

7. In any one of paragraphs 1 to 6, A step of obtaining a scaling value for the future velocity based on the direction of the future velocity is included, The step of predicting the future positions of both eyes corresponding to the target time based on the above future speed and the above future acceleration is: A method comprising the step of predicting future positions of both eyes corresponding to the target time point based on the future velocity, the future acceleration and the scaling value for the future velocity.

8. In any one of paragraphs 1 to 7, Including a step of obtaining the future position of both eyes corresponding to a second point in time after the above target point in time, The step of outputting the image based on the future positions of the two eyes corresponding to the above target time point is as follows. A step of predicting the future positions of the two eyes for the target time point based on the future positions of the two eyes corresponding to the target time point and the future positions of the two eyes corresponding to the second time point; and A method comprising the step of outputting the image based on the future positions of the two eyes with respect to the target time point.

9. In paragraph 8, A step of obtaining the future positions of both eyes corresponding to a third time point between the target time point and the second time point is included. The step of predicting the future positions of the two eyes for the target time point based on the future positions of the two eyes corresponding to the target time point and the future positions of the two eyes corresponding to the second time point is as follows. A method comprising a step of predicting the future positions of the two eyes for the target time point based on the future positions of the two eyes corresponding to the target time point, the future positions of the two eyes corresponding to the second time point, and the future positions of the two eyes corresponding to the third time point.

10. In any one of paragraphs 1 to 9, A step of predicting a future position of the binocular center corresponding to the target time point based on the future acceleration of the binocular center and the future velocity of the binocular center, The target area includes at least one of the left eye or the right eye, The step of outputting the image based on the future positions of the two eyes corresponding to the above target time point is as follows. A step of predicting the future positions of the two eyes for the target time point based on the future positions of the two eyes corresponding to the target time point and the future positions of the centers of the two eyes corresponding to the target time point; and A method comprising the step of outputting the image based on the future positions of the two eyes with respect to the target time point.

11. In paragraph 10, The step of predicting the future positions of the two eyes for the target time point based on the future positions of the two eyes corresponding to the target time point and the future positions of the centers of the two eyes corresponding to the target time point is as follows. A step of predicting the future position of the binocular center for the target time point based on the future position of the binocular center corresponding to the target time point and the future position of the binocular center corresponding to the target time point; A step of filtering out noise from the future position of the binocular center for the above target time point; and A method comprising the step of predicting the future positions of the binocular centers for the target viewpoint based on the future positions of the binocular centers from which the noise is filtered.

12. One or more computer-readable storage media, Storing one or more computer programs containing computer-executable instructions that, when individually or collectively executed by one or more processors of an electronic device, cause the electronic device to perform operations; The above actions are, By the above electronic device, obtaining location information of a target part corresponding to a past point in time and location information of the target part corresponding to a reference point in time from an image including a viewer's face area input through a camera; By the electronic device, obtaining location change information of the target portion corresponding to the reference point in time based on location information of the target portion corresponding to the past point in time and location information of the target portion corresponding to the reference point in time; Predicting the future speed of the target portion based on position change information of the target portion corresponding to the reference point in time and position change information of the target portion corresponding to the past point in time by the electronic device; By the electronic device, obtaining information on change in speed of the target portion corresponding to the reference point in time based on information on change in position of the target portion corresponding to the reference point in time and information on change in position of the target portion corresponding to the past point in time; Predicting future acceleration of the target portion based on velocity change information of the target portion corresponding to the reference point in time and velocity change information of the target portion corresponding to the past point in time by the electronic device; Predicting the future positions of both eyes corresponding to the target time based on the future speed and the future acceleration by the electronic device; and One or more computer-readable storage media, comprising outputting an image based on the future positions of the two eyes corresponding to the target time point by the electronic device.

13. In electronic devices, A memory (1520) storing one or more computer programs; and comprising one or more processors (1510) communicatively connected to the memory (1520); The one or more computer programs, when individually or collectively executed by the one or more processors (1510), the electronic device, From an image including a viewer's face area input through a camera, location information of a target area corresponding to a past point in time and location information of the target area corresponding to a reference point in time are obtained, Based on the location information of the target part corresponding to the past point in time and the location information of the target part corresponding to the reference point in time, location change information of the target part corresponding to the reference point in time is acquired, Based on the position change information of the target part corresponding to the above reference point in time and the position change information of the target part corresponding to the past point in time, the future speed of the target part is predicted, Based on the position change information of the target part corresponding to the above reference point in time and the position change information of the target part corresponding to the past point in time, the speed change information of the target part corresponding to the above reference point in time is acquired, Based on the velocity change information of the target part corresponding to the above reference point in time and the velocity change information of the target part corresponding to the past point in time, the future acceleration of the target part is predicted, Based on the above future speed and the above future acceleration, the future positions of the two eyes corresponding to the target time are predicted, An electronic device comprising computer-executable instructions for outputting an image based on the future positions of the two eyes corresponding to the target time point.

14. In paragraph 13, The one or more computer programs, when individually or collectively executed by the one or more processors (1510), the electronic device, Based on the above future velocity and the above future acceleration, a first future position of the target portion corresponding to the target time is predicted, Based on the first future position of the target portion corresponding to the target point in time and the future position of the target portion corresponding to the first point in time before the target point in time, a second future position of the target portion corresponding to the target point in time is obtained, An electronic device further comprising computer-executable instructions for predicting future positions of both eyes corresponding to the target time point based on the second future position of the target portion.

15. In paragraph 13 or 14, The one or more computer programs, when individually or collectively executed by the one or more processors (1510), the electronic device, Based on the above future speed and the above future acceleration, the first future position of the center of the binocular corresponding to the target time point is predicted, An electronic device further comprising computer-executable instructions for predicting future positions of the two eyes corresponding to the target time point based on first future positions of the two eyes corresponding to the target time point and inter-pupillary distance (IPD) information.

Images