Electronic device and control method therefor

Abstract

According to one embodiment, an electronic device comprises: a memory including one or more storage media for storing instructions; and at least one processor including a processing circuit, wherein the instructions, when individually or collectively executed by the at least one processor, instruct the electronic device to perform at least one operation, and the at least one operation includes: an operation of obtaining first fatigue information on the basis of frames played during a first time period; an operation of obtaining second fatigue information on the basis of frames played during a second time period; and an operation of controlling a stereoscopic image to be displayed through a display on the basis of the first fatigue information and the second fatigue information, wherein the second time period may include at least a part of the first time period.

Description

Electronic device and control method thereof

[0001] The present disclosure relates to an electronic device and method for displaying 2D images and / or 3D images.

[0002] 3D display technology is being developed to provide realistic stereoscopic images. For example, 2D-to-3D image conversion technology is being developed to convert 2D images into 3D images. 2D-to-3D image conversion technology can estimate the depth of objects included in an image and use the estimated depth to generate pairs of images including a left eye image and a right eye image.

[0003] Methods for displaying 3D images include viewpoint display methods such as stereoscopic display and autostereoscopic display. Viewpoint display methods enable the perception of stereoscopic images by showing different images with disparity to the viewer's left and right eyes, respectively. In viewpoint display methods, each viewpoint image can be displayed on a display screen, similar to a 2D display method.

[0004] In the case of 2D displays, the accommodation of the actual eye and the convergence corresponding to the depth of the image coincide, but in 3D displays, accommodation and convergence do not coincide. Therefore, viewers may experience visual fatigue due to this discrepancy between accommodation and convergence. The degree of visual fatigue can increase as the depth of the stereoscopic image increases, as the discrepancy between accommodation and convergence becomes greater.

[0005] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. None of the foregoing is to be claimed as prior art related to the present disclosure, nor is it to be used to determine prior art.

[0006] The present disclosure relates to an electronic device and method for displaying 2D images and / or 3D images.

[0007] According to one embodiment, the electronic device comprises a memory including one or more storage media for storing instructions; and at least one processor including a processing circuit, wherein when the instructions are executed individually or collectively by the at least one processor, the electronic device is caused to perform at least one operation, wherein the at least one operation includes: an operation of obtaining first fatigue information based on frames played during a first time interval; an operation of obtaining second fatigue information based on frames played during a second time interval; and an operation of controlling a stereoscopic image to be displayed through a display based on the first fatigue information and the second fatigue information, wherein the second time interval may include at least a portion of the first time interval.

[0008] According to one embodiment, a control method for an electronic device comprises: an operation of acquiring first fatigue information based on frames played during a first time interval; an operation of acquiring second fatigue information based on frames played during a second time interval; and an operation of controlling a stereoscopic image to be displayed through a display based on the first fatigue information and the second fatigue information, wherein the second time interval may include at least a portion of the first time interval.

[0009] Figure 1 is a block diagram of an electronic device according to various examples.

[0010] Figure 2 illustrates an exemplary method for displaying a 3D image on a display.

[0011] Figure 3 is a flowchart illustrating a method for controlling stereoscopic images in an electronic device according to one example.

[0012] Figure 4 is a block diagram of an electronic device according to an example that can reduce fatigue.

[0013] Figure 5 illustrates a method for controlling a stereoscopic image in an electronic device according to one example.

[0014] Figure 6 illustrates a method for controlling a stereoscopic image in an electronic device according to one example.

[0015] Figure 7 is an example of a graph showing experimental results with the application of a stereoscopic image control method.

[0016] Figure 8 is an example of a graph showing experimental results with the application of a stereoscopic image control method.

[0017] Figure 9 illustrates a method for acquiring image fatigue in an electronic device according to one example.

[0018] Figure 10 illustrates a method for acquiring image fatigue in an electronic device according to one example.

[0019] Figure 11 explains the factors affecting fatigue.

[0020] FIG. 12 illustrates a method for determining a weighted disparity range in an electronic device according to one example.

[0021] Figure 13 illustrates the factors affecting the weighted disparity range.

[0022] FIG. 14 illustrates an exemplary method for controlling a stereoscopic image in an electronic device according to one example.

[0023] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of other embodiments. A singular expression may include a plural expression unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art described in this disclosure. Terms used in this disclosure that are defined in a general dictionary may be interpreted as having the same or similar meaning as they have in the context of the relevant technology, and are not to be interpreted in an ideal or overly formal sense unless explicitly defined in this disclosure. In some cases, even terms defined in this disclosure are not to be interpreted to exclude the embodiments of this disclosure.

[0024] In the various embodiments of the present disclosure described below, a hardware-based approach is described as an example. However, since the various embodiments of the present disclosure include techniques using both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.

[0025] Terms used in the following description to refer to the configuration of a device (e.g., processor, camera, display, module, communication circuit, etc.), terms for operation states (e.g., step, operation, procedure), terms for signals (e.g., signal, information, data, stream, user input, input, etc.), and terms for data (e.g., parameter, value, etc.) are provided as examples for the convenience of explanation. Accordingly, the present disclosure is not limited to the terms described below, and other terms having equivalent technical meanings may be used.

[0026] Additionally, in this disclosure, expressions of "greater than" or "less than" may be used to determine whether a specific condition is satisfied or fulfilled; however, this is merely for the purpose of expressing an example and does not exclude descriptions of "greater than" or "less than." Conditions described as "greater than" may be replaced with "greater than," conditions described as "less than" may be replaced with "less than," and conditions described as "greater than and less than" may be replaced with "greater than and less than." Furthermore, "A" to "B" below refer to at least one of the elements from A (including A) to B (including B).

[0027] Figure 1 is a block diagram of an electronic device according to various examples.

[0028] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one example, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one example, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some examples, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some examples, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0029] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), and can perform various data processing or operations. According to one example, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one example, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0030] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one example, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one example, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0031] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0032] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0033] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0034] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one example, the receiver may be implemented separately from the speaker or as part thereof.

[0035] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one example, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0036] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one example, the audio module (170) can acquire sound through an input module (150) or output sound through an audio output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0037] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one example, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0038] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one example, the interface (177) may include, for example, a high definition multi-media interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0039] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one example, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0040] The haptic module (179) can convert electrical signals into mechanical stimulation (e.g., vibration or movement) or electrical stimulation that can be perceived by the user through tactile or kinesthetic senses. According to one example, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0041] The camera module (180) can capture still images and video. According to one example, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0042] The power management module (188) can manage power supplied to the electronic device (101). According to one example, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0043] The battery (189) can supply power to at least one component of the electronic device (101). According to one example, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0044] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one example, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0045] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or largescale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one example, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0046] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one example, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one example, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some examples, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0047] According to various examples, the antenna module (197) can form a mmWave antenna module. According to one example, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0048] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0049] According to one example, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one example, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another example, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one example, the external electronic device (104) or the server (108) may be included within the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and / or IoT-related technology.

[0050] The electronic devices according to the various examples disclosed in this document may be of various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or consumer electronics. The electronic devices according to the examples in this document are not limited to the devices described above.

[0051] FIG. 2 illustrates an exemplary method of displaying a 3D image on a display (e.g., the display (160) of FIG. 1) (200).

[0052] Stereoscopic imaging technology enables the three-dimensional perception of images by utilizing binocular disparity between the left and right visual fields. This leverages the human visual system, which acquires two independent visual fields through the left and right eyes, respectively. Stereoscopic imaging technology analyzes the disparity between the left and right visual fields and utilizes the brain's ability to measure the relative distance of objects within the image from the observer.

[0053] In FIG. 2, it is assumed that points A (210), B (220), and F (230) are objects. A viewer may gaze at point F (230). Point F (230) may be an object presented on a display (e.g., display (160) in FIG. 1) (200) that is indicated by a horizontal line. A user may gaze at point A (210). Point A (210) may be an object presented on the back of the display (200), in the direction opposite to the direction in which the image is displayed on the display (200). A user may gaze at point B (220). Point B (220) may be an object presented on the front of the display (200), in the direction in which the image is displayed on the display (200). The distance between the user and point B (220) may be shorter than the distance between the user and point A (210). The electronic device (101) may display a corresponding left eye pixel on AL (211) and a corresponding right eye pixel on AR (213) to display point A (210). The electronic device (101) may display corresponding left eye and right eye pixels on BL (223) and BR (221), respectively, to display point B (220). Point A (210) may appear to be virtually located at a first distance (240) on the back of the display (200), and point B (230) may appear to be virtually located at a second distance (250) on the front of the display (200). The first distance (240) may depend on the difference (disparity or parallax) between AL (211) and AR (213) and the viewing distance (distance between the viewer's eyes and the display screen). The second distance (250) may depend on the difference (disparity or parallax) between BL (223) and BR (221) and the viewing distance (distance between the viewer's eyes and the display screen). For example, when a user looks at point F (230), the viewer's eyes may focus on the distance of the display (200) and converge on point F (230). In this case, there is no discrepancy between the convergence of the gaze and the focus adjustment.When looking at point A (210), the viewer's eyes may focus on the distance of the display (200), but may converge on point A (210), which is a first distance (240) farther than the distance of the display (200). In this case, there is a mismatch between the convergence of the gaze and the focus adjustment. When the user looks at point B (220), the viewer's eyes may focus on the distance of the display (200), but may converge on point B (220), which is a second distance (250) closer than the distance of the display (200). In this case, there is a mismatch between the convergence of the gaze and the focus adjustment.

[0054] A mismatch between eye convergence and focus adjustment can cause visual fatigue. According to one embodiment, an electronic device (101) can reduce such fatigue. Hereinafter, in the present disclosure, visual fatigue will be abbreviated as fatigue.

[0055] FIG. 3 is a flowchart illustrating a method for controlling a stereoscopic image in an electronic device according to one example (e.g., the electronic device (101) of FIG. 1).

[0056] In the following examples of operations, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, or at least two operations may be performed in parallel.

[0057] Referring to FIG. 3, the electronic device (101) may acquire first fatigue information based on frames played during a first time interval in operation 310. For example, the number of frames played during the first time interval may be 10 to 30 or 10 to 20, but is not limited thereto. The electronic device (101) may acquire first fatigue information for each first time interval. The first fatigue information may be information regarding the fatigue of the frames played during the first time interval. The first fatigue information may include a first ratio of frames having a fatigue exceeding a first threshold among the frames played during the first time interval and / or the sum of the fatigue of the frames played during the first time interval.

[0058] The electronic device (101) may, in operation 320, obtain second fatigue information based on frames played during a second time interval. The second time interval may include at least a part or all of the first time interval. For example, the number of frames played during the second time interval may be 100 to 300 or 100 to 200, but is not limited thereto. The electronic device (101) may obtain second fatigue information for each second time interval. The second fatigue information may be information regarding the fatigue of the frames played during the second time interval. The second fatigue information may include a second ratio of frames having a fatigue level exceeding a second threshold among the second frames played during the second time interval.

[0059] The electronic device (101) can control a stereoscopic image to be displayed through a display based on the first fatigue information and the second fatigue information in operation 330. The stereoscopic image to be displayed through the display may be an image after the first time interval has elapsed. The stereoscopic image to be displayed through the display may be an image after the second time interval has elapsed. The electronic device (101) can adjust the disparity range frame by frame based on the first fatigue information. The disparity range may refer to a range from minimum disparity to maximum disparity. Increasing the disparity range can amplify the sense of depth (i.e., stereoscopic effect). Narrowing the disparity range can reduce the sense of depth (stereoscopic effect). The electronic device (101) can determine a method for adjusting the disparity range based on the second fatigue information. For example, the electronic device (101) can increase the disparity range at once based on the second fatigue information. For example, the electronic device (101) can increase the disparity range stepwise based on the second fatigue information.

[0060] FIG. 4 is a block diagram of an electronic device (e.g., the electronic device (101) of FIG. 1) according to an example that can reduce fatigue.

[0061] Referring to FIG. 4, according to one embodiment, the electronic device (101) may include or be composed of at least some of a parameter determination module (410), a frame processing module (420), a first time interval analysis module (430), or a second time interval analysis module (440). The components included in the electronic device (101) may be software or firmware executed by a processor (e.g., the processor (120) of FIG. 1). The components included in the electronic device (101) may be hardware modules included in the processor (120) or existing independently.

[0062] According to one embodiment, the parameter determination module (410) can obtain parameters such as display size, display resolution, frame rate of the display, size of objects included in the displayed image, or viewing distance.

[0063] According to one embodiment, the display size may refer to the width of the display (e.g., the display (160) of FIG. 1). The larger the display size, the greater the three-dimensional effect may be expressed, and as a result, high fatigue may be caused.

[0064] According to one embodiment, display resolution may refer to the total number of pixels that can be displayed on a screen. For example, the resolution of a display may be measured in pixels per inch (PPI). If the horizontal width of the pixels is large, a difference of the same number of pixels can cause a greater sense of depth. If the horizontal width of the pixels is large, the number of pixels per inch may decrease. The smaller the display resolution (e.g., PPI), the greater the sense of depth can be expressed. A low display resolution may cause high eye strain.

[0065] According to one embodiment, the frame rate of the display (e.g., in Hz) may refer to the number of frames provided by the display (160) per second. An image (e.g., video) is a set of still images that are sequential in time, and one still image may refer to a frame. The higher the frame rate of the display, the more severe the conflict between gaze convergence and focus adjustment becomes, which may result in high fatigue.

[0066] According to one embodiment, the frame may include a plurality of objects. As the size of the objects included in the frame increases, the conflict between gaze convergence and focus adjustment becomes more severe, which may result in high fatigue.

[0067] According to one embodiment, the viewing distance may refer to the distance between the viewer's eyes and the display (160). The longer the viewing distance, the smaller the disparity of the three-dimensional object displayed at the same distance may be. The viewing distance may affect the calculation of the 'weighted disparity range'. A detailed explanation thereof will be provided later with reference to FIG. 12.

[0068] According to one embodiment, the frame processing module (420) can obtain fatigue values ​​on a frame-by-frame basis. A known method may be used to obtain fatigue values ​​on a frame-by-frame basis. According to one example, the frame processing module (420) can obtain fatigue on a frame-by-frame basis by applying weight map information that assigns weights according to disparity to temporal disparity information (see FIG. 10). The frame processing module (420) can apply weights to fatigue based on the frame rate of the display. For example, the frame processing module (420) can set different thresholds for determining fatigue based on the frame rate. For example, if the frame rate is 30 Hz, the disparity difference must be greater than when the frame rate is 60 Hz for the user to feel fatigue.

[0069] According to one embodiment, the frame processing module (420) may transmit information regarding fatigue acquired on a frame-by-frame basis to the first time interval analysis module (430). The information regarding fatigue may be, for example, binary information. The binary information may be information indicating whether the frame is fatigued or not fatigued. The electronic device (101) may determine information regarding fatigue indicating that a frame having a fatigue level exceeding a first threshold is 'fatigued'. The electronic device (101) may determine information regarding fatigue indicating that a frame having a fatigue level below the first threshold is 'not fatigued'. The information regarding fatigue acquired by the frame processing module (420) on a frame-by-frame basis may be the fatigue value itself.

[0070] According to one embodiment, the first time interval analysis module (430) acquires first fatigue information based on frames played during the first time interval (hereinafter referred to as 'first frames'), and can adjust the disparity range for each frame using the acquired first fatigue information. The first time interval analysis module (430) can acquire the first fatigue information by collecting information regarding fatigue based on the determination of the frame processing module (410) for the first frames. For example, the number of first frames may be 10 to 30 or 10 to 20, but is not limited thereto. As an example, the first fatigue information may include a first ratio of frames having fatigue exceeding a first threshold among the first frames and / or the sum of the fatigue of the first frames. As an example, the first time interval analysis module (430) can acquire and output a control coefficient for adjusting the disparity range using the first fatigue information. A control coefficient for adjusting the disparity range can be called the 3D ratio. The 3D ratio can be a value between 0 and 1. If the 3D ratio is 0, the left eye image and the right eye image may be identical. That is, if the 3D ratio is 0, a 2D image can be provided. If the 3D ratio is 1, the disparity, which is the difference between the left eye image and the right eye image, can be maximized. That is, if the 3D ratio is 1, a 3D image with maximized stereoscopic effect can be provided. The first time interval analysis module (430) can lower the disparity range. Lowering the disparity range of the stereoscopic image can be called depth smoothing. Depth smoothing can cause a transition of the image playback mode from a 3D image playback mode to a 2D image playback mode. The first time interval analysis module (430) can increase the disparity range. Increasing the disparity range of stereoscopic images can be called depth sharpening.Depth sharpening can cause a transition of the video playback mode from 2D video playback mode to 3D video playback mode.

[0071] According to one embodiment, the first time interval analysis module (430) can filter out outliers. For example, the first time interval analysis module (430) can filter information regarding frames having disparity (or depth) exceeding a threshold. For example, the first time interval analysis module (430) can filter information regarding frames containing information indicating the degree of scene change. Scene change can cause abrupt changes in temporal disparity information, and consequently, fatigue may be overestimated. The first time interval analysis module (430) can detect scene change. The first time interval analysis module (430) can detect image similarity between first frames. The first time interval analysis module (430) can detect scene change when the image similarity between first frames exceeds a threshold.

[0072] According to one embodiment, the first time interval analysis module (430) can apply weights according to the temporal order of the first frames. For example, the first time interval analysis module (430) can apply weights to information regarding the fatigue of the most recently played frames among the first frames. The first time interval analysis module (430) can immediately control the video using information regarding the fatigue of the most recently played frames.

[0073] According to one embodiment, the first time interval analysis module (430) may apply different weights to the disparity difference by region by considering spatial elements when calculating fatigue. The first time interval analysis module (430) may divide the display area of ​​the display (160) into multiple regions. The first time interval analysis module (430) may obtain the disparity difference by multiple regions. For example, viewers tend to focus their gaze on the central region of the display area of ​​the display (160), while focusing relatively less on the side region. The first time interval analysis module (430) may assign a high weight to the disparity difference in the central region and a low weight to the disparity difference in the side region. For example, viewers tend to focus on objects with bright luminescence. The first time interval analysis module (430) may obtain the brightness by multiple regions. The first time interval analysis module (430) can give high weight to the disparity difference in high brightness areas.

[0074] According to one embodiment, the first time interval analysis module (430) may obtain a 3D ratio based on the sum of the first ratio and / or the fatigue of the first frames. For example, the first time interval analysis module (430) may determine (1 - the first ratio) (value obtained by subtracting the first ratio from 1) as the 3D ratio. For example, the first time interval analysis module (430) may determine the 3D ratio to be 0 if the accumulated fatigue value obtained from the frame processing module (420) during a predetermined time interval exceeds a threshold. The first time interval analysis module (430) may determine the 3D ratio to be 1 if the fatigue value obtained from the frame processing module (420) during a predetermined time interval becomes less than or equal to a specific value.

[0075] According to one embodiment, the first time interval analysis module (430) can obtain second fatigue information from the second time interval analysis module (440). The first time interval analysis module (430) can control a stereoscopic image to be displayed through a display based on the second fatigue information. Based on the second fatigue information, the first time interval analysis module (430) can determine a time interval in which the 3D ratio determined based on the first fatigue information is maintained and / or a step of restoring the 3D ratio.

[0076] According to one embodiment, the first time interval analysis module (430) can lower the 3D ratio all at once when lowering the disparity range of the stereoscopic image (when converting from 3D image playback mode to 2D image playback mode). The first time interval analysis module (430) can determine the time to maintain the lowered 3D ratio according to the second ratio. When increasing the disparity range of the stereoscopic image (when converting from 2D image playback mode to 3D image playback mode), the first time interval analysis module (430) can determine which step(s) should be used to recover the lowered 3D ratio according to the second ratio. The speed at which the first time interval analysis module (430) increases the 3D ratio may be slower than the speed at which it decreases the 3D ratio.

[0077] According to one embodiment, the second time interval analysis module (440) can collect the results of the frame processing module (410) for frames played during the second time interval (hereinafter referred to as 'second frames'). For example, the number of second frames may be 100 to 300 or 100 to 200, but is not limited thereto. The second time interval analysis module (440) can acquire second fatigue information for each second time interval. For example, the second time interval analysis module (440) can acquire second fatigue information for every 100 frames. For example, the second time interval analysis module (440) can acquire second fatigue information based on 100 second frames (frames 1 to 100) and then update the second fatigue information based on 100 second frames (frames 101 to 200). For example, the second fatigue information may be information regarding the ratio of frames having a fatigue level exceeding a threshold among the second frames. Hereinafter, in the present disclosure, the ratio of frames having a fatigue level exceeding a threshold among the second frames may be referred to as the "second ratio." For example, the second ratio may be within a first range, a second range, a third range, or a fourth range. For example, the first range may be 10% or less. For example, the second range may be greater than 10% and less than or equal to 20%. For example, the third range may be greater than 20% and less than or equal to 30%. For example, the fourth range may be greater than 30%. For example, if the second ratio is within the first range, the second fatigue information may be referred to as 1. For example, if the second ratio is within the second range, the second fatigue information may be referred to as 2. For example, if the second ratio is within the third range, the second fatigue information may be referred to as 3. The second time interval analysis module (440) can transmit the second fatigue information to the first time interval analysis module (430).

[0078] According to one embodiment, when the second ratio is within the first range, the first time interval analysis module (430) can lower the 3D ratio at once when lowering the disparity range of the stereoscopic image (when converting from 3D image playback mode to 2D image playback mode). For example, the first time interval analysis module (430) can lower the 3D ratio from 1 to 0 at once. For example, the first time interval analysis module (430) can lower the 3D ratio from 1 to 0.3 at once. The first time interval analysis module (430) can maintain a low 3D ratio for a first unit time. The first time interval analysis module (430) can maintain the 2D image playback mode for a first unit time. For example, the first unit time may be 2 seconds, but is not limited thereto. The first time interval analysis module (430) can increase the 3D ratio at once when increasing the disparity range of the stereoscopic image (when converting from 2D image playback mode to 3D image playback mode) if the second ratio is within the first range. For example, the first time interval analysis module (430) can increase the 3D ratio from 0 to 1 at once. For example, the first time interval analysis module (430) can increase the 3D ratio from 0.3 to 1 at once. The first time interval analysis module (430) can increase the 3D ratio over a predetermined time interval (e.g., see time interval 820 of FIG. 8).

[0079] According to one embodiment, when the second ratio is within the second range, the first time interval analysis module (430) can immediately lower the 3D ratio when lowering the disparity range of the stereoscopic image (when converting from 3D image playback mode to 2D image playback mode). For example, the first time interval analysis module (430) can immediately lower the 3D ratio from 1 to 0. The first time interval analysis module (430) can maintain a low 3D ratio for a first unit time. The first time interval analysis module (430) can maintain the 2D image playback mode for a first unit time. When the second ratio is within the second range, the first time interval analysis module (430) can gradually increase the 3D ratio when increasing the disparity range of the stereoscopic image (when converting from 2D image playback mode to 3D image playback mode). For example, the first time interval analysis module (430) can increase the 3D ratio from 0 to 0.5 and maintain it for a first unit time, and then increase the 3D ratio from 0.5 to 1 (e.g., see time interval 810 in FIG. 8).

[0080] According to one embodiment, when the second ratio is within the third range, the first time interval analysis module (430) can immediately lower the 3D ratio when lowering the disparity range of the stereoscopic image (when converting from 3D image playback mode to 2D image playback mode). For example, the first time interval analysis module (430) can immediately lower the 3D ratio from 1 to 0. The first time interval analysis module (430) can maintain the 2D image playback mode for a second unit time. For example, the second unit time may be 2 to 3 times the first unit time. When the second ratio is within the third range, the first time interval analysis module (430) can gradually increase the 3D ratio when increasing the disparity range of the stereoscopic image (when converting from 2D image playback mode to 3D image playback mode). For example, the first time interval analysis module (430) can increase the 3D ratio in three steps. For example, the first time interval analysis module (430) can increase the 3D ratio stepwise from 0 to 0.3 (step 1), 0.6 (step 2), and 1 (step 3). For example, the first time interval analysis module (430) can set the holding time of each step as the first unit time. The first time interval analysis module (430) can set the holding time of each step as the second unit time.

[0081] According to one embodiment, for a first image, the electronic device (101) can reduce fatigue caused by the first image by acquiring first fatigue information based on frames played during a first time interval and acquiring second fatigue information based on frames played during a second time interval, and by controlling a stereoscopic image to be displayed through a display based on the first fatigue information and the second fatigue information. For a third image in which a second image is added prior to the first image, the electronic device (101) can reduce fatigue caused by the third image by acquiring first fatigue information based on frames played during a first time interval and acquiring second fatigue information based on frames played during a second time interval, and by controlling a stereoscopic image to be displayed through a display based on the first fatigue information and the second fatigue information. Depending on the fatigue level of the second image, the second fatigue information of the first image included in the first image and the third image may differ, and accordingly, the electronic device (101) may control the first image included in the first image and the third image differently. Hereinafter, the first image included in the third image may be referred to as the first' image. If the fatigue level of the second image is low, the electronic device (101) may rapidly adjust the 3D ratio of the first' image compared to the 3D ratio of the first image. If the fatigue level of the second image is high, the electronic device (101) may relatively slowly and gradually adjust the 3D ratio of the first' image compared to the 3D ratio of the first image.

[0082] FIG. 5 illustrates a method for controlling a stereoscopic image in an electronic device according to one example (e.g., the electronic device (101) of FIG. 1).

[0083] Referring to FIG. 5, according to one embodiment, an electronic device (101) can play content. The electronic device (101) can receive content streaming from at least one server (e.g., a content provider). The electronic device (101) can play content streamed from at least one server (e.g., a content provider). The electronic device (101) can play content during a time interval including a first time interval. The electronic device (101) can play content during a time interval including a second time interval. The electronic device (101) can obtain first fatigue information based on first frames played during the first time interval. The electronic device (101) can obtain first fatigue information for each first time interval. The electronic device (101) can obtain second fatigue information based on second frames played during the second time interval. The electronic device (101) can acquire second fatigue information for each second time interval. Before the second time interval has elapsed, the electronic device (101) cannot acquire second fatigue information based on the second frames. The electronic device (101) can set initial second fatigue information. Before the second time interval has elapsed, the electronic device (101) can control the stereoscopic image to be displayed through the display (e.g., the display (160) of FIG. 1) based on first fatigue information based on the first frames played during the first time interval and initial second fatigue information. When the second time interval has elapsed, the electronic device (101) can acquire second fatigue information based on the second frames. When the second time interval has elapsed, the electronic device (101) can control the stereoscopic image to be displayed through the display (160) based on the acquired second fatigue information and first fatigue information.

[0084] According to one embodiment, an electronic device (101) may receive content (501; 502) streamed from at least one server (e.g., a content provider). The content (501; 502) may be content included in the same video. The time during which the content (501) is played may be shorter than the second time interval. While the content (501) is being played, the second time interval analysis module (e.g., the second time interval analysis module (440) of FIG. 4) (540) may transmit initial second fatigue information (503) to the first time interval analysis module (e.g., the first time interval analysis module (430) of FIG. 4) (530). For example, the initial second fatigue information (503) may be information that the second ratio is within the first range. The first time interval analysis module (530) may not transmit a signal (504) requesting second fatigue information to the second time interval analysis module (540).

[0085] According to one embodiment, the electronic device (101) may receive content (502) after a second time interval has elapsed. While playing the content (502), the second time interval analysis module (e.g., the second time interval analysis module (440) of FIG. 4) (540) may provide second fatigue information (505) to the first time interval analysis module (e.g., the first time interval analysis module (430) of FIG. 4) (530). The first time interval analysis module (530) may transmit a signal (506) requesting the second fatigue information to the second time interval analysis module (540).

[0086] FIG. 6 illustrates a method for controlling a stereoscopic image in an electronic device according to one example (e.g., the electronic device (101) of FIG. 1).

[0087] In FIG. 6, according to one embodiment, the content (600) may be content stored in the electronic device (101). For example, the content (600) may be content downloaded from at least one server. The electronic device (101) may play the content (600). Before completing the playback of the entire content (600), the electronic device (101) may obtain second fatigue information determined at each second time interval from the content (600).

[0088] According to one embodiment, while playing content (600), a second time interval analysis module (e.g., the second time interval analysis module (440) of FIG. 4) (640) may provide second fatigue information (603) to a first time interval analysis module (e.g., the first time interval analysis module (430) of FIG. 4) (630). The first time interval analysis module (530) may transmit a signal (604) requesting the second fatigue information to the second time interval analysis module (540). Since the electronic device (101) can acquire the second fatigue information (603) in advance without calculating it in real time, the acquired second fatigue information (603) can be used more immediately. When playing stored content (600), the electronic device (101) can control the stereoscopic image more accurately based on the first fatigue information and the second fatigue information compared to when playing streamed content.

[0089] Figure 7 is an example of a graph showing experimental results with the application of a stereoscopic image control method.

[0090] In FIG. 7, a first content (e.g., a movie) was played by applying a stereoscopic image control method to an electronic device (e.g., the electronic device (101) of FIG. 1). According to one embodiment, the electronic device (101) automatically converted the first content, which was originally a 2D image, into a 3D image. When playing the first content converted into a 3D image in the electronic device (101), a stereoscopic image control method according to one example was applied. The electronic device (101) played the first content while adjusting the 3D ratio of the first content. As a result of the experiment, the electronic device (101) lowered the 3D ratio at time interval 700 and then restored it.

[0091] In FIG. 7, the x-axis represents the number of frames included in the first content, and the y-axis represents the 3D ratio. According to one embodiment, an electronic device (101) continuously acquired first fatigue information based on first frames played during a first time interval. The first fatigue information was a first ratio of frames among the first frames that had a fatigue level exceeding a first threshold. The electronic device (101) continuously acquired second fatigue information based on second frames played during a second time interval. The second fatigue information was a second ratio of frames among the second frames that had a fatigue level exceeding a threshold. Before time interval 700, the electronic device (101) recognized a first scene in which there was a lot of object movement in an area with large disparity (an area far from the display in the front direction of the display). The electronic device (101) acquired a 3D ratio to play the video in time interval 700 based on the first fatigue information acquired based on frames played before time interval 700. The electronic device (101) reduced the 3D ratio from 1 to 0.3 from frame a (710) to frame b (720). Frames a (710) and b (720) are consecutive frames. The electronic device (101) recognized a second scene between frame b (720) and frame c (730) in which there is a lot of movement of the object in an area of ​​large disparity. The electronic device (101) identified that the second ratio is within the second range. The electronic device (101) maintained the 3D ratio at 0.3 during a predetermined time interval from frame b (720) to frame c (730). The electronic device (101) gradually increased the 3D ratio from 0.3 to 0.6 and then to 1. The electronic device (101) increased the 3D ratio to 0.6 over a predetermined time interval from frame c (730) to frame d (740). The electronic device (101) maintained a 3D ratio of 0.6 during a predetermined time interval from frame d (740) to frame e (750).The electronic device (101) increased the 3D ratio to 1 over a predetermined time interval from frame e (750) to frame f (760). After time interval 700, the electronic device (101) did not recognize a scene that causes high fatigue. After time interval 700, the electronic device (101) maintained the 3D ratio of the first content at 1.

[0092] According to one embodiment, as a result of the experiment, if a high-fatigue image (a high-fatigue image between frame b (720) and frame c (730)) persists after the disparity range is lowered by the first time interval module (e.g., the first time interval module (430) of FIG. 4), the electronic device (101) slowly and stepwise restored the 3D ratio based on the second fatigue information obtained by the second time interval module (e.g., the second time interval module (440) of FIG. 4).

[0093] Figure 8 is an example of a graph showing experimental results with the application of a stereoscopic image control method.

[0094] In FIG. 8, an electronic device (e.g., the electronic device (101) of FIG. 1) played a second content (e.g., a music video). According to one embodiment, the second content may be a 2D image. The electronic device (101) may automatically convert the second content into a 3D image. The electronic device (101) played the second content converted into a 3D image. The electronic device (101) played the second content while adjusting the 3D ratio of the second content. The second content included many scenes with a lot of object movement in areas with large disparity. As a result of the experiment, the electronic device (101) lowered the 3D ratio at time intervals 810, 820, 830, and 840 and then restored it.

[0095] In FIG. 8, the electronic device (101) continuously acquired first fatigue information based on first frames played during a first time interval. According to one embodiment, the first fatigue information was a first ratio of frames among the first frames having a fatigue level exceeding a first threshold. The electronic device (101) continuously acquired second fatigue information based on second frames played during a second time interval. The second fatigue information was a second ratio of frames among the second frames having a fatigue level exceeding a threshold. Prior to time interval 810, the electronic device (101) recognized a first scene in which there was a lot of object movement in an area with large disparity. Based on the first fatigue information acquired based on frames played prior to time interval 810, the electronic device (101) acquired a 3D ratio to play the video in time interval 810. The electronic device (101) reduced the 3D ratio from 1 to 0.3 from frame a1 (811) to frame a2 (812). The electronic device (101) recognized a second scene in which there is a lot of movement of an object in an area of ​​large disparity between frame a2 (812) and frame a3 (813). The electronic device (101) identified that the second ratio is within the second range. The electronic device (101) maintained the 3D ratio at 0.3 during a predetermined time interval from frame a2 (812) to frame a3 (813). The electronic device (101) gradually increased the 3D ratio from 0.3 to 0.7 and 1. The electronic device (101) increased the 3D ratio to 0.7 over a predetermined time interval from frame a3 (813) to frame a4 (814). The electronic device (101) maintained the 3D ratio at 0.7 during a predetermined time interval from frame a4 (814) to frame a5 (815). The electronic device (101) increased the 3D ratio to 1 over a predetermined time interval from frame a5 (815) to frame a6 (816).

[0096] According to one embodiment, as a result of the experiment, if a high-fatigue image (a high-fatigue image between frame a2 (812) and frame a3 (813)) persists after the disparity range is lowered by the first time interval module (e.g., the first time interval module (430) of FIG. 4), the electronic device (101) slowly and stepwise restores the 3D ratio based on second fatigue information obtained by the second time interval module (e.g., the second time interval module (440) of FIG. 4). This prevents fatigue caused by the high-fatigue image that may exist even after frame a3 (813).

[0097] According to one embodiment, prior to time interval 820, the electronic device (101) recognized a third scene in which there is a lot of movement of an object in an area with large disparity. The electronic device (101) obtained a 3D ratio for playing the video in time interval 820 based on first fatigue information obtained based on frames played prior to time interval 820. The electronic device (101) identified that the second ratio is within the first range based on frames played prior to time interval 820. The electronic device (101) reduced the 3D ratio from 1 to 0.3 from frame b1 (821) to frame b2 (822). The electronic device (101) maintained the 3D ratio at 0.3 during a predetermined time interval from frame b2 (822) to frame b3 (823). The electronic device (101) increased the 3D ratio from 0.3 to 1. The electronic device (101) increased the 3D ratio to 1 over a predetermined time interval from frame b3 (813) to frame b4 (814).

[0098] According to one embodiment, after the 3D ratio was adjusted to 0.3 in time interval 830, there were no scenes with a lot of object movement in areas with large disparity. In time interval 830, the 3D ratio was adjusted in the same way as in time interval 820.

[0099] According to one embodiment, there was a scene with a lot of object movement in an area with large disparity after the 3D ratio was adjusted to 0.3 at time interval 840. The 3D ratio at time interval 840 was adjusted in the same way as at time interval 810.

[0100] Table 1 shows the experimental results of applying the control method for stereoscopic images according to one embodiment.

[0101] No. Stereoscopic image control method not applied Stereoscopic image control method applied Difference 1 2.4 33.8 31.40 22.19 3.50 1.31 32.42 3.50 1.08 43.00 3.43 0.43 52.76 3.90 1.146 2.74 3.90 1.177 2.62 3.310.69 82.95 3.95 1.00 93.10 3.48 0.38 10 2.36 3.29 0.93 112.93 3.57 0.64 Average 2.68 3.610.92

[0102] Referring to Table 1, a total of 11 videos were used in this experiment. For each of the 11 videos, both the case with and without the stereoscopic image control method according to one example were evaluated. Twenty-one participants took part in the evaluation. Videos with and without the stereoscopic image control method applied were randomly mixed, and the 21 participants were instructed to watch each video. After watching the two versions (the version with and without the stereoscopic image control method applied), the participants were asked to provide a Mean Opinion Score (MOS). MOS is a subjective evaluation metric used to assess video quality. MOS is expressed as a score ranging from 1 (Very Poor), 2 (Poor), 3 (Average), 4 (Good), to 5 (Very Good). The experimental results showed that when the stereoscopic image control method was applied, the MOS was, on average, 0.92 points higher. This means that when a stereoscopic image control method was applied, experiment participants evaluated the image quality more positively. In one example, the stereoscopic image control method was able to effectively adjust the sense of depth while reducing fatigue caused by watching stereoscopic images.

[0103] FIG. 9 illustrates a method for obtaining image fatigue in an electronic device according to one example (e.g., the electronic device (101) of FIG. 1).

[0104] Referring to FIG. 9, according to one embodiment, an electronic device (101) can measure (or acquire) the fatigue of an image. For example, the electronic device (101) can acquire the degree of temporal change as the fatigue of the image. The degree of temporal change may refer to the amount of motion within the image. The electronic device (101) can acquire the degree of temporal change of the image by using the difference (930) between the depth of the frame (910) at time point t-1 and the depth of the frame (920) at time point t. Time points t-1 and time point t may be consecutive. For example, for an image containing N frames, the electronic device (101) can acquire the degree of temporal change of the image by summing the variance of the depth difference between consecutive frames belonging to the N frames during the (N-1) interval. If the depth difference between consecutive frames is small, the degree of temporal change of the image is low. If the depth difference between consecutive frames is large, the degree of temporal change of the image is high. A high degree of temporal variation can cause high fatigue. When the degree of temporal variation is high, the electronic device (101) can reduce fatigue by adjusting the depth of the image.

[0105] According to one embodiment, depth differences at all depths between consecutive frames can contribute equally to the degree of temporal change and visual fatigue. As previously described in FIG. 2, depth differences at depths close to the display of the electronic device (101) (e.g., the display (160) of FIG. 1) may cause relatively less fatigue. Depth differences at depths close to the display (160) may be caused, for example, by the movement of an object displayed in an area close to the display (160). Depth differences at depths far from the display (160) may cause relatively more fatigue. Depth differences at depths far from the display (160) may be caused by the movement of an object displayed in an area far from the display (160).

[0106] FIG. 10 illustrates a method for obtaining image fatigue in an electronic device according to one example (e.g., the electronic device (101) of FIG. 1).

[0107] Referring to FIG. 10, according to one embodiment, an electronic device (101) can measure (or acquire) the fatigue of an image. For example, the electronic device (101) can acquire the fatigue on a frame-by-frame basis by applying weight map information that assigns weights according to disparity to temporal disparity information. Hereinafter, the fatigue acquired by applying weight map information that assigns weights according to disparity to temporal disparity information may be referred to as 'temporal fatigue'. The electronic device (101) can acquire the temporal fatigue on a frame-by-frame basis.

[0108] In FIG. 9, it was explained that an electronic device (101) can obtain the degree of temporal change as fatigue, and the degree of temporal change can be obtained by using the depth difference between frames per unit of time. In the same principle, according to one embodiment, the electronic device (101) can obtain temporal disparity information by using the disparity difference between frames per unit of time. This is because, as explained in FIG. 2, disparity creates depth, and the disparity difference between frames is correlated with the depth difference between frames.

[0109] In FIG. 10, according to one embodiment, an electronic device (101) can obtain temporal disparity information between frames by using the difference (1030) between the disparity of a frame (1010) at time t-1 and the disparity of a frame (1020) at time t. Time t-1 and time t may be consecutive. Time t-1 and time t may not be consecutive. The temporal disparity information may include the degree of change in disparity per unit of time.

[0110] According to one embodiment, the electronic device (101) can estimate the disparity (1040) for each pixel within a frame at time t. The electronic device (101) can obtain disparity information (1050) of the pixels within a frame at time t. The electronic device (101) can generate a weight map (1060) by considering the disparity information (1050) of the frame at time t. The electronic device (101) can obtain fatigue on a frame-by-frame basis by applying the weight map (1060) information, which assigns weights according to the disparity, to the temporal disparity information (1030). In the weight map (1060), the disparity of an object displayed on the front of the display can be expressed as a negative disparity (-1, -2, -3, -4) to the right of the position (0) corresponding to the display. In the weight map (1060), the disparity of an object displayed on the back of the display can be represented as positive disparity (1, 2, 3, 4) to the left of the position (0) corresponding to the display. In the weight map (1060), the weights can be represented by a sigmoid curve in which the weights increase steeply based on a specific negative disparity at the position (0) corresponding to the display, and a sigmoid curve in which the weights increase steeply based on a specific positive disparity. The electronic device (101) can determine the sigmoid curve by considering the viewing environment (e.g., display size and viewing distance). The range between the negative disparity and the positive disparity that serve as the reference for the steep increase in weights in the sigmoid curve may be referred to as the 'weighted disparity range'. A method for determining the sigmoid curve and / or the weighted disparity range will be described later with reference to FIGS. 11 through 13.

[0111] According to one embodiment, when calculating fatigue, disparity at a depth close to the display (e.g., the display (160) of FIG. 1) (movement of an object close to the display (160)) may contribute little or no contribution. When calculating fatigue, disparity at a depth far from the display (e.g., the display (160) of FIG. 1) (movement of an object far from the display (160)) may contribute significantly.

[0112] Figure 11 explains the factors affecting fatigue.

[0113] Referring to the left portion of FIG. 11, according to one embodiment, a viewer can comfortably view an object displayed in a first area (1100) between the dotted lines. The first area (1100) may be an area close to the display (e.g., the display (160) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1). A second or third area (1101; 1102) outside the dotted lines may be an area far from the display (160) in the front or rear direction of the display (160). Disparity (hereinafter referred to as 'first disparity') in the first area (1100) may be low. For example, the first disparity of the portion displayed on the display (160) may be 0. An object displayed within the first area (1100) may not cause fatigue. An object displayed within the first area (1100) may not cause fatigue even with large movements.

[0114] Referring to the middle portion of FIG. 11, according to one embodiment, an object expressed within a first region (1100) may not cause fatigue. If the disparity (1120) detected in a second or third region (1101; 1102) (hereinafter referred to as "second or third disparity") is small, the fatigue may be low even if the movement of the object expressed in the second or third region (1101; 1102) is fast.

[0115] Referring to the right portion of FIG. 11, according to one embodiment, an object expressed within a first region (1100) may not cause fatigue. However, if the second or third disparity (1130) detected in a second or third region (1101; 1102) is large and the movement of the object expressed in the region (1101; 1102) is fast, it may cause high fatigue. Viewers may feel fatigue (e.g., dizziness) due to images having movement exceeding a threshold in a region with large disparity.

[0116] FIG. 12 illustrates a method for determining a weighted disparity range in an electronic device according to one example (e.g., the electronic device (101) of FIG. 1).

[0117] In FIG. 12, according to one embodiment, the viewer may perceive the expressed stereoscopic image as being displayed on the front of the display (1201), that is, between the display (1201) and the viewer. The electronic device (101) may obtain a distance (1209) at which the viewer feels fatigue. An object seen at a distance closer to the viewer than the distance (1209) at which the viewer feels fatigue may cause fatigue to the viewer. For example, the electronic device (101) may receive the distance (1209) at which the viewer feels fatigue.

[0118] According to one embodiment, the electronic device (101) can obtain actual disparity (1207) on the display (1201) based on the distance at which the viewer feels fatigue (1209), the viewing distance (1203), and the distance between the viewer's eyes (1205).

[0119] According to one embodiment, the actual disparity ds phy (1207) can be calculated from the following equation.

[0120] ds phy : d_e = d_o : d_u - d_o

[0121] In the above formula,

[0122] d_o = distance where the viewer feels fatigued (1209);

[0123] d_u = Viewing Distance(1203);

[0124] d_e = the distance between the viewer's eyes (1205).

[0125] According to one embodiment, the electronic device (101) can calculate a viewing distance (1203) based on the size of the display (1201). The viewing distance (1203) may be the distance (1203) between the viewer's eyes and the display. For example, if the size of the display (1201) is 32 inches, the electronic device (101) can obtain a viewing distance of 70 cm.

[0126] According to one embodiment, the electronic device (101) can obtain the distance between the eyes of a viewer (1205). The electronic device (101) can receive the average distance between the eyes of a viewer. For example, the average distance between the eyes of a viewer may be 6.5 cm.

[0127] According to one embodiment, the electronic device (101) has an actual disparity ds phy Pixel disparity ds from (1207) pix You can obtain.

[0128] ds pix = ds phy *R / W

[0129] In the above formula,

[0130] R = horizontal resolution of the display (1201);

[0131] W = can mean the width of the display (1213).

[0132] In FIG. 12, the stereoscopic image is shown as being displayed on the front of the display (1201), but according to one embodiment, pixel disparity can be obtained in the same way even when the stereoscopic image is displayed on the back of the display (1201).

[0133] According to one embodiment, the electronic device (101) obtains pixel disparity ds from the front direction of the display (1201). pix and pixel disparity ds obtained from the rear direction of the display (1201) pix Weight map information can be obtained based on the weighted disparity range between. The weighted disparity range may represent a boundary for applying weights to temporal disparity differences according to the size of the disparity. For example, the temporal disparity information may be the disparity difference between a frame at time t-1 and a frame at time t. The electronic device (101) may apply a low weight to the temporal disparity information in pixels having disparity within the weighted disparity range. The electronic device (101) may apply a high weight to the temporal disparity information in pixels having disparity outside the weighted disparity range.

[0134] According to one embodiment, the electronic device (101) may obtain temporal disparity information from pixels having disparity outside the weighted disparity range. The electronic device (101) may not obtain temporal disparity information from pixels having disparity within the weighted disparity range. For example, the temporal disparity information may be the disparity difference between the frame at time t-1 and the frame at time t. The electronic device (101) may obtain information regarding the fatigue of the image based on the obtained temporal disparity information.

[0135] Figure 13 illustrates the factors affecting the weighted disparity range.

[0136] In FIG. 13, according to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) can determine a weighted disparity range based on the display size and display resolution. The electronic device (101) can adjust the disparity based on the display size and display resolution. The electronic device (101) can display a stereoscopic image on a first display (1310), a second display (1320), and a third display (1330). The width of the first display (1310) may be a first size. The width of the second display (1320) may be a second size. The width of the third display (1330) may be a third size. The first size may be smaller than the second size. The second size may be smaller than the third size. It may be assumed that the horizontal resolutions of the first display (1310), the second display (1320), and the third display (1330) are all the same. For example, when a stereoscopic image is displayed on the first display (1310) with a pixel disparity range of -10 to 10, the actual disparity range may be -3 to 3. For example, when a stereoscopic image is displayed on the third display (1330) with a pixel disparity range of -10 to 10, the actual disparity range may be -20 to 20. The larger the display size, the larger the actual disparity range may be compared to the pixel disparity range. Viewers can perceive a sense of depth based on the actual disparity. The larger the display size, the greater the sense of depth can be conveyed to the viewer.

[0137] According to one embodiment, a graph drawn below the second display (1320) represents the pixel disparity range (1321; 1323) when a stereoscopic image is played through the second display (1320). Two graphs are drawn below the first display (1310). Among these, the graph drawn on the left represents the actual disparity range (1311; 1313) when a stereoscopic image is played through the first display (1310). The graph drawn on the right represents the pixel disparity range (1315; 1317) when a stereoscopic image is played through the first display (1310). Two graphs are drawn below the third display (1330). Among these, the graph drawn on the left represents the pixel disparity range (1335; 1337) when a stereoscopic image is played through the third display (1330). The graph drawn on the right shows the actual disparity range (1331; 1333) when a stereoscopic image is played through the third display (1330). When a stereoscopic image is played through the third display (1330), the pixel disparity range may be smaller than when a stereoscopic image is played through the second display (1320). When a stereoscopic image is played through the second display (1320), the pixel disparity range may be smaller than when a stereoscopic image is played through the first display (1310). The electronic device (101) can determine the pixel disparity range as a weighted disparity range.

[0138] According to one embodiment, the electronic device (101) can generate a weight map based on a weight disparity range. A larger size of the display can amplify the sense of depth. A greater sense of depth can cause greater fatigue for the viewer. In the weight map, the weight disparity range may become narrower as the size of the display increases. The weight disparity range may also be determined by the resolution of the display. For example, the resolution of the display may be measured in pixels per inch (PPI). A larger horizontal width of the pixels may result in a larger pixel disparity. Large pixel disparity can amplify the sense of depth. A larger horizontal width of the pixels may result in a decrease in pixels per inch. In the weight map, the weight disparity range may increase as the number of pixels per inch increases.

[0139] FIG. 14 illustrates an exemplary method for controlling a stereoscopic image in an electronic device (101) according to one example.

[0140] According to one embodiment, an electronic device (101) may receive user input regarding a first object (1420) included in a stereoscopic image. For example, while watching a stereoscopic image, a user (1410) may touch a point on a display corresponding to an area where the first object (1420) to which the user wants to enhance the stereoscopic effect is displayed. Upon receiving user input, the electronic device (101) may increase the disparity of the binocular image for displaying the first object (1420). In this case, the total amount of fatigue in the stereoscopic image may increase. The electronic device (101) may adjust the speed of playback of the stereoscopic image (e.g., frame rate) to control the fatigue in the stereoscopic image to within a threshold. Although the stereoscopic effect of a specific object in the stereoscopic image may be enhanced, the fatigue in the stereoscopic image may not increase because the speed of the stereoscopic image is slowed down.

[0141] According to one embodiment, an electronic device (101) comprises a memory (130) including one or more storage media for storing instructions; and at least one processor (120) including a processing circuit, wherein when the instructions are executed individually or collectively by at least one processor, the electronic device is caused to perform at least one operation, the at least one operation comprising: an operation of obtaining first fatigue information based on frames played during a first time interval; an operation of obtaining second fatigue information based on frames played during a second time interval; and an operation of controlling a stereoscopic image to be displayed through a display based on the first fatigue information and the second fatigue information, wherein the second time interval may include at least a portion of the first time interval.

[0142] According to one embodiment, the first fatigue information may include a first ratio of frames having a fatigue level exceeding a first threshold among the first frames played during the first time interval and / or the sum of the fatigue levels of the first frames.

[0143] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may be caused to perform an operation of filtering information regarding a frame having a disparity exceeding a threshold among the first frames played during the first time interval.

[0144] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may be caused to perform an operation of applying a weight to information regarding the fatigue of one or more frames most recently played among the first frames played during the first time interval.

[0145] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may be caused to perform an operation of adjusting the disparity range on a frame-by-frame basis based on the first fatigue information.

[0146] According to one embodiment, the second fatigue information may include a second ratio of frames having a fatigue level exceeding a second threshold among the second frames played during the second time interval.

[0147] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may be caused to perform an operation to determine a method for adjusting the disparity range according to the second ratio.

[0148] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may be caused to perform: an operation to decrease the disparity range at once and an operation to increase the disparity range at once if the second ratio is within the first range; and an operation to decrease the disparity range at once and an operation to increase the disparity range in steps if the second ratio is outside the first range (e.g., within the second range or within the third range).

[0149] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may be caused to perform an operation of acquiring fatigue on a frame-by-frame basis by applying weight map information that assigns weights according to disparity to temporal disparity information.

[0150] According to one embodiment, when the instructions are executed individually or collectively by at least one processor, the electronic device may be caused to perform: an operation of obtaining a distance at which a viewer feels fatigued; an operation of obtaining an actual disparity corresponding to the distance at which a viewer feels fatigued; an operation of obtaining a pixel disparity corresponding to the actual disparity; and an operation of obtaining weight map information based on the pixel disparity.

[0151] According to one embodiment, a control method for an electronic device (101) comprises: an operation of acquiring first fatigue information based on frames played during a first time interval; an operation of acquiring second fatigue information based on frames played during a second time interval; and an operation of controlling a stereoscopic image to be displayed through a display based on the first fatigue information and the second fatigue information, wherein the second time interval may include at least a portion of the first time interval.

[0152] According to one embodiment, in the control method, the first fatigue information may include a first ratio of frames having a fatigue level exceeding a first threshold among the first frames played during the first time interval and / or the sum of the fatigue levels of the first frames.

[0153] According to one embodiment, in the control method, the operation of acquiring the first fatigue information may include the operation of filtering information regarding a frame having a disparity exceeding a threshold among the first frames played during the first time interval.

[0154] According to one embodiment, in the control method, the operation of acquiring the first fatigue information may include the operation of applying a weight to information regarding the fatigue of one or more frames that were most recently played among the first frames played during the first time interval.

[0155] According to one embodiment, in the control method, the operation of controlling a stereoscopic image to be displayed through the display may include the operation of adjusting the disparity range frame by frame based on the first fatigue information.

[0156] According to one embodiment, in the control method, the second fatigue information may include a second ratio of frames having a fatigue level exceeding a second threshold among the second frames played during the second time interval.

[0157] According to one embodiment, in the control method, the operation of controlling a stereoscopic image to be displayed through the display may include an operation of determining a method for adjusting the disparity range according to the second ratio.

[0158] According to one embodiment, the control method may include, if the second ratio is within a first range, an operation to decrease the disparity range at once and an operation to increase the disparity range at once; and if the second ratio is outside the first range (e.g., within a second range or within a third range), an operation to decrease the disparity range at once and an operation to increase the disparity range in steps.

[0159] According to one embodiment, the control method may include an operation of obtaining fatigue on a frame-by-frame basis by applying weight map information that assigns weights according to disparity to temporal disparity information.

[0160] According to one embodiment, the control method may include: an operation of obtaining a distance at which a viewer feels fatigue; an operation of obtaining an actual disparity corresponding to the distance at which the viewer feels fatigue; an operation of obtaining a pixel disparity corresponding to the actual disparity; and an operation of obtaining weight map information based on the pixel disparity.

[0161] The various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features described in the present disclosure to specific examples, and should be understood to include various modifications, equivalents, or substitutions of such examples. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In the present disclosure, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” each may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as “first,” “second,” or “first” or “second” may be used simply to distinguish a component from another corresponding component and do not limit the components in other aspects (e.g., importance or order). Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0162] The term “module” as used in various embodiments of the present disclosure may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one example, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0163] Various embodiments of the present disclosure may be implemented as software (e.g., a program) comprising one or more instructions stored in a storage medium (e.g., internal memory or external memory) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0164] According to one embodiment, the method according to the various examples disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). ™ It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0165] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various examples, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various examples, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device, A memory comprising one or more storage media for storing instructions; and It includes at least one processor including a processing circuit, and When the above instructions are executed individually or collectively by at least one processor, the electronic device is caused to perform at least one operation, and The above at least one operation is: An operation to acquire first fatigue information based on frames played during a first time interval; An operation to acquire second fatigue information based on frames played during a second time interval; and Operation of controlling a stereoscopic image to be displayed through a display based on the first fatigue information and the second fatigue information. Includes, An electronic device in which the second time interval comprises at least a portion of the first time interval.

2. In Paragraph 1, An electronic device comprising at least one of the first fatigue information, the first ratio of frames having a fatigue level exceeding a first threshold among the first frames played during the first time interval, or the sum of the fatigue levels of the first frames.

3. In Paragraph 1 or 2, When the above instructions are executed individually or collectively by at least one processor, the electronic device: An operation to filter information regarding frames having a disparity exceeding a threshold among the first frames played during the first time interval. An electronic device that causes to perform.

4. In any one of paragraphs 1 through 3, When the above instructions are executed individually or collectively by at least one processor, the electronic device: An operation of applying weights to information regarding the fatigue of one or more frames most recently played among the first frames played during the first time interval. An electronic device that causes to perform.

5. In any one of paragraphs 1 through 4, When the above instructions are executed individually or collectively by at least one processor, the electronic device: Operation of adjusting the disparity range frame by frame based on the above-mentioned first fatigue information An electronic device that causes to perform.

6. In any one of paragraphs 1 through 5, The electronic device, wherein the second fatigue information includes a second ratio of frames having a fatigue level exceeding a second threshold among the second frames played during the second time interval.

7. In Paragraph 6, When the above instructions are executed individually or collectively by at least one processor, the electronic device: Operation of determining a method for adjusting the disparity range according to the second ratio above An electronic device that causes to perform.

8. In Paragraph 6 or 7, When the above instructions are executed individually or collectively by at least one processor, the electronic device: If the second ratio is within the first range, an operation to decrease the disparity range at once and an operation to increase the disparity range at once; and If the second ratio deviates from the first range, the operation of decreasing the disparity range all at once and the operation of increasing the disparity range step by step An electronic device that causes to perform.

9. In any one of paragraphs 1 through 8, When the above instructions are executed individually or collectively by at least one processor, the electronic device: An operation to acquire fatigue on a frame-by-frame basis by applying weight map information that assigns weights according to disparity to temporal disparity information. An electronic device that causes to perform.

10. In Paragraph 9, When the above instructions are executed individually or collectively by at least one processor, the electronic device: Action of acquiring a distance at which the viewer feels fatigued; An operation to obtain actual disparity corresponding to the distance at which the viewer feels fatigued; The operation of obtaining pixel disparity corresponding to the actual disparity above; and Operation of obtaining the weight map information based on the pixel disparity An electronic device that causes to perform.

11. In a method for controlling an electronic device, An operation to acquire first fatigue information based on frames played during a first time interval; An operation to acquire second fatigue information based on frames played during a second time interval; and Operation of controlling a stereoscopic image to be displayed through a display based on the first fatigue information and the second fatigue information. Includes, A control method in which the second time interval comprises at least a portion of the first time interval.

12. In Paragraph 11, A control method comprising at least one of the first fatigue information, wherein the first fatigue information comprises a first ratio of frames having a fatigue level exceeding a first threshold among the first frames played during the first time interval, or the sum of the fatigue levels of the first frames.

13. In Paragraph 11 or 12, A control method comprising an operation to control a stereoscopic image to be displayed through the above display, wherein the operation to control the stereoscopic image includes an operation to adjust the disparity range frame by frame based on the first fatigue information.

14. In any one of paragraphs 11 through 13, A control method comprising the second fatigue information including a second ratio of frames having a fatigue level exceeding a second threshold among the second frames played during the second time interval.

15. In Paragraph 14, If the second ratio is within the first range, an operation to decrease the disparity range at once and an operation to increase the disparity range at once; and If the second ratio deviates from the first range, the operation of decreasing the disparity range all at once and the operation of increasing the disparity range step by step A control method including

Images