Electronic device including display device and operating method of electronic device

Abstract

This electronic device comprises a display device, a memory, and a processor. The display device comprises a base layer, a circuit layer disposed on the base layer, a pixel layer disposed on the circuit layer and including a plurality of light-emitting pixels and a plurality of light-receiving pixels, an optical layer disposed on the pixel layer, and a lens layer disposed on the optical layer and including a plurality of lenses corresponding to the plurality of light-receiving pixels. The plurality of lenses are disposed such that the optical axes of the plurality of lenses do not pass through the centers of the plurality of light receiving pixels. According to the electronic device, an image may be displayed using the plurality of light-emitting pixels and depth information may be obtained on the basis of disparity between a plurality of view images obtained using the plurality of light-receiving pixels.

Description

Electronic device including a display device and method of operation of the electronic device

[0001] The present disclosure relates to an electronic device including a display device and a method of operating the electronic device. Specifically, it relates to an electronic device including a display device that displays an image and acquires depth information, and a method of operating the electronic device.

[0002] Thanks to the advancement of electronic technology, various types of electronic devices are being developed and distributed. Furthermore, through the development of camera technology, cameras can be incorporated into electronic devices.

[0003] Conventional electronic devices are equipped with a display that displays images, and can display various images to provide to the user.

[0004] In addition, technology is being utilized in which the user can capture images through a camera included in the electronic device, and the electronic device displays the acquired images or stores and manages them within the electronic device.

[0005] In addition, the electronic device includes both a display and a camera, acquires user input based on an image acquired through the camera, and can control the image displayed on the display based on the acquired user input.

[0006] In addition, depth information of the user utilizing the electronic device can be acquired through depth sensors, etc., and utilized to control the image displayed on the display.

[0007] One embodiment of the present disclosure provides an electronic device. The electronic device may include a display device. The electronic device may include a memory in which a program or at least one instruction is stored. The electronic device may include at least one processor including processing circuitry. The display device may include a base layer. The display device may include a circuit layer disposed on the base layer. The display device may include a pixel layer disposed on the circuit layer and comprising a plurality of light-emitting pixels and a plurality of light-receiving pixels. The display device may include an optical layer disposed on the pixel layer. The display device may include a lens layer disposed on the optical layer and comprising a plurality of lenses corresponding to a plurality of light-receiving pixels. The plurality of lenses may be arranged so that the optical axis of the plurality of lenses does not pass through the center of the plurality of light-receiving pixels. By having at least one processor execute a program or at least one instruction stored in memory individually or collectively, the electronic device may display an image through a plurality of light-emitting pixels. By having at least one processor execute a program stored in memory or at least one instruction individually or collectively, the electronic device can acquire depth information based on the disparity between multiple view images acquired through multiple light-receiving pixels.

[0008] In one embodiment of the present disclosure, a method of operation of an electronic device including a display device may be provided. The display device may include a base layer. The display device may include a circuit layer disposed on the base layer. The display device may include a pixel layer disposed on the circuit layer and comprising a plurality of light-emitting pixels and a plurality of light-receiving pixels. The display device may include an optical layer disposed on the pixel layer. The display device may include a lens layer disposed on the optical layer and comprising a plurality of lenses corresponding to a plurality of light-receiving pixels. The plurality of lenses may be arranged so that the optical axis of the plurality of lenses does not pass through the center of the plurality of light-receiving pixels. The method of operation of the electronic device may include the step of displaying an image through a plurality of light-emitting pixels. The method of operation of the electronic device may include the step of acquiring depth information based on the disparity between a plurality of view images acquired through a plurality of light-receiving pixels.

[0009] In one embodiment of the present disclosure, a computer-readable recording medium may be provided on which a program for performing at least one of the embodiments of the method of operating the disclosed electronic device is recorded on a computer.

[0010] The technical problems to be solved in this document are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs from the description below.

[0011] The present disclosure may be understood from the combination of the following detailed description and the accompanying drawings, where reference numerals denote structural elements.

[0012] FIG. 1 is a drawing for explaining the operation of an electronic device according to one embodiment of the present disclosure.

[0013] FIG. 2 is a block diagram for explaining the configuration of an electronic device according to one embodiment of the present disclosure.

[0014] FIG. 3 is a block diagram for explaining the configuration of a display device according to one embodiment of the present disclosure.

[0015] FIG. 4 is a drawing for explaining the configuration of a display device according to one embodiment of the present disclosure.

[0016] FIG. 5 is a drawing for explaining the arrangement of a plurality of components included in a display device according to one embodiment of the present disclosure.

[0017] FIG. 6 is a drawing for explaining a circuit layer and a light-emitting layer according to one embodiment of the present disclosure.

[0018] FIG. 7 is a drawing for explaining the arrangement of a plurality of components included in a display device according to one embodiment of the present disclosure.

[0019] FIG. 8 is a flowchart for explaining a method of operation of an electronic device according to one embodiment of the present disclosure.

[0020] FIG. 9 is a flowchart for explaining the operation of acquiring depth information through an electronic device according to one embodiment of the present disclosure.

[0021] FIG. 10a is a drawing for explaining that a light-receiving pixel is arranged in each of a plurality of lenses according to one embodiment of the present disclosure.

[0022] FIG. 10b is a drawing for explaining that a light-receiving pixel is arranged in each of a plurality of lenses according to one embodiment of the present disclosure.

[0023] FIG. 11a is a drawing for explaining that two light-receiving pixels are arranged in each of a plurality of lenses according to one embodiment of the present disclosure.

[0024] FIG. 11b is a drawing for explaining that two light-receiving pixels are arranged in each of a plurality of lenses according to one embodiment of the present disclosure.

[0025] FIG. 12a is a drawing for explaining that four light-receiving pixels are arranged in each of a plurality of lenses according to one embodiment of the present disclosure.

[0026] FIG. 12b is a drawing for explaining that four light-receiving pixels are arranged in each of a plurality of lenses according to one embodiment of the present disclosure.

[0027] FIG. 12c is a drawing for explaining that four light-receiving pixels are arranged in each of a plurality of lenses according to one embodiment of the present disclosure.

[0028] FIG. 13 is a flowchart illustrating an operation to acquire depth information according to the rotational state of an electronic device according to one embodiment of the present disclosure.

[0029] FIG. 14 is a drawing for explaining the operation of acquiring depth information according to the rotational state of an electronic device according to one embodiment of the present disclosure.

[0030] FIG. 15 is a drawing for explaining the operation of acquiring depth information according to the rotational state of an electronic device according to one embodiment of the present disclosure.

[0031] The terms used in this disclosure will be briefly explained, and an embodiment of this disclosure will be described in detail.

[0032] Throughout this disclosure, unless specifically stated otherwise, "or" is inclusive and not exclusive. Accordingly, "A or B" may mean "A, B, or both" unless clearly indicated otherwise by the context.

[0033] In the present disclosure, the expression “at least one of a, b, or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “a, b, and c all”, or variations thereof.

[0034] The terms used in this disclosure have been selected to be as widely used as possible, taking into account the functions in the embodiments of this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description section of the relevant embodiments of this disclosure. Therefore, the terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.

[0035] Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art as described in this specification.

[0036] Throughout this disclosure, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part," "module," etc., as used in this disclosure refer to a unit that processes at least one function or operation, and may be implemented in hardware or software, or as a combination of hardware and software.

[0037] The expression “configured to” as used in this disclosure may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware. Instead, in some situations, the expression “system configured to” may mean that the system is “capable of” together with other devices or components. For example, the phrase “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing said operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or an application processor) capable of performing said operations by executing one or more software programs stored in memory.

[0038] In addition, when a component is described in the present disclosure as being “connected” or “connected” to another component, it should be understood that the component may be directly connected to or directly connected to the other component, but unless otherwise specifically stated, it may also be connected or connected through another component in between.

[0039] In this specification, where a component (or region, layer, part, etc.) is described as being “on,” “connected,” or “joined” another component, it means that it may be directly placed / connected / joined on the other component, or that a third component may be placed between them.

[0040] Meanwhile, in the present application, "direct placement" may mean that there are no additional layers, films, regions, plates, etc. added between a part such as a layer, film, region, or plate and another part. For example, "direct placement" may mean that two layers or two members are placed without using additional members such as adhesive members between them.

[0041] In addition, terms such as “below,” “lower side,” “above,” “upper side,” “on,” “upper surface,” and “lower surface” are used to describe the relationships between components illustrated in the drawings. These terms are relative concepts and are described based on the directions indicated in the drawings.

[0042] It should be understood that the blocks in each flowchart and combinations of flowcharts can be executed by one or more computer programs containing computer-executable instructions. One or more computer programs may be stored all in a single memory or may be partitioned and stored in multiple different memories.

[0043] All functions or operations described in this document may be processed by a single processor or a combination of multiple processors.

[0044] Embodiments of the present disclosure are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, an embodiment of the present disclosure may be implemented in various different forms and is not limited to the embodiment described herein. Furthermore, in order to clearly explain an embodiment of the present disclosure in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the present disclosure are denoted by similar reference numerals.

[0045] Embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0046] FIG. 1 is a drawing for explaining the operation of an electronic device according to one embodiment of the present disclosure.

[0047] Referring to FIG. 1, in one embodiment of the present disclosure, FIG. 1 illustrates an electronic device (100) and a user (200) using the electronic device (100).

[0048] In one embodiment of the present disclosure, the electronic device (100) may include a display device (110). The display device (110) may have a shape parallel to the x-axis direction (10) and the y-axis direction (20).

[0049] The electronic device (100) in FIG. 1 is depicted as a bar-shaped rigid type electronic device, but is not particularly limited thereto. In one embodiment of the present disclosure, the electronic device (100) may be a foldable, rollable, or sliderable type electronic device. In this case, the display device (110) may also have a bent or folded shape based on a plane defined by the x-axis direction (10) and the y-axis direction (20).

[0050] Hereinafter, the normal direction substantially perpendicular to the plane defined by the x-axis direction (10) and the y-axis direction (20) is defined as the z-axis direction (30). In this specification, the meaning of “when viewed in a plane” may mean the state viewed from the z-axis direction (30). That is, the plane may be parallel to the plane defined by the x-axis direction (10) and the y-axis direction (20).

[0051] In one embodiment of the present disclosure, the electronic device (100) may be implemented as an electronic device of various shapes, such as a smartphone, a laptop computer, a tablet PC, a television, a mobile device, a digital signage device, and a head-mounted display device.

[0052] In one embodiment of the present disclosure, the display device (110) may include a pixel layer comprising a plurality of light-emitting pixels. The electronic device (100) may display an image (111) by controlling through the plurality of light-emitting pixels.

[0053] In one embodiment of the present disclosure, an electronic device (100) may display an image (111) in the z-axis direction (30) through a display device (110) and provide it to a user (200). The direction in which the image (111) is displayed through the display device (110) may correspond to the front surface of the electronic device (100).

[0054] In one embodiment of the present disclosure, the display device (110) may include a pixel layer comprising a plurality of light-receiving pixels. The electronic device (100) can sense a user (200) using the electronic device (100) through the plurality of light-receiving pixels and acquire a plurality of images (120, 121) corresponding to the user (200).

[0055] However, the present disclosure is not limited thereto, and the electronic device (100) can acquire a plurality of images (120, 121) by sensing the environment around the electronic device (100) (e.g., an object or background located around the electronic device (100)) through a plurality of light-receiving pixels.

[0056] Hereinafter, each of the plurality of images (120, 121) obtained through the display device (110) will be referred to as a view image.

[0057] In one embodiment of the present disclosure, the display device (110) may include a lens layer disposed on a pixel layer and comprising a plurality of lenses corresponding to a plurality of light-receiving pixels. In one embodiment of the present disclosure, the lens layer may be disposed on the upper surface of the pixel layer with respect to the z-axis direction (30).

[0058] In one embodiment of the present disclosure, light incident from outside the electronic device (100) to the display device (110) can be concentrated through a plurality of lenses included in the lens layer and provided to a plurality of light-receiving pixels.

[0059] In one embodiment of the present disclosure, a plurality of lenses may be arranged so that the optical axis of the plurality of lenses does not pass through the center of the plurality of light-receiving pixels.

[0060] In this case, the "optical axis" may refer to an imaginary line passing through the geometric center of the lens. The optical axis may refer to the axis of rotational symmetry of the lens.

[0061] In one embodiment of the present disclosure, the optical axis of each of the plurality of lenses may extend in a direction parallel to the z-axis direction (30). However, it is understood that the direction of the optical axis may vary depending on the shape or arrangement of each of the plurality of lenses. Hereinafter, for convenience of explanation, the optical axis of each of the plurality of lenses will be described as an imaginary line extending in a direction parallel to the z-axis direction (30).

[0062] In one embodiment of the present disclosure, the optical axis of each lens may be positioned so as not to pass through the center of at least one corresponding light-receiving pixel. Specifically, each lens may be positioned to correspond to one light-receiving pixel or to correspond to two or more light-receiving pixels.

[0063] In one embodiment of the present disclosure, when a light-receiving pixel is positioned to correspond to each lens, each lens may be positioned so that its optical axis does not pass through the center of the light-receiving pixel. When two or more light-receiving pixels are positioned to correspond to each lens, each lens may be positioned so that its optical axis does not pass through the center of two or more light-receiving pixels.

[0064] Hereinafter, the arrangement relationship between a plurality of lenses and a plurality of light-receiving pixels will be described later in FIGS. 3 to 5 and FIGS. 10a to 12c.

[0065] In one embodiment of the present disclosure, as the optical axes of a plurality of lenses are arranged so as not to pass through the center of a plurality of light-receiving pixels, the position of the user located in a plurality of view images (120, 121) obtained through a display device (110) may be different.

[0066] Specifically, depending on the direction in which the optical axis of the lens and the center of at least one light-receiving pixel corresponding to the lens are shifted, the position of the user (200) included in the view image sensed through at least one light-receiving pixel may differ.

[0067] At this time, "view" may correspond to a position where a different side of the user (200) can be seen. In one embodiment of the present disclosure, the appearance of the user (200) viewed from different views may be different. "View image" may mean an image sensed from different views.

[0068] In one embodiment of the present disclosure, a plurality of light-receiving pixels may include a plurality of first light-receiving pixels having a center shifted in a first direction from the optical axis of the lens and a plurality of second light-receiving pixels having a center shifted in a direction opposite to the first direction from the optical axis of the lens. In this case, the first direction may mean the x-axis direction. However, this is not limited, and the first direction may mean any direction within a plane formed by the x-axis direction (10) and the y-axis direction (20). Additionally, the direction opposite to the first direction may be referred to as the second direction, and is not limited to either one.

[0069] In one embodiment of the present disclosure, a plurality of view images (120, 121) may include a first view image (120) and a second view image (121).

[0070] In one embodiment of the present disclosure, a user (200) may be positioned in a first view image (120) obtained through a plurality of first light-receiving pixels, shifted in a first direction. In a second view image (121) obtained through a plurality of first light-receiving pixels, the user (200) may be positioned in a direction opposite to the first direction, shifted in the opposite direction.

[0071] In one embodiment of the present disclosure, the position of the user (200) included in the first view image (120) within the first view image (120) and the position of the user (200) included in the second view image (121) within the second view image (121) may be different.

[0072] In one embodiment of the present disclosure, the positional difference between pixels corresponding to objects located at the same distance from the electronic device (100) in the first view image (120) and the second view image (121) obtained by sensing the user (200) can be defined as "disparity".

[0073] In one embodiment of the present disclosure, the parallax may vary depending on the distance between the display device (110) and the object to be sensed through the display device (110). Specifically, as the distance between the electronic device (100) including the display device (110) and the user (200) increases, the parallax of the user (200) in the first view image (120) and the second view image (121) may decrease. As the distance between the electronic device (100) including the display device (110) and the user (200) decreases, the parallax of the user (200) in the first view image (120) and the second view image (121) may increase.

[0074] In one embodiment of the present disclosure, an electronic device (100) can obtain depth information (130) for a user (200) based on a first view image (120) and a second view image (121). The electronic device (100) can obtain depth information (130) based on the parallax between the first view image (120) and the second view image (121).

[0075] However, the present disclosure is not limited thereto, and it is obvious that the types of view images used by the electronic device (100) to acquire depth information (130) may vary depending on the positional relationship between the plurality of lenses and the plurality of light-receiving pixels.

[0076] Additionally, in one embodiment of the present disclosure, the display device (110) may include an optical layer disposed between a pixel layer and a lens layer. Light provided to the display device (110) from the outside may be provided to a plurality of light-receiving pixels through a plurality of lenses and an optical layer.

[0077] At this time, the thickness of the optical layer can be determined such that a plurality of light-receiving pixels are located at the focal length of each of the plurality of lenses. Through the optical layer, the focal point of each of the plurality of lenses can be aligned with the corresponding plurality of light-receiving pixels. Accordingly, a clear view image can be obtained through the display device (110).

[0078] In one embodiment of the present disclosure, the electronic device (100) can sense an object (e.g., a user's (200) finger, etc.) that is in contact with the display device (110) (e.g., touching the display device (110) or a part of a body area in contact with the display device (110)).

[0079] In addition, in one embodiment of the present disclosure, the electronic device (100) can sense an object (e.g., a user (200) who is viewing an image (111) through the display device (110)) that is spaced apart from the display device (110) by a certain distance, and acquire a plurality of view images (120, 121) including the user (200).

[0080] In addition, in one embodiment of the present disclosure, the electronic device (100) may acquire depth information (130) about the user (200) based on a plurality of view images (120, 121) and sense the distance of the user (200) from the electronic device (100) or the location of the user (200).

[0081] In one embodiment of the present disclosure, the electronic device (100) can obtain a plurality of view images (120, 121) by capturing the surroundings through a display device (110) without having a separate configuration such as a camera or an IR (Infrared Ray) sensor.

[0082] In one embodiment of the present disclosure, the electronic device (100) may perform an operation of displaying an image (111) to a user (200) through a display device (110). The electronic device (100) may perform an operation of recognizing the user (200) and acquiring a plurality of view images (120, 121). Additionally, the electronic device (100) may perform the operation of displaying an image (111) to the user (200) through a display device (110) and recognizing the user (200) to acquire a plurality of view images (120, 121).

[0083] Through this, the electronic device (100) may acquire the movements, facial expressions, or instructions of a user (200) watching an image (111) through multiple view images (120, 121) and perform interaction with the user (200). Additionally, the electronic device (100) may improve the accuracy in performing interaction with the user (200) based on depth information (130) acquired through multiple view images (120, 121).

[0084] Hereinafter, the operation of the electronic device (100), the configuration of the display device (110) included in the electronic device (100), and the method of operation of the electronic device (100) will be described in FIGS. 2 to 15.

[0085] FIG. 2 is a block diagram for explaining the configuration of an electronic device according to one embodiment of the present disclosure.

[0086] Referring to FIGS. 1 and FIGS. 2, in one embodiment of the present disclosure, an electronic device (100) may include a display device (110), a memory (140), at least one processor (150), an input / output interface (160), and a communication interface (170).

[0087] However, not all of the components shown in FIG. 2 are essential components. The electronic device (100) may be implemented with more components than those shown in FIG. 2, or with fewer components.

[0088] A display device (110), memory (140), at least one processor (150), an input / output interface (160), and a communication interface (170) included in the electronic device (100) can each be electrically connected to each other.

[0089] In one embodiment of the present disclosure, as at least one processor (150) controls the display device (110), the electronic device (100) can display an image. Additionally, as at least one processor (150) controls the display device (110), the electronic device (100) can acquire an image by sensing light provided from the surroundings.

[0090] In one embodiment of the present disclosure, a display device (110) may include a plurality of light-emitting pixels. The plurality of light-emitting pixels included in the display device (110) may include organic light-emitting diodes. However, the present disclosure is not limited thereto, and the plurality of light-emitting pixels included in the display device (110) may include inorganic light-emitting diodes. However, the present disclosure is not limited thereto, and the display device (110) may include other types of light-emitting pixels capable of displaying images (e.g., a combination of a backlight and a color filter).

[0091] In one embodiment of the present disclosure, the display device (110) may include a plurality of light-receiving pixels. The plurality of light-receiving pixels included in the display device (110) may include inorganic photodiodes, for example, thin film transistor (TFT) photodiodes. The TFT photodiodes may include amorphous silicon PIN diodes or oxide thin film transistor (oxide TFT) photodiodes.

[0092] However, the present disclosure is not limited thereto, and a plurality of light-receiving pixels included in the display device (110) may include organic photodiodes.

[0093] Hereinafter, the components included in the display device (110) and the arrangement of the components in the display device (110) will be described later in FIGS. 3 to 7.

[0094] In one embodiment of the present disclosure, the memory (140) may store instructions, data structures, and program code that can be read by at least one processor (150). In one embodiment of the present disclosure, the memory (140) may be one or more. Operations performed by the electronic device (100) may be implemented by at least one processor (150) executing the instructions or code of a program stored in the memory (140).

[0095] In one embodiment of the present disclosure, the memory (140) may include at least one of a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), Mask ROM, Flash ROM, etc.), a hard disk drive (HDD), or a solid-state drive (SSD).

[0096] In one embodiment of the present disclosure, the memory (140) may not exist separately and may be configured to be included in at least one processor (150).

[0097] In one embodiment of the present disclosure, instructions or program code for performing functions or operations of an electronic device (100) may be stored in the memory (140). The instructions, algorithms, data structures, program code, and application programs stored in the memory (140) may be implemented in a programming or scripting language such as, for example, C, C++, Java, Python, assembler, etc.

[0098] In one embodiment of the present disclosure, various types of modules that can be used to perform the operation of the electronic device (100) may be stored in the memory (140).

[0099] In one embodiment of the present disclosure, the memory (140) may store an image display module (141), an image acquisition module (142), and a depth information acquisition module (143). However, not all modules illustrated in FIG. 2 are required. More modules than those illustrated in FIG. 2 may be stored in the memory (140), or fewer modules may be stored.

[0100] In one embodiment of the present disclosure, a 'module' included in the memory (140) may mean a unit that processes a function or operation performed by at least one processor (150). The 'module' included in the memory (140) may be implemented as software such as instructions, algorithms, data structures, or program code.

[0101] In one embodiment of the present disclosure, the image display module (141) may be composed of instructions or program code regarding an operation or function of displaying an image (111) through a display device (110).

[0102] In one embodiment of the present disclosure, by having at least one processor (150) execute instructions or program code of an image display module (141), the electronic device (100) can display an image (111) through a display device (110).

[0103] In one embodiment of the present disclosure, the image acquisition module (142) may be composed of instructions or program code regarding the operation or function of acquiring a plurality of view images (120, 121) through a display device (110).

[0104] In one embodiment of the present disclosure, by having at least one processor (150) execute instructions or program code of an image acquisition module (142), the electronic device (100) can sense and acquire a plurality of view images (120, 121) through a display device (110).

[0105] In one embodiment of the present disclosure, the depth information acquisition module (143) may be composed of instructions or program code regarding an operation or function of acquiring depth information based on a plurality of view images (120, 121). In this case, the depth information may include a depth map that includes a depth value corresponding to each of a plurality of pixels included in each view image.

[0106] In one embodiment of the present disclosure, the depth information acquisition module (143) may be composed of instructions or program code regarding an operation or function of acquiring depth information based on the parallax between a plurality of view images (120, 121).

[0107] In one embodiment of the present disclosure, the depth information acquisition module (143) may be composed of instructions or program code regarding an operation or function to acquire depth information based on the parallax between a plurality of view images, the degree to which the optical axes of a plurality of lenses are shifted from the center of a corresponding light-receiving pixel, the shape of a plurality of lenses, or the thickness of an optical layer.

[0108] In one embodiment of the present disclosure, the depth information acquisition module (143) may include a correction function for acquiring depth information based on the disparity between a plurality of view images (120, 121). In this case, the correction function may be a function for calculating the distance between the electronic device (100) and the object according to the disparity, which is pre-calculated based on the change in disparity between a plurality of view images including the object according to the change in distance between the electronic device (100) and the object.

[0109] In one embodiment of the present disclosure, the depth information acquisition module (143) may include a stereo matching algorithm that estimates the disparity between a first view image (120) and a second view image (121). The stereo matching algorithm may be an algorithm that performs preprocessing, such as distortion correction or epipolar alignment, on two view images, estimates the disparity between two images, and acquires depth information according to a triangulation relationship.

[0110] However, the present disclosure is not limited thereto, and the depth information acquisition module (143) may be composed of various commands or program codes capable of performing the operation of acquiring depth information based on a plurality of view images.

[0111] In one embodiment of the present disclosure, the depth information acquisition module (143) may include an artificial intelligence model. The artificial intelligence model included in the depth information acquisition module (143) may include a machine learning or deep learning model. In one embodiment of the present disclosure, the artificial intelligence model included in the depth information acquisition module (143) may include a CNN (Convolutional Neural Network) or a transformer, and may be an artificial intelligence model trained to infer depth information, such as a depth map, by receiving a plurality of view images as input.

[0112] In one embodiment of the present disclosure, by having at least one processor (150) execute instructions or program code of a depth information acquisition module (143), the electronic device (100) can acquire depth information based on the parallax between a plurality of view images.

[0113] In one embodiment of the present disclosure, at least one processor (150) may be configured to control a series of processes to operate an electronic device (100) according to the embodiments described below, and may be composed of one or more processors.

[0114] In one embodiment of the present disclosure, at least one processor (150) may be composed of at least one of a Central Processing Unit, a microprocessor, a Graphic Processing Unit, an Application Processor (AP), an Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), or a Communication Processor (CP), but is not limited thereto.

[0115] In one embodiment of the present disclosure, at least one processor (150) may be composed of circuitry such as a System on Chip (SoC) or an Integrated Circuit (IC). At least one processor (150) may include processing circuitry.

[0116] In one embodiment of the present disclosure, at least one processor (150) can execute various types of modules stored in memory (140). At least one processor (150) can execute at least one instruction constituting the various types of modules stored in memory (140) individually or collectively.

[0117] By executing a program or at least one instruction stored in memory (140), at least one processor (150) can process data according to a predefined operation rule.

[0118] In one embodiment of the present disclosure, at least one processor (150) may include a plurality of processors. In one embodiment of the present disclosure, at least one of a plurality of modules in memory (140) may be executed by any one of the plurality of processors. The remaining modules among the plurality of modules stored in memory (140) may be executed by another of the plurality of processors.

[0119] In one embodiment of the present disclosure, at least one processor (150) may include a controller (301, see FIG. 3), a data driver (302, see FIG. 3), and a sensing driver (303, see FIG. 3). However, the present disclosure is not limited thereto, and at least one of the controller (301), the data driver (302), or the sensing driver (303) may be a component included in the display device (110).

[0120] Additionally, in one embodiment of the present disclosure, at least one component of a scan driver (304, see FIG. 3) or a light-emitting driver (305, see FIG. 3) included in a display device (110) may be a component included in at least one processor (150).

[0121] Hereinafter, the controller (301), data driver (302), sensing driver (303), scan driver (304) and light-emitting driver (305) will be described later in FIG. 3.

[0122] In one embodiment of the present disclosure, the input / output interface (160) may perform input / output operations with an external electronic device using at least one of an input / output method including an HDMI port (High-Definition Multimedia Interface port), DVI (Digital Visual Interface), a component jack, a PC port, or a USB port (Universal Serial Bus port). However, the present disclosure is not limited to the above-mentioned input / output methods.

[0123] In one embodiment of the present disclosure, at least one processor (150) controls an input / output interface (160), so that the electronic device (100) can obtain an image signal and a scan control signal from an external electronic device, etc., through the input / output interface (160).

[0124] In one embodiment of the present disclosure, at least one processor (150) may provide depth information obtained through a depth information acquisition module (143) to an external electronic device through an input / output interface (160).

[0125] In one embodiment of the present disclosure, the electronic device (100) can display an image (111) through a display device (110) based on an image signal obtained through an input / output interface (160).

[0126] In one embodiment of the present disclosure, the communication interface (170) can perform data communication with an external server or an external electronic device using at least one of a data communication method including, for example, wired LAN, wireless LAN, Wi-Fi, Bluetooth, Zigbee, WFD (Wi-Fi Direct), infrared communication (IrDA, infrared Data Association), BLE (Bluetooth Low Energy), NFC (Near Field Communication), Wibro (Wireless Broadband Internet), WiMAX (World Interoperability for Microwave Access), SWAP (Shared Wireless Access Protocol), WiGig (Wireless Gigabit Alliance), and RF communication.

[0127] In one embodiment of the present disclosure, at least one processor (150) controls a communication interface (170), so that the electronic device (100) can perform data communication with an external server or an external electronic device.

[0128] In one embodiment of the present disclosure, at least one processor (150) may provide depth information obtained through the depth information acquisition module (143) to an external electronic device or an external server through a communication interface (170).

[0129] FIG. 3 is a block diagram for explaining the configuration of a display device according to one embodiment of the present disclosure.

[0130] Referring to FIGS. 1, 2 and 3, in one embodiment of the present disclosure, FIG. 3 shows a display device (110) included in an electronic device (100). FIG. 3 also shows a controller (301), a data driver (302), and a sensing driver (303) among a plurality of components included in at least one processor (150).

[0131] In one embodiment of the present disclosure, the display device (110) may include a scan driver (304) and a light-emitting driver (305).

[0132] In one embodiment of the present disclosure, the display device (110) may include a display area where an image (111) is displayed and a non-display area (e.g., a bezel area) adjacent to the display area where the image (111) is not displayed. In one embodiment of the present disclosure, a scan driver (304) and a light-emitting driver (305) may be placed in the non-display area.

[0133] In FIG. 3, the controller (301), data driver (302), and sensing driver (303) are shown as distinct components, but the present disclosure is not limited thereto. Operation by two or more of the controller (301), data driver (302), and sensing driver (303) may be performed in a single component.

[0134] Additionally, although the scan driver (304) and the light-emitting driver (305) are depicted as separate components, it is obvious that the operation of the scan driver (304) and the light-emitting driver (305) may be performed in a single component. Furthermore, a component that performs the operation of the scan driver (304) and the light-emitting driver (305) together may be composed of two or more components and placed within a non-display area.

[0135] In one embodiment of the present disclosure, the display device (110) may include a pixel layer (540, see FIG. 5) including a plurality of light-emitting pixels (311) and a plurality of light-receiving pixels (320), and a lens layer (570, see FIG. 5) including a plurality of lenses (330).

[0136] At this time, the lens layer (570) may represent an arrangement of multiple lenses (330). That is, if each of the multiple lenses (330) is a micro lens having a size in the micrometer unit, the lens layer may represent a micro lens array (MLA) representing an arrangement of multiple micro lenses.

[0137] In one embodiment of the present disclosure, the lens layer (570) may be disposed on the pixel layer (540). The optical axes of the plurality of lenses (330) may be disposed so as not to pass through the center of the corresponding plurality of light-receiving pixels (320).

[0138] In one embodiment of the present disclosure, a plurality of lenses (350) may include a plurality of first lenses (331) having a first optical axis shifted in a first direction from a plurality of light-receiving pixels (320). A plurality of lenses (350) may include a plurality of second lenses (332) having a second optical axis shifted in a direction opposite to the first direction from a plurality of light-receiving pixels (320). In this case, the first direction may mean the x-axis direction (10).

[0139] However, FIG. 3 illustrates that each lens corresponds to one light-receiving pixel and that the optical axis of each lens is shifted in the first direction or the opposite direction of the first direction of the corresponding light-receiving pixel, but the present disclosure is not limited thereto.

[0140] FIG. 3 is an example for explaining the arrangement relationship between a plurality of lenses (350) and a plurality of light-receiving pixels (320), and of course, the arrangement relationship between the plurality of lenses (350) and the plurality of light-receiving pixels (320) may include various examples in which the optical axis of the plurality of lenses (350) is arranged so as not to pass through the center of the corresponding plurality of light-receiving pixels.

[0141] In one embodiment of the present disclosure, FIG. 3 shows a plurality of light-emitting pixels (311), a plurality of light-receiving pixels (320), and a plurality of lenses (330) among the components included in the display device (110), but the present disclosure is not limited thereto, and the display device (110) may include more components than those shown in FIG. 3 (e.g., a base layer (500, see FIG. 5) or an optical layer (560, see FIG. 5), etc.).

[0142] In one embodiment of the present disclosure, each of the plurality of light-emitting pixels (311) may include a plurality of subpixels including a light-emitting diode that generates different color light. In this case, the light-emitting diode may refer to a light-emitting element including a light-emitting diode.

[0143] Specifically, each of the plurality of light-emitting pixels (311) may include a red subpixel including a red light-emitting diode that generates red color light, a green subpixel including a green light-emitting diode that generates green color light, and a blue subpixel including a blue color diode that generates blue color light.

[0144] However, the present disclosure is not limited thereto, and each of the plurality of light-emitting pixels (311) may include at least one subpixel comprising light-emitting diodes that generate various combinations of color light for displaying an image (111).

[0145] Although not shown in FIG. 3, each of the plurality of light-emitting pixels (311) may be electrically connected to at least one transistor and at least one capacitor. At least one transistor and at least one capacitor electrically connected to each of the plurality of light-emitting pixels (311) may be referred to as a light-emitting pixel driving circuit.

[0146] In one embodiment of the present disclosure, a light-emitting pixel driving circuit may be included in a circuit layer (510, see FIG. 5) included in a display device (110). In one embodiment of the present disclosure, a pixel layer (540) may be disposed on the circuit layer (510). Each of a plurality of light-emitting pixels (311) included in the pixel layer (540) may be electrically connected to a light-emitting pixel driving circuit included in the circuit layer (510).

[0147] In one embodiment of the present disclosure, the scan driver (304) and the light-emitting driver (305) may also include a plurality of transistors and may be included in the circuit layer (510). However, the present disclosure is not limited thereto, and if the data driver (302) is included in the display device (110), the data driver (302) may also be included in the circuit layer (510).

[0148] In one embodiment of the present disclosure, the display device (110) may include a plurality of first scan lines that extend from the scan driver (304) in the x-axis direction (10) and are spaced apart from each other in the y-axis direction (20) that intersects the x-axis direction (10). Additionally, the display device (110) may include a plurality of light-emitting lines that extend from the light-emitting driver (305) in the direction opposite to the x-axis direction (10) and are spaced apart from each other in the y-axis direction (20).

[0149] In one embodiment of the present disclosure, the display device (110) may include a plurality of data lines that extend in the opposite direction of the y-axis direction (20) from the data driver (302) and are spaced apart from each other in the x-axis direction (10).

[0150] In one embodiment of the present disclosure, a plurality of first scan lines, a plurality of light-emitting lines and a plurality of data lines may be included in the circuit layer (510).

[0151] In one embodiment of the present disclosure, each of the plurality of light-emitting pixels (311) may be electrically connected to a corresponding plurality of first scan lines, a plurality of light-emitting lines, and a plurality of data lines. In one embodiment of the present disclosure, depending on the configuration of the light-emitting pixel driving circuit included in each of the plurality of light-emitting pixels (311), the connection relationship between each of the plurality of light-emitting pixels (311) and the plurality of first scan lines, the plurality of light-emitting lines, and the plurality of data lines may be changed.

[0152] In one embodiment of the present disclosure, the controller (301) can acquire an image signal and a scan control signal from an external source. The controller (301) can provide an output image signal, converted in the data format of the image signal, to the data driver (302) so as to be suitable for the data driver (302) and the display device (110).

[0153] In one embodiment of the present disclosure, the data driver (302) may convert an output image signal into a plurality of data signals and provide the plurality of data signals to each of the plurality of data lines. The plurality of data signals may be analog voltages converted to correspond to the grayscale level of the output image signal.

[0154] In one embodiment of the present disclosure, the scan control signal may include a vertical start signal for initiating the operation of the scan driver (304), a vertical start signal for initiating the operation of the light-emitting driver (305), and a clock signal for determining the timing of the output of the vertical start signals.

[0155] In one embodiment of the present disclosure, the scan driver (304) may generate a plurality of first scan signals based on a scan control signal. The scan driver (304) may provide a plurality of first scan signals to each of a plurality of first scan lines.

[0156] In one embodiment of the present disclosure, the light-emitting driver (305) can generate a plurality of light-emitting signals based on a scan control signal. The light-emitting driver (305) can provide a plurality of light-emitting signals to each of a plurality of light-emitting lines.

[0157] In one embodiment of the present disclosure, the electronic device (100) controls the timing of applying a first scan signal, a data signal, and a light-emitting signal to each of a plurality of light-emitting pixels (311), and adjusts the length or size of the signal interval included in each signal, so that an image (111) can be displayed through a display device (110).

[0158] In one embodiment of the present disclosure, a plurality of light-receiving pixels (320) may be arranged to correspond to each of a plurality of light-emitting pixels (311). Specifically, when the area containing each light-emitting pixel is referred to as a pixel area (310), each light-receiving pixel may be arranged to correspond to the pixel area (310).

[0159] In one embodiment of the present disclosure, a light-receiving pixel may be arranged to correspond to each pixel area (310). A light-receiving pixel may be arranged to be included in each pixel area (310). However, the present disclosure is not limited thereto, and it is understood that two or more light-receiving pixels may be arranged to correspond to or included in each pixel area (310), or that a light-receiving pixel may be arranged to correspond to or included in two or more pixel areas.

[0160] In one embodiment of the present disclosure, each of the plurality of light-receiving pixels (320) may include a photodiode that senses light provided from the outside and converts it into an electrical signal. Each of the plurality of light-receiving pixels (320) may sense light reflected from the surrounding environment of the display device (110) or light provided from the outside to obtain an electrical signal regarding the surrounding environment.

[0161] In one embodiment of the present disclosure, each of the plurality of lenses (330) may be arranged to correspond to each light-receiving pixel. Specifically, one lens may be arranged to correspond to one light-receiving pixel. However, the present disclosure is not limited thereto, and it is understood that two or more light-receiving pixels (e.g., two or four, etc.) may be arranged to correspond to one lens.

[0162] Although not shown in FIG. 3, each of the plurality of light-receiving pixels (320) may be electrically connected to at least one transistor. At least one transistor electrically connected to each of the plurality of light-receiving pixels (320) may be referred to as a light-receiving pixel driving circuit.

[0163] In one embodiment of the present disclosure, a light-receiving pixel driving circuit may be included in a circuit layer (510) included in a display device (110). The light-receiving pixel driving circuit may be formed through the same process as the light-emitting pixel driving circuit and included in the circuit layer (510).

[0164] In one embodiment of the present disclosure, each of the plurality of light-receiving pixels (320) included in the pixel layer (540) may be electrically connected to a light-receiving pixel driving circuit included in the circuit layer (510).

[0165] In one embodiment of the present disclosure, the display device (110) may include a plurality of second scan lines that extend from the scan driver (304) in the x-axis direction (10) and are spaced apart from each other in the y-axis direction (20) that intersects the x-axis direction (10).

[0166] In one embodiment of the present disclosure, the display device (110) may include a plurality of sensing lines that extend in the opposite direction of the y-axis direction (20) from the sensing driver (303) and are spaced apart from each other in the x-axis direction (10).

[0167] In one embodiment of the present disclosure, a plurality of second scan lines and a plurality of sensing lines may be included in the circuit layer (510).

[0168] In one embodiment of the present disclosure, each of the plurality of light-receiving pixels (320) may be electrically connected to each of the corresponding plurality of second scan lines and plurality of sensing lines. In one embodiment of the present disclosure, depending on the configuration of the light-receiving pixel driving circuit included in each of the plurality of light-receiving pixels (320), the connection relationship between each of the plurality of light-receiving pixels (320) and the plurality of second scan lines and plurality of sensing lines may be changed.

[0169] In one embodiment of the present disclosure, the scan driver (304) may generate a plurality of second scan signals based on a scan control signal. The scan driver (304) may provide a plurality of second scan signals to each of a plurality of second scan lines.

[0170] In one embodiment of the present disclosure, the electronic device (100) controls the timing of applying a second scan signal to each of the plurality of light-receiving pixels (320) and adjusts the length or size of the signal interval included in the second scan signal, thereby obtaining an electrical signal for obtaining a plurality of view images (120, 121) through a display device (110).

[0171] In one embodiment of the present disclosure, the sensing driver (303) can acquire an electrical signal sensed from a plurality of light receiving pixels (320) through a plurality of sensing lines. The sensing driver (303) can convert the acquired electrical signal to acquire an image signal corresponding to a plurality of view images (120, 121).

[0172] In one embodiment of the present disclosure, a plurality of light-receiving pixels (320) may include a plurality of first light-receiving pixels corresponding to a plurality of second lenses (332) and a plurality of second light-receiving pixels corresponding to a plurality of first lenses (331).

[0173] In one embodiment of the present disclosure, a plurality of first light-receiving pixels may refer to a light-receiving pixel having a center shifted in a first direction from the optical axis of a plurality of lenses (330) among a plurality of light-receiving pixels (320). A plurality of second light-receiving pixels may refer to a light-receiving pixel having a center shifted in a direction opposite to the first direction from the optical axis of a plurality of lenses (330) among a plurality of light-receiving pixels (320).

[0174] In one embodiment of the present disclosure, a sensing driver (303) can acquire a first electrical signal sensed from a plurality of first light-receiving pixels through a plurality of first sensing lines connected to a plurality of first light-receiving pixels. The sensing driver (303) can convert the acquired first electrical signal to acquire a first image signal corresponding to a first view image (120).

[0175] In one embodiment of the present disclosure, the sensing driver (303) can acquire a second electrical signal sensed from a plurality of second light-receiving pixels through a plurality of second sensing lines connected to a plurality of second light-receiving pixels. The sensing driver (303) can convert the acquired second electrical signal to acquire a second image signal corresponding to a second view image (121).

[0176] In one embodiment of the present disclosure, a sensing driver (303) provides a first image signal and a second image signal to a controller (301), and the controller (301) can obtain a first view image (120) and a second view image (121) using the first image signal and the second image signal.

[0177] FIG. 3 is illustrated as providing a first view image (120) and a second view image (121) to a sensing driver (303), but this may be illustrated for convenience of explanation as a view image obtained using the first image signal and the second image signal provided to the sensing driver (303).

[0178] In one embodiment of the present disclosure, an electronic device (100) can acquire a first view image (120) through a plurality of first light-receiving pixels. The electronic device (100) can acquire a second view image (121) through a plurality of second light-receiving pixels.

[0179] In one embodiment of the present disclosure, an object (e.g., user (200)) included in the first view image (120) may be shifted in a first direction (i.e., x-axis direction (10)) compared with an object included in an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (330).

[0180] In one embodiment of the present disclosure, an object (e.g., user (200)) included in the second view image (121) may be shifted in the opposite direction of the first direction compared to an object included in an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (330).

[0181] In one embodiment of the present disclosure, the electronic device (100) can obtain depth information of an object (e.g., user (200), etc.) around the electronic device (100) obtained through a plurality of light-receiving pixels (320) based on the obtained first view image (120) and second view image (121). The electronic device (100) may also obtain depth information in the z-axis direction (30) relative to the x-axis direction (10) based on the obtained first view image and second view image.

[0182] FIG. 4 is a drawing for explaining the configuration of a display device according to one embodiment of the present disclosure.

[0183] Referring to FIGS. 1, FIGS. 3 and FIGS. 4, in one embodiment of the present disclosure, FIG. 4 shows a display device (110) included in an electronic device (100).

[0184] In one embodiment of the present disclosure, the display device (110) may include a display module (400), a lens layer (410), and a window (420). In this case, the display module (400) may include a base layer (500), a circuit layer (510), a pixel layer (540), an optical adhesive layer (550), and an optical layer (560) as illustrated in FIG. 5.

[0185] FIG. 4 illustrates that the display module (400) includes a driving circuit (401), a plurality of light-emitting pixels (402), and a plurality of light-receiving pixels (403), but it is obvious that more components may be included. In this case, the driving circuit (401) may include a light-emitting pixel driving circuit and a light-receiving pixel driving circuit.

[0186] In one embodiment of the present disclosure, the lens layer (410) may include a plurality of lenses (411) and a light blocking film (412) surrounding the plurality of lenses (411). In this case, the light blocking film (412) may include a plurality of openings that overlap with the plurality of lenses (411).

[0187] In one embodiment of the present disclosure, the optical axis of each of the plurality of lenses (411) may be arranged so as not to pass through the center of the plurality of light-receiving pixels (403) included in the display module (400). With respect to the z-axis direction (30), the center of the plurality of light-receiving pixels (403) may not pass through the optical axis of each of the plurality of lenses (411). On a plane defined by the x-axis and y-axis, the center of the plurality of light-receiving pixels (403) may be arranged to be spaced apart from the point where the optical axis of each of the plurality of lenses (411) intersects the plane.

[0188] In this case, the center of the light-receiving pixel may refer to the point where light refracted through the lens corresponding to each light-receiving pixel is focused on the light-receiving pixel.

[0189] In one embodiment of the present disclosure, the window (420) may be made of a transparent material capable of displaying an image (111) through the display device (110) or providing light provided from the outside to the display device (110). For example, it may be composed of glass, sapphire, plastic, etc. The window (420) is illustrated as a single layer, but is not limited thereto and may include a plurality of layers.

[0190] In one embodiment of the present disclosure, a lens layer (410) may be laminated on a display module (400). At this time, "on the display module (400)" may mean that the lens layer (410) is laminated on the upper surface of the display module (400) with respect to the z-axis direction (30).

[0191] In one embodiment of the present disclosure, a window (420) may be laminated on a lens layer (410). At this time, "on the lens layer (410)" may mean that a window (420) is laminated on the upper surface of the lens layer (410) with respect to the z-axis direction (30).

[0192] FIG. 5 is a drawing for explaining the arrangement of a plurality of components included in a display device according to one embodiment of the present disclosure.

[0193] Referring to FIGS. 3, 4, and 5, in one embodiment of the present disclosure, FIG. 5 shows a plurality of components included in a display device (110).

[0194] In one embodiment of the present disclosure, FIG. 5 illustrates a first case (590) in which an object is located at a first distance (591) from a display device (110), and a second case (594) in which an object is located at a second distance (595) from a display device (110). In one embodiment of the present disclosure, the first distance (591) and the second distance (595) may refer to distances between an electronic device (100) and an object. Additionally, the object may refer to a user (200) using the electronic device (100).

[0195] Hereinafter, for the convenience of explanation, a plurality of components included in the display device (110) will be described based on the first case (590), and then the second case (594) will be described.

[0196] In one embodiment of the present disclosure, the display device (110) may include a base layer (500).

[0197] In one embodiment of the present disclosure, the display device (110) may include a circuit layer (510) disposed on a base layer (500). In this case, the circuit layer (510) may include small elements such as a plurality of transistors (511) for driving a plurality of light-emitting pixels (531) and a plurality of light-receiving pixels (521).

[0198] In one embodiment of the present disclosure, the circuit layer (510) may be disposed on the upper surface of the base layer (500). The circuit layer (510) may be stacked in the z-axis direction (30) with respect to the base layer (500).

[0199] Hereinafter, the base layer (500) and the circuit layer (510) will be described later in FIG. 6.

[0200] In one embodiment of the present disclosure, the display device (110) may include a pixel layer (540) disposed on a circuit layer (510). The pixel layer (540) may be disposed on the upper surface of the circuit layer (510). The pixel layer (540) may be stacked in the z-axis direction (30) with respect to the circuit layer (510).

[0201] In one embodiment of the present disclosure, the pixel layer (540) may include a light-emitting layer (530) including a plurality of light-emitting pixels (311) and a light-receiving layer (520) including a plurality of light-receiving pixels (521). In one embodiment of the present disclosure, the light-receiving layer (520) may be disposed on the upper surface of the circuit layer (510), and the light-emitting layer (530) may be disposed on the upper surface of the light-receiving layer (520).

[0202] However, the stacking order and arrangement relationship of the light-emitting layer (530) and the light-receiving layer (520) may vary depending on the type of the plurality of light-emitting pixels (531) and the type of the plurality of light-receiving pixels (521). In one embodiment of the present disclosure, when the plurality of light-receiving pixels (521) include inorganic photodiodes and the plurality of light-emitting pixels (531) include organic light-emitting diodes, the light-emitting layer (530) may be disposed on the upper surface of the light-receiving layer (520).

[0203] On the other hand, when a plurality of light-receiving pixels (521) include inorganic photodiodes and a plurality of light-emitting pixels (531) include inorganic light-emitting diodes, the light-emitting layer (530) and the light-receiving layer (520) may be disposed on the same layer. That is, a plurality of light-receiving pixels (521) and a plurality of light-emitting pixels (531) may be included on the same layer.

[0204] In addition, even when a plurality of light-receiving pixels (521) include organic photodiodes and a plurality of light-emitting pixels (531) include organic light-emitting diodes, it is obvious that the plurality of light-receiving pixels (521) and the plurality of light-emitting pixels (531) may be included on the same layer.

[0205] Hereinafter, for the convenience of explanation, a plurality of light-emitting pixels (531) are described as including organic light-emitting diodes, and a plurality of light-receiving pixels (521) are described as including inorganic photodiodes.

[0206] In one embodiment of the present disclosure, each light-receiving pixel may include an active layer (522) that senses light and converts it into an electrical signal, and an electrode (523) for applying voltage to the active layer. In one embodiment of the present disclosure, if the light-receiving pixel is a PIN diode, the active layer (522) may refer to a PIN layer. However, the present disclosure is not limited thereto, and it is understood that the structure and arrangement of each light-receiving pixel included in the light-receiving layer (520) may vary depending on the type of photodiode included in the light-receiving pixel.

[0207] In one embodiment of the present disclosure, the light-emitting layer (530) may include a plurality of light-emitting pixels (531) and a protective film (532) covering the plurality of light-emitting pixels (531).

[0208] In one embodiment of the present disclosure, the protective film (532) may be a film for protecting a plurality of light-emitting pixels (531) from the outside. The protective film (532) may refer to a thin film encapsulation (TFE). Although the protective film (532) is depicted as a single layer, it may have a structure in which a plurality of inorganic layers and organic layers are alternately stacked. In this case, the inorganic layer protects the plurality of light-emitting pixels (531) from external moisture, and the organic layer prevents defects in the plurality of light-emitting pixels (531) caused by foreign substances introduced during the manufacturing process. In this case, the protective film (532) may be referred to as an encapsulation layer.

[0209] However, the present disclosure is not limited thereto, and in the case where a plurality of light-emitting pixels (531) include inorganic light-emitting diodes, the protective film (532) may stabilize surface states, suppress leakage current, or provide an insulating function. In this case, the protective film (532) may be referred to as a passivation layer.

[0210] In addition, a flat surface can be provided when an optical layer (560), etc. is combined on the pixel layer (540) through the protective film (532).

[0211] In one embodiment of the present disclosure, the display device (110) may include an optical adhesive layer (550) on a pixel layer (540). The optical adhesive layer (550) may be a layer that serves to bond the pixel layer (540) and the optical layer (560). The optical adhesive layer (550) may be disposed between the pixel layer (540) and the optical layer (560).

[0212] However, the optical adhesive layer (550) shown in FIG. 5 is illustrated for illustrative purposes only, and the thickness of the optical adhesive layer (550) included in the display device (110) may be thinner than that shown in FIG. 5. Additionally, it is obvious that the configuration of the optical adhesive layer (550) may be omitted when describing the configuration of the display device (110).

[0213] In one embodiment of the present disclosure, the optical adhesive layer (550) has excellent light transmission properties, so that light loss between the multiple layers being bonded is minimized and light reflection or refraction is minimized.

[0214] In one embodiment of the present disclosure, the optical adhesive layer (550) may be a layer formed by applying and curing an optically clear resin (OCR) in the form of a liquid resin. Additionally, the optical adhesive layer (550) may be a layer formed of an optically clear adhesive (OCA) in the form of a film.

[0215] However, the present disclosure is not limited thereto, and the optical adhesive layer (550) may include other types of materials with excellent light transmission properties and adhesive strength.

[0216] In one embodiment of the present disclosure, the display device (110) may include an optical layer (560) disposed on a pixel layer (540). The optical layer (560) may be stacked in the z-axis direction (30) with respect to the pixel layer (540). In one embodiment of the present disclosure, the optical layer (560) may be bonded to the pixel layer (540) using an optical adhesive layer (550).

[0217] In one embodiment of the present disclosure, the optical layer (560) may include an optically transparent material, such as glass or polyimide (PI).

[0218] In one embodiment of the present disclosure, the thickness (561) of the optical layer (560) may be determined to correspond to the focal length of each of the plurality of lenses (571, 572).

[0219] In one embodiment of the present disclosure, an optical layer (560) having a thickness (561) corresponding to the focal length of a plurality of lenses (571, 572) is disposed between the lens layer (570) and the pixel layer (540), so that the focus of light passing through each lens can be formed on a corresponding light receiving element.

[0220] In one embodiment of the present disclosure, as the display device (110) includes an optical layer (560), the focus of light reflected from an object located at a certain distance from the display device (110) passes through a plurality of lens layers (570) and is formed on a plurality of light receiving pixels (521), so that the display device (110) may acquire a view image.

[0221] In one embodiment of the present disclosure, the lens layer (570) may include a plurality of first lenses (571) and a plurality of second lenses (572). In FIG. 5, for convenience of explanation, one first lens and one second lens are shown, but these may refer to a plurality of first lenses (571) and a plurality of second lenses (572), respectively.

[0222] In one embodiment of the present disclosure, the first optical axis (573) of the plurality of first lenses (571) may not pass through the center of the light-receiving pixel corresponding to the plurality of first lenses (571). The plurality of first lenses (571) may be arranged so that the first optical axis (573) extending in the z-axis direction (30) is shifted in a first direction from the center of the light-receiving pixel. In this case, the first direction may mean the x-axis direction (10).

[0223] In one embodiment of the present disclosure, the second optical axis (574) of the plurality of second lenses (572) may not pass through the center of the light-receiving pixel corresponding to the plurality of second lenses (572). The plurality of second lenses (572) may be arranged so that the second optical axis (574) extending in the z-axis direction (30) is shifted in the opposite direction of the first direction from the center of the light-receiving pixel.

[0224] In one embodiment of the present disclosure, a protective layer (580) may be disposed on the lens layer (570) to protect a plurality of lenses (571, 572) included in the lens layer (570) from the outside. At this time, although the protective layer (580) is shown as a single layer, it may have a structure in which a plurality of inorganic layers and organic layers are alternately stacked. However, the present disclosure is not limited thereto, and the protective layer (580) may refer to the window (420) shown in FIG. 4.

[0225] In one embodiment of the present disclosure, in a first case (590), a first view image (120) obtained through a plurality of light-receiving pixels (521) via a plurality of first lenses (571) may include a first scene (592) of viewing an object in the first view. In a first case (590), a second view image (121) obtained through a plurality of light-receiving pixels (521) via a plurality of second lenses (572) may include a second scene (593) of viewing an object in the second view.

[0226] In one embodiment of the present disclosure, in a second case (594), a first view image (120) obtained through a plurality of light-receiving pixels (521) via a plurality of first lenses (571) may include a third scene (596) of viewing an object in the first view. In a second case (594), a second view image (121) obtained through a plurality of light-receiving pixels (521) via a plurality of second lenses (572) may include a second scene (597) of viewing an object in the second view.

[0227] In one embodiment of the present disclosure, when the reference scene is defined as viewing an object from the front, the first scene (592) and the third scene (596) may refer to scenes in which the object is viewed from a direction opposite to the first direction compared to the reference scene. The second scene (593) and the fourth scene (597) may refer to scenes in which the object is viewed from the first direction compared to the reference scene.

[0228] In one embodiment of the present disclosure, the difference between the third scene (596) and the fourth scene (597) may be greater than the difference between the first scene (592) and the second scene (593). In one embodiment of the present disclosure, as the distance of an object from the electronic device (100) increases, the parallax between the scene included in the first view image (120) and the scene included in the second view image (121) may increase.

[0229] FIG. 6 is a drawing for explaining a circuit layer and a light-emitting layer according to one embodiment of the present disclosure.

[0230] Referring to FIG. 5 and FIG. 6, in one embodiment of the present disclosure, FIG. 6 illustrates a base layer (500), a circuit layer (510), and a light receiving layer (520) among a plurality of components included in a display device (110).

[0231] In one embodiment of the present disclosure, the circuit layer (510) may include a plurality of transistors, and the light receiving layer (520) may also include a plurality of light receiving pixels. In FIG. 6, for convenience of explanation, one transistor and one light receiving pixel are shown arranged.

[0232] In one embodiment of the present disclosure, the base layer (500) may include a synthetic resin layer. The synthetic resin layer may include a thermosetting resin. In particular, the synthetic resin layer may be a polyimide-based resin layer, and the material thereof is not particularly limited. The synthetic resin layer may include at least one of an acrylic resin, a methacrylate resin, a polyisoprene, a vinyl resin, an epoxy resin, a urethane resin, a cellulose resin, a siloxane resin, a polyamide resin, and a perylene resin. In addition, the base layer (500) may include a glass substrate, a metal substrate, or an organic / inorganic composite material substrate, etc.

[0233] In one embodiment of the present disclosure, a circuit layer (510) stacked on a base layer (500) may include a first semiconductor pattern (600), a first insulating layer (610), a gate electrode (611), a second insulating layer (620), a source electrode (621), a drain electrode (622), and a third insulating layer (630).

[0234] In one embodiment of the present disclosure, the insulating layer included in the circuit layer (510) and the light receiving layer (520) may be an inorganic layer and / or an organic layer and may have a single layer or a multilayer structure. The insulating layer included in the circuit layer (510) and the light receiving layer (520) may include at least one of aluminum oxide, titanium oxide, silicon oxide, silicon nitride, silicon oxynitride, zirconium oxide, and hafnium oxide.

[0235] Additionally, the insulating layer included in the circuit layer (510) and the light receiving layer (520) may include general-purpose polymers such as BCB (Benzocyclobutene), polyimide, HMDSO (Hexamethyldisiloxane), polymethylmethacrylate (PMMA), or polystyrene (PS), polymer derivatives having a phenolic group, acrylic polymers, imide polymers, aryl ether polymers, amide polymers, fluorine polymers, p-xylene polymers, vinyl alcohol polymers, and blends thereof, but is not particularly limited thereto.

[0236] In one embodiment of the present disclosure, a first semiconductor pattern (600) may be disposed on a base layer (500). In one embodiment of the present disclosure, the first semiconductor pattern (600) may be divided into a source region, a gate region, and a drain region according to the doping concentration or conductivity of the semiconductor pattern. In one embodiment of the present disclosure, the source region and the drain region may be a p-type semiconductor layer or an n-type semiconductor layer. The gate region may be an active layer or a channel layer disposed between the source region and the drain region.

[0237] In one embodiment of the present disclosure, the first semiconductor pattern (600) may include a silicon semiconductor. For example, the silicon semiconductor may include amorphous silicon, polycrystalline silicon, etc. Or the first semiconductor pattern (600) may include an oxide of a material selected from group 12, 13, 14 metals such as indium (In), gallium (Ga), tin (Sn), cadmium (Cd), aluminum (Al), germanium (Ge), zinc (Zinc) or hafnium (Hf), and combinations thereof.

[0238] In one embodiment of the present disclosure, a first insulating layer (610) may be disposed on a base layer (500). The first insulating layer (610) may cover a first semiconductor pattern (600).

[0239] In one embodiment of the present disclosure, the first insulating layer (610) may be an inorganic layer and / or an organic layer and may have a single layer or a multilayer structure.

[0240] In one embodiment of the present disclosure, a gate electrode (611) may be disposed on a first insulating layer (610). The gate electrode (611) may be disposed on top of a first semiconductor pattern (600). The gate electrode (611) may overlap with the gate region of the first semiconductor pattern (600).

[0241] In one embodiment of the present disclosure, the gate electrode (611) may include titanium (Ti), silver (Ag), a silver-containing alloy, molybdenum (Mo), a molybdenum-containing alloy, aluminum (Al), an aluminum-containing alloy, aluminum nitride (AlN), tungsten (W), tungsten nitride (WN), copper (Cu), indium tin oxide (ITO), indium zinc oxide (IZO), etc., but is not particularly limited thereto.

[0242] In one embodiment of the present disclosure, a second insulating layer (620) may be disposed on a first insulating layer (610). The second insulating layer (620) may cover a gate electrode (611). The second insulating layer (620) may have a single layer or a multilayer structure. The second insulating layer (620) may be an inorganic layer and / or an organic layer.

[0243] In one embodiment of the present disclosure, a source electrode (621) and a drain electrode (622) may be disposed on a second insulating layer (620). The source electrode (621) and the drain electrode (622) may include titanium (Ti), silver (Ag), a silver-containing alloy, molybdenum (Mo), a molybdenum-containing alloy, aluminum (Al), an aluminum-containing alloy, aluminum nitride (AlN), tungsten (W), tungsten nitride (WN), copper (Cu), indium tin oxide (ITO), indium zinc oxide (IZO), etc., but are not particularly limited thereto.

[0244] In one embodiment of the present disclosure, the source electrode (621) may be electrically connected to the source region of the first semiconductor pattern (600) through a through hole penetrating the first insulating layer (610) and the second insulating layer (620). However, the present disclosure is not limited thereto, and the source electrode (621) may be electrically connected to the source region through a connecting electrode penetrating the first insulating layer (610) and the second insulating layer (620).

[0245] In one embodiment of the present disclosure, the drain electrode (622) may be electrically connected to the drain region of the first semiconductor pattern (600) through a through hole penetrating the first insulating layer (610) and the second insulating layer (620). However, the present disclosure is not limited thereto, and the drain electrode (622) may be electrically connected to the drain region through a connecting electrode penetrating the first insulating layer (610) and the second insulating layer (620).

[0246] In one embodiment of the present disclosure, a third insulating layer (630) may be disposed on a second insulating layer (620). The third insulating layer (630) may cover a source electrode (621) and a drain electrode (622). The third insulating layer (630) may have a single layer or a multilayer structure. The third insulating layer (630) may be an inorganic layer and / or an organic layer.

[0247] In one embodiment of the present disclosure, a light receiving layer (520) stacked on a circuit layer (510) may include a second semiconductor pattern (631), a transparent electrode (632), a fourth insulating layer (640), a first electrode (641), a second electrode (642), and a fifth insulating layer (650).

[0248] In one embodiment of the present disclosure, a second semiconductor pattern (631) may be disposed on a drain electrode (622). In one embodiment of the present disclosure, the second semiconductor pattern (631) may be formed in an opening formed by etching a portion of a third insulating layer (630) covering the drain electrode (622). The second semiconductor pattern (631) may be electrically connected to the drain electrode (622).

[0249] In one embodiment of the present disclosure, the second semiconductor pattern (631) may include a silicon semiconductor. For example, the silicon semiconductor may include amorphous silicon, polycrystalline silicon, etc. Or the second semiconductor pattern (631) may include an oxide of a material selected from group 12, 13, 14 metals such as indium (In), gallium (Ga), tin (Sn), cadmium (Cd), aluminum (Al), germanium (Ge), zinc (Zinc) or hafnium (Hf), and combinations thereof.

[0250] In one embodiment of the present disclosure, the second semiconductor pattern (631) may be a PIN layer in which a p-type semiconductor layer, an i-type (intrinsic type) semiconductor layer, and an n-type semiconductor layer are sequentially stacked. A photocurrent corresponding to light incident on the second semiconductor pattern (631) may be generated and provided to the drain electrode (622).

[0251] In one embodiment of the present disclosure, a transparent electrode (632) may be disposed on a second semiconductor pattern (631). The transparent electrode (632) may refer to an electrode that is optically transparent and electrically conductive. The transparent electrode (632) may include an ITO (Indium Tin Oxide) electrode. However, the present disclosure is not limited thereto, and the transparent electrode (632) may include a transparent conductive oxide such as IZO (Indium Zinc Oxide) or GZO (Gallium-doped Zinc Oxide).

[0252] In one embodiment of the present disclosure, a fourth insulating layer (640) may be disposed on a transparent electrode (632). The fourth insulating layer (640) may cover the second semiconductor pattern (631) and the transparent electrode (632). The fourth insulating layer (640) may have a single layer or a multilayer structure. The fourth insulating layer (640) may be an inorganic layer and / or an organic layer.

[0253] In one embodiment of the present disclosure, a first electrode (641) and a second electrode (642) may be disposed on a fourth insulating layer (640). The first electrode (641) and the second electrode (642) may include titanium (Ti), silver (Ag), a silver-containing alloy, molybdenum (Mo), a molybdenum-containing alloy, aluminum (Al), an aluminum-containing alloy, aluminum nitride (AlN), tungsten (W), tungsten nitride (WN), copper (Cu), indium tin oxide (ITO), indium zinc oxide (IZO), etc., but are not particularly limited thereto.

[0254] In one embodiment of the present disclosure, the second electrode (642) may be electrically connected to the transparent electrode (632) through a through hole penetrating the fourth insulating layer (640). However, the present disclosure is not limited thereto, and the second electrode (642) may be electrically connected to the transparent electrode (632) through a connecting electrode penetrating the fourth insulating layer (640).

[0255] In one embodiment of the present disclosure, the first electrode (641) may be an electrode constituting another light-receiving element included in the light-receiving layer (520) or another light-emitting element included in the light-emitting layer (530).

[0256] In one embodiment of the present disclosure, a fifth insulating layer (650) may be disposed on a fourth insulating layer (640). The fifth insulating layer (650) may cover the first electrode (641) and the second electrode (642). The fifth insulating layer (650) may have a single layer or a multilayer structure. The fifth insulating layer (650) may be an inorganic layer and / or an organic layer.

[0257] In one embodiment of the present disclosure, the illustration in FIG. 6 is an example of a cross-sectional view for explaining the positional relationship between the base layer (500), the circuit layer (510), and the light receiving layer (520). Of course, the cross-sectional view of each layer may vary depending on the configuration of the driving circuit included in the circuit layer (510) or the type of light receiving pixel included in the light receiving layer (520).

[0258] In addition, it goes without saying that a light-emitting layer (530) may be disposed on the light-receiving layer (520). A plurality of light-emitting pixels (531) included in the light-emitting layer (530) and a plurality of electrodes for driving the plurality of light-emitting pixels (531) may be disposed on the light-receiving layer (520).

[0259] FIG. 7 is a drawing for explaining the arrangement of a plurality of components included in a display device according to one embodiment of the present disclosure.

[0260] Referring to FIG. 5 and FIG. 7, in one embodiment of the present disclosure, a display device (110) may include a base layer (500), a circuit layer (510), a pixel layer (540), an optical adhesive layer (550), an optical layer (560), a lens layer (570), and a protective layer (580) that are sequentially stacked in the z-axis direction (30).

[0261] In one embodiment of the present disclosure, the circuit layer (510) may include a plurality of transistors (511) for driving a plurality of light-receiving pixels (521) and a plurality of light-emitting pixels included in the pixel layer (540).

[0262] In one embodiment of the present disclosure, the pixel layer (540) may include a light receiving layer (520) and a light emitting layer (530). In this case, for convenience of explanation, in FIG. 7, two of the plurality of light receiving pixels (521) included in the light receiving layer (520) are shown, and the plurality of light emitting pixels included in the light emitting layer (530) are not shown. However, FIG. 7 is a cross-sectional view of a part of the display device (110) shown for the purpose of explanation, and it is obvious that the number or arrangement of light receiving pixels and light emitting pixels included in the light receiving layer (520) and the light emitting layer (530) may vary.

[0263] In one embodiment of the present disclosure, the display device (110) may include a light blocking film (700) disposed on an optical layer (560). The light blocking film (700) may be stacked in the z-axis direction (30) with respect to the optical layer (560). The light blocking film (700) may be disposed on the top of the optical layer (560). In this case, the top of the optical layer (560) may refer to a surface adjacent to an object (730) in the z-axis direction (30) with respect to the bottom of the optical layer (560).

[0264] In one embodiment of the present disclosure, the light blocking film (700) may include a plurality of openings that overlap with a plurality of lenses included in the lens layer (570). In one embodiment of the present disclosure, each opening may overlap with a corresponding lens. In one embodiment of the present disclosure, an opening and a lens corresponding to the opening may overlap in the z-axis direction (30).

[0265] In one embodiment of the present disclosure, a light-receiving pixel corresponding to a lens in the z-axis direction (30) and an aperture may overlap. In the z-axis direction (30), the light-receiving pixel and the area that is not an aperture of the light-blocking film (700) may not overlap.

[0266] However, although not illustrated in FIG. 7, the present disclosure is not limited thereto and the light blocking film (700) may further include a plurality of openings that overlap with a plurality of light-emitting pixels included in the pixel layer (540). Through this, light provided through the plurality of light-emitting pixels may not be blocked by the light blocking film (700) and may be provided as an image (111). Hereinafter, the light blocking film included in the display device (110) of the present invention may include a plurality of openings that overlap with a plurality of light-emitting pixels.

[0267] In one embodiment of the present disclosure, the opening width (710) of each of the plurality of openings may be equal to the width of each lens. In this case, the width of the lens may refer to the diameter of the lens. However, the present disclosure is not limited thereto, and the opening width (710) of each opening may be smaller than the diameter of the corresponding lens.

[0268] In one embodiment of the present disclosure, the width of each of the plurality of light-receiving pixels (521) may be referred to as the pixel width (720). The aperture width (710) may be larger than the pixel width (720).

[0269] The light-blocking film described in one embodiment of the present disclosure may be made of an optically opaque material. In one embodiment of the present disclosure, the light-blocking film may include a metal such as chromium (Cr), molybdenum (Mo), aluminum (Al), etc., or a resin including carbon black, etc. However, the present disclosure is not limited thereto, and the light-blocking film may include an inorganic material or an organic material having absorptive or reflective properties.

[0270] In one embodiment of the present disclosure, the light blocking film may be patterned through a photolithography process to block external light in areas other than specific apertures. In one embodiment of the present disclosure, the light blocking film may be referred to as a Black Matrix (BM).

[0271] In one embodiment of the present disclosure, light that has not passed through a plurality of lenses can be prevented from being provided to a plurality of light-receiving pixels (521) through a light blocking film (700). Specifically, among the light provided to the display device (110) from the outside, light that has passed through each lens can be provided to a light-receiving pixel corresponding to each lens through an optical layer (560). Light that has not passed through each lens can be prevented from being provided to a light-receiving pixel through an optical layer (560) through a light blocking film (700).

[0272] Through this, when the display device (110) acquires a view image, light of noise components that has not passed through the lens is provided to the light receiving pixel, thereby preventing image quality degradation such as crosstalk.

[0273] FIG. 8 is a flowchart for explaining a method of operation of an electronic device according to one embodiment of the present disclosure. FIG. 9 is a flowchart for explaining an operation of acquiring depth information through an electronic device according to one embodiment of the present disclosure.

[0274] Referring to FIGS. 1, FIGS. 2, FIGS. 3 and FIGS. 8, in one embodiment of the present disclosure, a method of operating an electronic device (100) may include a step (S100) of displaying an image (111) through a plurality of light-emitting pixels (311).

[0275] In step S100, the electronic device (100) can control the display device (110) to display an image (111). The electronic device (100) can display the image (111) through a plurality of light-emitting pixels (311) included in the display device (110). The electronic device (100) can display the image (111) through the display device (110) based on an image signal obtained from an external source.

[0276] In one embodiment of the present disclosure, the method of operation of an electronic device (100) may include the step (S200) of acquiring depth information based on the parallax between a plurality of view images (120, 121) acquired through a plurality of light-receiving pixels (320).

[0277] In step S200, the electronic device (100) can acquire depth information based on the parallax between a plurality of view images (120, 121) through a depth information acquisition module (143).

[0278] Referring to FIGS. 3, 8 and 9, in one embodiment of the present disclosure, the step of acquiring depth information (S200) may include the step (S210) of acquiring a first view image (120) of an object from a plurality of first light-receiving pixels having a center shifted in a first direction from the optical axis of a plurality of lenses (330) among a plurality of light-receiving pixels (320).

[0279] In step S210, the electronic device (100) can obtain a first view image (120) of an object from a plurality of first light-receiving pixels having a center shifted in a first direction from the optical axis of a plurality of lenses (330) among a plurality of light-receiving pixels (320).

[0280] In one embodiment of the present disclosure, the step of acquiring depth information (S200) may include the step (S220) of acquiring a second view image (121) of an object from a plurality of second light-receiving pixels having a center shifted in a second direction different from the first direction from the optical axis of a plurality of lenses (330) among a plurality of light-receiving pixels (320).

[0281] In step S220, the electronic device (100) can obtain a second view image (121) of an object from a plurality of second light-receiving pixels having a center shifted in a second direction different from the first direction from the optical axis of a plurality of lenses (330) among a plurality of light-receiving pixels (320).

[0282] In one embodiment of the present disclosure, the electronic device (100) may include the step (S230) of obtaining depth information about an object from a first view image (120) and a second view image (121) through a stereo matching algorithm.

[0283] In step S230, the electronic device (100) can obtain depth information about an object from the first view image (120) and the second view image (121) through the depth information acquisition module (143).

[0284] In one embodiment of the present disclosure, the electronic device (100) can accurately recognize the position, speed, or direction of the user (200)'s finger, or recognize changes in the position, head direction, etc. of the user (200)'s face, by obtaining depth information about the user (200) through step S230.

[0285] Based on this, the electronic device (100) can accurately recognize gestures or control actions of a user (200) using the electronic device (100). In addition, the electronic device (100) can also accurately recognize gestures or control actions of a user (200) controlling the electronic device (100) while separated from the electronic device (100) by a certain distance.

[0286] In addition, in one embodiment of the present disclosure, the electronic device (100) may acquire gaze information of the user (200) based on two view images (120, 121) including a parallax. The electronic device (100) may recognize feature points included in the pupils of the user (200) included in each of the two view images (120, 121) and track the gaze of the user (200) based thereon. Based on this, the electronic device (100) may utilize the gaze information of the user (200) in recognizing the user's (200) gestures or control actions.

[0287] FIG. 10a is a drawing for explaining that a light-receiving pixel is disposed in each of a plurality of lenses according to one embodiment of the present disclosure. FIG. 10b is a drawing for explaining that a light-receiving pixel is disposed in each of a plurality of lenses according to one embodiment of the present disclosure. Hereinafter, the same reference numerals are assigned to configurations identical to those described in FIG. 3, and redundant descriptions are omitted.

[0288] Referring to FIG. 1, FIG. 3 and FIG. 10a, in one embodiment of the present disclosure, FIG. 10a shows a display device (110) included in an electronic device (100). FIG. 10a shows a plurality of light-emitting pixels (311), a plurality of light-receiving pixels (320), and a plurality of lenses (1000, 1010) among the components included in the display device (110) for convenience of explanation.

[0289] In one embodiment of the present disclosure, a plurality of light-emitting pixels (311) may include a red subpixel, a green subpixel, and a blue subpixel. Although the three subpixels included in each light-emitting pixel are illustrated as being arranged in the x-axis direction (10), the present disclosure is not limited thereto, and it is understood that the types, arrangements, shapes, etc. of the subpixels included in each light-emitting pixel may vary.

[0290] In one embodiment of the present disclosure, a plurality of light-receiving pixels (320) may be arranged to correspond to each of a plurality of light-emitting pixels (311). Each light-receiving pixel may be arranged to correspond to a pixel area (310).

[0291] In one embodiment of the present disclosure, a plurality of lenses (1000, 1010) may include a plurality of first lenses (1000) having a first optical axis shifted in a first direction from a plurality of light-receiving pixels (320) and a plurality of second lenses (1010) having a second optical axis shifted in the opposite direction to the first direction. In this case, the first direction may mean the y-axis direction (20).

[0292] In one embodiment of the present disclosure, a plurality of light-receiving pixels (320) may include a plurality of first light-receiving pixels corresponding to a plurality of second lenses (1010) and a plurality of second light-receiving pixels corresponding to a plurality of first lenses (1000).

[0293] In one embodiment of the present disclosure, the electronic device (100) can acquire a first view image through a plurality of first light-receiving pixels and acquire a second view image through a plurality of second light-receiving pixels.

[0294] At this time, the first view image may be shifted in the y-axis direction (20) compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1000, 1010). The second view image may be shifted in the opposite direction of the y-axis direction (20) compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1000, 1010).

[0295] In one embodiment of the present disclosure, an electronic device (100) can obtain depth information of an object (e.g., a user (200), etc.) around the electronic device (100) based on a first view image and a second view image obtained through a plurality of light-receiving pixels (320). Based on the obtained first view image and second view image, the electronic device (100) may also obtain depth information in the z-axis direction (30) relative to the y-axis direction (20).

[0296] Referring to FIG. 1, FIG. 3, and FIG. 10b, in one embodiment of the present disclosure, FIG. 10b shows a plurality of light-emitting pixels (311), a plurality of light-receiving pixels (320), and a plurality of lenses (1020, 1030, 1040, 1050) among the components included in a display device (110).

[0297] In one embodiment of the present disclosure, a plurality of light-receiving pixels (320) may be arranged to correspond to each of a plurality of light-emitting pixels (311). Each light-receiving pixel may be arranged to correspond to a pixel area (310).

[0298] In one embodiment of the present disclosure, a plurality of lenses (1020, 1030, 1040, 1050) may include a plurality of first lenses (1020) having a first optical axis shifted in a first direction from a plurality of light-receiving pixels (320), a plurality of second lenses (1030) having a second optical axis shifted in the opposite direction of the first direction, a plurality of third lenses (1040) having a third optical axis shifted in the second direction, and a plurality of fourth lenses (1050) having a fourth optical axis shifted in the opposite direction of the second direction.

[0299] At this time, the first direction may mean any direction within the second quadrant on the plane defined by the x-axis and the y-axis. The second direction may mean any direction within the first quadrant on the plane defined by the x-axis and the y-axis.

[0300] In one embodiment of the present disclosure, a plurality of light-receiving pixels (320) may include a plurality of first light-receiving pixels corresponding to a plurality of second lenses (1030), a plurality of second light-receiving pixels corresponding to a plurality of first lenses (1020), a plurality of third light-receiving pixels corresponding to a plurality of fourth lenses (1050), and a plurality of fourth light-receiving pixels corresponding to a plurality of third lenses (1040).

[0301] In one embodiment of the present disclosure, the electronic device (100) may acquire a first view image through a plurality of first light-receiving pixels and acquire a second view image through a plurality of second light-receiving pixels. The electronic device (100) may acquire a third view image through a plurality of third light-receiving pixels and acquire a fourth view image through a plurality of fourth light-receiving pixels.

[0302] In one embodiment of the present disclosure, the first view image may be shifted in the direction of the second quadrant in a plane defined by the x-axis and y-axis by comparing it with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1020, 1030, 1040, 1050). The second view image may be shifted in the direction of the fourth quadrant in a plane defined by the x-axis and y-axis by comparing it with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1020, 1030, 1040, 1050).

[0303] In one embodiment of the present disclosure, the third view image may be shifted in the direction of the first quadrant in a plane defined by the x-axis and y-axis by comparing it with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1020, 1030, 1040, 1050). The fourth view image may be shifted in the direction of the third quadrant in a plane defined by the x-axis and y-axis by comparing it with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1020, 1030, 1040, 1050).

[0304] Through this, the electronic device (100) can acquire an image corresponding to a wide field of view (FOV) of the surroundings of the electronic device (100) through a plurality of light-receiving pixels (320). The electronic device (100) can acquire information about situations occurring around the electronic device (100) and can increase the accuracy of the information acquired regarding the movements or gestures of the user (200).

[0305] Additionally, in one embodiment of the present disclosure, the electronic device (100) can obtain depth information of an object around the electronic device (100) based on a first view image and a second view image obtained through a plurality of light-receiving pixels (320). Based on the obtained first view image and second view image, the electronic device (100) may also obtain depth information in the z-axis direction (30) with respect to a first direction.

[0306] Additionally, the electronic device (100) can obtain depth information of an object around the electronic device (100) obtained through a plurality of light-receiving pixels (320) based on the obtained third view image and fourth view image. Based on the obtained third view image and fourth view image, the electronic device (100) may also obtain depth information in the z-axis direction (30) based on the second direction.

[0307] However, the present disclosure is not limited thereto, and the electronic device (100) may use all of the acquired first to fourth view images to obtain depth information in the z-axis direction (30) based on a plane defined by the x-axis and y-axis.

[0308] FIG. 11a is a drawing for explaining that two light-receiving pixels are arranged in each of a plurality of lenses according to one embodiment of the present disclosure. FIG. 11b is a drawing for explaining that two light-receiving pixels are arranged in each of a plurality of lenses according to one embodiment of the present disclosure. Hereinafter, the same reference numerals are assigned to configurations identical to those described in FIG. 3, and redundant descriptions are omitted.

[0309] Referring to FIG. 1, FIG. 3 and FIG. 11a, in one embodiment of the present disclosure, FIG. 11a shows a display device (110) included in an electronic device (100). FIG. 11a shows a plurality of light-emitting pixels (311), a plurality of light-receiving pixels (1110, 1120), and a plurality of lenses (1100) among the components included in the display device (110) for convenience of explanation.

[0310] In one embodiment of the present disclosure, a plurality of light-receiving pixels (1110, 1120) may be arranged to correspond to a plurality of light-emitting pixels (311). Two light-receiving pixels may be arranged to correspond to a pixel area (310) containing each light-emitting pixel.

[0311] In one embodiment of the present disclosure, two light-receiving pixels may be arranged to correspond to each of the plurality of lenses (1100).

[0312] In one embodiment of the present disclosure, two light-receiving pixels included in each lens may be arranged by shifting from the optical axis of the lens in a first direction and in a direction opposite to the first direction, respectively. In this case, the first direction may mean the x-axis direction (10).

[0313] In one embodiment of the present disclosure, when two light-receiving pixels included in each lens are referred to as a first light-receiving pixel and a second light-receiving pixel, the optical axis of each lens may have a center that is shifted in a first direction from the first light-receiving pixel and shifted in the opposite direction from the second light-receiving pixel to the first direction.

[0314] In one embodiment of the present disclosure, a plurality of lenses (1100) may include a plurality of first light-receiving pixels (1110) and a plurality of second light-receiving pixels (1120).

[0315] In one embodiment of the present disclosure, the electronic device (100) can acquire a first view image through a plurality of first light-receiving pixels (1110) and acquire a second view image through a plurality of second light-receiving pixels (1120).

[0316] At this time, the first view image may be shifted in the x-axis direction (10) compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1100). The second view image may be shifted in the opposite direction of the x-axis direction (10) compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1100).

[0317] In one embodiment of the present disclosure, the electronic device (100) can obtain depth information of an object around the electronic device (100) based on a first view image and a second view image obtained through a plurality of light-receiving pixels (1110, 1120). Based on the obtained first view image and second view image, the electronic device (100) may also obtain depth information in the z-axis direction (30) relative to the x-axis direction (10).

[0318] Referring to FIG. 1, FIG. 3 and FIG. 11b, in one embodiment of the present disclosure, FIG. 11b shows a plurality of light-emitting pixels (311), a plurality of light-receiving pixels (1140, 1150) and a plurality of lenses (1130) among the components included in a display device (110).

[0319] Multiple light-receiving pixels (1140, 1150) may be arranged to correspond to multiple light-emitting pixels (311). Two light-receiving pixels may be arranged to correspond to a pixel area (310) containing each light-emitting pixel.

[0320] In one embodiment of the present disclosure, two light-receiving pixels may be arranged to correspond to each of the plurality of lenses (1130).

[0321] In one embodiment of the present disclosure, two light-receiving pixels included in each lens may be arranged by shifting from the optical axis of the lens in a first direction and in a direction opposite to the first direction, respectively. In this case, the first direction may mean the y-axis direction (20).

[0322] In one embodiment of the present disclosure, when two light-receiving pixels included in each lens are referred to as a first light-receiving pixel and a second light-receiving pixel, the optical axis of each lens may have a center shifted in a first direction from the first light-receiving pixel. Additionally, the optical axis of each lens may have a center shifted in the opposite direction from the second light-receiving pixel.

[0323] In one embodiment of the present disclosure, a plurality of lenses (1130) may include a plurality of first light-receiving pixels (1140) and a plurality of second light-receiving pixels (1150).

[0324] In one embodiment of the present disclosure, the electronic device (100) can acquire a first view image through a plurality of first light-receiving pixels (1140) and acquire a second view image through a plurality of second light-receiving pixels (1150).

[0325] At this time, the first view image may be shifted in the y-axis direction (20) compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1130). The second view image may be shifted in the opposite direction of the y-axis direction (20) compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1130).

[0326] In one embodiment of the present disclosure, the electronic device (100) can obtain depth information of an object around the electronic device (100) based on a first view image and a second view image obtained through a plurality of light-receiving pixels (1140, 1150). Based on the obtained first view image and second view image, the electronic device (100) may also obtain depth information in the z-axis direction (30) relative to the y-axis direction (20).

[0327] FIG. 12a is a drawing for explaining that four light-receiving pixels are arranged in each of a plurality of lenses according to one embodiment of the present disclosure. FIG. 12b is a drawing for explaining that four light-receiving pixels are arranged in each of a plurality of lenses according to one embodiment of the present disclosure. FIG. 12c is a drawing for explaining that four light-receiving pixels are arranged in each of a plurality of lenses according to one embodiment of the present disclosure. Hereinafter, the same reference numerals are assigned to configurations identical to those described in FIG. 3, and redundant descriptions are omitted.

[0328] Referring to FIG. 1, FIG. 3 and FIG. 12a, in one embodiment of the present disclosure, FIG. 12a shows a display device (110) included in an electronic device (100). FIG. 12a shows a plurality of light-emitting pixels (311), a plurality of light-receiving pixels (1201, 1202, 1203, 1204), and a plurality of lenses (1200) among the components included in the display device (110) for convenience of explanation.

[0329] In one embodiment of the present disclosure, a plurality of light-receiving pixels (1201, 1202, 1203, 1204) may be arranged to correspond to a plurality of light-emitting pixels (311). Four light-receiving pixels may be arranged to correspond to a pixel area (310) containing each light-emitting pixel.

[0330] In one embodiment of the present disclosure, four light-receiving pixels may be arranged to correspond to each of the plurality of lenses (1200).

[0331] In one embodiment of the present disclosure, four light-receiving pixels included in each lens may be arranged by shifting from the optical axis of the lens in a first direction, a direction opposite to the first direction, a second direction, and a direction opposite to the second direction. At this time, the first direction may mean the x-axis direction (10). The second direction may mean the y-axis direction (20).

[0332] In one embodiment of the present disclosure, each lens may include a first light-receiving pixel positioned by shifting in a first direction from the optical axis of the lens, a second light-receiving pixel positioned by shifting in a direction opposite to the first direction from the optical axis of the lens, a third light-receiving pixel positioned by shifting in a second direction from the optical axis of the lens, and a fourth light-receiving pixel positioned by shifting in a direction opposite to the second direction from the optical axis of the lens.

[0333] In one embodiment of the present disclosure, a plurality of lenses (1200) may include a plurality of first light-receiving pixels (1201), a plurality of second light-receiving pixels (1202), a plurality of third light-receiving pixels (1203), and a plurality of fourth light-receiving pixels (1204).

[0334] In one embodiment of the present disclosure, an electronic device (100) can acquire a first view image through a plurality of first light-receiving pixels (1201). The electronic device (100) can acquire a second view image through a plurality of second light-receiving pixels (1202). The electronic device (100) can acquire a third view image through a plurality of third light-receiving pixels (1203). The electronic device (100) can acquire a fourth view image through a plurality of fourth light-receiving pixels (1204).

[0335] At this time, the first view image may be shifted in the x-axis direction (10) compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1200). The second view image may be shifted in the opposite direction of the x-axis direction (10) compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1200). The third view image may be shifted in the y-axis direction (20) compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1200). The fourth view image may be shifted in the opposite direction of the y-axis direction (20) compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1200).

[0336] In one embodiment of the present disclosure, the electronic device (100) can obtain depth information of an object around the electronic device (100) based on a first view image and a second view image obtained through a plurality of first light-receiving pixels (1201) and a plurality of second light-receiving pixels (1202). Based on the obtained first view image and second view image, the electronic device (100) may also obtain depth information in the z-axis direction (30) relative to the x-axis direction (10).

[0337] In one embodiment of the present disclosure, the electronic device (100) can obtain depth information of an object around the electronic device (100) based on a third view image and a fourth view image obtained through a plurality of third light-receiving pixels (1203) and a plurality of fourth light-receiving pixels (1204). Based on the obtained third view image and fourth view image, the electronic device (100) may also obtain depth information in the z-axis direction (30) relative to the y-axis direction (20).

[0338] However, the present disclosure is not limited thereto, and the electronic device (100) may use all of the acquired first to fourth view images to obtain depth information in the z-axis direction (30) based on a plane defined by the x-axis and y-axis.

[0339] Referring to FIG. 1, FIG. 3 and FIG. 12b, in one embodiment of the present disclosure, FIG. 12b shows a plurality of light-emitting pixels (311), a plurality of light-receiving pixels (1211, 1212, 1213, 1214) and a plurality of lenses (1210) among the components included in a display device (110).

[0340] In one embodiment of the present disclosure, a plurality of light-receiving pixels (1211, 1212, 1213, 1214) may be arranged to correspond to each of a plurality of light-emitting pixels (311). Each light-receiving pixel may be arranged to correspond to a pixel area (310).

[0341] In one embodiment of the present disclosure, four light-receiving pixels may be arranged to correspond to each of the plurality of lenses (1200).

[0342] In one embodiment of the present disclosure, four light-receiving pixels included in each lens may be arranged by shifting from the optical axis of the lens in a first direction, a direction opposite to the first direction, a second direction, and a direction opposite to the second direction, respectively.

[0343] In one embodiment of the present disclosure, each lens may include a first light-receiving pixel positioned by shifting in a first direction from the optical axis of the lens, a second light-receiving pixel positioned by shifting in a direction opposite to the first direction from the optical axis of the lens, a third light-receiving pixel positioned by shifting in a second direction from the optical axis of the lens, and a fourth light-receiving pixel positioned by shifting in a direction opposite to the second direction from the optical axis of the lens.

[0344] At this time, the first direction may mean any direction within the second quadrant on the plane defined by the x-axis and the y-axis. The second direction may mean any direction within the first quadrant on the plane defined by the x-axis and the y-axis.

[0345] In one embodiment of the present disclosure, a plurality of lenses (1210) may include a plurality of first light-receiving pixels (1211), a plurality of second light-receiving pixels (1212), a plurality of third light-receiving pixels (1213), and a plurality of fourth light-receiving pixels (1214).

[0346] In one embodiment of the present disclosure, an electronic device (100) can acquire a first view image through a plurality of first light-receiving pixels (1211). The electronic device (100) can acquire a second view image through a plurality of second light-receiving pixels (1212). The electronic device (100) can acquire a third view image through a plurality of third light-receiving pixels (1213). The electronic device (100) can acquire a fourth view image through a plurality of fourth light-receiving pixels (1214).

[0347] At this time, the first view image may be shifted in the direction of the second quadrant in a plane defined by the x-axis and y-axis by comparing it with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1200).

[0348] The second view image may be shifted in the direction of the fourth quadrant in a plane defined by the x-axis and y-axis, compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1200).

[0349] The third view image may be shifted in the direction of the first quadrant in a plane defined by the x-axis and y-axis, compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1200).

[0350] The fourth view image may be shifted in the direction of the third quadrant in a plane defined by the x-axis and y-axis, compared with an image obtained through a plurality of light-receiving pixels having a center that overlaps with the optical axis of a plurality of lenses (1200).

[0351] In one embodiment of the present disclosure, the electronic device (100) can obtain depth information of an object around the electronic device (100) based on a first view image and a second view image obtained through a plurality of first light-receiving pixels (1211) and a plurality of second light-receiving pixels (1212). Based on the obtained first view image and second view image, the electronic device (100) may also obtain depth information in the z-axis direction (30) with respect to a first direction.

[0352] In one embodiment of the present disclosure, the electronic device (100) can obtain depth information of an object around the electronic device (100) based on a third view image and a fourth view image obtained through a plurality of third light-receiving pixels (1213) and a plurality of fourth light-receiving pixels (1214). Based on the obtained third view image and fourth view image, the electronic device (100) may also obtain depth information in the z-axis direction (30) with respect to a second direction.

[0353] However, the present disclosure is not limited thereto, and the electronic device (100) may use all of the acquired first to fourth view images to obtain depth information in the z-axis direction (30) based on a plane defined by the x-axis and y-axis.

[0354] Referring to FIGS. 1, FIGS. 3, FIGS. 12b and FIGS. 12c, in one embodiment of the present disclosure, FIG. 12c shows a plurality of light-emitting pixels (311), a plurality of light-receiving pixels (1221, 1222, 1223, 1224), and a plurality of lenses (1220) included in a display device (110).

[0355] In one embodiment of the present disclosure, a plurality of light-receiving pixels (1221, 1222, 1223, 1224) may be arranged to correspond to a plurality of light-emitting pixels (311). Four light-receiving pixels may be arranged to correspond to a pixel area (310) containing each light-emitting pixel.

[0356] In one embodiment of the present disclosure, each of the plurality of lenses (1220) may be arranged such that each light-emitting pixel corresponds to four light-receiving pixels.

[0357] In one embodiment of the present disclosure, a light-emitting pixel corresponding to each lens may be arranged to overlap with the optical axis of the lens. Each light-emitting pixel may be arranged to be located on the optical axis of the lens. As a plurality of light-emitting pixels (311) are arranged to overlap with the optical axis of a plurality of lenses (1220), a degradation in image quality and resolution of the image (111) provided through the display device (110) may not occur.

[0358] In one embodiment of the present disclosure, a plurality of lenses (1210) may include a plurality of first light-receiving pixels (1211) each shifted in a first direction from the optical axis of the lens, a plurality of second light-receiving pixels (1212) each shifted in a direction opposite to the first direction from the optical axis of the lens, a plurality of third light-receiving pixels (1213) each shifted in a second direction from the optical axis of the lens, and a plurality of fourth light-receiving pixels (1214) each shifted in a direction opposite to the second direction from the optical axis of the lens.

[0359] At this time, the first direction may mean any direction within the second quadrant on the plane defined by the x-axis and the y-axis. The second direction may mean any direction within the first quadrant on the plane defined by the x-axis and the y-axis.

[0360] In one embodiment of the present disclosure, the electronic device (100) can obtain depth information around the electronic device (100) through a plurality of light-receiving pixels (1221, 1222, 1223, 1224) as shown in FIG. 12b.

[0361] In one embodiment of the present disclosure, the positional relationship between the plurality of light-emitting pixels (311), the plurality of light-receiving pixels, and the plurality of lenses shown in FIGS. 10a to 12c is, as an example, not limited thereto. The plurality of light-emitting pixels (311), the plurality of light-receiving pixels, and the plurality of lenses included in the display device (110) may have various positional relationships that can acquire multiple view images through the plurality of light-receiving pixels and acquire depth information using the parallax between the plurality of view images.

[0362] FIG. 13 is a flowchart illustrating an operation for acquiring depth information according to the rotational state of an electronic device according to an embodiment of the present disclosure. Hereinafter, the same reference numerals are assigned to steps identical to those described in FIG. 8, and redundant descriptions are omitted. In addition, FIG. 13 may represent an operation in which four light-receiving pixels shifted in four different directions from the optical axis of the lens are correspondingly arranged in each lens included in the display device (110).

[0363] Referring to FIGS. 1, FIGS. 2, FIGS. 3, FIGS. 8 and FIGS. 13, in one embodiment of the present disclosure, a method of operating an electronic device (100) may include a step (S240) of detecting a rotational state of the electronic device (100).

[0364] In one embodiment of the present disclosure, the rotational state of the electronic device (100) may include a first rotational state (1500, see FIG. 15) in which the first direction is the right direction of the electronic device (100) and the second direction is the left direction of the electronic device (100). Additionally, the rotational state of the electronic device (100) may include a second rotational state (1510, see FIG. 15) in which the third direction is the right direction of the electronic device (100) and the fourth direction is the left direction of the electronic device (100).

[0365] In one embodiment of the present disclosure, the first direction may be a column direction in which a plurality of light-emitting pixels (311) and a plurality of light-receiving pixels (320) included in the display device (110) are arranged. The third direction may be a row direction in which a plurality of light-emitting pixels (311) and a plurality of light-receiving pixels (320) included in the display device (110) are arranged.

[0366] In one embodiment of the present disclosure, with reference to FIG. 14, the first direction may mean the x-axis direction (10). The second direction may mean the direction opposite to the x-axis direction (10). The third direction may mean the y-axis direction (20). The fourth direction may mean the direction opposite to the y-axis direction (20).

[0367] In step S240, the electronic device (100) can detect the rotational state of the electronic device (100).

[0368] In one embodiment of the present disclosure, the electronic device (100) may include an inertial sensor.

[0369] In one embodiment of the present disclosure, the inertial sensor may include an acceleration sensor, a gyroscope, a magnetometer, or an Inertial Measurement Unit (IMU) including these. In one embodiment of the present disclosure, the electronic device (100) may obtain a 3DoF (Degree of Freedom) or 6DoF value by measuring a change in the motion state of the electronic device (100) through the inertial sensor.

[0370] In one embodiment of the present disclosure, the electronic device (100) can sense the rotational state of the electronic device (100) using an inertial sensor. The electronic device (100) can sense the movement of a user (200) using the electronic device (100) or the movement of the electronic device (100) to detect whether the electronic device (100) is in a first rotational state or a second rotational state.

[0371] In step S240, the electronic device (100) can detect the rotational state of the electronic device (100).

[0372] In one embodiment of the present disclosure, when the rotational state of the electronic device (100) is detected to be a first rotational state, the method of operation of the electronic device (100) may include the step (S250) of acquiring depth information based on the parallax between the view image acquired through the first light receiving pixel and the view image acquired through the second light receiving pixel.

[0373] In step S250, the electronic device (100) can acquire a view image through a first light-receiving pixel and acquire a view image through a second light-receiving pixel as the rotation state of the electronic device (100) is detected as a first rotation state. At this time, the view image acquired from the first light-receiving pixel and the view image acquired from the second light-receiving pixel may be a view image at the left viewpoint and a view image at the right viewpoint of the electronic device (100) based on the first rotation state.

[0374] In one embodiment of the present disclosure, the electronic device (100) can obtain depth information based on the parallax between a view image obtained through a first light-receiving pixel and a view image obtained through a second light-receiving pixel.

[0375] In one embodiment of the present disclosure, when the rotation state of the electronic device (100) is detected to be a second rotation state, the method of operation of the electronic device (100) may include the step (S260) of obtaining depth information based on the parallax between the view image obtained through the third light receiving pixel and the view image obtained through the fourth light receiving pixel.

[0376] In step S260, the electronic device (100) can acquire a view image through a third light-receiving pixel and acquire a view image through a fourth light-receiving pixel as the rotation state of the electronic device (100) is detected as a second rotation state. At this time, the view image acquired from the third light-receiving pixel and the view image acquired from the fourth light-receiving pixel may be a view image of the left viewpoint and a view image of the right viewpoint of the electronic device (100) based on the second rotation state.

[0377] Below, the description of steps S240, S250, and S260 will be provided in FIGS. 14 and FIGS. 15.

[0378] FIG. 14 is a drawing for explaining an operation of acquiring depth information according to the rotational state of an electronic device according to one embodiment of the present disclosure. FIG. 15 is a drawing for explaining an operation of acquiring depth information according to the rotational state of an electronic device according to one embodiment of the present disclosure. Hereinafter, the same reference numerals are assigned to configurations identical to those described in FIG. 3, and redundant descriptions are omitted.

[0379] Referring to FIGS. 2, FIGS. 3 and FIGS. 14, in one embodiment of the present disclosure, FIG. 14 shows a display device (110) included in an electronic device (100). FIG. 14 also shows a controller (301), a scan control unit (1400), and a sensing driver (303) among a plurality of components included in at least one processor (150).

[0380] However, the present disclosure is not limited thereto, and at least one processor (150) may further include a data driver (302), etc.

[0381] In one embodiment of the present disclosure, the display device (110) may include a first scan driver (1410) and a second scan driver (1420). However, the present disclosure is not limited thereto, and at least one processor (150) may further include a light-emitting driver (305), etc.

[0382] In FIG. 14, the first scan driver (1410) and the second scan driver (1420) are shown as separate components, but it is obvious that the operation of the first scan driver (1410) and the second scan driver (1402) may be performed on a single component.

[0383] In one embodiment of the present disclosure, the display device (110) may include a plurality of light-emitting pixels (311), a plurality of light-receiving pixels (1430, 1431, 1440, 1441), and a plurality of lenses (1450).

[0384] In one embodiment of the present disclosure, four light-receiving pixels and light-emitting pixels may be arranged to correspond to each lens. The light-emitting pixels may be arranged on the optical axis of each lens.

[0385] In one embodiment of the present disclosure, a plurality of light-receiving pixels (1430, 1431, 1440, 1441) may include a plurality of first light-receiving pixels (1430) arranged by shifting in a first direction from the optical axis of the lens, a plurality of second light-receiving pixels (1431) arranged by shifting in a direction opposite to the first direction from the optical axis of the lens, a plurality of third light-receiving pixels (1440) arranged by shifting in a second direction from the optical axis of the lens, and a plurality of fourth light-receiving pixels (1441) arranged by shifting in a direction opposite to the second direction from the optical axis of the lens.

[0386] At this time, the first direction may mean the x-axis direction (10), and the second direction may mean the y-axis direction (20).

[0387] In one embodiment of the present disclosure, the display device (110) may include a plurality of first light-receiving scan lines (1412) that extend from a first scan driver (1410) in a direction opposite to the x-axis direction (10) and are spaced apart from each other in the y-axis direction (20). The display device (110) may include a plurality of second light-receiving scan lines (1422, 1423) that extend from a second scan driver (1410) in the x-axis direction (10) and are spaced apart from each other in the y-axis direction (20). At this time, FIG. 14 is an example, and the arrangement and connection relationship of the plurality of first light-receiving scan lines (1412) and the plurality of second light-receiving scan lines (1422, 1423) may vary.

[0388] In one embodiment of the present disclosure, each of the plurality of first light-receiving pixels (1430) and the plurality of second light-receiving pixels (1431) may be electrically connected to a corresponding plurality of first light-receiving scan lines (1412). Each of the plurality of second light-receiving pixels (1440) and the plurality of second light-receiving pixels (1441) may be electrically connected to a corresponding plurality of second light-receiving scan lines (1422, 1423).

[0389] At this time, it goes without saying that each of the multiple light-receiving pixels (1430, 1431, 1440, 1441) may be electrically connected to multiple sensing lines.

[0390] In one embodiment of the present disclosure, depending on the configuration of the light receiving pixel driving circuit included in each of the plurality of light receiving pixels (1430, 1431, 1440, 1441), the connection relationship between each of the plurality of first light receiving sensing plurality of light receiving pixels (1430, 1431, 1440, 1441) and the plurality of sensing lines and the plurality of light receiving scan lines may be changed.

[0391] In one embodiment of the present disclosure, a scan control unit (1400) can control the operation of a first scan driver (1410) and a second scan driver (1420). Specifically, the scan control unit (1400) can control the operation of the first scan driver (1410) through a first scan control signal (1411). The scan control unit (1400) can control the operation of the second scan driver (1420) through a second scan control signal (1421).

[0392] In one embodiment of the present disclosure, the scan control unit (1400) can selectively control the operation of either the first scan driver (1410) or the second scan driver (142) according to the rotational state of the electronic device (100).

[0393] In one embodiment of the present disclosure, the scan control unit (1400) may provide a first scan control signal (1411) to a first scan driver (1410) as it is determined that the electronic device (100) is in a first rotational state. The scan control unit (1400) may provide a second scan control signal (1421) to a second scan driver (1420) as it is determined that the electronic device (100) is in a second rotational state.

[0394] In one embodiment of the present disclosure, the scan control unit (1400) includes a switching element such as a transistor and can selectively provide a first scan control signal (1411) or a second scan control signal (1421) depending on the rotational state of the detected electronic device (100).

[0395] However, as an example, depending on the settings of the electronic device (100), the scan control unit (1400) may provide a first scan control signal (1411) to the first scan driver (1410) and provide a second scan control signal (142) to the second scan driver (1420), regardless of the rotation state of the electronic device (100).

[0396] In one embodiment of the present disclosure, the first scan driver (1410) may generate a plurality of first sub-scan signals based on a first scan control signal (1411). The first scan driver (1410) may provide a first sub-scan signal to each of a plurality of first light receiving scan lines (1412).

[0397] In one embodiment of the present disclosure, the electronic device (100) controls the timing of applying a first sub-scan signal to each of the plurality of first light-receiving pixels (1430) and the plurality of second light-receiving pixels (1431), and adjusts the length or size of the signal interval included in the first sub-scan signal, thereby obtaining an electrical signal for obtaining a first view image and a second view image through a display device (110).

[0398] In one embodiment of the present disclosure, a sensing driver (303) can acquire an electrical signal sensed from a plurality of first light-receiving pixels (1430) and a plurality of second light-receiving pixels (1431) through a plurality of sensing lines. The sensing driver (303) can convert the acquired electrical signal to acquire an image signal corresponding to a first view image and a second view image.

[0399] In one embodiment of the present disclosure, the second scan driver (1420) may generate a plurality of second sub-scan signals based on the second scan control signal (1421). The second scan driver (1420) may provide the second sub-scan signals to each of the plurality of second light receiving scan lines (1422, 1423).

[0400] In one embodiment of the present disclosure, the electronic device (100) controls the timing of applying a second sub-scan signal to each of the plurality of third light-receiving pixels (1440) and the plurality of fourth light-receiving pixels (1441), and adjusts the length or size of the signal interval included in the second sub-scan signal, thereby obtaining an image signal for obtaining a third view image and a fourth view image through a display device (110).

[0401] In one embodiment of the present disclosure, the electronic device (100) controls the timing of applying a second scan signal to each of the plurality of light-receiving pixels (320) and adjusts the length or size of the signal interval included in the second scan signal, thereby obtaining an image signal for obtaining a plurality of view images (120, 121) through a display device (110).

[0402] In one embodiment of the present disclosure, the sensing driver (303) can acquire an electrical signal sensed from a plurality of third light-receiving pixels (1440) and a plurality of fourth light-receiving pixels (1441) through a plurality of sensing lines. The sensing driver (303) can convert the acquired electrical signal to acquire an image signal corresponding to a third view image and a fourth view image.

[0403] Referring to FIGS. 13, 14 and 15, in one embodiment of the present disclosure, FIG. 15 illustrates a case where the electronic device (100) is in a first rotational state (1500) and a case where the electronic device (100) is in a second rotational state (1510).

[0404] In one embodiment of the present disclosure, the first rotation state (1500) may mean a case where the user (200) uses the electronic device (100) in a state where the length of the electronic device (100) in the y-axis direction (20) is longer than the length of the electronic device (100) in the first direction. The second rotation state (1510) may mean a case where the user (200) uses the electronic device (100) in a state where the length of the electronic device (100) in the first direction is longer than the length of the electronic device (100) in the y-axis direction (20).

[0405] In one embodiment of the present disclosure, the first rotation state (1500) may be referred to as portrait mode or vertical mode. The second rotation state (1510) may be referred to as landscape mode or horizontal mode.

[0406] In one embodiment of the present disclosure, a user (300) can rotate the electronic device (100) clockwise or counterclockwise to use it in a first rotation state (1500) or a second rotation state (1510).

[0407] In one embodiment of the present disclosure, when a user (300) rotates the electronic device (100) clockwise or counterclockwise, the image (111) may also be rotated clockwise or counterclockwise and displayed on the display device (110). Accordingly, the user (300) can view the image (111) without moving their head to correspond to the rotation of the electronic device (100).

[0408] In one embodiment of the present disclosure, when acquiring depth information using the parallax between view images through a stereo matching algorithm, accuracy may be higher when using a left view image and a right view image rather than using an upside view image and a downside view image.

[0409] Accordingly, the electronic device (100) can select a scan driver to control for acquiring a view image between the first scan driver (1410) or the second scan driver (1420) depending on the rotation state of the electronic device (100) in order to acquire a view image of the left view and a view image of the right view.

[0410] Specifically, based on the plurality of lenses (1450) and the plurality of light-receiving pixels (1430, 1431, 1440, 1441) shown in FIG. 14, the electronic device (100) can control the operation of the first scan driver (1410) as the electronic device (100) determines that it is in a first rotation state (1500). The electronic device (100) can provide a first scan control signal (1521) including a turn-on signal to the first scan driver (1410) to control the first transistor (1522) electrically connected to the plurality of first light-receiving pixels (1430) and the second transistor (1523) electrically connected to the plurality of second light-receiving pixels (1431) to turn on.

[0411] Through this, the electronic device (100) can obtain a first view image and a second view image through a plurality of first light-receiving pixels (1430) and a plurality of second light-receiving pixels (1431). The electronic device (100) can obtain depth information of the user (200) through the first view image and the second view image corresponding to the left view image and the right view image in the first rotation state (1500).

[0412] In one embodiment of the present disclosure, the electronic device (100) can control the operation of the second scan driver (1420) as the electronic device (100) determines that it is in a second rotation state (1510). The electronic device (100) can provide a second scan control signal (1531) including a turn-on signal to the second scan driver (1420) to control the third transistor (1524) electrically connected to a plurality of third light-receiving pixels (1440) and the fourth transistor (1525) electrically connected to a plurality of fourth light-receiving pixels (1441) to turn on.

[0413] Through this, the electronic device (100) can obtain a third view image and a fourth view image through a plurality of third light-receiving pixels (1440) and a plurality of second light-receiving pixels (1441). The electronic device (100) can obtain depth information of the user (200) through the third view image and the fourth view image corresponding to the left view image and the right view image in the second rotation state (1510).

[0414] The electronic device (100) of the present disclosure can acquire high-accuracy depth information by controlling the operation of the first scan driver (1410) and the second scan driver (1420) so as to acquire a view image of the left viewpoint and a view image of the right viewpoint by taking into account the rotation state of the electronic device (100).

[0415] Additionally, the electronic device (100) can reduce computational power consumption and power consumption by selectively using only some of the light-receiving pixels among the plurality of light-receiving pixels (1430, 1431, 1440, 1441) included in the display device (110) to obtain depth information of the user (200).

[0416] In order to solve the technical problem described above, an electronic device is provided in one embodiment of the present disclosure. The electronic device may include a display device. The electronic device may include a memory in which a program or at least one instruction is stored. The electronic device may include at least one processor including processing circuitry. The display device may include a base layer. The display device may include a circuit layer disposed on the base layer. The display device may include a pixel layer disposed on the circuit layer and comprising a plurality of light-emitting pixels and a plurality of light-receiving pixels. The display device may include an optical layer disposed on the pixel layer. The display device may include a lens layer disposed on the optical layer and comprising a plurality of lenses corresponding to a plurality of light-receiving pixels. The plurality of lenses may be arranged so that the optical axis of the plurality of lenses does not pass through the center of the plurality of light-receiving pixels. By having at least one processor execute a program or at least one instruction stored in memory individually or collectively, the electronic device may display an image through a plurality of light-emitting pixels. By having at least one processor execute a program stored in memory or at least one instruction individually or collectively, the electronic device can acquire depth information based on the disparity between multiple view images acquired through multiple light-receiving pixels.

[0417] In one embodiment of the present disclosure, an electronic device may acquire a first view image of an object from a plurality of first light-receiving pixels having a center shifted in a first direction from the optical axis of a plurality of lenses among a plurality of light-receiving pixels. The electronic device may acquire a second view image of an object from a plurality of second light-receiving pixels having a center shifted in a second direction different from the first direction from the optical axis of a plurality of lenses among a plurality of light-receiving pixels. The electronic device may acquire depth information of an object from the first view image and the second view image through a stereo matching algorithm.

[0418] In one embodiment of the present disclosure, a light-receiving pixel may be disposed to correspond to each of the plurality of lenses. The plurality of lenses may include a plurality of first lenses having a first optical axis shifted in a first direction from the plurality of light-receiving pixels, and a plurality of second lenses having a second optical axis shifted in a second direction different from the first direction from the plurality of light-receiving pixels.

[0419] In one embodiment of the present disclosure, two or more light-receiving pixels may be arranged to correspond to each of the number of lenses.

[0420] In one embodiment of the present disclosure, a first light-receiving pixel and a second light-receiving pixel may be arranged to correspond to each lens. The optical axis of the lens may be shifted in a first direction from the first light-receiving pixel and shifted in a second direction different from the first direction from the second light-receiving pixel.

[0421] In one embodiment of the present disclosure, a first light-receiving pixel, a second light-receiving pixel, a third light-receiving pixel, and a fourth light-receiving pixel may be arranged to correspond to each lens. The optical axis of the lens may be shifted in a first direction from the first light-receiving pixel, shifted in a second direction opposite to the first direction from the second light-receiving pixel, shifted in a third direction different from the first and second directions from the third light-receiving pixel, and shifted in a fourth direction opposite to the third direction from the fourth light-receiving pixel.

[0422] In one embodiment of the present disclosure, the electronic device can detect the rotational state of the electronic device. When the rotational state of the electronic device is a first rotational state in which the first direction is the right direction of the electronic device (100) and the second direction is the left direction of the electronic device (100), depth information can be obtained based on the parallax between the view image obtained through the first light-receiving pixel and the view image obtained through the second light-receiving pixel. When the rotational state of the electronic device is a second rotational state in which the third direction is the right direction of the electronic device (100) and the fourth direction is the left direction of the electronic device (100), depth information can be obtained based on the parallax between the view image obtained through the third light-receiving pixel and the view image obtained through the fourth light-receiving pixel.

[0423] In one embodiment of the present disclosure, the display device may include a light blocking film disposed on an optical layer. The light blocking film may include a plurality of openings superimposed with a plurality of lenses.

[0424] In one embodiment of the present disclosure, the thickness of the optical layer may correspond to the focal length of each of the plurality of lenses.

[0425] In one embodiment of the present disclosure, the pixel layer may include a protective film for protecting a plurality of light-emitting pixels from the outside. The display device may include an optical adhesive layer disposed on the pixel layer and the optical layer.

[0426] To solve the technical problem described above, in one embodiment of the present disclosure, a method of operation of an electronic device including a display device may be provided. The display device may include a base layer. The display device may include a circuit layer disposed on the base layer. The display device may include a pixel layer disposed on the circuit layer and comprising a plurality of light-emitting pixels and a plurality of light-receiving pixels. The display device may include an optical layer disposed on the pixel layer. The display device may include a lens layer disposed on the optical layer and comprising a plurality of lenses corresponding to a plurality of light-receiving pixels. The plurality of lenses may be arranged so that the optical axis of the plurality of lenses does not pass through the center of the plurality of light-receiving pixels. The method of operation of the electronic device may include the step of displaying an image through a plurality of light-emitting pixels. The method of operation of the electronic device may include the step of acquiring depth information based on the disparity between a plurality of view images acquired through a plurality of light-receiving pixels.

[0427] In one embodiment of the present disclosure, the step of acquiring depth information may include acquiring a first view image of an object from a plurality of first light-receiving pixels having a center shifted in a first direction from the optical axis of a plurality of lenses among a plurality of light-receiving pixels. The step of acquiring depth information may include acquiring a second view image of an object from a plurality of second light-receiving pixels having a center shifted in a second direction different from the first direction from the optical axis of a plurality of lenses among a plurality of light-receiving pixels. The step of acquiring depth information may include acquiring depth information of an object from the first view image and the second view image through a stereo matching algorithm.

[0428] In one embodiment of the present disclosure, a light-receiving pixel may be disposed to correspond to each of the plurality of lenses. The plurality of lenses may include a plurality of first lenses having a first optical axis shifted in a first direction from the plurality of light-receiving pixels, and a plurality of second lenses having a second optical axis shifted in a second direction different from the first direction from the plurality of light-receiving pixels.

[0429] In one embodiment of the present disclosure, two or more light-receiving pixels may be arranged to correspond to each of the plurality of lenses.

[0430] In one embodiment of the present disclosure, a first light-receiving pixel and a second light-receiving pixel may be arranged to correspond to each lens. The optical axis of the lens may be shifted in a first direction from the first light-receiving pixel and shifted in a second direction different from the first direction from the second light-receiving pixel.

[0431] In one embodiment of the present disclosure, a first light-receiving pixel, a second light-receiving pixel, a third light-receiving pixel, and a fourth light-receiving pixel may be arranged to correspond to each lens. The optical axis of the lens may be shifted in a first direction from the first light-receiving pixel, shifted in a second direction opposite to the first direction from the second light-receiving pixel, shifted in a third direction different from the first and second directions from the third light-receiving pixel, and shifted in a fourth direction opposite to the third direction from the fourth light-receiving pixel.

[0432] In one embodiment of the present disclosure, a method of operating an electronic device may include a step of detecting a rotational state of the electronic device. When the rotational state of the electronic device is a first rotational state in which the first direction is the right direction of the electronic device (100) and the second direction is the left direction of the electronic device (100), the method may include a step of obtaining depth information based on the parallax between a view image obtained through a first light-receiving pixel and a view image obtained through a second light-receiving pixel. When the rotational state of the electronic device is a second rotational state in which the third direction is the right direction of the electronic device (100) and the fourth direction is the left direction of the electronic device (100), the method may include a step of obtaining depth information based on the parallax between a view image obtained through a third light-receiving pixel and a view image obtained through a fourth light-receiving pixel.

[0433] In one embodiment of the present disclosure, the display device may include a light blocking film disposed on an optical layer. The light blocking film may include a plurality of apertures superimposed with a plurality of lenses. The thickness of the optical layer may correspond to the focal length of each of the plurality of lenses.

[0434] In one embodiment of the present disclosure, the pixel layer may include a protective film for protecting a plurality of light-emitting pixels from the outside. The display device may include an optical adhesive layer disposed on the pixel layer and the optical layer.

[0435] In order to solve the technical problem described above, a display device may be provided in one embodiment of the present disclosure. The display device may include a base layer. The display device may include a circuit layer disposed on the base layer. The display device may include a pixel layer disposed on the circuit layer and comprising a plurality of light-emitting pixels and a plurality of light-receiving pixels. The display device may include an optical layer disposed on the pixel layer. The display device may include a first light-blocking film disposed on the optical layer and comprising a plurality of first apertures. The display device may include a lens layer disposed on the first light-blocking film and comprising a plurality of lenses corresponding to each of the plurality of light-receiving pixels. The plurality of first apertures may overlap with the plurality of lenses.

[0436] A program executed by an electronic device described in this disclosure may be implemented by hardware components, software components, and / or a combination of hardware components and software components. The program may be executed by any system capable of executing computer-readable instructions.

[0437] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively.

[0438] Software can be implemented as a computer program containing instructions stored on a computer-readable storage medium. Examples of computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROMs, DVDs (Digital Versatile Discs)). Computer-readable recording media can be distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner. The recording medium is readable by a computer, stored in memory, and can be executed by a processor.

[0439] Computer-readable storage media may be provided in the form of non-transitory storage media. Here, 'non-transitory storage media' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily. For example, 'non-transitory storage media' may include a buffer in which data is stored temporarily.

[0440] In addition, the program according to the embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product.

[0441] A computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, a computer program product may include a product in the form of a software program (e.g., a downloadable application) that is distributed electronically through a manufacturer of an electronic device or an electronic market (e.g., Samsung Galaxy Store). For electronic distribution, at least a portion of the software program may be stored on a storage medium or temporarily created. In this case, the storage medium may be a server of the manufacturer of the electronic device, a server of the electronic market, or a storage medium of a relay server that temporarily stores the software program.

[0442] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, appropriate results can be achieved even if the described techniques are performed in a different order than described, and / or components such as the described computer system or module are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

Claims

1. In an electronic device (100), Display device (110); Memory (140) where a program or at least one instruction is stored; and It includes at least one processor (150) including a processing circuitry, The above display device (110) is, Base layer; A circuit layer disposed on the above base layer; A pixel layer disposed on the circuit layer and comprising a plurality of light-emitting pixels and a plurality of light-receiving pixels; An optical layer disposed on the pixel layer above; and It includes a lens layer disposed on the optical layer and comprising a plurality of lenses corresponding to the plurality of light-receiving pixels, and The plurality of lenses are arranged such that the optical axis of the plurality of lenses does not pass through the center of the plurality of light-receiving pixels, and By having the above at least one processor (150) execute the above program or the above at least one instruction stored in the memory (140) individually or collectively, the electronic device (100) An electronic device (100) that displays an image (111) through the plurality of light-emitting pixels and obtains depth information (130) based on the disparity between the plurality of view images obtained through the plurality of light-receiving pixels.

2. In Paragraph 1, The above electronic device (100) is, Among the plurality of light-receiving pixels, a first view image (120) of an object is obtained from a plurality of first light-receiving pixels having a center shifted in a first direction from the optical axis of the plurality of lenses, and Among the plurality of light-receiving pixels, a second view image (121) of an object is obtained from a plurality of second light-receiving pixels having a center shifted in a second direction different from the first direction from the optical axis of the plurality of lenses, and An electronic device (100) that obtains depth information (130) for the object from the first view image (120) and the second view image (121) through a stereo matching algorithm.

3. In either Paragraph 1 or Paragraph 2, In each of the above plurality of lenses, a light-receiving pixel is arranged to correspond to it, and The above plurality of lenses comprises a plurality of first lenses having a first optical axis shifted in a first direction from the plurality of light-receiving pixels and a plurality of second lenses having a second optical axis shifted in a second direction different from the first direction from the plurality of light-receiving pixels, an electronic device (100).

4. In any one of paragraphs 1 to 3, An electronic device (100) having two or more light-receiving pixels arranged to correspond to each of the plurality of lenses.

5. In Paragraph 4, In each lens, a first light-receiving pixel and a second light-receiving pixel are arranged to correspond to each other, and The optical axis of the above lens is shifted in a first direction from the first light-receiving pixel and shifted in a second direction different from the first direction from the second light-receiving pixel, in an electronic device (100).

6. In either Paragraph 4 or Paragraph 5, In each lens, a first light-receiving pixel, a second light-receiving pixel, a third light-receiving pixel, and a fourth light-receiving pixel are arranged to correspond to each other, and The optical axis of the above lens is shifted in a first direction from the first light-receiving pixel, shifted in a second direction opposite to the first direction from the second light-receiving pixel, shifted in a third direction different from the first direction and the second direction from the third light-receiving pixel, and shifted in a fourth direction opposite to the third direction from the fourth light-receiving pixel.

7. In Paragraph 6, The above electronic device (100) is, Detecting the rotational state of the above electronic device (100), When the rotational state of the electronic device (100) is a first rotational state in which the first direction is the right direction of the electronic device (100) and the second direction is the left direction of the electronic device (100), The depth information (130) is obtained based on the parallax between the view image obtained through the first light-receiving pixel and the view image obtained through the second light-receiving pixel, and When the rotational state of the electronic device (100) is a second rotational state in which the third direction is the right direction of the electronic device (100) and the fourth direction is the left direction of the electronic device (100), An electronic device (100) that obtains depth information (130) based on the parallax between the view image obtained through the third light receiving pixel and the view image obtained through the fourth light receiving pixel.

8. In any one of paragraphs 1 through 7, The above display device (110) is, It further includes a light blocking film disposed on the optical layer, and The light blocking film is an electronic device (100) comprising a plurality of openings superimposed on the plurality of lenses.

9. In any one of paragraphs 1 through 8, The thickness of the optical layer corresponds to the focal length of each of the plurality of lenses in the electronic device (100).

10. In any one of paragraphs 1 through 9, The pixel layer above is, It further includes a protective film for protecting the plurality of light-emitting pixels mentioned above, and The above display device (110) is, An electronic device (100) further comprising an optical adhesive layer disposed between the pixel layer and the optical layer.

11. A method of operating an electronic device (100) including a display device (110), The above display device (110) comprises a base layer, a circuit layer disposed on the base layer, a pixel layer disposed on the circuit layer and including a plurality of light-emitting pixels and a plurality of light-receiving pixels, an optical layer disposed on the pixel layer, and a lens layer disposed on the optical layer and including a plurality of lenses corresponding to the plurality of light-receiving pixels. The plurality of lenses are arranged such that the optical axis of the plurality of lenses does not pass through the center of the plurality of light-receiving pixels, and The method of operation of the above electronic device (100) is, A step (S100) of displaying an image (111) through the plurality of light-emitting pixels; and A method of operation of an electronic device (100) comprising the step (S200) of acquiring depth information (130) based on the disparity between multiple view images acquired through the plurality of light-receiving pixels.

12. In Paragraph 11, The step (S200) of obtaining the depth information (130) above is, A step of obtaining a first view image (120) of an object from a plurality of first light-receiving pixels having a center shifted in a first direction from the optical axis of the plurality of lenses among the plurality of light-receiving pixels; A step of obtaining a second view image (121) of an object from a plurality of second light-receiving pixels having a center shifted in a second direction different from the first direction from the optical axis of the plurality of lenses among the plurality of light-receiving pixels; and A method of operation of an electronic device (100) comprising the step of obtaining depth information (130) for the object from the first view image (120) and the second view image (121) through a stereo matching algorithm.

13. In either Article 11 or Article 12, In each of the above plurality of lenses, a light-receiving pixel is arranged to correspond to it, and A method of operation of an electronic device (100) comprising the plurality of lenses, the plurality of first lenses having a first optical axis shifted in a first direction from the plurality of light-receiving pixels and the plurality of second lenses having a second optical axis shifted in a second direction different from the first direction from the plurality of light-receiving pixels.

14. In any one of paragraphs 11 through 13, A method of operation of an electronic device (100) in which two or more light-receiving pixels are arranged to correspond to each of the plurality of lenses.

15. A computer-readable recording medium having a program recorded thereon for performing the method of operation described in any one of paragraphs 11 through 14 on a computer.

Images