Electronic device comprising plurality of cameras, and control method therefor

Abstract

An electronic device according to one embodiment may comprise: a first housing including a first surface and a second surface opposite to the first surface; a second housing including a third surface and a fourth surface opposite to the third surface; a hinge structure for rotatably connecting the first housing and the second housing; a flexible display arranged on the first surface and the third surface; a first rear camera and a second rear camera arranged on the second surface; a front camera arranged on the fourth surface; at least one processor including a processing circuit; and a memory for storing instructions. According to one embodiment, when executed individually or collectively by the at least one processor, the instructions can cause the electronic device to: acquire a first image through the first rear camera, acquire a second image through the second rear camera and acquire a third image through the front camera on the basis that a user input for capturing a stereo image is received; acquire a left eye image and a right eye image of the stereo image on the basis of the first image and the second image; and acquire first depth information of the stereo image on the basis of the first image and the third image.

Description

Electronic device including multiple cameras and method for controlling the same

[0001] Various embodiments of the present invention relate to an electronic device including a plurality of cameras and a method for controlling the same.

[0002] The variety of services and additional features offered through electronic devices, such as smartphones, is steadily increasing. To enhance the utility of these devices and satisfy the diverse needs of users, telecommunications service providers and electronic device manufacturers are competitively developing electronic devices to offer a variety of features and differentiate themselves from competitors. Consequently, the various functions offered through electronic devices are also becoming increasingly sophisticated.

[0003] Recently, technologies have been developed that allow electronic devices to capture images in various ways, including using multiple cameras. Electronic devices can capture stereoscopic images using multiple cameras. Stereoscopic images capture a pair of 2D images that utilize the visual differences between the two eyes to create a three-dimensional sense of depth. Stereoscopic images are fundamentally based on the principle that the images shown to each human eye differ due to parallax (e.g., binocular parallax) based on the distance between the two eyes, and that this difference is recognized to create a sense of depth. Therefore, two images, a left image (left-eye image) and a right image (right-eye image), are expressed as a pair.

[0004] The above information may be provided as background information to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0005] According to one embodiment, an electronic device may include a first housing including a first side and a second side opposite the first side, a second housing including a third side and a fourth side opposite the third side, a hinge structure rotatably connecting the first housing and the second housing, a flexible display disposed on the first side and the third side, a first rear camera and a second rear camera disposed on the second side, a front camera disposed on the fourth side, at least one processor including a processing circuit, and a memory storing instructions. According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to acquire a first image through the first rear camera, a second image through the second rear camera, and a third image through the front camera, based on a user input for capturing a stereo image being received. In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain a left-eye image and a right-eye image of the stereo image based on the first image and the second image. In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain first depth information of the stereo image based on the first image and the third image.

[0006] According to one embodiment, a method of operating an electronic device may include: a first housing including a first side and a second side opposite to the first side; a second housing including a third side and a fourth side opposite to the third side; a hinge structure rotatably connecting the first housing and the second housing; a flexible display disposed on the first side and the third side; a first rear camera and a second rear camera disposed on the second side; and a front camera disposed on the fourth side. According to one embodiment, the method of operating the electronic device may include an operation of acquiring a first image through the first rear camera, a second image through the second rear camera, and a third image through the front camera, based on a user input for capturing a stereo image being received. According to one embodiment, the method of operating the electronic device may include an operation of acquiring a left-eye image and a right-eye image of a stereo image based on the first image and the second image. According to one embodiment, the method of operating the electronic device may include an operation of obtaining first depth information of the stereo image based on the first image and the third image.

[0007] According to one embodiment, a non-transitory computer-readable storage medium storing instructions, wherein the instructions, when collectively or individually executed by at least one processor, cause an electronic device, wherein the electronic device comprises a first housing including a first side and a second side opposite the first side, a second housing including a third side and a fourth side opposite the third side, a hinge structure rotatably connecting the first housing and the second housing, a flexible display disposed on the first side and the third side, a first rear camera and a second rear camera disposed on the second side, and a front camera disposed on the fourth side, wherein, based on a user input for capturing a stereo image, a first image is acquired through the first rear camera, a second image is acquired through the second rear camera, and a third image is acquired through the front camera, and based on the first image and the second image, a left-eye image and a right-eye image of a stereo image are acquired, and the first image and the second image are Based on the third image, first depth information of the stereo image can be obtained.

[0008] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0009] FIG. 2A is a diagram illustrating an unfolded state of an electronic device according to an embodiment of the present disclosure.

[0010] FIG. 2b is a diagram illustrating a folded state of an electronic device according to an embodiment of the present disclosure.

[0011] FIG. 3 is a diagram illustrating a state in which multiple cameras are arranged in an electronic device according to one embodiment of the present disclosure.

[0012] FIG. 4 is a block diagram of an electronic device including a plurality of cameras according to an embodiment of the present disclosure.

[0013] FIG. 5 is a diagram illustrating a concept of a depth information generation method according to an embodiment of the present disclosure.

[0014] FIG. 6 is a diagram illustrating the concept of a depth information estimation method using stereo images according to an embodiment of the present disclosure.

[0015] FIG. 7 is a diagram illustrating a method for generating depth information according to an embodiment of the present disclosure.

[0016] FIG. 8 is a diagram illustrating an example of a data format of a stereo video including depth information according to an embodiment of the present disclosure.

[0017] FIGS. 9A and 9B are diagrams illustrating a method of displaying stereo video using depth information in an electronic device or a wearable electronic device according to an embodiment of the present disclosure.

[0018] FIGS. 10A and 10B are diagrams illustrating a method of displaying stereo video using depth information in an electronic device or a wearable electronic device according to an embodiment of the present disclosure.

[0019] FIG. 11 is a diagram showing an example of depth information estimation according to a baseline length difference, according to an embodiment of the present disclosure.

[0020] FIG. 12 is a diagram illustrating a method for displaying stereo video using depth information in a wearable electronic device according to an embodiment of the present disclosure.

[0021] FIG. 13 is a diagram illustrating a method for displaying stereo video using depth information in a wearable electronic device according to an embodiment of the present disclosure.

[0022] FIG. 14 is a flowchart illustrating a method for an electronic device to acquire a stereo image according to an embodiment of the present disclosure.

[0023] FIG. 1 is a block diagram of an electronic device within a network environment according to one embodiment.

[0024] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0025] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0026] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0027] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0028] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0029] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0030] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0031] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0032] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0033] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0034] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0035] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0036] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0037] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0038] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0039] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0040] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0041] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0042] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0043] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0044] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0045] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0046] The artificial intelligence-related functions according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, one or more processors may be a general-purpose processor such as a central processing unit (CPU), an application processor (AP), or a digital signal processor (DSP), a graphics-only processor such as a graphics processing unit (GPU) or a vision processing unit (VPU), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0047] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is trained using a learning algorithm using a plurality of learning data, thereby creating a predefined operation rules or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0048] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks.

[0049] FIG. 2A is a diagram illustrating an unfolded state of an electronic device according to one embodiment of the present disclosure. FIG. 2B is a diagram illustrating a folded state of an electronic device according to one embodiment of the present disclosure.

[0050] Referring to FIGS. 2A and 2B, an electronic device (101) (e.g., the electronic device (101) of FIG. 1) may include a housing (201), a hinge cover (240) covering a foldable portion of the housing (201), and a display (230) (e.g., the display module (160) of FIG. 1) disposed within a space formed by the housing (201).

[0051] According to one embodiment, the surface on which the screen output from the display (230) is exposed may be defined as the front surface of the electronic device (101) (e.g., the first front surface (210a) and the second front surface (220a)). The surface opposite to the front surface may be defined as the back surface of the electronic device (101) (e.g., the first back surface (210b) and the second back surface (220b)). In one embodiment, the surface surrounding the space between the front surface and the back surface may be defined as the side surface of the electronic device (101) (e.g., the first side surface (210c) and the second side surface (220c)). The side surface of the electronic device (101) may be the side surface of at least one of the first housing (210) and the second housing (220). The electronic device (101) of FIGS. 2A and 2B may be referred to as a foldable electronic device, a portable electronic device, or a portable foldable electronic device. In one embodiment, the housing (201) may be referred to as a foldable housing. The display (230) may be referred to as a “flexible display.”

[0052] According to one embodiment, the housing (201) may form at least a portion of the exterior of the electronic device (101). The housing (201) may include a first housing (210), a second housing (220) that can rotate with respect to the first housing (210), a first rear cover (280), and a second rear cover (290). The housing (201) of the electronic device (101) is not limited to the shape and combination shown in FIGS. 2A and 2B and may be implemented by a combination and / or combination of other shapes or parts. For example, in one embodiment, the first housing (210) and the first rear cover (280) may be formed integrally, and the second housing (220) and the second rear cover (290) may be formed integrally. According to one embodiment, the first rear cover (280) may be defined and / or referred to as a first rear plate (280). The second rear cover (290) may be defined and / or referred to as a second rear plate (290).

[0053] According to one embodiment, the first housing (210) is connected to a hinge structure (e.g., the hinge structure (202) of FIG. 4) and may include a first front surface (210a) facing a first direction and a first rear surface (210b) facing a second direction opposite to the first direction. The second housing (220) is connected to the hinge structure and includes a second front surface (220a) facing a third direction and a second rear surface (220b) facing a fourth direction opposite to the third direction, and may rotate with respect to the first housing (210) about the hinge structure (202). Accordingly, the electronic device (101) may be variable between a folded state and an unfolded state. The folding or unfolding motion of the electronic device (101) can be understood as the rotation of the first housing (210) with respect to the hinge structure or the rotation of the second housing (220) with respect to the hinge structure. When the electronic device (101) is in a folded state, the first front side (210a) can face the second front side (220a). When the electronic device (101) is in an unfolded state, the third direction can be the same as the first direction. In the following, unless otherwise stated, the directions are described based on the unfolded state of the electronic device (101).

[0054] According to one embodiment, the first housing (210) and the second housing (220) are arranged on both sides with respect to the folding axis (Ax) as the center, and may have an overall symmetrical shape with respect to the folding axis (Ax). As described below, the angle or distance between the first housing (210) and the second housing (220) may vary depending on whether the state of the electronic device (101) is in an unfolded state, a folded state, or an intermediate state. According to one embodiment, the second housing (220) additionally includes a sensor area (224) in which sensors (e.g., cameras) are arranged, but may have a mutually symmetrical shape in other areas.

[0055] According to one embodiment, the folding axis (Ax) may be a plurality of parallel folding axes (e.g., two). In the present disclosure, the folding axis (Ax) is provided along the longitudinal direction (Y-axis direction) of the electronic device (101), but the direction of the folding axis (Ax) is not limited thereto. For example (not shown), an embodiment may be implemented in which the electronic device (101) includes a folding axis extending along the width direction (e.g., X-axis direction).

[0056] According to one embodiment, the electronic device (101) may include a structure into which a digital pen (not shown) can be attached. For example, the electronic device (101) may include a magnetic body configured to attach the digital pen to a side of the first housing (210) or a side of the second housing (220). According to one embodiment, the electronic device (101) may include a structure into which a digital pen can be inserted. For example, a hole (not shown) into which a digital pen can be inserted may be formed in a side of the first housing (210) or a side of the second housing (220) of the electronic device (101).

[0057] According to one embodiment, at least a portion of the first housing (210) and the second housing (220) may be formed of a metallic or non-metallic material having a rigidity of a size selected to support the display (230). At least a portion formed of the metallic material may provide a ground plane of the electronic device (101) and may be electrically connected to a ground line formed on a printed circuit board (e.g., the board portion (260) of FIG. 4).

[0058] According to one embodiment, the sensor area (224) may be formed to have a predetermined area adjacent to one edge or one corner of the second housing (220). However, the arrangement, shape, and size of the sensor area (224) are not limited to the illustrated example. According to one embodiment, the sensor area (224) may be provided in another corner of the second housing (220) or any area between the upper and lower corners or in the first housing (210). In one embodiment, components for performing various functions built into the electronic device (101) may be exposed to the front of the electronic device (101) through the sensor area (224) or through one or more openings provided in the sensor area (224). In one embodiment, the components may include various types of sensors. The sensor(s) may include, for example, at least one of a front camera, a receiver, or a proximity sensor.

[0059] According to one embodiment, the first rear cover (280) is disposed on one side of the folding axis (Ax) at the rear of the electronic device (101) and may have, for example, a substantially rectangular periphery, the periphery of which may be wrapped by another structure of the first housing (210). Similarly, the second rear cover (290) is disposed on the other side of the folding axis (Ax) at the rear of the electronic device (101) and the periphery of which may be wrapped by another structure of the second housing (220).

[0060] According to one embodiment, the first rear cover (280) and / or the second rear cover (290) may have a shape that is substantially symmetrical about the folding axis (Ax). However, the first rear cover (280) and the second rear cover (290) do not necessarily have mutually symmetrical shapes, and in one embodiment, the electronic device (101) may include the first rear cover (280) and the second rear cover (290) of different shapes that are not symmetrical.

[0061] According to one embodiment, the first rear cover (280), the second rear cover (290), the first housing (210), and the second housing (220) may provide a space in which various components of the electronic device (101) (e.g., a printed circuit board or a battery) may be placed. According to one embodiment, one or more components may be placed or visually exposed on the rear surface of the electronic device (101). For example, at least a portion of the sub-display (234) may be visually exposed through at least a portion of the first rear cover (280). According to one embodiment, one or more components or sensors may be visually exposed through at least a portion of the second rear cover (290). In various embodiments, the sensors may include a proximity sensor and / or a camera module (206) (e.g., a rear camera).

[0062] According to one embodiment, a front camera exposed to the front of the electronic device (101) through one or more openings provided in the sensor area (224) or a camera module (206) (e.g., a rear camera) exposed through at least a portion of the second rear cover (290) may include one or more lenses, image sensors, and / or image signal processors. In some embodiments, two or more lenses (infrared camera, wide-angle and telephoto lenses) and image sensors may be arranged on one side of the electronic device (101).

[0063] In one embodiment, the camera module (206) exposed through at least a portion of the second rear cover (290) may face opposite the second display area (232) of the display (230).

[0064] According to one embodiment, the hinge cover (240) may be disposed between the first housing (210) and the second housing (220) to cover internal components (e.g., the hinge structure (202) of FIG. 4). According to one embodiment, the hinge cover (240) may be covered by a portion of the first housing (210) and the second housing (220) or exposed to the outside, depending on the state of the electronic device (101) (flat state or folded state).

[0065] According to one embodiment, as illustrated in FIG. 2, when the electronic device (101) is in an unfolded state, the hinge cover (240) may not be exposed because it is covered by the first housing (210) and the second housing (220). According to one embodiment, as illustrated in FIG. 2B, when the electronic device (101) is in a folded state (e.g., a fully folded state), the hinge cover (240) may be exposed to the outside between the first housing (210) and the second housing (220). According to one embodiment, when the first housing (210) and the second housing (220) are in an intermediate state where they are folded with a certain angle, the hinge cover (240) may be partially exposed to the outside between the first housing (210) and the second housing (220). However, in this case, the exposed area may be less than that in the fully folded state. In one embodiment, the hinge cover (240) may include a curved surface.

[0066] According to one embodiment, the display (230) may be positioned on a space formed (or defined) by the housing (201). For example, the display (230) may be seated on a recess provided by the housing (201) and may form a majority of the front surface of the electronic device (101). Accordingly, the front surface of the electronic device (101) may include the display (230), a portion of the first housing (210) adjacent to the display (230) and a portion of the second housing (220). The back surface of the electronic device (101) may include a first back cover (280), a portion of the first housing (210) adjacent to the first back cover (280), a second back cover (290), and a portion of the second housing (220) adjacent to the second back cover (290).

[0067] In one embodiment, the display (230) may include a plurality of display areas spaced apart from each other. For example, the display (230) may include a first display area (231) disposed on a first housing (210), a second display area (232) disposed on a second housing (220), and a folding area (233). In one embodiment, the first display area (231) and the second display area (232) may rotate about a folding axis (Ax).

[0068] According to one embodiment, the display (230) may refer to a display in which at least a portion of the display can be transformed into a flat or curved surface. For example, the display (230) may be a foldable or flexible display. According to one embodiment, the display (230) may include a folding area (233), a first display area (231) arranged on one side (e.g., the left side of the folding area (233) illustrated in FIG. 2) with respect to the folding area (233), and a second display area (232) arranged on the other side (e.g., the right side of the folding area (233) illustrated in FIG. 2). However, the division of the areas of the display (230) is exemplary, and the display (230) may be divided into a plurality of areas (e.g., four or more or two) depending on the structure or function. For example, in the embodiment illustrated in FIG. 2, the display (230) may be divided into regions by a folding region (233) or a folding axis (Ax) extending parallel to the Y-axis. In one embodiment, the display (230) may also be divided into regions based on another folding region (e.g., a folding region parallel to the X-axis) or another folding axis (e.g., a folding axis parallel to the X-axis). In one embodiment, the display (230) may be coupled to or disposed adjacent to a touch detection circuit, a pressure sensor capable of measuring the intensity (pressure) of a touch, and / or a digitizer (not shown) configured to detect a magnetic field-type stylus pen.

[0069] According to one embodiment, the first display area (231) and the second display area (232) may have an overall symmetrical shape centered on the folding area (233). According to one embodiment (not shown), the second display area (232), unlike the first display area (231), may include a cut notch depending on the presence of the sensor area (224), but may have a shape that is substantially symmetrical with respect to the first display area (231) in other areas. For example, the first display area (231) and the second display area (232) may include a portion having a symmetrical shape with respect to each other and a portion having an asymmetrical shape with respect to each other.

[0070] Hereinafter, the operation of the first housing (210) and the second housing (220) and each area of ​​the display (230) according to the state of the electronic device (101) (e.g., flat state or unfolded state and folded state) will be described.

[0071] According to one embodiment, when the electronic device (101) is in a flat state (e.g., FIG. 2), the first housing (210) and the second housing (220) may be arranged to face the same direction at a substantially 180-degree angle. The surface of the first display area (231) of the display (230) and the surface of the second display area (232) may form a 180-degree angle with each other and face the same direction (e.g., toward the front of the electronic device). The folding area (233) may form the same plane as the first display area (231) and the second display area (232).

[0072] According to one embodiment, when the electronic device (101) is in a folded state (e.g., FIG. 2b), the first housing (210) and the second housing (220) may be arranged to face each other. The surface of the first display area (231) of the display (230) and the surface of the second display area (232) may form a narrow angle (e.g., between about 0 and 10 degrees) with each other and may face each other. When the electronic device (101) is in a folded state, the folding area (233) may be formed as a curved surface having at least a predetermined curvature.

[0073] According to one embodiment, when the electronic device (101) is in an intermediate state (not shown), the first housing (210) and the second housing (220) may be arranged at a certain angle with respect to each other. The surface of the first display area (231) of the display (230) and the surface of the second display area (232) may form an angle that is greater than the angle in the folded state and less than the angle in the unfolded state. The folding area (233) may be formed as a curved surface having at least a certain curvature, and the curvature at this time may be less than that in the folded state.

[0074] FIG. 3 is a diagram illustrating a state in which multiple cameras are arranged in an electronic device according to one embodiment of the present disclosure.

[0075] Referring to FIG. 3, an electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1) may include a plurality of cameras (e.g., camera module (190) of FIG. 1). However, the electronic device (101) according to one embodiment may further include other components described in FIG. 1.

[0076] According to one embodiment, an electronic device can capture a stereo image using a plurality of rear cameras (e.g., a first rear camera (310), a second rear camera (320)) included in a camera module (206) in a state where the housing (210, 220) is unfolded as illustrated in FIG. 3. For example, the electronic device can capture a stereo image using a plurality of rear cameras (e.g., a first rear camera (310), a second rear camera (320)) included in the camera module (206) in a state where the electronic device is positioned in a horizontal direction (e.g., a state where the electronic device is rotated 90 degrees).

[0077] According to one embodiment, a "stereo image" may be used to mean a pair of 2D images that are images acquired through two or more cameras and allow the perception of a three-dimensional stereoscopic effect (depth) by utilizing the difference in visual angles between the two eyes. For example, the stereo image may include a left-eye image corresponding to the left eye and a right-eye image corresponding to the right eye. According to one embodiment, the stereo image may be played back through an electronic device developed to be wearable by a user (e.g., a wearable electronic device such as an augmented reality glass (AR glass), a video see-through (VST) device, or a head-mounted display (HMD) device) and displayed to the user in the form of a 3D image. The stereo image may also be replaced with the term "spatial video," but is not limited to a specific term.

[0078] Meanwhile, the second housing (210) described below may be identical to the first housing (210) of FIGS. 2a and 2b. In addition, the first housing (220) may be identical to the second housing (220) of FIGS. 2a and 2b.

[0079] According to one embodiment, the camera may include a plurality of cameras, and among the plurality of cameras, a front camera (330) may be disposed on a fourth side of the second housing (210) (e.g., a front side when the electronic device (101) is in a folded state), and among the plurality of cameras, a plurality of rear cameras (310, 320) may be disposed on a second side of the first housing (220) (e.g., a rear side when the electronic device (101) is in a folded state). For example, when the electronic device is placed in a horizontal direction (a direction rotated 90 degrees from the vertical direction illustrated in FIG. 2A), a stereo image may be captured using at least two (e.g., a first rear camera (310) and a second rear camera (320)) of the plurality of rear cameras (392) arranged in a row in a first direction (e.g., a Y direction). According to one embodiment, the plurality of rear cameras (310, 320) may include a first rear camera (310) and a second rear camera (320) of different or the same type for capturing stereo images. The first rear camera (310) and the second rear camera (320) of different or the same type may be rear cameras arranged on the rear of the electronic device. For example, when the first rear camera (310) and the second rear camera (320) are different types of cameras, the first rear camera (310) may be a wide-angle camera, and the second rear camera (320) may be an ultra-wide-angle camera. The first rear camera (310) configured as the wide-angle camera may be referred to as a main camera, but is not limited to the above term.

[0080] According to one embodiment, when the electronic device is in a foldable form, as illustrated in FIGS. 2A and 3 , when the electronic device is fully unfolded, the front camera (330) and the rear camera (310 and / or 320) may face the same direction. Accordingly, depth information may be acquired using the front camera (330) positioned at the front and one of the plurality of rear cameras (310, 320) positioned at the rear (e.g., the first rear camera (310) or the second rear camera (320)).

[0081] According to one embodiment, in the embodiments described below, for convenience of explanation, among the multiple cameras used for stereo image capturing, the first rear camera (310) may be set as a camera corresponding to the left eye, and the second rear camera (320) may be set as a camera corresponding to the right eye. According to various embodiments, the first rear camera (310) may be set as a camera corresponding to the right eye, and the second rear camera (320) may be set as a camera corresponding to the left eye.

[0082] According to one embodiment, the electronic device may acquire a first image through the first rear camera (310), a second image through the second rear camera (320), and a third image through the front camera (330) based on receiving a user input for capturing a stereo image. According to one embodiment, the electronic device may acquire a left-eye image and a right-eye image of the stereo image based on the first image and the second image. The electronic device may acquire first depth information of the stereo image based on the first image and the third image. According to various embodiments, the precision of the first depth information may be higher than the precision of the second depth information acquired based on the first image and the second image. A detailed description thereof will be provided later in the description of FIG. 11.

[0083] According to various embodiments, the electronic device may store data of the stereo image acquired based on the first image and the second image in a memory. The electronic device may store the first depth information acquired based on the first image and the third image in the memory. For example, the electronic device may store the first depth information and a left-eye image (e.g., the first image) and a right-eye image (e.g., the second image) for a stereo image together. The electronic device may display the stereo image through a display (e.g., a flexible display) using the first depth information based on receiving a user input for reproducing the stereo image. Alternatively, the electronic device may display the stereo image through a display of a wearable electronic device (e.g., an HMD device, a VST device, a VR device, or an XR device) using the first depth information based on receiving a user input for reproducing the stereo image. Various embodiments of displaying stereo images using the above first depth information will be described in detail later in the descriptions of FIGS. 9a, 9b, 10a, and 10b.

[0084] According to various embodiments, when a user captures a stereo image, the user may capture the image while holding the electronic device horizontally, as illustrated in FIG. 3. At this time, since the user typically perceives that the image is captured through the rear camera (310 and / or 320), the front camera (330) may be covered by the hand. Even if the front camera (330) is covered by the hand, a stereo image may be acquired through a plurality of rear cameras (310, 320). However, since the front camera (330) may be used to acquire the high-precision first depth information, the electronic device may display a guide screen to allow the user to change the grip position when the front camera (330) is covered. The user may obtain depth information through the front camera (330) by checking the displayed guide screen and adjusting the grip position. According to various embodiments, the guide screen may include a screen displaying the position of the front camera (330) or a screen displaying a hand position guide.

[0085] FIG. 4 is a block diagram of an electronic device including a plurality of cameras according to an embodiment of the present disclosure.

[0086] Referring to FIG. 4, an electronic device (101) including a plurality of cameras may include a processor (420) (e.g., the processor (120) of FIG. 1), a memory (430) (e.g., the memory (130) of FIG. 1), a flexible display (200) (e.g., the display module (160) of FIG. 1), and a plurality of cameras (e.g., the camera module (180) of FIG. 1). The plurality of cameras may include a first rear camera (310) and a second rear camera (320). Although FIG. 4 illustrates two rear cameras, this is for convenience of explanation, and the electronic device (101) may include three or more rear cameras. The plurality of cameras may include at least one front camera (330).

[0087] According to one embodiment, the processor (420) may control the overall operation of the electronic device (101). For example, the processor (420) may be implemented in the same or similar manner as the processor (120) of FIG. 1. The processor (420) according to one embodiment may execute software (e.g., the program (140) of FIG. 1) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (420), and may perform data processing or calculations based on the instructions. The instructions according to one embodiment may include instructions configured in a machine language that can be processed by the electronic device (101) or the processor (420). For example, the instructions may include instructions corresponding to operation instructions used in the program.

[0088] Meanwhile, although FIG. 2 illustrates that the electronic device (101) includes one processor (420), this is merely exemplary and the technical concept of the present invention may not be limited thereto. For example, the electronic device (101) may include at least one processor. For example, the processor (420) may be implemented as at least one processor.

[0089] According to one embodiment, the memory (430) (e.g., the memory (130) of FIG. 1) may store at least one instruction (or command) that causes at least one operation of the electronic device (101). The at least one instruction, when executed individually or collectively by the processor (420), may cause the electronic device (101) to perform a corresponding operation.

[0090] According to one embodiment, the memory (430) may store data of the electronic device (101). For example, the data may include data about content, and the content may include images and / or videos.

[0091] According to one embodiment, the generative AI model may be stored in the memory (430). Alternatively, the generative AI model may be stored in an external electronic device (e.g., a server). For example, the processor (220) may use the generative AI model to obtain a three-dimensional space image (or a three-dimensional space image) using an image obtained from at least one of the first rear camera (310), the second rear camera (320), or the front camera (330).

[0092] According to one embodiment, when a user inputs execution of an application related to taking pictures or videos on an electronic device (101), the processor (420) may execute a program corresponding to the application related to taking pictures or videos stored in the memory (430) (e.g., program (140) of FIG. 1).

[0093] According to one embodiment, the processor (420) may display a preview image through the flexible display (200) according to the execution of an application related to the photo or video recording. The preview image displayed through the flexible display (200) may correspond to a first image (e.g., a left-eye image) acquired (or captured) through the first rear camera (310) or a second image (e.g., a right-eye image) acquired (or captured) through the second rear camera (320).

[0094] According to one embodiment, when a user input for starting shooting of a stereo image is received through the application, and a user input for ending shooting is received after a certain period of time, the processor (420) may store a first image acquired through the first rear camera (310) and a second image acquired through the second rear camera (320) from the start of shooting to the end of shooting as a stereo image in the memory (430). According to one embodiment, the stereo image stored in the memory (430) may be stored in the form of a compressed file based on a specified coding method (e.g., HV-HEVC (multi-view high efficiency video coding)). According to one embodiment, the user input for starting shooting of the stereo image may be set or confirmed by an input of a video shooting start button when the shooting mode in the application is set to a stereo image shooting mode, or may be set or confirmed by an input of a separately set stereo image shooting start button. According to various embodiments, the processor (420) may acquire a third image through the front camera (330) when capturing the stereo image. The processor (420) may acquire depth information based on the first image acquired through the first rear camera (310) and the third image acquired through the front camera (330). According to another embodiment, the processor (420) may acquire depth information based on the second image acquired through the second rear camera (320) and the third image acquired through the front camera (330). The processor (420) may store the acquired depth information in the memory (430). The processor (420) may synchronize the depth information with the stereo image and store it in the memory (430). A specific embodiment in which the processor (420) generates depth information based on the first image (or the second image) and the third image will be described in detail later in the description of FIG. 6.

[0095] FIG. 5 is a diagram illustrating a concept of a depth information generation method according to an embodiment of the present disclosure.

[0096] Referring to FIG. 5, according to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 4) may capture and acquire a stereo image by using a stereo image capturing module (510), a depth information generating module (520), an additional information storage module (530), and a codec (540). For example, the stereo image capturing module (510), the depth information generating module (520), the additional data storage module (530), and the codec (540) may be included in the electronic device (101) and may be implemented as at least a part of software or hardware.

[0097] According to one embodiment, the electronic device (101) can capture a stereo image using a plurality of rear cameras (e.g., a first rear camera (310) and a second rear camera (320)) when the electronic device (101) is positioned in a horizontal direction (e.g., when the electronic device is rotated 90 degrees) as illustrated in FIG. 5. According to various embodiments, the electronic device (e.g., the electronic device (101) of FIG. 4) can also capture a stereo image using a plurality of rear cameras (e.g., a first rear camera (310) and a second rear camera (320)) when the electronic device (101) is positioned in a vertical direction (e.g., when the electronic device is rotated 0 degrees or 180 degrees) as illustrated in FIG. 5. However, for convenience of explanation, the operation of capturing a stereo image will be described with emphasis on the operation performed when the electronic device (101) is positioned in a horizontal direction.

[0098] According to one embodiment, the stereo image capturing module (510) may acquire a first image (e.g., a left-eye image) and a second image (e.g., a right-eye image) using a plurality of rear cameras (310, 320). For example, the stereo image capturing module (510) may include an application related to taking pictures or videos. For example, when an input requesting execution of the stereo image capturing module (510) (e.g., an application related to taking pictures or videos) is confirmed, the electronic device (101) may execute the stereo image capturing module (510) (e.g., an application related to taking pictures or videos). When a user input for starting shooting of a stereo image is confirmed through a stereo image shooting module (510) (e.g., an application related to taking pictures or videos), a first image (e.g., a left-eye image) and a second image (e.g., a right-eye image) can be acquired through the first rear camera (310) and the second rear camera (320) from the start of shooting to the end of shooting.

[0099] According to one embodiment, the depth information generation module (520) may acquire a first image and a third image using the first rear camera (310) and the front camera (330) when capturing the stereo image. The depth information generation module (520) may acquire depth information related to the stereo image using the acquired first image and the third image. For example, the depth information may include depth information of a space where the stereo image is captured. The depth information generation module (520) may acquire camera parameters related to the first rear camera (310) and the front camera (330). The depth information generation module (520) may use the camera parameters when acquiring the depth information. For example, the camera parameters may include intrinsic parameters and / or extrinsic parameters. For example, the intrinsic parameters may include information about the arrangement distance (e.g., baseline distance) between the first rear camera (310) and the front camera (330), the focal length of the first rear camera (310), and the focal length of the front camera (330). The extrinsic parameters may include information about the external brightness of the electronic device (101), the presence or absence of lighting, and whether the electronic device is indoors. According to another embodiment, the depth information generation module (520) may also obtain depth information using a second image obtained through the second rear camera (320) and a third image obtained through the front camera (330).

[0100] According to one embodiment, the additional information storage module (530) may acquire additional information (hereinafter, “additional information”) related to stereo image capturing. For example, the additional information may include time information (e.g., information about the time at which the stereo image was captured), location information (e.g., information about the location at which the stereo image was captured), and / or person information (e.g., information about a person included in the stereo image).

[0101] According to one embodiment, the codec (540) may generate data (550) for a stereo image and store the generated data (550) in a memory (e.g., memory (430) of FIG. 4). For example, the data (550) may include a left-eye image (or a left-eye image image), a right-eye image (or a right-eye image image), depth information, and / or additional information based on a specified format. For example, the codec (540) may store the first image and the second image acquired from the stereo image capturing module (510) as a stereo image. For example, the stereo image may be stored in the form of a compressed file based on a specified coding method (e.g., HV-HEVC (multi-view high efficiency video coding)). The codec (540) may store depth information acquired from the depth information generation module (520) in the data (550). For example, the codec (540) can synchronize depth information with a stereo image and store it in the data (550). In addition, the codec (540) can store additional information obtained from the additional information storage module (530) in the data (550).

[0102] According to one embodiment, the electronic device (101) may reproduce or display a corresponding stereo image through a display (e.g., a display (200) of FIG. 4) based on the stored data (550). According to another embodiment, after a wearable electronic device (e.g., an HMD device, a VST device, a VR device, or an XR device) obtains data for a stereo image stored by the electronic device (101), the wearable electronic device may display the stereo image using the first depth information through the display of the wearable electronic device. For example, the wearable electronic device may display the stereo image using the first depth information based on confirming a user input for reproduction of the stereo image. For example, the electronic device (101) may transmit data for a stereo image to the wearable electronic device according to a request of the wearable electronic device.

[0103] FIG. 6 is a diagram illustrating the concept of a depth information estimation method using stereo images according to an embodiment of the present disclosure.

[0104] Referring to FIG. 6, according to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 4) can acquire depth information based on a first image (611) (e.g., first rear camera (310)) and a third image (512) (e.g., front camera (330)).

[0105] According to one embodiment, the electronic device (101) can identify and compare similar pixels included in the first image (611) and the third image (612). For example, the electronic device (101) can identify a first pixel (621) included in the first image (611) and a first pixel (622) included in the third image (612), and compare a horizontal distance (DIS1) between the identified pixels (621, 622). The electronic device (101) can identify a second pixel (626) included in the first image (611) and a second pixel (627) included in the third image (612), and compare a horizontal distance (DIS2) between the identified pixels (626, 627). For example, the horizontal distances (DIS1, DIS2) can be determined on a pixel basis.

[0106] According to one embodiment, the electronic device (101) can obtain a disparity map (640) for the first image (611) and the third image (612) using a stereo disparity map estimation model. The electronic device (101) can obtain the disparity map (640) based on disparity distances (DIS1, DIS2). For example, the stereo disparity map estimation model (630) can be set or trained to obtain the disparity map based on the disparity distances (DIS1, DIS2).

[0107] According to one embodiment, the electronic device (101) may convert the displacement map (640) into a depth map (660) using the depth information conversion model (650). For example, the depth information conversion model (650) may be set or trained to convert the displacement map into a depth map. For example, the electronic device (101) may obtain information about the depth (D) based on a focal length (f) of at least one of the first rear camera (310) or the front camera (320), a distance (b) (e.g., a baseline) between the first rear camera (310) and the front camera (320), and a displacement distance (d). For example, the depth (D) may be determined as in Mathematical Expression 1 below. For example, the depth (D) may be determined in units of mm. For example, the displacement distance (d) can be determined based on the horizontal distance (DIS1) between the first pixels (621, 622) and / or the horizontal distance (DIS2) between the second pixels (626, 627).

[0108]

[0109] According to one embodiment, the electronic device (101) can obtain a depth map (660) as depth information for the first image (611) and the third image (612).

[0110] Based on the above-described method, the electronic device (101) can acquire depth information using at least two images acquired from different cameras (310, 330). The method by which the electronic device (101) acquires depth information will be specifically described in FIG. 7 below.

[0111] FIG. 7 is a diagram illustrating a method for generating depth information according to an embodiment of the present disclosure.

[0112] Referring to FIG. 7, according to one embodiment, the depth information generation module (520) may include a first image modification module (710), an image correction module (720), a depth information estimation module (730), and a second image modification module (740).

[0113] According to one embodiment, the first image modification module (710) may receive a first image acquired from the first rear camera (310) and a third image acquired from the front camera (330). The first image modification module (710) may modify, change, or rotate the captured images based on the vertical arrangement of the first rear camera (310) and the front camera (330). For example, the first image modification module (710) may rotate the vertically arranged images by 90 degrees (or 270 degrees). In addition, the first image modification module (710) may reduce the rotated images to a specified size (e.g., 0.25 times the size). For example, the operation of reducing the size of the images may be omitted to reduce the amount of computation.

[0114] According to one embodiment, the image rectification module (720) may acquire or output horizontally aligned images based on information about pre-calibrated camera parameters (e.g., intrinsic parameters and / or extrinsic parameters) and receiving the first image and the third image. For example, the first image and the third image may be images that have been modified (e.g., rotated and / or scaled down) by the first image rectification module. For example, the image rectification module (720) may acquire and output a plurality of binocular image images (e.g., two images corresponding to the first image and the third image) based on horizontally aligning the first image and the third image in a virtual plane space.

[0115] According to one embodiment, the depth information estimation module (730) may estimate and output depth information based on receiving horizontally aligned binocular video images. For example, the depth information estimation module (730) may estimate depth information based on a deep learning model or a non-deep learning model (e.g., a convolutional method). For example, the depth information estimation model (730) may include the stereo disparity map estimation module (630) and the depth map conversion model (650) described in FIG. 6. For example, when the depth information estimation module (730) uses a deep learning model, at least one of the stereo disparity map estimation module (630) and the depth map conversion model (650) described in FIG. 6 may include a deep learning model.

[0116] According to one embodiment, the second image modification module (740) may modify and output depth information received from the depth information estimation module (730) based on changes (e.g., rotation and scale reduction) made by the first image modification module (710). For example, the second image modification module (740) may reversely rotate the depth information by 90 degrees (or 270 degrees) and adjust the size of the depth information (e.g., increase the size) to fit the target size. The second image modification module (740) may output the modified depth information as depth information for the first image and the third image.

[0117] Based on the above-described method, the electronic device (101) can obtain depth information using at least two images obtained from different cameras (310, 330).

[0118] FIG. 8 is a diagram illustrating an example of a data format of a stereo video including depth information according to an embodiment of the present disclosure.

[0119] Referring to FIG. 8, according to one embodiment, data for a stereo image (e.g., data (550) of FIG. 5) may include a left-eye image (810), a right-eye image (820), and a depth image (830) corresponding to depth information, which are captured or acquired in time sequence. For example, the left-eye image (810) may include first images (e.g., left-eye image images) acquired through the first rear camera (310), and the right-eye image (820) may include second images (e.g., left-eye image images) acquired through the second rear camera (320).

[0120] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 4) can match the storage time of the left-eye image (810) and the right-eye image (820) with the storage time of the depth image (830). For example, the electronic device (101) can determine the storage time of the depth image (830) based on the capturing time of the left-eye image corresponding to the left-eye image (810). For example, since the left-eye image corresponding to the left-eye image (810) is necessary for generating both a stereo image and a depth image, the storage time of the left-eye image (810) and the right-eye image (820) and the storage time of the depth image (830) can be matched or synchronized based on the capturing time of the left-eye image corresponding to the left-eye image (810).

[0121] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 4) may compress a left-eye image (810) and a right-eye image (820) based on a base view. For example, when an image corresponding to a base view is a left-eye image (810), the electronic device (101) may store data (550) based on compressing right-eye image images (820) based on the left-eye image (810).

[0122] According to one embodiment, the electronic device (101) may store a depth image (830) in addition to the left-eye image (810) and the right-eye image (820) of the stereo image in the data (550). For example, the electronic device (101) may synchronize and add and store the depth image to the left-eye image (810) and the right-eye image (820) in the video track of MV-HEVC. For example, since the depth image (830) is composed of a single channel depth value that is not similar to the binocular images (810, 820) of the stereo image, it may not be possible to compress it based on the base view (Base view0). Accordingly, the electronic device (101) may store the depth image (830) together with the binocular images (810, 820) of the compressed stereo image in the video track of MV-HEVC, as shown in FIG. 8.

[0123] FIGS. 9A and 9B are diagrams illustrating a method of displaying stereo video using depth information in an electronic device or a wearable electronic device according to an embodiment of the present disclosure.

[0124] Referring to FIGS. 9A and 9B , according to one embodiment, an electronic device (101) or a wearable electronic device (e.g., a wearable electronic device that obtains data for a stereo video from the electronic device (101) of FIG. 4 ) may display a stereo video using depth information. For example, the wearable electronic device may include an HMD device, a VST device, a VR device, or an XR device.

[0125] Referring to FIG. 9A, according to one embodiment, the electronic device (101) or the wearable electronic device may generate parallax-adjusted binocular image content (e.g., binocular image) (930) based on a first image (e.g., an image acquired through a first rear camera (310)) (910) for a left-eye image and a second image (e.g., an image acquired through a second rear camera (320)) for a right-eye image. At this time, the electronic device (101) or the wearable electronic device can generate a spatial binocular video image (930) using depth information (e.g., depth information generated based on the first image (910) and the third image (e.g., an image acquired through the front camera (330)). For example, the electronic device (101) or the wearable electronic device can obtain a warped image (920) based on a disparity (or depth) between the first image (910) and the second image. The warped image can include at least a portion of the first image (910) and the second image (920) based on the disparity. For example, at least a portion of the warped image can include an uncolored portion. The electronic device (101) or the wearable electronic device can generate a spatial binocular video image (930) using the first image (910), the second image, and the depth information (or the third image). An in-painted image (930) can be generated or acquired. For example, the in-painted image (930) can include an image in which an unpainted portion of the warped image (920) is additionally colored. The electronic device (101) or the wearable electronic device can acquire the in-painted image (930) as parallax-adjusted binocular image content (e.g., a binocular image).

[0126] Based on the above-described method, the electronic device (101) or the wearable electronic device can obtain a more three-dimensional and spatial image for binocular imaging using depth information.

[0127] Referring to FIG. 9B, according to one embodiment, the electronic device (101) or the wearable electronic device may display a stereo image using a first image (940) for a left-eye image adjusted for parallax as described above in FIG. 9A and a second image (e.g., an image acquired through the second rear camera (320)) (950) for a right-eye image. The electronic device (101) or the wearable electronic device may adjust the three-dimensionality of the stereo image based on a user input. The electronic device (101) or the wearable electronic device may check the degree of three-dimensionality corresponding to the user input and generate a stereo image corresponding to the checked degree of three-dimensionality.

[0128] According to one embodiment, the electronic device (101) or the wearable electronic device may display a control bar (960) for adjusting the three-dimensional effect. For example, the wearable electronic device may display the control bar (960) on a screen displaying a stereoscopic image. Based on a user input for the control bar (960), the electronic device (101) or the wearable electronic device may display a screen with adjusted three-dimensional effect. For example, if a user input for moving the control bar (960) to the right (e.g., a user input for increasing the three-dimensional effect) is confirmed, the electronic device (101) may display a stereoscopic image (e.g., a stereoscopic image with enhanced depth based on the corresponding length) on the screen based on the length of the rightward movement.

[0129] Based on the above-described method, the electronic device (101) or the wearable electronic device can obtain a binocular image with a more three-dimensional and spatial sense by using depth information. In addition, the electronic device (101) or the wearable electronic device can display a stereoscopic image with adjusted three-dimensionality based on user input.

[0130] FIGS. 10A and 10B are diagrams illustrating a method of displaying stereo video using depth information in an electronic device or a wearable electronic device according to an embodiment of the present disclosure.

[0131] Referring to FIG. 10A, according to one embodiment, an electronic device (101) or a wearable electronic device (e.g., a wearable electronic device that obtains data for a stereo video from the electronic device (101) of FIG. 4) may display a stereo video using depth information. For example, the wearable electronic device may include an HMD device, a VST device, a VR device, or an XR device.

[0132] Referring to FIG. 10A, according to one embodiment, the electronic device (101) or the wearable electronic device may generate a stereo image representing a three-dimensional space based on a plurality of images (e.g., images acquired through a first rear camera (310), a second rear camera (320), and a front camera (330)) (1011, 1012, 1013, 1014) captured by the electronic device (101). At this time, the electronic device (101) or the wearable electronic device may generate an image (1020) representing a three-dimensional space with a sense of space by using depth information. For example, the electronic device (101) or the wearable electronic device may acquire depth information by using images acquired through the first rear camera (310) and the front camera (330) included in the electronic device (101). The electronic device (101) or the wearable electronic device can generate a stereo image representing a three-dimensional space using a plurality of images (1011, 1012, 1013, 1014) and depth information. For example, the electronic device (101) or the wearable electronic device can perform in-painting based on the plurality of images (1011, 1012, 1013, 1014) and depth information to generate an image of a portion not expressed in the plurality of images (1011, 1012, 1013, 1014). Through this, the electronic device (101) or the wearable electronic device can generate a stereo image representing a three-dimensional space. In addition, the electronic device (101) can also support a free view viewing function for the three-dimensional space. For example, the electronic device (101) or the wearable electronic device can display a screen representing the three-dimensional space viewed from a location requested by the user through a preview viewing function.

[0133] Referring to FIG. 10B, according to one embodiment, an electronic device (101) or the wearable electronic device can generate and display an image (1030) representing a three-dimensional space based on the method for generating a three-dimensional space of FIG. 10A described above. The electronic device (101) can generate a three-dimensional space with a more three-dimensional and spatial sense by using depth information. In addition, the electronic device (101) or the wearable electronic device can provide a preview function for the three-dimensional space by using depth information.

[0134] According to one embodiment, the electronic device (101) or the wearable electronic device may display a control window (1040) to support a free view viewing function on an image (1030) representing a three-dimensional space. For example, the electronic device (101) or the wearable electronic device may display a screen representing the three-dimensional space viewed from a location indicated by the user input, based on a user input to the control window (1040).

[0135] Based on the above-described method, the electronic device (101) or the wearable electronic device can display a screen that represents a three-dimensional space with a more three-dimensional and spatial sense by using depth information.

[0136] FIG. 11 is a diagram showing an example of depth information estimation according to a baseline length difference, according to one embodiment of the present disclosure.

[0137] Referring to FIG. 11, according to one embodiment, a first distance (or first baseline) (e.g., 85 mm) between the first rear camera (310) and the front camera (330) may be longer than a second distance (or second baseline) (e.g., 19 mm) between the first rear camera (310) and the second rear camera (320).

[0138] Referring to FIG. 11, according to a comparative example, the electronic device (101) may not estimate depth information based on the first image (1110) and the second image (1120) acquired using the rear cameras (310, 320). For example, the second distance (or the second baseline distance) between the first rear camera (310) and the second rear camera (320) may be 19 mm, and the disparity between the first image (1110) and the second image (1120) may be confirmed as 3 (e.g., 13 pixels). For example, the electronic device (101) may reduce the sizes of the first image (1110) and the second image (1120) to a specified size (e.g., reduced to 0.25 times) for high-speed calculation of depth information. For example, the depth information estimation model (1150) can confirm that the displacement difference between the first image (1115) and the second image (1125) reduced to a specified size is approximately 0, and can confirm that the corresponding length is approximately 0 mm. Accordingly, the electronic device (101) may not be able to obtain depth information based on the first image (1110) and the second image (1120) obtained using the rear cameras (310, 320) using the depth information estimation model (1150).

[0139] According to one embodiment, the electronic device (101) may estimate depth information based on a first image (1110) and a third image (1130) acquired using a first rear camera (310) and a front camera (330). For example, a first distance (or a first baseline distance) between the first rear camera (310) and the front camera (330) may be 85 mm, a disparity between the first image (1110) and the third image (1130) may be confirmed as 13 (e.g., 13 pixels), and a corresponding length may be confirmed as approximately 6800 mm. For example, the electronic device (101) may reduce the sizes of the first image (1110) and the third image (1130) to a specified size (e.g., reduced to 0.25 times) for high-speed calculation of depth information. For example, the depth information estimation model (1150) can confirm that the displacement difference between the first image (1115) and the third image (1135) reduced to a specified size is about 3, and the corresponding length can be confirmed to be about 1700 mm. Accordingly, the electronic device (101) can obtain depth information based on the first image (1110) and the third image (1130) obtained using the first rear camera (310) and the front camera (330) using the depth information estimation model (1150).

[0140] FIG. 12 is a diagram illustrating a method for displaying stereo video using depth information in a wearable electronic device according to an embodiment of the present disclosure.

[0141] Referring to (a) of FIG. 12, according to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 4) may estimate first depth information based on a first image (1210) and a third image (1220) acquired using a first rear camera (310) and a front camera (330). For example, the electronic device (101) may generate a first depth image (1240) corresponding to the estimated first depth information. Thereafter, a wearable electronic device (e.g., an HMD device, a VST device, a VR device, or an XR device) that acquires the first depth image (1240) and images for stereo images (e.g., images captured by the first rear camera (310) and the second rear camera (320)) from the electronic device (101) may acquire and display a first stereo image (1260) based on the first depth image (1240).

[0142] Referring to (b) of FIG. 12, according to a comparative example, the electronic device (101) may estimate second depth information based on the first image (1210) and the second image (1230) acquired using the rear cameras (310, 320). For example, the electronic device (101) may generate a second depth image (1250) corresponding to the estimated second depth information. The electronic device (101) may acquire a second stereo image (1270) based on the second depth image (1250). For example, the precision of the second depth information may be lower than the precision of the first depth information. Accordingly, the precision of the second depth image (1250) may be lower than the precision of the first depth image (1240). In addition, the precision of the second stereo image (1270) may be lower than the precision of the first stereo image (1260). For example, the precision of a portion (1275) of the second stereo image (1270) may be lower than the precision of a portion (1265) of the first stereo image (1260).

[0143] Based on the above-described method, the electronic device (101) can generate a stereo image with a more three-dimensional and spatial sense by using depth information based on images acquired through the first rear camera (310) and the front camera (330). A wearable electronic device (e.g., a wearable electronic device that receives data regarding a stereo image from the electronic device (101) of FIG. 4) can receive a stereo image generated simply and effectively by the electronic device (101) and reproduce the received stereo image.

[0144] FIG. 13 is a diagram illustrating a method for displaying stereo video using depth information in a wearable electronic device according to an embodiment of the present disclosure.

[0145] Referring to FIG. 13, according to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 4) may estimate first depth information based on a first image (1310) and a third image (1320) acquired using a first rear camera (310) and a front camera (330). For example, the electronic device (101) may generate a first depth image (1340) corresponding to the estimated first depth information. Thereafter, a wearable electronic device (e.g., an HMD device, a VST device, a VR device, or an XR device) that has acquired the first depth image (1240) and images for stereo images (e.g., images captured by the first rear camera (310) and the second rear camera (320)) from the electronic device (101) can acquire and display a first space image (e.g., a space cloud image) (1360) for restoring (or generating) a three-dimensional space (or a three-dimensional space model) based on the first depth image (1340).

[0146] Referring to (b) of FIG. 12, according to a comparative embodiment, the electronic device (101) can estimate second depth information based on the first image (1315) and the second image (1330) acquired using the rear cameras (310, 320). For example, the electronic device (101) can generate a second depth image (1350) corresponding to the estimated second depth information. The electronic device (101) can acquire a second space image (e.g., a space cloud image) (1370) for restoring (or generating) a three-dimensional space (or a three-dimensional space model) based on the second depth image (1350). For example, the precision of the second depth information may be lower than the precision of the first depth information. Accordingly, the precision of the second depth image (1350) may be lower than the precision of the first depth image (1340). Additionally, the precision of the second spatial image (1370) may be lower than the precision of the first spatial image (1360).

[0147] Based on the above-described method, the electronic device (101) can generate (or acquire) a more precise spatial image by using depth information based on images acquired through the first rear camera (310) and the front camera (330). The electronic device (101) can generate (or restore) an image representing a three-dimensional space with a more three-dimensional and spatial sense by using the more precise spatial image. A wearable electronic device (e.g., a wearable electronic device that receives data regarding a stereoscopic image from the electronic device (101) of FIG. 4) can receive a stereoscopic image generated simply and effectively by the electronic device (101) and reproduce the received stereoscopic image.

[0148] FIG. 14 is a flowchart illustrating a method for an electronic device to acquire a stereo image according to an embodiment of the present disclosure.

[0149] Referring to FIG. 14, according to one embodiment, in operation 1401, an electronic device (e.g., the electronic device (101) of FIG. 4) may acquire a first image through a first rear camera (e.g., the first rear camera (310) of FIG. 4), a second image through a second rear camera (e.g., the second rear camera (320) of FIG. 4), and a third image through a front camera (e.g., the front camera (330) of FIG. 4) based on a user input for capturing a stereo image being received. For example, the electronic device (101) may acquire or capture the first image, the second image, and the third image when the housings of the electronic device (101) are fully unfolded (e.g., an unfolded state of the electronic device). For example, the electronic device (101) may acquire or capture the first image, the second image, and the third image when the electronic device (101) is positioned in a designated direction in which the first rear camera (310) and the second rear camera (320) are vertically arranged (or when the electronic device (101) is positioned in a horizontal direction). For example, the electronic device (101) may acquire the first image, the second image, and the third image based on determining that the electronic device (101) is positioned in the designated direction and the electronic device is fully unfolded. For example, the electronic device (101) may display guide information that guides the electronic device (101) to be positioned in the designated direction and to be fully unfolded based on determining that the electronic device (101) is not positioned in the designated direction or is not fully unfolded.

[0150] According to one embodiment, the electronic device (101) may acquire a first image through the first rear camera (310) and acquire or capture a second image through the second rear camera (320) when the housings of the electronic device (101) are not fully unfolded. For example, the electronic device (101) may acquire only the first image and the second image based on determining that the electronic device (101) is not fully unfolded. For example, the electronic device (101) may acquire and store a general image based on the first image and the second image. Alternatively, the electronic device (101) may display guide information to induce the user to fully unfold the electronic device (101) for a stereo image based on determining that the electronic device (101) is not fully unfolded. According to an implementation, the electronic device (101) may acquire and store a left-eye image (or left-eye image image) and a right-eye image (or right-eye image image) of a stereo image based on the first image and the second image without depth information. According to one embodiment, in operation 1403, the electronic device (101) may acquire and store a left-eye image and a right-eye image of the stereo image based on the first image and the second image. For example, the first image may be a left-eye image for a left-eye image, and the second image may be a right-eye image for a right-eye image.

[0151] According to one embodiment, in operation 1405, the electronic device (101) may acquire and store first depth information related to the stereo image based on the first image and the third image. For example, operation 1405 may be performed in parallel or simultaneously with operation 1403.

[0152] According to one embodiment, in operation 1407, the electronic device (101) may store data of a stereo image including first depth information. For example, the electronic device (101) may store the first depth information and the left-eye image and the right-eye image of the stereo image together. For example, the electronic device (101) may store data of the stereo image based on a specified format (e.g., a format based on the HV-HEVC (multi-view high efficiency video coding) method).

[0153] According to one embodiment, in operation 1409, after a wearable electronic device (e.g., an HMD device, a VST device, a VR device, or an XR device) receives or acquires data of a stereo image from an electronic device (101), the wearable electronic device may display the stereo image using the first depth information through a display of the wearable electronic device. For example, the wearable electronic device may display the stereo image using the first depth information based on confirming a user input for reproduction of the stereo image. For example, the wearable electronic device may request transmission of data for the stereo image to the electronic device (101) based on confirming a user input for reproduction of the stereo image. For example, the electronic device (201) may transmit data of the stereo image to the wearable electronic device according to a request of the wearable electronic device. For example, the wearable electronic device may display the stereo image using the first depth information included in the data of the stereo image. The wearable electronic device can generate (or restore) a more precise and three-dimensional stereoscopic image or an image representing a three-dimensional space using the first depth information, and can display the stereoscopic image or an image representing a three-dimensional space.

[0154] According to another embodiment, the electronic device (101) may display a stereo image (e.g., a flexible display (200) of FIG. 4) using the first depth information based on a user input for playing back a stereo image being received. For example, the electronic device (101) may display a more precise and immersive stereo image using the first depth information. In addition, the electronic device (101) may generate (or restore) an image representing a more precise and three-dimensional space using the first depth information, and may display an image representing the three-dimensional space.

[0155] Based on the above-described method, the electronic device (101) can capture and generate more precise and spatial stereo images by using depth information based on images acquired through the first rear camera (310) and the front camera (330). In addition, the electronic device (101) can capture and generate stereo images simply and efficiently by using depth information based on images acquired through the first rear camera (310) and the front camera (330).

[0156] According to one embodiment, an electronic device (101) may include a first housing including a first side and a second side opposite to the first side, a second housing including a third side and a fourth side opposite to the third side, a hinge structure rotatably connecting the first housing and the second housing, a flexible display (200) disposed on the first side and the third side, a first rear camera (310) and a second rear camera (320) disposed on the second side, a front camera (330) disposed on the fourth side, at least one processor (420) including a processing circuit, and a memory (430) storing instructions. In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to acquire a first image through the first rear camera, a second image through the second rear camera, and a third image through the front camera based on a user input for capturing a stereo image being received. In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to acquire a left-eye image and a right-eye image of the stereo image based on the first image and the second image. In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to acquire first depth information of the stereo image based on the first image and the third image.

[0157] According to one embodiment, the precision of the first depth information may be higher than the precision of the second depth information obtained based on the first image and the second image.

[0158] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to store data of the stereo image in the memory. According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display the stereo image through the flexible display using the first depth information based on receiving a user input for reproduction of the stereo image.

[0159] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to generate information about a three-dimensional space of the stereo image using the first image, the second image, and the first depth information, based on a user input for reproduction of the stereo image being received. According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display the stereo image using the information about the three-dimensional space.

[0160] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to store the data of the stereo image including the first depth information based on synchronizing the first depth information with the first image and the second image.

[0161] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to estimate a disparity map for the first image and the third image based on horizontal distances between pixels corresponding to the same subject included in each of the first image and the third image. In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain the first depth information based on the disparity map.

[0162] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to obtain the first depth information based on providing the first image and the third image to a pre-trained model.

[0163] In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to determine whether the electronic device is positioned in a specified direction and whether the electronic device is fully unfolded. In one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to acquire the first image, the second image, and the third image based on determining that the electronic device is positioned in the specified direction and the electronic device is fully unfolded.

[0164] According to one embodiment, the instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to display guide information to guide the electronic device to be positioned in the designated direction and to fully unfold the electronic device based on determining that the electronic device is not positioned in the designated direction or is not fully unfolded.

[0165] According to one embodiment, the first distance between the first rear camera and the front camera may be longer than the second distance between the first rear camera and the second rear camera.

[0166] According to one embodiment, in a method of operating an electronic device (101), the electronic device may include a first housing including a first side and a second side opposite to the first side, a second housing including a third side and a fourth side opposite to the third side, a hinge structure rotatably connecting the first housing and the second housing, a flexible display (200) disposed on the first side and the third side, a first rear camera (310) and a second rear camera (320) disposed on the second side, and a front camera (330) disposed on the fourth side. According to one embodiment, the method of operating the electronic device may include an operation of acquiring a first image through the first rear camera, acquiring a second image through the second rear camera, and acquiring a third image through the front camera, based on receiving a user input for capturing a stereo image. According to one embodiment, the method of operating the electronic device may include an operation of acquiring a left-eye image and a right-eye image of a stereo image based on the first image and the second image. According to one embodiment, the method of operating the electronic device may include an operation of acquiring first depth information of the stereo image based on the first image and the third image.

[0167] According to one embodiment, the precision of the first depth information may be higher than the precision of the second depth information obtained based on the first image and the second image.

[0168] According to one embodiment, the method of operating the electronic device may further include an operation of storing data of the stereo image in the memory. According to one embodiment, the method of operating the electronic device may further include an operation of displaying the stereo image through the flexible display using the first depth information based on a user input for playing the stereo image being received.

[0169] According to one embodiment, the operation of displaying the stereo image may include an operation of generating information about a three-dimensional space of the stereo image using the first image, the second image, and the first depth information based on a user input for playing the stereo image being received. According to one embodiment, the operation of displaying the stereo image may include an operation of displaying the stereo image using the information about the three-dimensional space.

[0170] In one embodiment, the act of storing the data may include an act of storing the data of the stereo image including the first depth information based on synchronizing the first depth information with the first image and the second image.

[0171] In one embodiment, the operation of obtaining the first depth information may include an operation of estimating a disparity map for the first image and the third image based on horizontal distances between pixels corresponding to the same subject included in each of the first image and the third image. In one embodiment, the operation of obtaining the first depth information may include an operation of obtaining the first depth information based on the disparity map.

[0172] In one embodiment, the operation of obtaining the first depth information may include an operation of obtaining the first depth information based on providing the first image and the third image to a pre-learned model.

[0173] According to one embodiment, the operations of acquiring the first image, the second image, and the third image may include an operation of determining whether the electronic device is positioned in a specified direction and whether the electronic device is fully unfolded. According to one embodiment, the operations of acquiring the first image, the second image, and the third image may include an operation of acquiring the first image, the second image, and the third image based on determining that the electronic device is positioned in the specified direction and the electronic device is fully unfolded.

[0174] According to one embodiment, the method of operating the electronic device may further include an operation of displaying guide information for guiding the electronic device to be positioned in the specified direction and to be fully unfolded based on determining that the electronic device is not positioned in the specified direction or is not fully unfolded.

[0175] According to one embodiment, a non-transitory computer-readable storage medium (130, 430) storing instructions, wherein the instructions, when collectively or individually executed by at least one processor (420), cause an electronic device (101) to: a first housing including a first side and a second side opposite to the first side, a second housing including a third side and a fourth side opposite to the third side, a hinge structure rotatably connecting the first housing and the second housing, a flexible display (200) disposed on the first side and the third side, a first rear camera (310) and a second rear camera (320) disposed on the second side, and a front camera (330) disposed on the fourth side, wherein, based on a user input for capturing a stereo image, the electronic device acquires a first image through the first rear camera, acquires a second image through the second rear camera, and captures a first image through the front camera. A third image can be acquired through the first image and the second image, and a left-eye image and a right-eye image of a stereo image can be acquired based on the first image and the second image, and first depth information of the stereo image can be acquired based on the first image and the third image.

[0176] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0177] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0178] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0179] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0180] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0181] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101), A first housing comprising a first side and a second side opposite to the first side; A second housing comprising a third side and a fourth side opposite to the third side; A hinge structure that rotatably connects the first housing and the second housing; A flexible display (200) arranged on the first side and the third side; A first rear camera (310) and a second rear camera (320) arranged on the second surface; A front camera (330) arranged on the fourth side; At least one processor (420) comprising a processing circuit; and It includes a memory (430) for storing instructions; The above instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Based on the user input for shooting a stereo image being received, a first image is acquired through the first rear camera, a second image is acquired through the second rear camera, and a third image is acquired through the front camera. Based on the first image and the second image, a left-eye image and a right-eye image of a stereo image are acquired, An electronic device that obtains first depth information of the stereo image based on the first image and the third image.

2. In the first paragraph, the precision of the first depth information is An electronic device having a higher precision than the second depth information acquired based on the first image and the second image.

3. In any one of paragraphs 1 to 2, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Store the data of the above stereo image in the above memory, An electronic device that displays the stereo image through the flexible display using the first depth information based on a user input received for reproduction of the stereo image.

4. In any one of paragraphs 1 to 3, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Based on the user input for reproduction of the stereo image being received, information about the three-dimensional space of the stereo image is generated using the first image, the second image, and the first depth information, An electronic device that displays the stereo image using information about the three-dimensional space.

5. In any one of paragraphs 1 to 4, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that stores the data of the stereo image including the first depth information based on synchronizing the first depth information with the first image and the second image.

6. In any one of paragraphs 1 to 5, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Estimating a disparity map for the first image and the third image based on the horizontal distance between pixels corresponding to the same subject included in each of the first image and the third image, An electronic device that obtains the first depth information based on the displacement difference map.

7. In any one of paragraphs 1 to 6, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that obtains the first depth information based on providing the first image and the third image to a pre-learned model.

8. In any one of paragraphs 1 to 7, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: Ensure that the electronic device is positioned in the specified direction and that the electronic device is fully unfolded; An electronic device that acquires the first image, the second image, and the third image based on the electronic device being positioned in the specified direction and confirming that the electronic device is fully unfolded.

9. In any one of paragraphs 1 to 8, the instructions, when individually or collectively executed by the at least one processor, cause the electronic device to: An electronic device that displays guide information to guide the electronic device to be positioned in the specified direction and to fully unfold the electronic device based on determining that the electronic device is not positioned in the specified direction or is not fully unfolded.

10. In any one of paragraphs 1 to 9, An electronic device wherein a first distance between the first rear camera and the front camera is longer than a second distance between the first rear camera and the second rear camera.

11. In the operating method of an electronic device (101), The electronic device comprises a first housing including a first side and a second side opposite to the first side, a second housing including a third side and a fourth side opposite to the third side, a hinge structure rotatably connecting the first housing and the second housing, a flexible display (200) disposed on the first side and the third side, a first rear camera (310) and a second rear camera (320) disposed on the second side, and a front camera (330) disposed on the fourth side. An operation of acquiring a first image through the first rear camera, a second image through the second rear camera, and a third image through the front camera based on a user input for capturing a stereo image; An operation of obtaining a left-eye image and a right-eye image of a stereo image based on the first image and the second image; and An operating method of an electronic device, comprising an operation of obtaining first depth information of the stereo image based on the first image and the third image.

12. In the 11th paragraph, the precision of the first depth information is A method of operating an electronic device, wherein the precision of second depth information obtained based on the first image and the second image is higher than that of the first image.

13. In any one of paragraphs 11 to 12, An operation of storing data of the stereo image in the memory; and An operating method of an electronic device further comprising an operation of displaying the stereo image through the flexible display using the first depth information based on a user input for reproduction of the stereo image.

14. In any one of the 11th to 13th clauses, the operation of displaying the stereo image is: An operation of generating information about a three-dimensional space of the stereo image using the first image, the second image, and the first depth information based on receiving a user input for reproduction of the stereo image; and An operating method of an electronic device including an operation of displaying the stereo image using information about the three-dimensional space.

15. In any one of paragraphs 11 to 14, the operation of storing the data comprises: An operating method of an electronic device, comprising an operation of storing the data of the stereo image including the first depth information based on synchronizing the first depth information with the first image and the second image.

Images