Method and electronic device using angle of view of camera module

Abstract

An electronic device according to one embodiment may include a first camera module that captures a first image at a first angle of view, a second camera module that captures a second image at a second angle of view different from the first angle of view, and a processor configured to activate the second camera module in response to the case where at least one object among the objects detected in the first image moves away from the first angle of view.

Description

Technology Field

[0001] The following disclosure relates to an electronic device and method for capturing an image using the field of view of a camera module. Background Technology

[0002] Generally, electronic devices provide camera functions to capture external objects and generate and save images or videos. Initially, camera functions were used only for basic recording, storage, and transmission; however, technological development is underway to improve performance, quality, and functionality to satisfy the diverse needs of users. Recently, electronic devices may include multiple camera modules, and each of these modules may have a different angle of view. The angle of view refers to the angle at which a camera can capture an image through its lens. A wide angle of view allows a camera to photograph a larger range of objects, while a narrow angle of view allows it to photograph objects within a limited range. Electronic devices can acquire different images by using multiple cameras with different angles of view and display the acquired images on a display area. means of solving the problem

[0003] An electronic device according to one embodiment may include a first camera module that captures a first image at a first angle of view, a second camera module that captures a second image at a second angle of view different from the first angle of view, and a processor configured to activate the second camera module in response to the case where at least one object among the objects detected in the first image moves away from the first angle of view.

[0004] The processor can display the first image in a first partial area within the display area and display the second image in a second partial area within the display area.

[0005] The processor can generate a cropped image by cropping an area including at least one object from the second image, display the first image in a first partial area within the display area, and display the generated cropped image in a second partial area within the display area.

[0006] The processor may be further configured to resize the size of the generated cropped image based on the first image.

[0007] The processor resizes the size of the generated cropped image based on a comparison between a first object region representing a reference object in the first image and a second object region representing the reference object in the second image, and the reference object may be different from the at least one object.

[0008] The processor may be further configured to determine the size and location of the first partial area and the second partial area within the display area in response to at least one or two combinations of the size, shape, ratio, and location of the first partial area, the size, shape, ratio, and location of the second partial area, and user input, resize the size of the first image according to the determined size of the first partial area, and resize the size of the generated cropped image according to the determined size of the second partial area.

[0009] The processor may activate the second camera module in response to the case where at least a portion of the target object among the plurality of objects detected from the first image deviates from the first angle of view, display an area containing the remaining objects among the plurality of objects excluding the target object in the first image in a first partial area, and display an area containing the target object in the second image in a second partial area.

[0010] An electronic device according to one embodiment further includes a third camera module that captures a third image at a third angle of view different from the second angle of view, and the processor may be further configured to activate the third camera module in response to at least one object among the objects detected in the second image moving away from the second angle of view.

[0011] The processor may be further configured to disable the first camera module in response to the second camera module being activated.

[0012] The first camera module includes a first image sensor having a plurality of sensing elements that receive light and convert it into an image signal, and the processor reads out a first sensing element corresponding to a first sensing area among the plurality of sensing elements to acquire a horizontal image, reads out a second sensing element corresponding to a second sensing area among the plurality of sensing elements to acquire a vertical image, and can store the acquired horizontal image and the acquired vertical image by mapping them to each other.

[0013] The processor may be further configured to select one of the objects detected in the first image, generate a cropped image by cropping an area including the selected object from the acquired vertical image, display the acquired horizontal image in a first partial area within the display area, and display the generated cropped image in a second partial area within the display area.

[0014] A method performed by an electronic device may include the operation of capturing a first image at a first angle of view of a first camera module, and the operation of activating a second camera module to capture a second image at a second angle of view different from the first angle of view in response to at least one object among the objects detected in the first image moving away from the first angle of view. Effects of the invention

[0015] An electronic device according to one embodiment may activate a second camera module when at least one object among the objects detected from a first image moves out of the first angle of view. An electronic device according to one embodiment may display an object moving out of the angle of view of the first camera module in a display area based on a second image captured by the second camera module.

[0016] An electronic device according to one embodiment can generate a cropped image by cropping a second image based on an object that deviates from the field of view of the first camera module. An electronic device according to one embodiment can adjust the size of the object by resizing the size of the generated cropped image, and can provide a sense of unity among multiple images displayed in the display area to the user by displaying the image with the adjusted size of the object in the display area. Brief explanation of the drawing

[0017] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments. FIG. 2 is a block diagram illustrating a camera module according to various embodiments. FIG. 3 is a flowchart illustrating the operation of an electronic device according to one embodiment. FIG. 4 describes the field of view of a camera module included in an electronic device according to one embodiment. FIG. 5 illustrates the process of an electronic device detecting an object from an image according to one embodiment. FIGS. 6a and 6b describe a case where, when an electronic device places a camera module horizontally to photograph an object, at least one of the objects detected from the first image is out of the first angle of view. FIGS. 7a and 7b describe a case where, when an electronic device photographs an object by arranging a camera module in a vertical direction, at least one of the objects detected from the first image is out of the first angle of view. FIGS. 8A and 8B illustrate the process of an electronic device according to one embodiment displaying a captured image in a display area within a display module. FIG. 9 illustrates the process of an electronic device according to one embodiment resizing the size of a cropped image. FIG. 10 describes the process of determining the size and location of a partial region within a display area using an electronic device according to one embodiment. FIG. 11 describes a case in which at least a portion of a target object among objects detected from an image of an electronic device according to one embodiment is out of the first angle of view. FIG. 12 describes the case where an electronic device according to one embodiment activates a third camera module. FIG. 13 illustrates an image taken by a conventional electronic device using a camera module. FIG. 14 illustrates the process of a conventional electronic device acquiring a horizontal image using a camera module. FIGS. 15a and FIGS. 15b illustrate the shape of an image sensor that a camera module of an electronic device according to one embodiment may have. FIG. 16 illustrates the process of displaying a horizontal image and a vertical image obtained by an electronic device according to one embodiment on a display area. Specific details for implementing the invention

[0018] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0019] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0020] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0021] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0022] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0023] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0024] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0025] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0026] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0027] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0028] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0029] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0030] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0031] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0032] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0033] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0034] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0035] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0036] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0037] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0038] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0039] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0041] FIG. 2 is a block diagram (200) illustrating a camera module (180) according to various embodiments. According to one embodiment, an electronic device (101) may include a plurality of camera modules (180) each having different attributes or functions. In this case, for example, at least one of the plurality of camera modules (180) may be a wide-angle camera and at least another may be a telephoto camera. Similarly, at least one of the plurality of camera modules (180) may be a front camera and at least another may be a rear camera.

[0042] Referring to FIG. 2, the camera module (180) may include a lens assembly (210), a flash (220), an image sensor (230), an image stabilizer (240), a memory (250) (e.g., a buffer memory), or an image signal processor (260). The lens assembly (210) may collect light emitted from a subject that is the target of image capture. The lens assembly (210) may include one or more lenses. According to one embodiment, the camera module (180) may include a plurality of lens assemblies (210). In this case, the camera module (180) may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies (210) may have the same lens properties (e.g., angle of view, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties different from the lens properties of other lens assemblies. The lens assemblies (210) may include, for example, a wide-angle lens or a telephoto lens.

[0043] A flash (220) may emit light used to enhance light emitted or reflected from a subject. According to one embodiment, the flash (220) may include one or more light-emitting diodes (e.g., RGB (red-green-blue) LED, white LED, infrared LED, or ultraviolet LED), or a xenon lamp. An image sensor (230) may acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through a lens assembly (210) into an electrical signal. According to one embodiment, the image sensor (230) may include one image sensor selected from image sensors with different properties, such as an RGB sensor, a BW (black and white) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same properties, or a plurality of image sensors having different properties. Each image sensor included in the image sensor (230) can be implemented using, for example, a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor.

[0044] The image stabilizer (240) may move at least one lens or image sensor (230) included in the lens assembly (210) in a specific direction or control the operational characteristics of the image sensor (230) (e.g., adjusting read-out timing, etc.) in response to the movement of the camera module (180) or the electronic device (101) containing it. This allows for compensating for at least some of the negative effects caused by the movement on the image being captured. According to one embodiment, the image stabilizer (240) may detect such movement of the camera module (180) or the electronic device (101) using a gyroscope sensor (not shown) or an accelerometer sensor (not shown) placed inside or outside the camera module (180). According to one embodiment, the image stabilizer (240) may be implemented as, for example, an optical image stabilizer. The memory (250) may temporarily store at least a portion of the image acquired through the image sensor (230) for the next image processing operation. For example, if image acquisition by the shutter is delayed or multiple images are acquired at high speed, the acquired original image (e.g., a Bayer-patterned image or a high-resolution image) is stored in the memory (250), and the corresponding copy image (e.g., a low-resolution image) can be previewed through the display module (160). Subsequently, when a specified condition is satisfied (e.g., user input or system command), at least a portion of the original image stored in the memory (250) may be acquired and processed by, for example, an image signal processor (260). According to one embodiment, the memory (250) may be configured as at least a portion of the memory (130) or as a separate memory that operates independently thereof.

[0045] The image signal processor (260) can perform one or more image processing operations on an image obtained through the image sensor (230) or an image stored in memory (250). One or more of the above image processing methods may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softing). Additionally or generally, the image signal processor (260) may perform control (e.g., exposure time control, or readout timing control, etc.) over at least one of the components included in the camera module (180) (e.g., image sensor (230)). The image processed by the image signal processor (260) may be stored back in memory (250) for further processing or provided to an external component of the camera module (180) (e.g., memory (130), display module (160), electronic device (102), electronic device (104), or server (108)). According to one embodiment, the image signal processor (260) is at least part of the processor (120). It may be configured as a separate processor that operates independently of the processor (120). If the image signal processor (260) is configured as a separate processor from the processor (120), at least one image processed by the image signal processor (260) may be displayed through the display module (160) as is or after additional image processing by the processor (120).

[0046] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0047] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0048] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0049] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0050] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0051] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0053] FIG. 3 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0054] In operation (310), an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can take a first image with a first angle of view using a first camera module.

[0055] An electronic device according to one embodiment may include a plurality of camera modules having different angles of view (e.g., camera modules (180) of FIGS. 1 and 2). An electronic device according to one embodiment may include a first camera module that captures a first image at a first angle of view, a second camera module that captures a second image at a second angle of view different from the first angle of view, and a third camera module that captures a third image at a third angle of view different from the first angle of view and the second angle of view. However, the number of camera modules included in the electronic device is not limited thereto.

[0056] In operation (320), a processor of an electronic device according to one embodiment (e.g., processor (120) of FIG. 1) can detect an object from a first image. The object may be, for example, a human face, a human, a vehicle, or an animal (e.g., a cat, a dog). In this specification, it is assumed that the electronic device extracts a human face as an object from an image.

[0057] In operation (330), an electronic device according to one embodiment can determine whether at least one object among the objects detected from the first image is out of the first angle of view.

[0058] In operation (340), an electronic device according to one embodiment may activate a second camera module having a second angle of view in response to at least one object among the objects detected in the first image moving away from the first angle of view.

[0059] In operation (350), an electronic device according to one embodiment may display an image in a display area within a display module (e.g., a display module (160) of FIG. 1). An electronic device according to one embodiment may display the first image in a display area within the display module when an object detected from the first image does not move out of the first angle of view. When at least one of the objects detected from the first image moves out of the first angle of view, the electronic device may activate a second camera module to capture a second image and display the first image and the second image in a display area within the display module.

[0061] FIG. 4 describes the field of view of a camera module included in an electronic device according to one embodiment.

[0062] A field of view (FOV) (410) may be an area where an object is seen when an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) takes a picture by positioning a camera module (e.g., the camera module (180) of FIG. 1 and FIG. 2) in a horizontal direction. A first FOV (411) may be an area where an object is seen corresponding to a first angle of view when the first camera module is positioned in a horizontal direction to take a picture. A second FOV (412) may be an area where an object is seen corresponding to a second angle of view when the second camera module is positioned in a horizontal direction to take a picture. A third FOV (413) may be an area where an object is seen corresponding to a third angle of view when the third camera module is positioned in a horizontal direction to take a picture. The camera module includes an image sensor having a plurality of sensing elements that receive light and convert it into an image signal. Horizontal arrangement of a camera module refers to positioning the image sensor in a direction where the number of horizontal axis sensing elements is greater than the number of vertical axis sensing elements. Similarly, vertical arrangement of a camera module refers to positioning the image sensor in a direction where the number of vertical axis sensing elements is greater than the number of horizontal axis sensing elements.

[0063] According to one embodiment, as illustrated in FIG. 4, the second FOV (412) is wider than the first FOV (411) and may include the first FOV (411). The third FOV (413) is wider than the second FOV (412) and may include the second FOV (412). However, the first FOV corresponding to the first angle of view, the second FOV corresponding to the second angle of view, and the third FOV corresponding to the third angle of view are not limited as in FIG. 4. According to another embodiment, only a portion of the first FOV (411) may be included in the second FOV (412). According to another embodiment, the first FOV (411) and the second FOV (412) may not have overlapping areas.

[0064] Conversely, FOV (420) may be an area where an object is seen when the electronic device according to one embodiment takes a picture with the camera module positioned vertically. The fourth FOV (424) may be an area where an object is seen when the first camera module is positioned vertically and taken a picture. The fifth FOV (425) may be an area where an object is seen when the second camera module is positioned vertically and taken a picture, and the sixth FOV (426) may be an area where an object is seen when the third camera module is positioned vertically and an image is taken with a third field of view.

[0066] FIG. 5 illustrates the process of an electronic device detecting an object from an image according to one embodiment.

[0067] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can take an image (510) using a camera module (e.g., the camera module (180) of FIG. 1 and FIG. 2). The electronic device can detect an object (e.g., a human face) from the image (510).

[0068] An electronic device according to one embodiment can detect an object by recognizing a human face on an image (510) captured by a camera module. Additionally, the electronic device can track the object detected in the image captured by the camera module. Specifically, the electronic device can detect object regions (511, 512, 513, 514, 515) representing an object from the image (e.g., a region including a human face). The electronic device can continuously track the object regions (511, 512, 513, 514, 515) representing the object within the image. An object region may be an area within the image that corresponds to an object. For example, if the electronic device detects a human face as an object, the object region may be an area including the human face. As an example, the object region may be a rectangular area. By tracking the object regions detected from the image, the electronic device can determine whether there is an object outside the field of view of the camera module.

[0069] According to one embodiment, the electronic device can calculate the number of objects detected in an image (e.g., 5) by tracking the object area detected from the image (510). If the number of objects detected in a series of images captured by the camera module decreases, the electronic device can determine that at least one object has moved out of the camera module's field of view.

[0071] FIGS. 6 and 7 describe the case where at least one of the objects detected in the image deviates from the first angle of view.

[0072] FIGS. 6a and 6b describe a case where, when an electronic device places a camera module horizontally to photograph an object, at least one of the objects detected from the first image is out of the first angle of view.

[0073] An electronic device can capture a first image at a first angle of view using a first camera module. At least one of the objects detected in the first image may be outside the first angle of view and within the second angle of view of a second camera module. The electronic device may activate the second camera module when at least one of the objects detected in the first image is outside the first angle of view and the at least one object outside the first angle of view is within the second angle of view of the second camera module. According to one embodiment, when at least one object is outside the first angle of view, the electronic device may additionally activate the second camera module while the first camera module remains activated. As will be described later, according to another embodiment, when at least one object is outside the first angle of view, the electronic device may activate the second camera module and deactivate the first camera module.

[0074] In this specification, the fact that an object detected in an image is within the field of view of a camera module is equivalent to the fact that the area of ​​the object detected in the image is in the FOV corresponding to the field of view. The fact that an object detected in an image is out of the field of view is equivalent to the fact that the area of ​​the object detected in the image is out of the FOV corresponding to the field of view.

[0075] An electronic device according to one embodiment can activate a first camera module to capture a first image at a first angle of view. If at least one object region (631, 632) among the object regions detected in the first image is outside the first FOV (611) corresponding to the first angle of view and is within the second FOV (612) corresponding to the second angle of view, the electronic device can activate a second camera module to capture a second image at a second angle of view. FIG. 6a illustrates the case where the object regions (631, 632) are outside the first FOV (611) in the same direction, and FIG. 6b illustrates the case where the object regions (641, 642) are outside the first FOV (611) in different directions. Although not illustrated in FIGS. 6a and 6b, an electronic device according to one embodiment can activate a third camera module when at least one object detected in a second image is out of the second angle of view of the second camera module and is in the third angle of view of the third camera module.

[0076] FIGS. 7a and 7b describe a case where, when an electronic device photographs an object by arranging a camera module in a vertical direction, at least one of the objects detected from the first image is out of the first angle of view.

[0077] As described in FIG. 6, the electronic device can activate a second camera module to capture a second image at the second angle of view when at least one object region (731, 732) among the object regions detected in the first image is out of the fourth FOV (724) corresponding to the first angle of view and is within the fifth FOV (725) corresponding to the second angle of view. FIG. 7a illustrates the case where the object regions (731, 732) are out of the fourth FOV (724) in the same direction, and FIG. 7b illustrates the case where the object regions (741, 742) are out of the fourth FOV (724) in different directions.

[0079] FIGS. 8A and 8B illustrate the process of an electronic device according to one embodiment displaying a captured image in a display area within a display module.

[0080] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can display a first image captured at a first angle of view of a first camera module (e.g., the camera module (180) of FIG. 1 and FIG. 2) in a first partial area (851) within a display area, and can display a second image captured at a second angle of view of a second camera module in a second partial area (852) within a display area. The first image may represent an image captured by the first camera module at a first angle of view, and the second image may represent an image captured by the second camera module at a second angle of view. The first image corresponds to the first FOV (811), and the second image corresponds to the second FOV (812).

[0081] An electronic device according to one embodiment may divide the entire display area (850) into a plurality of display areas (851, 852). The electronic device may not activate the second camera module if there are no objects detected from the first image that are outside the first FOV (811). In this case, the electronic device may not divide the entire display area (850) and may display the first image as a single unit on the entire display area (850). The electronic device may divide the entire display area into a first partial area (851) and a second partial area (852) in response to at least one object detected from the first image being outside the first FOV (811). The electronic device may also divide the entire display area into a first partial area (851) and a second partial area (852) in response to the second camera module being activated. The first partial area (851) and the second partial area (852) may be parts that do not overlap each other within the display area.

[0082] According to one embodiment, the electronic device may display a first image containing an object that does not extend beyond the first FOV (811) in a first partial area (851), and may display a second image containing an object that extends beyond the first FOV (811) in a second partial area (852). According to another embodiment, the electronic device may display the first image in the first partial area (851) and may display a cropped image created by cropping a portion of the second image in the second partial area (852). The created cropped image may include an object that extends beyond the first angle of view. The electronic device may create the cropped image by cropping the second image based on an object that extends beyond the first angle of view. The electronic device may create the cropped image by cropping an area (860) (e.g., an area containing a person) that includes at least one object that extends beyond the first angle of view in the second image. The electronic device may display the cropped image created from the second image in the second partial area (852). FIG. 8a illustrates the process of cropping a second image when object regions (831, 832) deviate in the same direction with respect to the first FOV (811). When object regions (831, 832) deviate in the same direction with respect to the first FOV (811) and object regions (831, 832) are within the second FOV (812), the electronic device may generate a cropped image by cropping one region (860) (e.g., a region containing a person) containing object regions (831, 832) from the second image. The electronic device may display the generated cropped image in the second partial region (852).

[0083] FIG. 8b illustrates a process of cropping a second image when the object area (841, 842) deviates in a different direction from the first FOV (811). When the object area (841, 842) deviates in a different direction from the first FOV (811) and the object area (841, 842) is within the second FOV (812), the electronic device may generate a plurality of cropped images by cropping the area (861) containing the object area (841) (e.g., an area containing a person) and the area (862) containing the object area (842) (e.g., an area containing a person) from the second image, respectively. An electronic device according to one embodiment may display the generated plurality of cropped images separately in a second partial area (852).

[0085] FIG. 9 illustrates the process of an electronic device according to one embodiment resizing the size of a cropped image.

[0086] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can resize the size of a cropped image (960) generated from a second image. Since the electronic device captures multiple images with multiple camera modules having different viewing angles and different magnifications, the size of the object detected from different images may differ. For example, the size of the object detected in the first image captured by the first camera module and the size of the object detected in the second image captured by the second camera module may differ from each other. The electronic device can adjust the size of the object by resizing the size of the generated cropped image (960). By displaying the image with the adjusted size of the object in the display area, the electronic device can provide the user with a sense of unity among the multiple images displayed in the display area. Hereinafter, the description focuses on the electronic device resizing the size of the cropped image (960), but the electronic device may resize the size of the first image or the second image as well as the cropped image (960).

[0087] An electronic device according to one embodiment can resize the size of a cropped image (960) based on a first image. According to one embodiment, the electronic device can resize the size of the cropped image (960) using a reference object that exists simultaneously within a first field of view of a first camera module and a second field of view of a second camera module. The reference object is an object different from an object that is outside the first. For example, the electronic device can select one of object regions (951, 952, 953) that exist simultaneously in a first FOV (911) corresponding to the first field of view and a second FOV (912) corresponding to the second field of view. The selected object region (951) is an object region corresponding to the reference object. The electronic device can take a first image containing the reference object at the first field of view using the first camera module, and can take a second image containing the reference object at the second field of view using the second camera module. The electronic device can resize the size of the cropped image (960) based on a comparison between a first object area representing a reference object in the first image (e.g., an area corresponding to a human face) and a second object area representing a reference object in the second image (e.g., an area corresponding to a human face).

[0088] According to one embodiment, the electronic device may have an object region representing an object that is in the shape of a rectangle. The electronic device may calculate the number of pixels corresponding to the height of a first object region representing a reference object in a first image and the number of pixels corresponding to the width of the first object region, respectively. Likewise, the electronic device may calculate the number of pixels corresponding to the height of a second object region representing a reference object in a second image and the number of pixels corresponding to the width of the second object region. The electronic device may resize the height of the cropped image (960) by the ratio of the number of pixels corresponding to the height of the first object region to the number of pixels corresponding to the height of the second object region. The electronic device may resize the width of the cropped image (960) by the ratio of the number of pixels corresponding to the width of the first object region to the number of pixels corresponding to the width of the second object region.

[0089] According to another embodiment, the electronic device may calculate the total number of pixels in a first object area representing a reference object from a first image, and the total number of pixels in a second object area representing a reference object from a second image. The electronic device may resize the height and width of the cropped image (960) in a ratio of the total number of pixels in the first object area to the total number of pixels in the second object area. The method by which the electronic device according to one embodiment resizes the size of the cropped image (960) is not limited to the examples above, and the cropped image may be resized in various other ways. The electronic device may display the first image in a first partial area (971) and display the resized cropped image in a second partial area (972).

[0090] According to another embodiment, the electronic device may resize the height of the second image by a ratio of the number of pixels corresponding to the height of the first object area to the number of pixels corresponding to the height of the second object area, and may resize the width of the second image by a ratio of the number of pixels corresponding to the width of the first object area to the number of pixels corresponding to the width of the second object area. The electronic device according to one embodiment may crop the resized second image based on an object that is out of the first viewing angle and display the cropped image in the second partial area (972).

[0092] FIG. 10 describes the process of determining the size and location of a partial region within a display area using an electronic device according to one embodiment.

[0093] A processor (e.g., processor (120) of FIG. 1) of an electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1) may determine the size and location of the first partial area (1051) and the second partial area (1052) within a display area in response to at least one or more combinations of the size, shape, ratio, and location of the first partial area (1051), the size, shape, ratio, and location of the second partial area (1052), and user input. For example, if the shape of the first partial area and the shape of the second partial area are determined to be circular, the electronic device may appropriately determine the size and location of the first partial area (1051) and the second partial area (1052) within a display module so that the first partial area (1051) and the second partial area (1052) each have a circular shape within the display area.

[0094] As another example, the electronic device may adjust the size of the first partial area and the second partial area according to received user input. The electronic device may receive a drag input from the user regarding the boundary line between the first partial area (1051) and the second partial area (1052). The drag input may represent an input in which a click or touch occurs at a point (1011) corresponding to the boundary line between the first partial area and the second partial area, moves while maintaining the click or touch, and releases the click or touch at another point (1012). When the electronic device receives the user's drag input, it may change the boundary line between the first partial area and the second partial area to correspond to the point (1012) where the click or touch is released, and may increase or decrease the size of the first partial area (1051) and the second partial area (1052) to match the changed boundary line. An electronic device according to one embodiment may resize the size of a first image displayed in a first partial area (1051) according to a determined size of a first partial area (1051), and resize the size of a cropped image displayed in a second partial area (1052) according to a determined size of a second partial area (1052). For example, as illustrated in FIG. 10, an electronic device according to one embodiment assumes that the partial area of ​​a display module is a rectangular area. The electronic device may calculate the number of pixels corresponding to the height and width of the first partial area before modification, and may calculate the number of pixels corresponding to the height and width of the first partial area after modification, respectively. The electronic device may resize the height of the first image by the ratio of the height of the first partial area after modification to the height of the first partial area before modification, and resize the width of the first image by the ratio of the width of the first partial area after modification to the width of the first partial area before modification. The electronic device may display the resized first image in the first partial area corresponding to the modification.Likewise, the electronic device can display the resized cropped image in a corresponding second partial area after modification.

[0096] FIG. 11 describes a case in which at least a portion of a target object among objects detected from an image of an electronic device according to one embodiment is out of the first angle of view.

[0097] An electronic device according to one embodiment may activate a second camera module in response to at least a portion of a target object among a plurality of objects detected from a first image deviating from a first field of view. At least a portion of the target object refers to a portion of a human face. The electronic device may activate a second camera module when a portion of a region (1135) representing a target object among a plurality of object regions (1131, 1132, 1133, 1134, 1135) detected from a first image deviates from a first FOV (1111). The electronic device may crop an area (1161) (e.g., an area including a person) containing the remaining object areas (1131, 1132, 1133, 1134) excluding the target object area (1135) among the plurality of object areas (1131, 1132, 1133, 1134, 1135) detected from the first image and display it in a first partial area (1151), and crop an area (1162) (e.g., an area including a person) containing the target object area (1135) from the second image and display it in a second partial area (1152). In one embodiment, if a part of the target object deviates from the first viewing angle, the electronic device may display an area containing the remaining objects excluding the target object among the plurality of detected objects in the first partial area, and distinguish and display an area containing the target object in the second partial area.

[0099] FIG. 12 describes the case where an electronic device according to one embodiment activates a third camera module.

[0100] An electronic device according to one embodiment may include a first camera module, a second camera module, and a third camera module. The third angle of view of the third camera module may be different from the first angle of view of the first camera module and the second angle of view of the second camera module. An electronic device according to one embodiment may activate the third camera module in response to at least one object among the objects detected in the second image moving away from the second angle of view. The electronic device may display images captured by each camera module in a display area. For example, if at least one object among the objects detected from the second image moves away from the second angle of view and the object moving away from the second angle of view is in the third angle of view, the electronic device may divide the display area into a first partial area (1251, 1261), a second partial area (1252, 1262), and a third partial area (1253, 1263). According to one embodiment, the electronic device may display a first image captured using a first camera module in a first partial area (1251, 1261), display a second image captured using a second camera module in a second partial area (1252, 1262), and display a third image captured using a third camera module in a third partial area (1253, 1263). According to another embodiment, the electronic device may display a first image in a first partial area (1251, 1261), display a second image in a second partial area (1252, 1262) by cropping it based on an object outside the first angle of view, and display a third image in a third partial area (1253, 1263) by cropping it based on an object outside the second angle of view.

[0102] According to one embodiment, the electronic device may activate a second camera module when at least one object among the objects detected from the first image moves out of the first angle of view, and may deactivate the first camera module in response to the activation of the second camera module. When the first camera module is activated, the electronic device may display the first image captured by the first camera module in a display area. When at least one object moves out of the first angle of view, the electronic device may activate the second camera module to capture a second image with the second camera module. By deactivating the first camera module simultaneously with activating the second camera module, the electronic device may display the second image instead of the first image in the display area. For example, assume a case where the second FOV corresponding to the second angle of view of the second camera module is wider than the first FOV corresponding to the first angle of view of the first camera module, and the second FOV includes the first FOV. By displaying the second image instead of the first image in the display area, the electronic device may display all objects detected from the first image in the display area.

[0104] FIG. 13 illustrates an image taken by a conventional electronic device using a camera module.

[0105] A camera module of an electronic device according to one embodiment may include an image sensor having a plurality of optical sensing elements (hereinafter referred to as "sensing elements") that receive light and convert it into an image signal. A sensing element is a device that senses optical information based on light reflected from a physical location on a subject and incident on the element. For example, a sensing element may output a value indicating the intensity of the incident light. A sensing element may include a photodiode.

[0106] An image sensor of a camera module included in a conventional electronic device may include a plurality of sensing elements arranged in an array form. An image sensor (1310) may be arranged in a vertical direction to receive light reflected from a subject, and an image sensor (1320) may be arranged in a horizontal direction to receive light reflected from a subject. The horizontal arrangement of the image sensor indicates that the number of sensing elements arranged along the horizontal axis (e.g., 1920) is greater than the number of sensing elements arranged along the vertical axis (e.g., 1440). The vertical arrangement of the image sensor indicates that the number of sensing elements arranged along the vertical axis (e.g., 1920) is greater than the number of sensing elements arranged along the horizontal axis (e.g., 1440). An image sensor of a conventional camera module may include 1920 × 1440 sensing elements. A vertical image (1310) may represent an image in which the number of pixels along the horizontal axis is greater than the number of pixels along the vertical axis. A horizontal image (1320) can represent an image in which the number of pixels on the vertical axis is greater than the number of pixels on the horizontal axis. Here, a pixel can represent basic unit information that constitutes an image.

[0107] A conventional electronic device can obtain a vertical image (1311) with a resolution of 1080×1920 by reading out 1080×1920 sensing elements from an image sensor having 1920×1440 sensing elements. The electronic device can change the resolution of the vertical image by changing the area from which the sensing elements are read from the image sensor. For example, the electronic device can obtain a vertical image (1312) with a maximum resolution of 1440×1920 by reading out all 1440×1920 sensing elements from the image sensor.

[0108] A conventional electronic device can obtain a horizontal image (1321) having a resolution of 1920×1080 by reading 1920×1080 sensing elements from an image sensor having 1920×1440 sensing elements. The electronic device can obtain a vertical image (1322) having a maximum resolution of 1920×1440 by reading all 1920×1440 sensing elements from the image sensor.

[0110] FIG. 14 illustrates the process of a conventional electronic device acquiring a horizontal image using a camera module.

[0111] Conventional electronic devices assume a case where an image sensor is positioned vertically to receive light reflected from a subject. Conventional electronic devices can obtain a vertical image (1411) by reading 1080×1920 sensing elements from an image sensor having 1920×1440 sensing elements, and can obtain a vertical image (1412) by reading 1440×1920 sensing elements. On the other hand, in the case where the image sensor is positioned vertically in a conventional electronic device, generally, to obtain a horizontal image, a horizontal image (1431) could be obtained by reading 1080×607 sensing elements from an image sensor having 1920×1440 sensing elements. In conventional electronic devices, when a horizontal image (1431) (e.g., having a resolution of 1080×607) is acquired using a camera module of an image sensor arranged in a vertical direction, the resolution of the horizontal image (1431) is inevitably significantly reduced because the number of sensing elements to be read is reduced.

[0113] FIGS. 15a and FIGS. 15b illustrate the shape of an image sensor that a camera module of an electronic device according to one embodiment may have.

[0114] Referring to FIG. 15a, a camera module of an electronic device according to one embodiment may include a square-shaped image sensor (1510). The image sensor (1510) may include 1920 × 1920 sensing elements. 1920 sensing elements may be arranged on a first side (1501) of the image sensor (1510), and 1920 sensing elements may be arranged on a second side (1502) perpendicular to the first side (1501).

[0115] An electronic device according to one embodiment can acquire a horizontal image by reading first sensing elements corresponding to a first sensing area (1511) among 1920×1920 sensing elements of an image sensor. An electronic device according to one embodiment can acquire a vertical image by reading second sensing elements corresponding to a second sensing area (1512) among 1920×1920 sensing elements of an image sensor. The first sensing area (1511) may be an area containing 1920×1080 sensing elements among a plurality of sensing elements included in the image sensor. The first sensing area (1511) is a rectangular area, where 1920 sensing elements may be arranged on one side and 1080 sensing elements may be arranged on the other side perpendicular to the one side. The second sensing area (1512) may be an area containing 1080 × 1920 sensing elements among the plurality of sensing elements included in the image sensor. The second sensing area (1512) is a rectangular area, and 1080 sensing elements may be placed on one side, and 1920 sensing elements may be placed on the other side perpendicular to the one side.

[0116] Referring to FIG. 15b, a camera module of an electronic device according to one embodiment may include a cross-shaped image sensor (1520). The sensing area of ​​the image sensor (1520) may include a square-shaped sensing area (1531) and a rectangular-shaped sensing area (1541, 1542, 1543, 1544) adjacent to the sensing area (1531). The square-shaped sensing area (1531) may be an area containing 1080×1080 sensing elements. The rectangular-shaped sensing area (1541, 1542, 1543, 1544) may be an area containing 1080×420 sensing elements.

[0117] An electronic device according to one embodiment can acquire a horizontal image by reading third sensing elements corresponding to a third sensing area (1521) from an image sensor (1520), and can acquire a vertical image by reading fourth sensing elements corresponding to a fourth sensing area (1522). The third sensing area (1521) is an area containing 1920 × 1080 sensing elements among a plurality of sensing elements included in the image sensor, and is an area corresponding to the first sensing area (1511). The third sensing area (1521) is an area containing a sensing area (1541), a sensing area (1531), and a sensing area (1542). 1080 sensing elements may be arranged on one side (1503) of the third sensing area (1521), and 1920 sensing elements may be arranged on the other side (1504) perpendicular to the one side (1503). The fourth sensing area (1522) is an area containing 1080 × 1920 sensing elements among the plurality of sensing elements included in the image sensor, and is an area corresponding to the second sensing area (1512). The fourth sensing area is an area including a sensing area (1544), a sensing area (1531), and a sensing area (1543). 1920 sensing elements may be placed on one side (1505) of the fourth sensing area (1522), and 1080 sensing elements may be placed on the other side (1506) perpendicular to the one side.

[0118] An electronic device according to one embodiment may store a horizontal image obtained by reading a first sensing area and a vertical image obtained by reading a second sensing area by mapping them to each other. The electronic device may also store the obtained horizontal image and the obtained vertical image by mapping them to each other as a single image file. The electronic device may include a plurality of camera modules, and each of the plurality of camera modules may include at least one image sensor among an image sensor (1510) and an image sensor (1520). Regardless of the orientation in which the image sensor is positioned, the electronic device may store the horizontal image and the vertical image 1920 Since it can be obtained by reading 1080 sensing elements, the degradation of image resolution can be prevented.

[0120] FIG. 16 illustrates the process of displaying a horizontal image and a vertical image obtained by an electronic device according to one embodiment on a display area.

[0121] An electronic device according to one embodiment may display an image captured by a camera module in a display area. An electronic device according to one embodiment may acquire a horizontal image by reading a first sensing element corresponding to a first sensing area (1611) of an image sensor (1600) of a camera module, and may acquire a vertical image by reading a second sensing element corresponding to a second sensing area (1612) of an image sensor (1600). According to another embodiment, the electronic device may acquire a horizontal image by capturing with one camera module and acquire a vertical image by capturing with another camera module. A plurality of camera modules included in the electronic device may include at least one image sensor among an image sensor (1510) and an image sensor (1520). The electronic device may acquire a horizontal image by reading a sensing element corresponding to a first sensing area of ​​an image sensor included in the first camera module, and may acquire a vertical image by reading a sensing element corresponding to a second sensing area of ​​an image sensor included in the second camera module.

[0122] An electronic device according to one embodiment can display a horizontal image and a vertical image in a display area. The electronic device can select one of the objects detected in the images (horizontal image and vertical image) captured by the first camera module. The electronic device can acquire a vertical image by reading a second sensing element corresponding to a second sensing area (1612). The electronic device can generate a cropped image by cropping an area (e.g., an area including a person) (1661) containing an object area representing the selected object from the acquired vertical image. The electronic device can display a horizontal image acquired by reading a first sensing element corresponding to a first sensing area (1611) in a first partial area (1651) within the display area. The electronic device can display a cropped image from the vertical image in a second partial area (1652) within the display area.

Claims

Claim 1 An electronic device comprising: a first camera module having a first angle of view; a second camera module having a second angle of view different from the first angle of view; and a processor, wherein the processor captures a first image at the first angle of view using the first camera module, detects a target object in the first image, and in response to at least a portion of the target object moving away from the first angle of view, activates the second camera module, captures a second image at the second angle of view using the activated second camera module, displays at least a portion of the first image in a first partial area within a display area, displays at least a portion of the second image in a second partial area within the display area, and the at least portion of the second image includes the at least portion of the target object moving away from the first angle of view. Claim 2 An electronic device according to claim 1, wherein the processor generates a cropped image by cropping an area including the target object from the second image, displays the first image in the first partial area, and displays the generated cropped image in the second partial area. Claim 3 An electronic device according to paragraph 2, wherein the processor determines a reference object existing simultaneously within the first angle of view and the second angle of view, and resizes the size of the generated cropped image based on a comparison between a first object region representing the reference object in the first image and a second object region representing the reference object in the second image. Claim 4 An electronic device according to paragraph 3, wherein the processor resizes the height of the cropped image by a ratio of the number of pixels corresponding to the height of the first object area to the number of pixels corresponding to the height of the second object area, and resizes the width of the cropped image by a ratio of the number of pixels corresponding to the width of the first object area to the number of pixels corresponding to the width of the second object area. Claim 5 In paragraph 3, the processor is an electronic device that resizes the cropped image by the ratio of the number of pixels in the first object area to the number of pixels in the second object area. Claim 6 An electronic device according to paragraph 3, wherein the processor is further configured to determine the size and position of the first partial area and the second partial area within the display area based on at least one of the size, shape, ratio, and position of the first partial area, the size, shape, ratio, and position of the second partial area, and user input, resize the size of the first image according to the determined size of the first partial area, and resize the size of the generated cropped image according to the determined size of the second partial area. Claim 7 An electronic device according to claim 1, wherein the processor detects a plurality of objects including the target object from the first image, displays an area in the first partial area that includes an object different from the target object among the plurality of objects in the first image, and displays an area in the second partial area that includes the target object in the second image. Claim 8 An electronic device according to claim 1, further comprising a third camera module that captures a third image at a third angle of view different from the second angle of view, and the processor further configured to activate the third camera module in response to the case where at least one object among the objects detected in the second image deviates from the second angle of view. Claim 9 An electronic device according to claim 1, wherein the processor is further configured to disable the first camera module in response to the second camera module being activated. Claim 10 An electronic device according to claim 1, wherein the first camera module includes a first image sensor having a plurality of sensing elements that receive light and convert it into an image signal, and the processor reads out a first sensing element corresponding to a first sensing area among the plurality of sensing elements to obtain a horizontal image, reads out a second sensing element corresponding to a second sensing area among the plurality of sensing elements to obtain a vertical image, and maps the obtained horizontal image and the obtained vertical image to each other and stores them. Claim 11 An electronic device according to claim 10, wherein the processor is further configured to select one of the objects detected in the first image, generate a cropped image by cropping an area including the selected object from the acquired vertical image, display the acquired horizontal image in a first partial area within a display area, and display the generated cropped image in a second partial area within a display area. Claim 12 A method performed by an electronic device comprising: an operation of capturing a first image at a first angle of view using a first camera module; an operation of detecting a target object in the first image; an operation of activating a second camera module in response to at least a portion of the target object moving away from the first angle of view; an operation of capturing a second image at a second angle of view different from the first angle of view using the activated second camera module; an operation of displaying at least a portion of the first image in a first partial area within a display area; and an operation of displaying at least a portion of the second image in a second partial area within the display area, wherein the at least portion of the second image includes the at least portion of the target object moving away from the first angle of view. Claim 13 A method according to claim 12, further comprising: generating a cropped image by cropping an area including the target object from the second image; and displaying the first image in the first partial area and displaying the generated cropped image in the second partial area. Claim 14 In claim 13, the operation of generating the cropped image further comprises: an operation of determining a reference object existing simultaneously within the first angle of view and the second angle of view; and an operation of resizing the size of the generated cropped image based on a comparison between a first object region representing the reference object in the first image and a second object region representing the reference object in the second image. Claim 15 A method according to claim 14, wherein the operation of resizing the size of the cropped image comprises: an operation of resizing the height of the cropped image by a ratio of the number of pixels corresponding to the height of the first object area to the number of pixels corresponding to the height of the second object area; and an operation of resizing the width of the cropped image by a ratio of the number of pixels corresponding to the width of the first object area to the number of pixels corresponding to the width of the second object area. Claim 16 In claim 14, the operation of resizing the size of the cropped image includes the operation of resizing the cropped image by the ratio of the number of pixels in the first object area to the number of pixels in the second object area. Claim 17 A method according to claim 14, further comprising: an operation of determining the size and position of the first partial area and the second partial area within the display area based on at least one of the size, shape, ratio, and position of the first partial area, the size, shape, ratio, and position of the second partial area, and user input; and an operation of resizing the first image according to the determined size of the first partial area and resizing the size of the generated cropped image according to the determined size of the second partial area. Claim 18 A method according to claim 12, further comprising the operation of deactivating the first camera module in response to the second camera module being activated. Claim 19 In claim 12, the method further comprises the operation of reading out a first sensing element corresponding to a first sensing area among a plurality of sensing elements having a first image sensor of the first camera module to obtain a horizontal image, and reading out a second sensing element corresponding to a second sensing area among the plurality of sensing elements to obtain a vertical image; and the operation of mapping the obtained horizontal image and the obtained vertical image to each other and storing them. Claim 20 A computer-readable storage medium storing one or more computer programs comprising instructions for performing the method of any one of paragraphs 12 through 19.

Images