Electronic device comprising camera and operation method thereof

Abstract

An electronic device according to various embodiments may be configured to: output a preview image acquired by a camera on a display; define a region of interest including at least one object image in the preview image; acquire angle information for identifying a direction in which the camera is oriented with respect to a reference plane, on the basis of a tilt of the electronic device detected by at least one sensor; determine a projection matrix to be applied to the region of interest, on the basis of the angle information and location information of the region of interest in the preview image; apply the projection matrix to the region of interest including the at least one object image; and display, on the display, another preview image including the region of interest to which the projection matrix is applied.

Description

Electronic device including a camera and method of operating the same

[0001] The present disclosure relates to an electronic device including a camera and a method of operating the same.

[0002] Images captured by a camera on an electronic device can feel unnatural to the user, depending on the tilt of the electronic device or the relative position of the camera relative to the device, which can occur due to various circumstances during the capture process. For example, if the camera captures a subject from an oblique angle rather than directly at it, the subject in the preview or captured image may appear stretched in some portion.

[0003] In order to provide users with images of subjects without any sense of incongruity in various shooting environments, it is necessary to correct image distortion in real time by taking into account the angle, position, or arrangement of the camera.

[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0005] An electronic device according to one embodiment of the present disclosure includes a camera, at least one sensor for acquiring sensing data for detecting an inclination of the electronic device, a display, a memory for storing instructions, and at least one processor, wherein the instructions, when executed by the at least one processor, cause the electronic device to display a preview image acquired by the camera on the display, define a region of interest including at least one object image in the preview image, acquire angle information used to identify a direction in which the camera faces with respect to a reference plane when the inclination of the electronic device is detected, determine a projection matrix to be applied to the region of interest based on the angle information and position information of the region of interest in the preview image, apply the projection matrix to the region of interest including the at least one object image, and display another preview image including the region of interest to which the projection matrix is ​​applied on the display.

[0006] An operating method of an electronic device according to one embodiment of the present disclosure may include an operation of displaying a preview image acquired by a camera on a display, an operation of defining a region of interest including at least one object image in the preview image, an operation of acquiring angle information used to identify a direction in which the camera faces with respect to a reference plane when an inclination of the electronic device is detected, an operation of determining a projection matrix to be applied to the region of interest based on the angle information and position information of the region of interest in the preview image, an operation of applying the projection matrix to the region of interest including the at least one object image, and an operation of displaying another preview image including the region of interest to which the projection matrix is ​​applied on the display.

[0007] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0008] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment.

[0009] FIG. 3 is a flowchart illustrating a process for a method of correcting distortion of a region of interest including an object image based on a projection matrix by an electronic device according to one embodiment.

[0010] FIG. 4 is a diagram illustrating a matrix for correcting distortion of an object image caused by curvature of a lens according to one embodiment.

[0011] FIG. 5 is a drawing for explaining the direction in which a camera faces with respect to a reference plane as an electronic device according to one embodiment is tilted.

[0012] FIG. 6 is a diagram illustrating a preview image output to a display based on the tilt of an electronic device according to one embodiment.

[0013] FIG. 7 is a diagram illustrating a matrix for correcting distortion of an object image when the tilt angle of an electronic device is in a range of 0 degrees or more and 90 degrees or less according to one embodiment.

[0014] FIG. 8 is a diagram illustrating a matrix for correcting distortion of an object image when the tilt angle of an electronic device is in a range of 90 degrees or more and 180 degrees or less according to one embodiment.

[0015] FIG. 9 is a flowchart illustrating a process for a method for an electronic device to switch a preview image to another preview image according to one embodiment.

[0016] FIG. 10 is a flowchart illustrating a process for a method for an electronic device to output a corrected object image to a wearable electronic device based on a control signal for controlling a correction level received from the wearable electronic device according to one embodiment.

[0017] FIG. 11 is a schematic diagram illustrating an electronic device according to one embodiment receiving a signal for controlling a compensation level from a wearable electronic device.

[0018] FIG. 12 is a flowchart illustrating a process for a method of changing a matrix based on the position of a camera by an electronic device according to one embodiment.

[0019] FIG. 13 is a diagram illustrating an outline of a region of interest and an image of an object within the region of interest, which is set based on the position of a camera according to one embodiment.

[0020] FIG. 14 is a flowchart illustrating a process for changing a projection matrix based on the position of a region of interest that has moved as the position of an image of an object changes, according to one embodiment of the present invention.

[0021] FIG. 15 is a diagram illustrating an example of an electronic device changing a region of interest based on movement of an object according to one embodiment.

[0022] FIG. 16 is a flowchart illustrating a process for identifying an object carrying a wearable electronic device connected to an electronic device among a plurality of objects according to one embodiment.

[0023] FIG. 17 is a diagram illustrating an example of identifying an object carrying a wearable electronic device connected to an electronic device among a plurality of objects according to one embodiment.

[0024] FIG. 18 is a flowchart illustrating a process for correcting distortion of a region of interest including an object image based on a projection matrix by a foldable electronic device according to one embodiment.

[0025] FIG. 19 is a drawing illustrating an example of a first folding state of a foldable electronic device according to one embodiment.

[0026] FIG. 20 is a drawing illustrating an example of a second folding state of a foldable electronic device according to one embodiment.

[0027] In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0028] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In addition, for the purpose of clearly explaining the present disclosure in the drawings, parts irrelevant to the description are omitted, and similar parts are designated with similar reference numerals throughout the specification.

[0029] The terms used in this disclosure are described as currently common terms, taking into account the functions mentioned herein. However, these terms may mean various other terms depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Therefore, the terms used in this disclosure should not be interpreted solely based on their names, but rather based on the meanings of the terms and the overall content of this disclosure.

[0030] Additionally, while terms such as first, second, etc. may be used to describe various components, the components should not be limited by these terms. These terms are used to distinguish one component from another.

[0031] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the cases where the parts are "directly connected" but also the cases where the parts are "electrically connected" with other elements intervening. Furthermore, when a part is said to "include" a component, this does not exclude other components, but rather includes other components, unless otherwise stated.

[0032] The phrases “in one embodiment” and the like appearing in various places throughout this disclosure do not necessarily all refer to the same embodiment.

[0033] An embodiment of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various hardware and / or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a given function. Furthermore, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented by algorithms that execute on one or more processors. Furthermore, the present disclosure may employ conventional techniques for electronic configuration, signal processing, and / or data processing. Terms such as "mechanism," "element," "means," and "configuration" may be used broadly and are not limited to mechanical and physical configurations.

[0034] Additionally, the connecting lines or connecting members between components depicted in the drawings are merely exemplary representations of functional connections and / or physical or circuit connections. In an actual device, connections between components may be represented by various functional connections, physical connections, or circuit connections that may be replaced or added.

[0035] The present disclosure will be described in detail with reference to the attached drawings below.

[0036] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0037] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0038] The processor (120) may control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing, for example, software (e.g., a program (140)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (120) may store a command or data received from another component (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the command or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0039] The auxiliary processor (123) may control at least a part of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where artificial intelligence is performed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0040] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0041] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0042] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0043] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0044] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0045] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0046] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0047] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0048] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0049] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0050] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0051] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0052] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0053] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi Direct (wireless fidelity direct), or IrDA (infrared data association)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0054] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beamforming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0055] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the selected at least one antenna. According to some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197). According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., a bottom surface) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent to a second surface (e.g., a top surface or a side surface) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0056] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0057] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service by itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0058] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0059] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0060] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0061] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more commands stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one command called. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0062] According to one embodiment, the method according to the various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0063] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Additionally or alternatively, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0064] According to one embodiment of the present disclosure, FIG. 2 is a block diagram illustrating a configuration of an electronic device according to one embodiment.

[0065] According to one embodiment of the present disclosure, the electronic device (200) of FIG. 2 may correspond to the electronic device (101) of FIG. 1. The electronic device (200) may include a processor (210), a memory (220), a camera (230), a display (240), and a motion sensor (250). For example, the processor (210) may correspond to the processor (120) of FIG. 1. For example, the memory (220) may correspond to the memory (130) of FIG. 1. For example, the camera (230) may be included in the camera module (180) of FIG. 1. For example, the display (240) may be included in the camera module (180) of FIG. 1. For example, the motion sensor (250) may be included in the sensor module (176) of FIG. 1.

[0066] According to one embodiment of the present disclosure, the processor (210) can output a preview image. The processor (210) can output at least a portion of an image frame acquired from the camera (230) as a preview image. The processor (210) can output the preview image through the display (240).

[0067] According to one embodiment of the present disclosure, the processor (210) can identify at least one object in a preview image output through the display (240). The processor (210) can identify at least one object in response to a user input. For example, in response to a touch input touching at least one object in the preview image, the processor (210) can identify at least one object. For example, in response to an input selecting a user interface (UI) for identifying at least one object, the processor (210) can identify at least one object. The processor (210) can identify at least one object in the preview image based on conditions pre-stored in the memory (220). For example, the types of objects (e.g., people, animals, etc.) that the processor (210) can identify may be pre-stored in the memory (220).

[0068] According to one embodiment of the present disclosure, the processor (210) may define a region of interest for an identified object. The region of interest within the preview image may be defined by adding a predetermined margin area to an area corresponding to the shape of the object. For example, the predetermined margin area may be preset in the electronic device (200). For example, the processor (210) may adjust the predetermined margin area in response to an input for adjusting the margin area. The processor (210) may display a visual object representing the region of interest on the display (240) to visually indicate the region of interest. For example, the visual object may be displayed to correspond to the shape of the object included in the region of interest.

[0069] According to one embodiment of the present disclosure, the processor (210) can track an object. For example, as the object moves, the processor (210) can define a region of interest (ROI) at the location where the image of the object has moved within the preview image. The processor (210) can define a ROI corresponding to an image of the object that has been deformed as the object moves. For example, the processor (210) can display a visual object representing the ROI on the display (240) so as to correspond to the image of the deformed object.

[0070] According to one embodiment of the present disclosure, when the processor (210) outputs a region of interest defined in an object as a preview image, the preview image may be distorted. For example, the angle between the camera (230) and the object may vary based on the angle between the electronic device (200) and the horizontal plane. For example, the position of the object with respect to the camera (230) may vary based on the position of the camera (230). Even if the angle between the electronic device (200) and the horizontal plane or the position of the camera (230) are the same, distortion may occur in the output preview image based on the movement of the object. The processor (210) may determine a projection matrix for correcting the distorted image based on the angle between the electronic device (200) and the horizontal plane, the position of the camera (230), or the movement of the object.

[0071] According to one embodiment of the present disclosure, based on angle information regarding the inclination of the electronic device (200) acquired through the motion sensor (250), the processor (210) can determine a projection matrix. The angle information regarding the inclination of the electronic device (200) can include angle information indicating the degree to which the electronic device (200) is inclining with respect to one or more preset reference planes. For example, information regarding an angle formed between the electronic device (200) and a horizontal plane can be acquired through the sensor (250). Based on the acquired angle information, the processor (210) can identify a direction in which the camera (230) faces with respect to one or more preset reference planes. The processor (210) can determine a projection matrix based on the identified angle.

[0072] According to one embodiment of the present disclosure, the motion sensor (250) may include at least one of a gyro sensor (not shown) or a hall sensor (not shown). Using the gyro sensor or the hall sensor, the processor (210) may identify the arrangement state of the electronic device (200). However, the above sensors are exemplary, and other types of sensors may be further included.

[0073] The electronic device (200) may include a Hall sensor and a magnet. The Hall sensor may detect changes in an electrical signal based on the proximity or distance of a magnetic force or an object having magnetic force. For example, if the electronic device (200) includes a first housing and a second housing connected by a hinge, the Hall sensor included in the first housing may measure a magnetic force value that changes as a magnet included in the second housing approaches or moves away from the first housing. The processor (210) may identify the arrangement state of the electronic device (200) based on the measured magnetic force value. The processor (210) may identify the position of the camera (230) based on the identified arrangement state of the electronic device (200).

[0074] The electronic device (200) may include a gyro sensor. The gyro sensor may measure acceleration acting along three axes (e.g., x-axis, y-axis, z-axis) of the electronic device (200). For example, the gyro sensor may measure a rotation angle or inclination of the electronic device (200) with respect to the three axes. The processor (210) may use the gyro sensor to identify a position of the camera (230) with respect to the electronic device (200) based on at least one of an inclination or rotation angle of the electronic device (200) with respect to a horizontal plane. In addition, the processor (210) may use the gyro sensor to identify a posture of the electronic device (200). For example, the processor (210) may receive acceleration data from the gyro sensor and obtain inclination information of the electronic device (200) with respect to the ground. The processor (210) may identify a position of an object with respect to the camera (230) based on the position of the camera (230). The processor (210) can determine a projection matrix based on the position of the object relative to the identified camera (230).

[0075] According to one embodiment of the present disclosure, the sensor (250) may include a motion sensor. The processor (210) may obtain information about the movement of an object from the motion sensor. The processor (210) may determine a projection matrix based on the information about the movement of the object.

[0076] According to one embodiment of the present disclosure, the processor (210) can output a distorted preview image as a corrected preview image through the display (240) based on the determined projection matrix. While outputting the corrected preview image, the processor (210) can change the projection matrix in response to an input for correcting a region of interest. The processor (210) can re-correct the corrected preview image based on the changed projection matrix.

[0077] According to one embodiment of the present disclosure, FIG. 3 is a flowchart (300) illustrating a process for a method for correcting distortion of a region of interest including an object image based on a projection matrix in an electronic device according to one embodiment. FIG. 4 is a diagram illustrating a matrix for correcting distortion of an object image caused by curvature of a lens according to one embodiment. FIG. 5 is a diagram conceptually illustrating a direction in which a camera faces with respect to a reference plane as an electronic device is tilted according to one embodiment. FIG. 6 is a diagram illustrating a preview image output to a display based on the tilt of an electronic device according to one embodiment. FIG. 7 is a diagram illustrating a matrix for correcting distortion of an object image when the tilted angle of an electronic device is in a range of 0 degrees or more and 90 degrees or less according to one embodiment. FIG. 8 is a diagram illustrating a matrix for correcting distortion of an object image when the tilted angle of an electronic device is in a range of 90 degrees or more and 180 degrees or less according to one embodiment.

[0078] Referring to FIG. 3, in operation 310, the processor (210) can output a preview image through the display (240). The processor (210) can output at least a portion of the image frames acquired from the camera (230) as preview images on the display (240). For example, the camera (230) can acquire image frames of an external object (e.g., a user) based on a user input (e.g., a touch input), and the processor (210) can display at least a portion (e.g., a preview image) of the image frames acquired from the camera (230) on the display (240).

[0079] In one embodiment of the present disclosure, at operation 320, the processor (210) may define a region of interest in an object image. For example, the region of interest may be a region defined based on data associated with a user or a region predefined in the electronic device (101). For example, the region of interest may be defined based on images previously acquired by the camera (230). That is, the processor (210) may identify a plurality of images previously acquired through the camera (230) and define an object (e.g., a user's face) commonly included in the plurality of images as a region of interest.

[0080] According to one embodiment of the present disclosure, the processor (210) can identify at least one object in a preview image output through the display (240). The processor (210) can identify at least one object in response to a user input. For example, in response to a touch input touching at least one object in the preview image, the processor (210) can identify at least one object. For example, in response to an input selecting a user interface (UI) for identifying at least one object, the processor (210) can identify at least one object. The processor (210) can identify at least one object in the preview image based on conditions pre-stored in the memory (220). For example, the types of objects (e.g., people, animals, etc.) that the processor (210) can identify may be pre-stored in the memory (220).

[0081] According to one embodiment of the present disclosure, the processor (210) may define an identified object as a region of interest. For example, in response to a user input for defining a region of interest, the processor (210) may define the region of interest in the identified object. The region of interest may be defined by adding a predetermined margin area to an area corresponding to the shape of the object. That is, the region of interest may be referred to as an image area that includes the shape of the object. For example, the predetermined margin area may be preset in the electronic device (200). According to one embodiment of the present disclosure, the predetermined margin area may be adjusted based on a user input or a pre-stored setting. For example, the processor (210) may display an region of interest including an object to the user through the display (240). In this case, the user may drag the displayed region of interest, and the processor (210) may increase or decrease the region of interest based on the direction and / or degree of the drag.

[0082] According to one embodiment of the present disclosure, the processor (210) can reduce or increase the region of interest based on the type of object included in the region of interest. That is, if the type of object included in the region of interest is a first type (e.g., a dog, a plant), the processor (210) can set the size of the margin area of ​​the region of interest to a first size. If the type of object included in the region of interest is a second type (e.g., a person), the processor (210) can set the size of the margin area of ​​the region of interest to a second size that is larger than the first size.

[0083] According to one embodiment of the present disclosure, the processor (210) may display a visual object representing the region of interest on the display (240) to visually indicate the region of interest. For example, the visual object may be displayed to correspond to the shape of an object included in the region of interest.

[0084] According to one embodiment of the present disclosure, the processor (210) can track an object included in images. For example, the processor (210) can capture images of an object using a camera (230) over a specified period of time and identify a change in the position of the object included in the captured images. In this case, the processor (210) can define an area including the object that has completed its movement as a region of interest when the object has completed its movement. In one example, the processor (210) can identify that the movement of the object has been completed based on identifying that there is no change in the position of the object in the captured images.

[0085] According to one embodiment of the present disclosure, the processor (210) may display a newly defined region of interest to the user through the display (240) as the object moves. For example, the processor (210) may identify the movement of the object while displaying a first region of interest including the object before moving through the display (240). In this case, the processor (210) may stop displaying the first region of interest while the object moves, and may display a second region of interest including the object that has completed moving on the display (240) when the object has completed moving.

[0086] According to one embodiment of the present disclosure, a preview image corresponding to a region of interest including an object may be an image corrected based on a projection matrix. For example, due to the shape of a convex lens included in the camera (230), a preview image including an object may have different degrees of distortion depending on which part of the convex lens the object corresponds to. That is, the degrees of distortion may be different when the object is photographed through a first part of the lens included in the camera (230) and when the object is photographed through a second part of the lens included in the camera (230).

[0087] Accordingly, before outputting the area of ​​interest set on the object as a preview image, the processor (210) can correct the area of ​​interest defined on the object using a projection matrix determined based on the curvature of the lens and output it as a preview image.

[0088] In one embodiment of the present disclosure, referring to FIG. 4, the image frame of an object acquired from a camera (230) may have a relatively greater degree of distortion from the center (405) of the preview image (400) before correction to the edge (415) depending on the convex shape of the lens.

[0089] An electronic device (200) according to one embodiment can perform LDC (Lens Distortion Correction) to correct distortion caused by the convex nature of a lens. LDC is a technique that compares a distorted image with a normal image, determines a distortion coefficient, and corrects by substituting a distortion opposite to the distortion. The distortion correction rate allows for correction of a captured distorted image by calibrating the camera's internal parameters.

[0090] Correction techniques for lens distortion are generally divided into "metric" and "non-metric" methods. The metric method refers to a method of correcting image distortion by measuring both intrinsic and extrinsic variables of the camera model that may affect image distortion based on the correspondence of the image, and the non-metric method refers to a method of correcting curved components caused by lens distortion into straight lines in the process of acquiring the image under the assumption that the straight line components of external objects should be projected into the image in the form of straight lines, without being based on the correspondence. It goes without saying that the lens distortion correction (LDC) applied in various embodiments of the present disclosure can be applied to various known techniques in addition to the above-described methods.

[0091] For example, as an object is located at the edge (410) of the preview image (400) before correction, an image frame that is stretched left and right or up and down more than the actual object may be obtained. The processor (210) may apply a projection matrix (415) for correcting distortion due to the curvature of the lens of the preview image (400) before correction of the object to the preview image (400) before correction of the object to output a corrected preview image (430). For example, the projection matrix (415) for correcting distortion due to the curvature of the lens may be formed to provide a concave effect as it goes from the edge (425) to the center (420). The processor (210) may output a preview image in which distortion due to the difference in the curvature of the lens at the edge (440) and the center (435) of the corrected preview image (430) is improved. The processor (210) can output the corrected preview image (430) by recalculating the coordinate values ​​of the pixels that have changed as a result of applying the projection matrix (415) to the uncorrected preview image (400) before outputting the corrected preview image (430).

[0092] In one embodiment of the present disclosure, the processor (210) may correct distortion of a preview image by correcting distortion due to curvature of a lens, and may also change a projection matrix based on distortion due to an angle of the camera (230) with respect to a horizontal plane or a position of an object within the preview image.

[0093] In one embodiment of the present disclosure, referring to FIG. 3, in operation 330, the processor (210) may obtain information on an angle (530) of the electronic device (200). The processor (210) may obtain information on an angle formed by the electronic device (200) with at least one preset reference plane using the sensor (250). For example, the processor (210) may obtain information on an angle (530) formed by the electronic device (200) with respect to a horizontal plane using the sensor (250). For example, the sensor (250) may include a gyro sensor. The processor (210) may identify a degree of inclination of the electronic device (200) with respect to a horizontal plane using the gyro sensor. The processor (210) may obtain an angle (530) of the electronic device (200) with respect to a horizontal plane based on the degree of inclination of the electronic device (200). For example, referring to FIG. 5, it can be seen that the angle (530) with respect to the horizontal plane (520) of the electronic device (500) is θ. For example, the electronic device (500) may correspond to at least one of the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2.

[0094] According to one embodiment of the present disclosure, the electronic device (500) can identify angle information of the electronic device (500) and a direction in which the camera (510) faces with respect to at least one reference plane. Referring to FIG. 5, for example, the electronic device (500) can identify a direction in which the camera (500) faces with respect to a vertical plane (540) based on angle information that the electronic device (500) forms with respect to a horizontal plane (520). In this case, the at least one reference plane may include the horizontal plane (520) and the vertical plane (540). For example, as the angle (530) between the electronic device (500) and the horizontal plane (520) increases, the direction in which the camera (510) faces with respect to the vertical plane (540) may change from a first direction (550) to a second direction (560). For example, as the angle (530) between the electronic device (500) and the horizontal plane (520) decreases, the direction in which the camera (510) faces with respect to the vertical plane (540) may change from the first direction (550) to the third direction (570). For example, the first direction (550) may be the direction in which the camera (510) faces with respect to the vertical plane (540) when the angle (530) between the electronic device (500) and the horizontal plane is 90 degrees. For example, the first direction (560) may be the direction parallel to the horizontal plane (520). For example, the second direction (560) may be the direction in which the camera (510) faces with respect to the vertical plane (540) in a range in which the angle (530) between the electronic device (500) and the horizontal plane is greater than 90 degrees. For example, the third direction (570) may be a direction in which the camera (510) faces the vertical plane (540) within a range where the angle (530) between the electronic device (500) and the horizontal plane is less than 90 degrees.

[0095] According to one embodiment of the present disclosure, the degree of distortion of a preview image that the processor (210) outputs to the display (240) may vary based on the angle (530) with respect to the horizontal plane (520) of the electronic device (500). As the angle (530) formed between the electronic device (500) and the horizontal plane (520) varies, the direction in which the camera (510) faces with respect to the vertical plane (540) may vary, and the degree of distortion of a preview image that the processor (210) outputs to the display (240) may vary based on the direction in which the camera (510) faces with respect to the vertical plane (540). For example, referring to FIG. 6, it can be seen that preview images (600, 610, 620) that output at least a portion of the image frames of the same object (630) to the display (240) are different depending on the angle (530) with respect to the horizontal plane (520) of the electronic device (500). As the angle (530) is 90 degrees, in a situation where the direction in which the camera (510) faces with respect to the vertical plane (540) is the first direction (550), the image of the lower part of the image of the object (600) within the second area of ​​interest (615) output by the processor (210) may be relatively more widely distorted than the image of the lower part of the image of the object (600) within the third area of ​​interest (625) output by the processor (210) in a situation where the direction in which the camera (510) faces with respect to the vertical plane (540) is the third direction (570), as the angle (530) is 70 degrees. In contrast, in a situation where the direction in which the camera (510) faces with respect to the vertical plane (540) is the first direction (550) as the angle (530) is 90 degrees, the image of the upper part of the image of the object (600) within the second area of ​​interest (615) output by the processor (210) may be relatively more widely distorted than in a situation where the direction in which the camera (510) faces with respect to the vertical plane (540) is the second direction (560) as the angle (530) is 110 degrees.The processor (210) can determine the projection matrix based on the degree of distortion of images obtained from the same object (600) at different angles (530).

[0096] In one embodiment of the present disclosure, referring to FIG. 3, in operation 340, the processor (210) may determine a projection matrix based on position information and angle information of the region of interest. The operation of determining the projection matrix based on the position information and angle information of the region of interest may include an operation of changing the projection matrix based on the position information and angle information of the region of interest. The processor (210) may correct the projection matrix based on an angle with respect to a horizontal plane (520) of the electronic device (500). For example, referring to FIG. 7, as the angle (530) becomes smaller than 90 degrees, the lower part of the object image within the region of interest may be widely distorted, so the projection matrix (700) having a rectangular shape may be changed to a correction matrix (700) having an inverted trapezoidal shape so that the length of the upper side (710) is formed relatively longer than the length of the lower side (720). For example, referring to FIG. 8, as the angle (530) becomes larger than 90 degrees, the upper part of the object image within the area of ​​interest may be widely distorted, so the rectangular projection matrix (800) may be changed to a trapezoidal correction matrix (800) so that the length of the lower side (810) is formed relatively longer than the length of the upper side (8200).

[0097] In one embodiment of the present disclosure, the processor (210) can change the projection matrix based on the top-bottom ratio (e.g., b / a value) of the image of the object within the region of interest. In the top-bottom ratio, b may refer to the length of the short axis of the image based on the image of the object, and a may refer to the length of the long axis. For example, in one embodiment of the present disclosure, the top-bottom ratio of the projection matrix may refer to the ratio between the length of the upper part and the length of the lower part of the projection matrix.

[0098] In one embodiment of the present disclosure, when the angle (530) formed by the electronic device (200) with respect to the horizontal plane (520) is less than 90 degrees, the processor (210) may change the projection matrix to increase the ratio of a relative to b based on identifying that the top / bottom ratio of the image of the object in the third preview image (620) of FIG. 6 is b / a<1. The processor (210) may correct the top / bottom ratio of the image of the object to be close to 1 based on the changed projection matrix. When the angle formed by the electronic device (200) with respect to the horizontal plane (520) is greater than 90 degrees, the processor (21) may change the projection matrix to increase the ratio of b relative to a based on identifying that the top / bottom ratio of the image of the object in the first preview image (600) of FIG. 6 is b / a>1.

[0099] However, the method of changing the projection matrix is ​​not limited to a method of correcting the length of the upper or lower side of a rectangular projection matrix, and may be changed based on the position of an object in the preview image, and may be changed within a range for achieving the purpose of correcting distortion of the preview image.

[0100] In one embodiment of the present disclosure, referring to FIG. 3, in operation 350, the processor (210) may apply the determined projection matrix to a region of interest including an object image. The processor (210) may apply the determined projection matrix to an image of a distorted object within the region of interest to output an undistorted image of the object as a preview image. For example, the processor (210) may apply the correction matrix (800) of FIG. 8 to the first region of interest (605) of FIG. 6 to output an undistorted image of the object as a preview image. For example, the processor (210) may apply the correction matrix (700) of FIG. 7 to the third region of interest (625) of FIG. 6 to output an undistorted image of the object as a preview image. The projection matrix is ​​not only applied to an image of an object, but may also be applied to a region of interest including an image of an object.

[0101] In one embodiment of the present disclosure, referring to FIG. 3, at operation 360, the processor (210) may display another preview image including a region of interest to which a projection matrix is ​​applied on the display. The processor (210) may reset the region of interest for another preview image including a corrected object image. For example, the image of the corrected object may be different from the image of the object before correction, and thus the processor (210) may reset the region of interest to correspond to the image of the corrected object. The processor (210) may output an image of the corrected object within the reset region of interest as a preview image.

[0102] According to one embodiment of the present disclosure, FIG. 9 is a flowchart (900) illustrating a process for a method for an electronic device according to one embodiment to switch a preview image to another preview image.

[0103] In one embodiment of the present disclosure, referring to FIG. 9, in operation 910, the processor (210) may convert a preview image to another preview image. The processor (210) may reset a region of interest based on the converted preview image. The processor (210) may output an object with a region of interest set among a plurality of objects in the preview image as a converted preview image including an image of the object that has been corrected based on a changed projection matrix. The processor (210) may execute a zooming or panning function to output an image of the corrected object as a converted preview image. For example, if an object with a region of interest set is located relatively far away from other objects in the preview image, the processor (210) may execute a zoom-in function to output a converted preview image including an image of the corrected object. For example, if an object with a region of interest set within a preview image is located relatively close to other objects, the processor (210) may execute a zoom-in function to output a converted preview image including an image of the corrected object. For example, if an object with a region of interest set within a preview image is located relatively to the side compared to other objects, the processor (210) may execute a panning function to move the preview image to be located at the center of the converted preview image including the corrected object, so as to output an image of the corrected object at the center of the preview image. Zooming or panning may be executed independently or in combination.

[0104] According to one embodiment of the present disclosure, when the processor (210) outputs a converted preview image including a corrected image based on a zooming or panning function, the processor (210) may reset the region of interest to correspond to the corrected image. For example, since the converted preview image including the image of the corrected object may not be the same image as the preview image before correction based on zooming or panning, the region of interest may be reset to correspond to the image of the object in the converted preview image after correction.

[0105] In one embodiment of the present disclosure, referring to FIG. 9 , at operation 920, the processor (210) may store at least a portion of the converted preview image including the reset region of interest as an image. In response to a user input for acquiring an image, the processor (210) may store at least a portion of the converted preview image including the reset region of interest as an image. For example, the user input may include selecting a UI for acquiring an image. For example, the user input may include a touch input touching the display (240).

[0106] According to one embodiment of the present disclosure, FIG. 10 is a flowchart (1000) illustrating a process for a method for an electronic device to output a corrected object image to a wearable electronic device based on a control signal for controlling a correction level received from the wearable electronic device according to one embodiment. According to one embodiment of the present disclosure, FIG. 11 is a diagram illustrating an overview of an electronic device receiving a signal for controlling a correction level from a wearable electronic device according to one embodiment.

[0107] In one embodiment of the present disclosure, referring to FIG. 10, after operation 360 is performed, in operation 1010, the processor (210) may identify a wearable electronic device (1110) connected to the electronic device (1100). For example, the electronic device (1100) may correspond to the electronic device (101) of FIG. 1. For example, the electronic device (1100) may correspond to the electronic device (200) of FIG. 2.

[0108] In one embodiment of the present disclosure, the processor (210) can identify whether a connection is made based on device information of the wearable electronic device (1110). For example, the information may include, but is not limited to, the model name of the wearable electronic device (1110), the international mobile equipment identity (IMEI) number, the manufacturer, the operating system, the version information, the screen resolution, and the identification value of the application installed in the wearable electronic device (1110). The processor (210) can request device information from the wearable electronic device (1110) through a communication unit capable of performing wireless communication, and the electronic device (1100) can receive device information from the wearable electronic device (1110).

[0109] In one embodiment of the present disclosure, if the processor (210) fails to identify the connected wearable electronic device (1110), the processor (210) may output a reset region of interest in the corrected object image to the display (240) of the electronic device (1100) according to operation 360.

[0110] In one embodiment of the present disclosure, when the processor (210) identifies a connected wearable electronic device (1110), the processor (210) may output a corrected preview image through the wearable electronic device (11110). For example, the processor (210) may output the preview image through a smart watch (1110-1) among the wearable electronic devices (1110). The preview processor (210) may control a communication unit so that at least a portion of the preview image is output to the wearable electronic device (1110). The processor (210) may transmit a signal (1130) for controlling the preview image to be output to the wearable electronic device (1110) to the wearable electronic device (1110).

[0111] In one embodiment of the present disclosure, the electronic device (1100) can receive a control signal for adjusting a correction level of an object image from a wearable electronic device (1110) through a communication unit. For example, the electronic device (110) can receive a control signal for adjusting a correction level of an object image from a smart watch (1110-1) and a smart pen (1110-3) among the wearable electronic devices (1110). For example, the processor (210) can adjust a projection matrix based on the reception of a control signal for adjusting a correction level. The processor (210) can adjust the projection matrix to reset a region of interest set for an object. The processor (210) can output a corrected preview image based on the adjustment through a wearable electronic device (1110) (e.g., a smart watch (1110-1). The processor (210) can transmit a signal to the wearable electronic devices (1110) (e.g., a smart watch (1110-1), smart earphones (1110-2), and smart pens (1110-3)) to indicate that the correction is complete.

[0112] According to one embodiment of the present disclosure, FIG. 12 is a flowchart (1200) illustrating a process for a method for an electronic device according to one embodiment to change a matrix based on the position of a camera. FIG. 13 is a diagram illustrating an outline of a region of interest and an object image within the region of interest set based on the position of a camera according to one embodiment.

[0113] In one embodiment of the present disclosure, after operation 330 is performed, in operation 340, when the processor (210) changes the projection matrix, the processor (210) may additionally consider the position of the camera. Operations 1210 to 1220 are not necessarily limited to operations that must be performed after operation 330.

[0114] In one embodiment of the present disclosure, referring to FIG. 12, in operation 1210, the processor (210) can identify the position of the camera (1330). The processor (210) can obtain rotation information of the electronic device (1330) using a gyro sensor. The processor (210) can identify the position of the camera (1310) based on the rotation information. For example, the electronic device (1300) may correspond to at least one of the electronic device (101) of FIG. 1, the electronic device (200) of FIG. 2, the electronic device (500) of FIG. 5, or the electronic device (1100) of FIG. 11. For example, the camera (1310) may correspond to the camera (230) of FIG. 2.

[0115] According to one embodiment of the present disclosure, the gyro sensor can measure the rotation angle or inclination of the electronic device (1300) about three axes (e.g., x-axis, y-axis, or z-axis). For example, when the electronic device (1300) rotates about the x-axis passing through the display (240) of the electronic device (1300) while facing in a direction parallel to the horizontal plane, the processor (210) can identify the relative position of the camera (1310) based on the rotation angle measured by the gyro sensor. For example, when the electronic device (1300) rotates 0 degrees about the x-axis, the camera (1310) can be located at a position relatively far from the ground. For example, the position of the camera (1310) within the electronic device (1300) can be located at a portion within the electronic device (1300) that is relatively far from the ground. For example, when the electronic device (1300) is rotated 180 degrees around the x-axis, the camera (1310) may be positioned relatively close to the ground. For example, the position of the camera within the electronic device (1300) may be positioned at a part within the electronic device (1300) that is relatively close to the ground.

[0116] According to one embodiment of the present disclosure, the angle of view (1320) through which the camera (1310) can capture an object (1370) may be 80 to 85 degrees. When the processor (210) captures the object (1370) through the camera (1310), a straight line perpendicular to the lens of the camera (1370) may intersect a wall (1380) formed perpendicular to the horizontal plane, thereby dividing the angle of view (1320) into two angles. The two angles may include an upper angle (1330) and a lower angle (1340). For example, the upper angle (1330) may be an angle formed relatively higher than the lower angle among the two angles. The lower angle and the upper angle may add up to 180 degrees.

[0117] In one embodiment of the present disclosure, when the electronic device (1300) is rotated 0 degrees around the x-axis and the processor identifies that the camera (1310) is positioned relatively far from the ground, the upper angle (1330) may be formed to be larger than the reference angle. For example, as the upper angle (1330) is formed to be large, when the object (1370) is a subject, the processor (210) may obtain an image frame of the subject having a wide upper part. For example, as the upper angle (1330) is formed to be large, when the object (1370) is a person, the processor (210) may obtain an image frame of the person having a wide forehead. When the electronic device (1300) is rotated 180 degrees around the x-axis and the processor identifies that the camera (1310) is positioned relatively close to the ground, the lower angle (1340) may be formed to be larger than the reference angle. For example, as the lower angle (1340) is formed to be large, if the object (1370) is a subject, the processor (210) can obtain an image frame of the subject having a wide lower part. For example, as the lower angle (1340) is formed to be large, if the object (1370) is a person, the processor (210) can obtain an image frame of a person having a wide lower part. However, in normal shooting, the lower angle (1340) may not be formed to be larger than the upper angle (1330).

[0118] According to one embodiment of the present disclosure, an image frame of an object (1370) may be acquired with distortion based on the position of the camera (1310). The processor (210) may change a projection matrix to correct the distorted image frame based on the position of the camera (1310).

[0119] In one embodiment of the present disclosure, referring to FIG. 12, in operation 1220, the processor (210) can change the matrix based on the position of the camera.

[0120] In one embodiment of the present disclosure, referring to (a) of FIG. 13, the position of the camera (1310) is relatively far from the ground, so that an image frame in which the upper part of the object (1370) appears relatively wide can be acquired. In order to correct the distortion of the image frame in which the upper part of the object (1370) appears relatively wide, the processor (210) can change the projection matrix into a matrix in which the lower side length (810) of the trapezoidal matrix is ​​relatively longer than the upper side length (820), as in the correction matrix (800) of FIG. 8. The processor (210) can correct the image of the object (1370) within the region of interest (1350) based on the changed projection matrix and output the corrected image as a preview image.

[0121] In one embodiment of the present disclosure, referring to (b) of FIG. 13, the position of the camera (1310) is relatively close to the ground, so that an image frame in which the lower part of the object (1370) appears relatively wide can be acquired. In order to correct the distortion of the image frame in which the lower part of the object (1370) appears relatively wide, the processor (210) can change the projection matrix into a matrix in which the upper side length (710) of a trapezoidal matrix is ​​relatively longer than the lower side length (720), as in the correction matrix (700) of FIG. 7. The processor (210) can correct the image of the object (1370) within the region of interest (1360) based on the changed projection matrix and output it as a preview image.

[0122] In one embodiment of the present disclosure, the processor (210) can change the projection matrix based on the vertical ratio (e.g., b / a value) of the image of the object within the region of interest. In the vertical ratio, b may refer to the length of the short axis of the image based on the image of the object, and a may refer to the length of the long axis. For example, in a situation where the position of the camera (1310) is located at the center of the electronic device (1300), the vertical ratio of the image of the object acquired may be set to b / a=1 in the electronic device (200).

[0123] In one embodiment of the present disclosure, in situation (a) where the camera (1310) is located relatively far from the ground, the processor (210) can change the projection matrix to increase the ratio of a relative to b based on identifying that the top / bottom ratio of the image of the object in the region of interest (1350) of FIG. 13 is b / a<1. The processor (210) can correct the top / bottom ratio of the image of the object to be close to 1 based on the changed projection matrix. In situation (b) where the camera (1310) is located relatively close to the ground, the processor (210) can change the projection matrix to increase the ratio of b relative to a based on identifying that the top / bottom ratio of the image of the object in the region of interest (1360) of FIG. 13 is b / a>1. The processor (210) can correct the top / bottom ratio of the image of the object to be close to 1 based on the changed projection matrix.

[0124] According to one embodiment of the present disclosure, FIG. 14 is a flowchart (1400) illustrating a process for a method for an electronic device according to one embodiment to change a region of interest based on the movement of an object. FIG. 15 is a diagram illustrating an example of an electronic device according to one embodiment to change a region of interest based on the movement of an object.

[0125] In one embodiment of the present disclosure, after operation 330 is performed, when the processor (210) changes the projection matrix in operation 340, the processor (210) may consider position information of the region of interest that has moved according to the movement of the object. Operations 1410 to 1420 are not necessarily limited to operations that must be performed after operation 330.

[0126] In one embodiment of the present disclosure, referring to FIG. 14, at operation 1410, the processor (210) may identify whether the region of interest has moved as the position of the image of the object has changed. If the processor (210) determines that the region of interest has not moved, the processor (210) may perform operation 340. If the processor (210) determines that the region of interest has moved, the processor (210) may perform operation 1420.

[0127] According to one embodiment of the present disclosure, the processor (210) may identify position information of a region of interest that moves within a preview image according to the movement of the object in order to track a moving object. The processor (210) may move the region of interest to define the position of the region of interest corresponding to the object after it has moved. The processor (210) may change the projection matrix based on the moved region of interest. For example, referring to FIG. 15, when an object (1500) before moving moves and becomes an object (1510) after moving, the position of the first region of interest (1520) corresponding to the position of the image of the object (1500) before moving may move to correspond to the position of the image of the object (1510) after moving. If the processor (210) applies the projection matrix for correcting distortion of the image of the object (1500) before moving to the image of the object (1510) after moving, distortion may occur. The processor (210) can change the projection matrix based on the moved position information of the first area of ​​interest and apply it to the image of the object (1510) after moving.

[0128] In one embodiment of the present disclosure, referring to FIG. 14, in operation 1420, the processor (210) can change the vertical ratio of the projection matrix based on the position of the moved region of interest. The processor (210) can change the projection matrix based on the amount of change in the position of the region of interest before and after the movement. For example, referring to FIG. 15, the object (1500) before the movement is biased toward the left side of the object (1500) based on the image of the object (1500), and thus the first region of interest (1520) can also be defined to correspond to the object (1500). In contrast, the object (1510) after the movement has a right ear portion that is more protruding than the left ear portion, and the center of gravity is biased to the right compared to the object (1500) before the movement. Accordingly, the second region of interest (1530) can be defined on the object by being shifted to the right based on the image of the object (1510) after movement compared to the first region of interest (1520). The processor (210) can change the projection matrix based on the result of comparing the information about the position where the second region of interest (1530) is defined with the information about the position where the first region of interest (1520) is defined. For example, referring to the corrected projection matrix (1540) of FIG. 15, the processor (210) can change the shape of the corrected projection matrix (1540) to a parallelogram shape shifted to the right based on the image of the object (1510) after movement, rather than a rectangle. The processor (210) can define a third region of interest including an image of the object (1510) after movement that is a corrected image based on the corrected projection matrix (1540). The processor (210) can output an image of an object included in the third area of ​​interest to the display (240) as a preview image.

[0129] In one embodiment of the present disclosure, the processor (210) can change the projection matrix based on the vertical ratio (e.g., b / a value) of the image of the object within the region of interest. In the vertical ratio, b may refer to the length of the short axis of the image based on the image of the object, and a may refer to the length of the long axis. For example, the vertical ratio of the image of the object acquired in a situation where the object does not move may be defined in the electronic device (200) as b / a=1.

[0130] In one embodiment of the present disclosure, for example, if the object is a person, and in a situation where the person's chin protrudes more than the forehead as the person moves, the processor (210) may change the projection matrix to increase the ratio of a relatively to b based on identifying that the top / bottom ratio of the image of the object within the region of interest is b / a<1. The processor (210) may correct the top / bottom ratio of the image of the object to approach 1 based on the changed projection matrix. For example, if the object is a person, and in a situation where the person's forehead protrudes more than the chin as the person moves, the processor (210) may change the projection matrix to increase the ratio of b relatively to a based on identifying that the top / bottom ratio of the image of the object within the region of interest is b / a>1. The processor (210) may correct the top / bottom ratio of the image of the object to approach 1 based on the changed projection matrix.

[0131] In one embodiment of the present disclosure, even if the object does not move, as in the example of FIG. 15, the processor (210) can reset the region of interest. For example, a first region of interest including an image of an object before correction can be defined as a first region of interest, a region of interest after the projection matrix is ​​applied as a second region of interest, and a region of interest corrected for the second region of interest as a third region of interest. When the processor (210) applies the projection matrix to the image of the object before correction to correct distortion of the image of the object before correction, the coordinate values ​​of a plurality of pixels included in the first region of interest may be distorted. The processor (210) can rearrange the coordinate values ​​of a plurality of pixels included in the second region of interest and output the third region of interest to the display (240). For example, referring to FIG. 15, the first region of interest may be the first region of interest (1520) of FIG. 15, and the third region of interest may be the second region of interest (1530) of FIG. 15. Although not shown in FIG. 15, the region of interest in which the coordinate values ​​of a plurality of pixels included in the second region of interest are rearranged may be the second region of interest (1530) of FIG. 15.

[0132] According to one embodiment of the present disclosure, FIG. 16 is a flowchart (1600) illustrating a process for identifying an object carrying a wearable electronic device connected to an electronic device among a plurality of objects according to one embodiment. FIG. 17 is a diagram illustrating an example for identifying an object carrying a wearable electronic device connected to an electronic device among a plurality of objects according to one embodiment.

[0133] According to one embodiment of the present disclosure, in operation 310, the processor (210) can identify multiple objects within a preview image output to a display. For example, referring to FIG. 17, even if the processor (210) has set the object for setting the region of interest in advance as a person in the electronic device (200), in response to a user's input for determining the type of object (e.g., person, animal, etc.) for setting the region of interest, the processor (210) can also recognize the face of a person in a portrait (1700) other than an actual person (1710), such as a portrait (1700) of a person drawn in a work of art, as an object for setting the region of interest.

[0134] In one embodiment of the present disclosure, referring to FIG. 16, in operation 1610, the processor (210) may set multiple regions of interest corresponding to multiple objects. The processor (210) may set multiple regions of interest for objects recognized as the same type of object. For example, referring to FIG. 17, the processor (210) may recognize, in addition to an actual person positioned at the center of the drawing, face drawings within multiple portraits arranged around the person as objects of the same type as the actual person. The processor (210) may set multiple regions of interest for multiple objects recognized as faces of the same person.

[0135] According to one embodiment of the present disclosure, images of objects within a region of interest set for a plurality of objects may be corrected images based on a changed projection matrix. For example, the processor (210) may change the projection matrices differently for each object based on at least one of a position of each object within a preview image, a position of the camera (230) with respect to an electronic device, an angle of the camera (230) photographing the objects with respect to a horizontal plane, or a movement of each object. The processor (210) may set a region of interest including images of objects in which each object is corrected based on the projection matrix that is changed differently for each object. The processor (210) may output images of the corrected objects as preview images to the display (240).

[0136] In one embodiment of the present disclosure, referring to FIG. 16, in operation 1620, the processor (210) may identify an object carrying a wearable device based on a signal generated from the wearable device. The processor (210) may identify only an object carrying a wearable device among a plurality of objects as an object for setting a region of interest. For example, referring to FIG. 17, among a plurality of objects, only an actual person wearing a wearable electronic device (e.g., a smartwatch) may be identified as an object for setting a region of interest.

[0137] In one embodiment of the present disclosure, the processor (210) can identify a wearable electronic device through a communication unit. The processor (210) can transmit a signal requesting device information of the wearable electronic device to the wearable electronic device. For example, the device information of the wearable electronic device may include, but is not limited to, the model name of the wearable electronic device, the international mobile equipment identity (IMEI) number, the manufacturer, the operating system, the version information, the screen resolution, and the identification value of the application installed on the wearable electronic device. The wearable electronic device can transmit the device information to the electronic device (200) based on the received signal. The processor (210) can identify the wearable electronic device based on the signal received from the wearable electronic device.

[0138] In one embodiment of the present disclosure, the processor (210) may identify only an object wearing a wearable electronic device as an object for setting a region of interest based on the identified wearable electronic device information. For example, the processor (210) may identify only the face of an actual person holding a wearable electronic device in the center of the drawing of FIG. 17 as an object for setting a region of interest.

[0139] In one embodiment of the present disclosure, the processor (210) may transmit the object identification result to the wearable electronic device. For example, referring to FIG. 11, if the processor (210) identifies an object carrying a wearable electronic device among a plurality of objects as an object to be set as a region of interest, the processor (210) may transmit the identification result to the wearable electronic devices (1110). The wearable electronic devices (1110) may transmit a signal to the electronic device (1100) to set a region of interest for the object identified. The electronic device (1100) may set a region of interest for the identified object based on the received signal. For example, if the object identification result is transmitted through the smart earphones (1110-2), the electronic device (1300) may set a region of interest for the identified object in response to an input for setting a region of interest using the smart pen (1110-3).

[0140] In one embodiment of the present disclosure, referring to FIG. 16, in operation 1630, the processor (210) may select one of a plurality of regions of interest based on a result of identifying an object. Based on identifying an object carrying a wearable electronic device as an object for setting a region of interest, the processor (210) may set a region of interest only for the object carrying the wearable electronic device among the plurality of regions of interest. The processor (210) may output only an image of an object for which a region of interest is set as a preview image to the display (240). For example, the processor (210) may output only an image of an actual person of FIG. 17 as a preview image to the display (240).

[0141] In one embodiment of the present disclosure, the processor (210) may select only one of the plurality of regions of interest as a region of interest in response to a user input selecting at least one of the plurality of regions of interest. For example, in response to a user touch input selecting one of the plurality of regions of interest displayed on the display (240), the processor (210) may set only the touched region of interest as a region of interest. For example, in response to an input selecting a UI displayed on the display (240), the processor (210) may set only a specific region of interest as a region of interest.

[0142] According to one embodiment of the present disclosure, FIG. 18 is a flowchart illustrating a process for a method for correcting distortion of a region of interest including an object image based on a projection matrix by a foldable electronic device according to one embodiment. FIG. 19 is a diagram illustrating an example of a first folding state of the foldable electronic device according to one embodiment. FIG. 20 is a diagram illustrating an example of a second folding state of the foldable electronic device according to one embodiment.

[0143] According to one embodiment of the present disclosure, referring to FIG. 18, operation 1810 may correspond to operation 310 of FIG. 3.

[0144] According to one embodiment of the present disclosure, referring to FIG. 18, operation 1820 may correspond to operation 320 of FIG. 3.

[0145] According to one embodiment of the present disclosure, referring to FIG. 18, in operation 1830, the processor (210) can obtain a folding angle formed by the housings. The foldable electronic device (1900) of FIG. 19 may correspond to at least one of the electronic device (101) of FIG. 1, the electronic device (200) of FIG. 2, the electronic device (500) of FIG. 5, the electronic device (1100) of FIG. 11, or the electronic device (1300) of FIG. 13.

[0146] According to one embodiment of the present disclosure, a foldable electronic device (1900) may include a first housing (1910). The first housing (1910) may include a first side facing a first direction and a second side facing a second direction opposite to the first direction. The foldable electronic device (1900) may include a second housing (1920). The second housing (1920) may include a third side facing a third direction and a fourth side facing a fourth direction opposite to the third direction. The first housing (1910) and the second housing (1920) may be connected via a hinge (1930). A first camera (1930) may be provided on at least a portion of a first side of the first housing (1910). A second camera (1940) may be provided on at least a portion of a second side of the first housing (1910). Although not shown in FIG. 19, the first display may be provided on at least a portion of the first side of the first housing (1910) and the third side of the second housing (1920). The second display may be provided on at least a portion of the second side of the first housing (1910).

[0147] According to one embodiment of the present disclosure, the first housing (1910) may include a first Hall sensor and a first magnet. The second housing (1920) may include a second Hall sensor and a second magnet. The Hall sensor may detect a change in an electrical signal based on the proximity or distance of an object having a magnetic force. The Hall sensor may measure a magnetic force value and transmit the measured magnetic force value to the processor (210). The processor (210) may obtain a folding angle formed by the housings based on the transmitted magnetic force value. For example, at least one of the first Hall sensor or the second Hall sensor may measure a changing magnetic force value based on the proximity or distance of the first housing (1910) to or from the second housing (1920). The processor (210) may receive the measured magnetic force value and obtain a folding angle of the housings.

[0148] According to one embodiment of the present disclosure, referring to FIG. 18, in operation 1840, the processor (210) may select one of a plurality of cameras based on a folding state of the electronic device. The folding state may include a first folding state and a second folding state. Referring to FIGS. 19(a) and (b), the first folding state may include, for example, a state in which the fourth side of the second housing (1920) of the foldable electronic device (1900) is placed parallel to a horizontal plane. For example, the first folding state may include a state in which the fourth side of the second housing (1920) of the foldable electronic device (1900) is placed on the floor. For example, the fourth side of the second housing (1920) may include a state in which the fourth side of the second housing (1920) is placed on the floor. The second folding state may include, for example, a state in which the hinge moves in a direction opposite to the horizontal plane as the folding angle θ decreases, as shown in (a) and (b) of FIG. 20. For example, the second folding state may include a state in which the hinge (1930) is positioned relatively farther from the ground than in the first folding state. For example, the second folding state may include a state in which at least a portion of the first housing (1910) and at least a portion of the second housing (1920) are in contact with the ground.

[0149] According to one embodiment of the present disclosure, the processor (210) may select at least one camera based on the folding state and the folding angle. The at least one camera may include a first camera (1935) and a second camera (1940). For example, in the first folding state of FIG. 19, when the folding angle θ is less than the reference angle α, the processor (210) may select the second camera (1940). For example, in the first folding state of FIG. 19, when the folding angle θ is greater than the reference angle α, the processor (210) may select the first camera (1935). For example, while in the second folding state of FIG. 20, the processor (210) may select the second camera (1940).

[0150] According to one embodiment of the present disclosure, with reference to FIG. 18, at operation 1850, the processor (210) may determine a projection matrix based on the folding angle and the position of the selected camera.

[0151] According to one embodiment of the present disclosure, the processor (210) may determine a projection matrix based on a change in a folding angle of the foldable electronic device (210). For example, referring to FIG. 19, when the folding angle θ is smaller than the reference angle α, the second camera (1940) may be selected. When the second camera (1940) is selected, as the folding angle θ1 decreases, the image of the object within the region of interest acquired through the second camera (1940) may be an image in which the lower part of the object is relatively distorted. The processor (210) may determine the projection matrix as the projection matrix (700) of FIG. 7 to correct the image of the distorted object. For example, referring to FIG. 19, when the folding angle θ is larger than the reference angle α, the first camera (1935) may be selected. When the first camera (1935) is selected, as the folding angle θ2 increases, the object image within the region of interest acquired through the first camera (1935) may be an image in which the lower part of the object is relatively wide and distorted. The processor (210) may determine the projection matrix as the projection matrix (700) of FIG. 7 to correct the image of the distorted object. For example, referring to FIG. 20, the second camera (1940) may be selected while in the second folding state. When the second camera (1940) is selected, as the folding angle decreases from θ2 to θ1, the object image within the region of interest acquired through the second camera (1940) may be an image in which the upper part of the object is relatively wide and distorted. The processor (210) may determine the projection matrix as the projection matrix (800) of FIG. 8 to correct the image of the distorted object.

[0152] According to one embodiment of the present disclosure, the processor (210) may determine a projection matrix based on the position of the camera of the foldable electronic device (1900). For example, the position of the second camera (1940) in the second folded state of FIG. 20 may be located relatively closer to the ground than the position of either the first camera (1935) or the second camera (1940) in FIG. 19. In the second folded state, the image of an object within a region of interest acquired through the second camera (1940) may be an image in which the lower part of the object is relatively wide and distorted. The processor (210) may determine a projection matrix for correcting the distorted image of the object acquired in the second folded state as a projection matrix that corrects the lower side of a trapezoid to be longer than the projection matrix for correcting the image of the object within the region of interest acquired through either the first camera (1935) or the second camera (1940) in the first folded state. The processor (210) can determine or change the projection matrix by taking into account the position of the camera based on the folding angle and the folding state.

[0153] According to one embodiment of the present disclosure, referring to FIG. 18, operation 1860 may correspond to operation 360 of FIG. 3.

[0154] According to one embodiment of the present disclosure, an electronic device may provide a method for correcting distortion of an image, which can correct distortion caused by tilting of a camera, movement of an object, or position of a camera.

[0155] According to one embodiment of the present disclosure, an electronic device can provide a method for correcting distortion of an image acquired from the electronic device by utilizing a wearable device even from a distance.

[0156] According to one embodiment of the present disclosure, an electronic device may include a camera, at least one sensor for acquiring sensing data for detecting an inclination of the electronic device, a display, a memory for storing instructions, and one or more processors. The instructions, when executed by the one or more processors, may cause the electronic device to display a preview image acquired by the camera on the display, define a region of interest including at least one object image in the preview image, acquire angle information used to identify a direction in which the camera faces with respect to a reference plane when an inclination of the electronic device is detected, determine a projection matrix to be applied to the region of interest based on the angle information and position information of the region of interest in the preview image, apply the matrix to the region of interest including the at least one object image, and display another preview image including the region of interest to which the projection matrix is ​​applied on the display.

[0157] According to one embodiment of the present disclosure, the instructions, when executed by one or more processors, may cause the electronic device to execute a zoom function and / or a pan function of a camera to switch from displaying a preview image to displaying another preview image.

[0158] According to one embodiment of the present disclosure, the instructions, when executed by one or more processors, may cause an electronic device to define a plurality of regions of interest including a region of interest within a preview image, and, in response to receiving a user input selecting a region of interest from among the plurality of regions of interest, determine a top-bottom ratio of a projection matrix to be applied to the region of interest.

[0159] According to one embodiment of the present disclosure, at least one sensor includes a gyro sensor, and instructions, when executed by one or more processors, can cause the electronic device to obtain rotation information of the electronic device detected by the gyro sensor, identify a position of a camera, and determine an up-down ratio of a projection matrix based on the identified position of the camera.

[0160] According to one embodiment of the present disclosure, the instructions, when executed by one or more processors, may cause an electronic device to move a region of interest as a position of at least one object image within a preview image changes, and to change a top-bottom ratio of a projection matrix based on the position of the moved region of interest.

[0161] According to one embodiment of the present disclosure, the electronic device may further include a communication unit that performs wireless communication with a wearable electronic device. The instructions, when executed by one or more processors, may cause the electronic device to control the communication unit so that at least a portion of a preview image including a region of interest is output to the wearable electronic device, receive a control signal from the wearable device through the communication unit that adjusts a correction level of at least one object image, and control the communication unit so that at least one object image corrected based on the control signal is output to the wearable electronic device.

[0162] According to one embodiment of the present disclosure, the instructions, when executed by one or more processors, may cause an electronic device to define a plurality of regions of interest corresponding to a plurality of objects in a preview image, identify an object carrying a wearable electronic device based on a signal generated by the wearable electronic device, and select one of the plurality of regions of interest based on the identified object.

[0163] According to one embodiment of the present disclosure, an electronic device may further include a first housing including a first Hall sensor and forming a first side facing a first direction and a second side facing a second direction opposite to the first direction, and a second housing connected to the first housing by a hinge, the second housing including a second Hall sensor and forming a third side facing a third direction and a fourth side facing a fourth direction opposite to the third direction. The instructions, when executed by one or more processors, may cause the electronic device to identify a folding angle formed by the first housing and the second housing based on a magnetic signal detected by at least one of the first Hall sensor or the second Hall sensor, and to determine a projection matrix based on the folding angle.

[0164] According to one embodiment of the present disclosure, the electronic device may further include another camera. The camera may be provided on at least a portion of a first surface of the first housing. The other camera may be provided on at least a portion of a second surface of the first housing. The at least one sensor may include a gyro sensor. The instructions, when executed by one or more processors, may cause the electronic device to identify a folding state of the electronic device based on movement of the first housing or the second housing detected by the gyro sensor, select one of the camera or the other camera based on the folding state, and determine a projection matrix based on the position of the selected camera.

[0165] According to one embodiment of the present disclosure, the folding state may include a first folding state in which the fourth side of the second housing is positioned parallel to the horizontal plane, and a second folding state in which the hinge moves in a direction opposite to the horizontal plane as the folding angle decreases. The instructions, when executed by one or more processors, may cause the electronic device to select another camera when the folding angle is less than or equal to a predetermined range while the electronic device is in the first folding state, and to select a camera when the folding angle is greater than the predetermined range. The instructions, when executed by the processor, may cause the electronic device to select another camera when the electronic device is in the second folding state.

[0166] According to one embodiment of the present disclosure, a method of operating an electronic device may include an operation of displaying a preview image acquired by a camera on a display, an operation of defining a region of interest including at least one object image in the preview image, an operation of acquiring angle information used to identify a direction in which the camera faces with respect to a reference plane as an inclination of the electronic device is detected, an operation of determining a projection matrix to be applied to the region of interest based on the angle information and position information of the region of interest in the preview image, an operation of applying the projection matrix to the region of interest including at least one object image, and an operation of displaying another preview image including the region of interest to which the projection matrix is ​​applied on the display.

[0167] According to one embodiment of the present disclosure, the method of operating an electronic device may further include an operation of executing a zoom function and / or a panning function of a camera to switch from displaying a preview image to displaying another preview image.

[0168] According to one embodiment of the present disclosure, a method of operating an electronic device may further include an operation of defining a plurality of regions of interest including a region of interest within a preview image, and an operation of determining a vertical ratio of a projection matrix to be applied to the region of interest in response to receiving a user input of selecting a region of interest from among the plurality of regions of interest.

[0169] According to one embodiment of the present disclosure, a method of operating an electronic device may further include an operation of obtaining rotation information of the electronic device detected by a gyro sensor, an operation of identifying a position of a camera based on the rotation information, and an operation of determining an up-down ratio of a projection matrix based on the identified position of the camera.

[0170] According to one embodiment of the present disclosure, a method of operating an electronic device may further include an operation of moving a region of interest within a preview image as a position of at least one object image within the preview image changes, and an operation of changing an up-down ratio of a projection matrix based on the position of the moved region of interest.

[0171] According to one embodiment of the present disclosure, a method of operating an electronic device may further include an operation of controlling a communication unit so that at least a portion of a preview image including a region of interest is output to the wearable electronic device, an operation of receiving a control signal for adjusting a correction level of at least one object image from the wearable device through the communication unit, and an operation of controlling the communication unit so that at least one object image corrected based on the control signal is output to the wearable electronic device.

[0172] According to one embodiment of the present disclosure, a method of operating an electronic device may further include an operation of defining a plurality of regions of interest corresponding to a plurality of objects in a preview image, an operation of identifying an object carrying a wearable electronic device based on a signal generated by the wearable electronic device, and an operation of selecting one of the plurality of regions of interest based on the identified object.

[0173] According to one embodiment of the present disclosure, an electronic device may further include a first housing including a first Hall sensor and forming a first side facing a first direction and a second side facing a second direction opposite to the first direction, and a second housing connected to the first housing by a hinge, including a second Hall sensor and forming a third side facing a third direction and a fourth side facing a fourth direction opposite to the third direction. A method of operating the electronic device may further include an operation of acquiring a folding angle formed by the first housing and the second housing based on a magnetic signal detected by at least one of the first Hall sensor and the second Hall sensor, and an operation of determining a projection matrix based on the folding angle.

[0174] According to one embodiment of the present disclosure, a camera may be provided on at least a portion of the first surface of the first housing, and another camera may be provided on at least a portion of the second surface of the first housing. At least one sensor may include a gyro sensor. The method of operating an electronic device may further include an operation of identifying a folding state of the electronic device based on a movement of the first housing or the second housing detected by the gyro sensor, an operation of selecting one of the camera or another camera based on the folding state, and an operation of determining a projection matrix based on a position of the selected camera.

[0175] According to one embodiment of the present disclosure, the folding state may include a first folding state in which the fourth side of the second housing is placed parallel to a horizontal plane, and a second folding state in which the hinge moves in a direction opposite to the horizontal plane as the folding angle decreases. The operating method of the electronic device may further include an operation of selecting another camera when the folding angle is below a predetermined range while the electronic device is in the first folding state, and an operation of selecting a camera when the folding angle is above the predetermined range, and an operation of selecting another camera when the electronic device is in the second folding state.

[0176] According to one embodiment of the present disclosure, an electronic device can provide an effect of effectively correcting image distortion by taking into account distortion due to tilt of a camera when correcting image distortion.

[0177] According to one embodiment of the present disclosure, an electronic device can correct image distortion even at a distance based on a control signal from a wearable device, thereby providing an effect of enhancing the user experience.

[0178] In addition, various effects may be provided directly or indirectly through the present disclosure.

Claims

1. In electronic devices, camera; At least one sensor for obtaining sensing data for detecting a tilt of the electronic device; display; Memory for storing instructions; and comprising one or more processors; The above instructions, when executed by the one or more processors, cause the electronic device to: Displaying a preview image acquired by the camera on the above display, Define a region of interest that includes at least one object image within the above preview image, Obtaining angle information used to identify the direction in which the camera is facing with respect to a reference plane as the tilt of the electronic device is detected, Determine a projection matrix to be applied to the region of interest based on the above angle information and the position information of the region of interest within the preview image, applying said projection matrix to said region of interest including said at least one object image, An electronic device causing another preview image including the region of interest to which the projection matrix is ​​applied to be displayed on the display.

2. In paragraph 1, The above instructions, when executed by the one or more processors, cause the electronic device to: An electronic device that causes the camera to perform a zoom function and / or a pan function to switch from displaying the preview image to displaying another preview image.

3. In paragraph 1, The above instructions, when executed by the one or more processors, cause the electronic device to: defining multiple regions of interest including the region of interest within the above preview image, An electronic device that determines the upper and lower ratios of a projection matrix to be applied to the region of interest in response to receiving a user input selecting the region of interest from among the plurality of regions of interest.

4. In paragraph 1, wherein at least one of the sensors comprises a gyro sensor, The above instructions, when executed by the one or more processors, cause the electronic device to: Obtaining rotation information of the electronic device detected by the above gyro sensor, Based on the above rotation information, the position of the camera is identified, An electronic device that determines the upper and lower ratio of the projection matrix based on the positions of the identified cameras.

5. In paragraph 1, The above instructions, when executed by the one or more processors, cause the electronic device to: Moving the region of interest within the preview image as the position of the at least one object image within the preview image changes, An electronic device for changing the upper and lower ratio of the projection matrix based on the position of the moved region of interest.

6. In paragraph 1, The electronic device further includes a communication unit that performs wireless communication with the wearable electronic device; The above instructions, when executed by the one or more processors, cause the electronic device to: Controlling the communication unit so that at least a portion of the preview image including the region of interest is output to the wearable electronic device; A control signal for adjusting the correction level of at least one object image is received from the wearable device through the communication unit, An electronic device that controls the communication unit so that at least one object image corrected based on the control signal is output to the wearable electronic device.

7. In paragraph 6, The above instructions, when executed by the one or more processors, cause the electronic device to: In the above preview image, multiple regions of interest corresponding to multiple objects are defined, Identifying an object carrying the wearable electronic device based on a signal generated from the wearable electronic device; An electronic device for selecting one of said plurality of regions of interest based on said identified object.

8. In paragraph 1, The above electronic device, A first housing including a first hall sensor and forming a first side facing a first direction and a second side facing a second direction opposite to the first direction; and Further comprising a second housing connected to the first housing by a hinge, including a second hall sensor, and forming a third side facing a third direction and a fourth side facing a fourth direction opposite to the third direction; The above instructions, when executed by the one or more processors, cause the electronic device to: Based on a magnetic signal detected by at least one of the first Hall sensor or the second Hall sensor, a folding angle formed by the first housing and the second housing is obtained, An electronic device that determines the projection matrix based on the folding angle.

9. In paragraph 8, The above electronic device, Including other cameras, The above camera is provided on at least a part of the first surface of the first housing, The above other camera is provided on at least a part of the second surface of the first housing, At least one sensor above, Includes a gyro sensor, The above instructions, when executed by the one or more processors, cause the electronic device to: Identifying the folding state of the electronic device based on the movement of the first housing or the second housing detected by the gyro sensor, Based on the above folding state, either the camera or the other camera is selected, An electronic device that determines the projection matrix based on the positions of the selected cameras.

10. In paragraph 9, The above folding state is, Including a first folding state in which the fourth surface of the second housing is placed parallel to the horizontal plane, and a second folding state in which the hinge moves in a direction opposite to the horizontal plane as the folding angle decreases. The above instructions, when executed by the one or more processors, cause the electronic device to: While the above electronic device is in the first folding state, If the above folding angle is below the specified range, select another camera, If the above folding angle exceeds the specified range, the camera is selected, An electronic device that allows said other camera to be selected while said electronic device is in said second folding state.

11. In the method of operating an electronic device, An action of displaying a preview image acquired by a camera on a display; An action of defining a region of interest including at least one object image within the above preview image; An operation of acquiring angle information used to identify a direction in which the camera is facing with respect to a reference plane when a tilt of the electronic device is detected; An operation of determining a projection matrix to be applied to the region of interest based on the angle information and the position information of the region of interest within the preview image; An operation of applying said projection matrix to said region of interest including said at least one object image; and A method of operation comprising displaying on the display another preview image including the region of interest to which the projection matrix is ​​applied.

12. In paragraph 11, An operating method further comprising: executing a zoom function and / or a panning function of the camera to switch from displaying the preview image to displaying the other preview image.

13. In paragraph 11, An operation of defining a plurality of regions of interest including the region of interest within the preview image; and A method of operation further comprising: in response to receiving a user input selecting a region of interest from among the plurality of regions of interest, determining an upper and lower ratio of a projection matrix to be applied to the region of interest.

14. In paragraph 11, An operation of obtaining rotation information of the electronic device detected by a gyro sensor; An operation of identifying the position of the camera based on the rotation information; and An operating method further comprising an operation of determining an upper and lower ratio of the projection matrix based on the position of the identified camera.

15. In paragraph 11, An operation of moving the region of interest within the preview image as the position of at least one object image within the preview image changes; and An operating method further comprising an operation of changing the upper and lower ratio of the projection matrix based on the position of the moved region of interest.

Images