Method and apparatus for adjusting camera by using image stabilization control

Abstract

This electronic device may comprise: a display; a processor; a camera comprising a lens, an image sensor, and an OIS module; a communication circuit; and a memory storing instructions. The instructions, when executed by the processor, may cause the electronic device to perform at least one operation. The at least one operation may comprise an operation of transmitting, to an external electronic device through the communication circuit, first data corresponding to a live preview image obtained by the camera. The at least one operation may comprise an operation of receiving a request transmitted by the external electronic device. The at least one operation may comprise an operation of controlling, on the basis of the request, the OIS module of the camera to change a relative position of the lens with respect to the image sensor in at least one axis perpendicular to the optical axis of the lens. The at least one operation may comprise an operation of transmitting, to the external electronic device, second data corresponding to the live preview image after the relative position of the lens with respect to the image sensor is adjusted.

Description

Method and device for adjusting a camera using image stabilization control

[0001] The present disclosure relates to a method and apparatus for adjusting a camera using image stabilization control.

[0002] Camera use is increasing in smartphones, tablets, and other mobile electronic devices. These devices offer a variety of user-friendly features for taking photos with their cameras.

[0003] For example, mobile electronic devices provide an auto-focusing (AF) function that automatically adjusts focus. Additionally, mobile electronic devices provide optical image stabilization (OIS), which utilizes sensor data related to the mobile electronic device's movement to compensate for undesirable movement (e.g., hand tremors) during photography.

[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0005] According to one embodiment of the present disclosure, an electronic device may include at least one processor, a camera module, and a communication module. The electronic device (or at least one processor) may transmit first preview data for previewing an image to be captured to an external electronic device. The electronic device (or at least one processor) may receive a composition adjustment request for adjusting a composition for capturing from the external electronic device. The electronic device (or at least one processor) may perform control on a camera module associated with an image stabilization function based on the composition adjustment request to adjust the composition. The electronic device (or at least one processor) may transmit second preview data for previewing an image associated with the adjusted composition to the external electronic device.

[0006] According to one embodiment of the present disclosure, an electronic device may include at least one processor and a camera module. The electronic device (or at least one processor) may identify an automatic composition adjustment request for automatically adjusting a composition for shooting. Based on the automatic composition adjustment request, the electronic device (or at least one processor) may control a camera module of the electronic device associated with an image stabilization function to adjust the composition to at least one composition. The electronic device (or at least one processor) may capture an image through the camera module in each of the at least one adjusted composition.

[0007] According to one embodiment of the present disclosure, a method of an electronic device may include an operation of transmitting first preview data for a preview of an image to be captured by an external electronic device. The method of the electronic device may include an operation of receiving a composition adjustment request for adjusting a composition for capturing from the external electronic device. The method of the electronic device may include an operation of controlling the camera module associated with an image stabilization function based on the composition adjustment request to adjust the composition. The method of the electronic device may include an operation of transmitting second preview data for a preview of an image associated with the adjusted composition to the external electronic device.

[0008] According to one embodiment of the present disclosure, a method of an electronic device may include an operation of identifying an automatic composition adjustment request for automatically adjusting a composition for shooting. The method of the electronic device may include an operation of controlling a camera module of the electronic device associated with an image stabilization function based on the automatic composition adjustment request to adjust the composition to at least one composition. The method of the electronic device may include an operation of capturing an image through the camera module in each of the at least one adjusted composition.

[0009] According to one embodiment of the present disclosure, a storage medium storing at least one computer-readable instruction, wherein the at least one instruction, when executed by at least one processor of an electronic device, causes the electronic device to perform at least one operation. The at least one operation may include transmitting first preview data for a preview of an image to be captured to an external electronic device. The at least one operation may include receiving a composition adjustment request for adjusting a composition for capturing from the external electronic device. The at least one operation may include controlling the camera module associated with an image stabilization function based on the composition adjustment request to adjust the composition. The at least one operation may include transmitting second preview data for a preview of an image associated with the adjusted composition to the external electronic device.

[0010] According to one embodiment of the present disclosure, a storage medium storing at least one computer-readable instruction, wherein the at least one instruction, when executed by at least one processor of an electronic device, causes the electronic device to perform at least one operation. The at least one operation may include identifying an automatic framing adjustment request for automatically adjusting a composition for shooting. The at least one operation may include controlling a camera module of the electronic device associated with an image stabilization function based on the automatic framing adjustment request to adjust the composition to at least one composition. The at least one operation may include capturing an image through the camera module in each of the at least one adjusted composition.

[0011] According to one embodiment of the present disclosure, an electronic device may include a display; a camera including a processor, a lens, an image sensor, and an optical image stabilization (OIS) module; communication circuitry; and a memory storing instructions. The instructions, when executed by the processor, may cause the electronic device to perform at least one operation. The at least one operation may include transmitting, via the communication circuitry, first data corresponding to a live preview image acquired by the camera to an external electronic device. The at least one operation may include receiving a request transmitted by the external electronic device, wherein the request may correspond to a user input received while the live preview image is displayed on the external electronic device. At least one action may include controlling the OIS module of the camera to change a relative position of the lens with respect to the image sensor in one or more axes perpendicular to an optical axis of the lens based on the request, and transmitting, to the external electronic device, second data corresponding to the live preview image after the relative position of the lens with respect to the image sensor has been adjusted.

[0012] According to one embodiment of the present disclosure, an electronic device may include a display; a camera including a processor, a lens, an image sensor, and an optical image stabilization (OIS) module; communication circuitry; and a memory storing instructions. The electronic device may include an operation of transmitting, through the communication circuitry, first data corresponding to a live preview image acquired by the camera to an external electronic device. The electronic device may include an operation of receiving a request transmitted by the external electronic device, wherein the request may correspond to a user input received while the live preview image is displayed on the external electronic device. The electronic device (101) may include an operation of controlling the OIS module of the camera to change a relative position of the lens with respect to the image sensor in one or more axes perpendicular to an optical axis of the lens based on the request, and transmitting, to the external electronic device, second data corresponding to the live preview image after the relative position of the lens with respect to the image sensor is adjusted.

[0013] According to one embodiment of the present disclosure, a method of an electronic device includes an operation of transmitting, through a communication circuit of the electronic device, first data corresponding to a live preview image acquired by a camera of the electronic device to an external electronic device, wherein the camera may include a lens, an image sensor, and an optical image stabilization (OIS) module. The method of the electronic device includes an operation of receiving a request transmitted by the external electronic device, wherein the request may correspond to a user input received while the live preview image is displayed on the external electronic device. The method of the electronic device may include an operation of controlling, based on the request, the OIS module of the camera to change a relative position of the lens with respect to the image sensor in one or more axes perpendicular to an optical axis of the lens. The method of the electronic device may include an operation of transmitting, to the external electronic device, second data corresponding to the live preview image after the relative position of the lens with respect to the image sensor has been adjusted.

[0014] According to one embodiment of the present disclosure, an electronic device may include a display; a processor; a camera including a lens, an image sensor, and an optical image stabilization (OIS) module; and a memory storing instructions. The instructions, when executed by the processor, may cause the electronic device to perform at least one operation. The at least one operation may include an operation of identifying whether a preset condition for adjusting the orientation of the camera is satisfied. The at least one operation may include an operation of controlling the OIS module of the camera based on the identification to adjust the orientation of the camera to at least one orientation. The at least one operation may include an operation of capturing an image through the camera in each of the at least one adjusted orientation.

[0015] According to one embodiment of the present disclosure, a method of an electronic device may include an operation of identifying whether a preset condition for adjusting the orientation of the camera is satisfied. Based on the identification, the method of the electronic device may include an operation of controlling the OIS module of the camera to adjust the orientation of the camera to at least one orientation. The method of the electronic device may include an operation of capturing an image through the camera in each of the at least one adjusted orientation.

[0016] According to one embodiment of the present disclosure, an electronic device may include a display; a processor; a camera including a lens, an image sensor, and an optical image stabilization (OIS) module; and a memory storing instructions. The electronic device may include an operation for identifying whether a preset condition for adjusting an orientation of the camera is satisfied. Based on the identification, the electronic device may include an operation for controlling the OIS module of the camera to adjust the orientation of the camera to at least one orientation. The electronic device may include an operation for capturing an image through the camera in each of the at least one adjusted orientation.

[0017] According to one embodiment of the present disclosure, a system may include an external electronic device including a smartwatch having a display configured to display a live preview image acquired by the electronic device and a camera of the electronic device.

[0018] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments of the present disclosure.

[0019] FIG. 2a illustrates the structure of a camera module according to one embodiment of the present disclosure.

[0020] FIG. 2b illustrates a method for controlling a camera module according to one embodiment of the present disclosure.

[0021] FIG. 3 illustrates a basic arrangement structure of a lens component and an image sensor component of a camera module according to one embodiment of the present disclosure.

[0022] FIG. 4a illustrates position adjustment of a lens component according to one embodiment of the present disclosure.

[0023] FIG. 4b illustrates position adjustment of an image sensor component according to one embodiment of the present disclosure.

[0024] FIG. 5a illustrates angle adjustment of a lens component according to one embodiment of the present disclosure.

[0025] FIG. 5b illustrates angle adjustment of an image sensor component according to one embodiment of the present disclosure.

[0026] FIG. 5c illustrates angle adjustment of a camera module according to one embodiment of the present disclosure.

[0027] FIG. 6 illustrates exemplary compositions for shooting according to one embodiment of the present disclosure.

[0028] FIG. 7 is a flowchart of a remote composition adjustment method according to one embodiment of the present disclosure.

[0029] FIG. 8 is a flowchart of a remote composition adjustment method according to one embodiment of the present disclosure.

[0030] FIG. 9a illustrates an example of a UI for composition adjustment according to one embodiment of the present disclosure.

[0031] FIG. 9b illustrates an example of a UI for composition adjustment according to one embodiment of the present disclosure.

[0032] FIG. 10 illustrates an example of a UI for composition adjustment according to one embodiment of the present disclosure.

[0033] FIG. 11 illustrates an example of a UI for composition adjustment according to one embodiment of the present disclosure.

[0034] FIG. 12 illustrates an example of a UI for composition adjustment according to one embodiment of the present disclosure.

[0035] FIG. 13a illustrates an example of a preview of an image of a desired composition according to one embodiment of the present disclosure.

[0036] FIG. 13b illustrates an example of a position and angle for a camera module for capturing a desired composition according to one embodiment of the present disclosure.

[0037] FIG. 14a illustrates an example of a preview for images with adjusted composition according to one embodiment of the present disclosure.

[0038] FIG. 14b illustrates an example of composition adjustment through position adjustment of a lens component of a camera module according to one embodiment of the present disclosure.

[0039] FIG. 14c illustrates an example of composition adjustment through position adjustment of an image sensor component of a camera module according to one embodiment of the present disclosure.

[0040] FIG. 14d illustrates an example of composition adjustment through angle adjustment of a camera module according to one embodiment of the present disclosure.

[0041] FIG. 15 is a flowchart of an automatic composition adjustment method according to one embodiment of the present disclosure.

[0042] FIG. 16 is a flowchart of an automatic composition adjustment method according to one embodiment of the present disclosure.

[0043] FIG. 17 is a flowchart of an automatic composition adjustment method according to one embodiment of the present disclosure.

[0044] FIG. 18 is a flowchart of an automatic composition adjustment method according to one embodiment of the present disclosure.

[0045] FIG. 19 shows examples of compositions adjusted according to an automatic composition adjustment method according to one embodiment of the present disclosure.

[0046] FIG. 20 illustrates an example of an optimal composition or optimal composition candidates adjusted according to an automatic composition adjustment method according to one embodiment of the present disclosure.

[0047] FIG. 21 is a flowchart of an operation of capturing an image in an adjusted composition of an automatic composition adjustment method according to one embodiment of the present disclosure.

[0048] FIG. 22 illustrates a flowchart of a method for remotely adjusting the composition of an electronic device according to one embodiment of the present disclosure.

[0049] FIG. 23 illustrates a flowchart of an automatic composition adjustment method of an electronic device according to one embodiment of the present disclosure.

[0050] FIG. 24 illustrates a remote control procedure according to one embodiment of the present disclosure.

[0051] FIG. 25 is a flowchart of a method of an electronic device according to one embodiment of the present disclosure.

[0052] FIG. 26 is a flowchart of a method of an electronic device according to one embodiment of the present disclosure.

[0053] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0054] At this time, it will be understood that each block of the processing flow diagrams and combinations of the flow diagrams can be performed by computer program instructions.

[0055] Additionally, each block may represent a module, segment, or portion of code that contains one or more executable instructions for performing a specific logical function(s). It should also be noted that in some alternative implementation examples, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may actually be executed substantially concurrently, or the blocks may sometimes be executed in reverse order, depending on their respective functions.

[0056] Here, the term '~ part' used in this embodiment means software or hardware components such as FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), and the '~ part' performs certain roles. However, the '~ part' is not limited to software or hardware. The '~ part' may be configured to be on an addressable storage medium or may be configured to reproduce one or more packet processing devices. Therefore, as an example, the '~ part' includes components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and '~ parts' may be combined into a smaller number of components and '~ parts' or further separated into additional components and '~ parts'. In addition, the components and '~parts' may be implemented to play one or more central processing units (CPUs) within the device or secure multimedia card. In addition, in the embodiment, the '~parts' may include one or more packet processing devices.

[0057] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments of the present disclosure.

[0058] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0059] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0060] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where artificial intelligence is performed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0061] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0062] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0063] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0064] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0065] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0066] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0067] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0068] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0069] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0070] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0071] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0072] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0073] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0074] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a communication module (192) (e.g., a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0075] The communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The communication module (192) may support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The communication module (192) may support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the communication module (192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0076] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0077] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0078] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface).

[0079] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0080] FIG. 2a illustrates the structure of a camera module according to one embodiment of the present disclosure.

[0081] Referring to FIG. 2A, a camera module (180) (e.g., the camera module (180) of FIG. 1) may include a lens component (181), an image sensor component (182), and / or an actuator component (183). In the present disclosure, the camera module (180) may be referred to as a camera.

[0082] According to one embodiment, the lens component (181) may include at least one lens. In the present disclosure, the lens component (181) may also be referred to as a lens system, a lens element component, or a lens assembly. In the present disclosure, the operation / characteristics of the lens component (181) may be understood as the operation / characteristics of a lens included in the lens component (181).

[0083] According to one embodiment, the image sensor component (182) may include at least one image sensor. The image sensor may be, for example, a complementary metal-oxide semiconductor (CMOS) image sensor or a charge coupled device (CCD) image sensor, but is not limited thereto. In the present disclosure, the image sensor component (182) may also be referred to as a photo sensor component. In the present disclosure, the operation / features of the image sensor component (182) may be understood as the operation / features of an image sensor included in the image sensor component (182).

[0084] In one embodiment, the image sensor component (182) can capture light passing through the lens component (181) and convert the captured light into image signal(s). For example, the image sensor component (182) can capture light refracted through at least one lens of the lens component (181) and convert the captured light into image signal(s). Through this, the image sensor component (182) can capture image(s) of scene(s) and / or subject(s) in front of the camera module (180).

[0085] According to one embodiment, the actuator component (183) may perform control over the camera module (180). For example, the actuator component (183) may perform control over the camera module (180) associated with an image stabilization function. The control over the camera module (180) may include, but is not limited to, movement control over the camera module (180) or optical control over the camera module (180). In the present disclosure, the actuator component (183) may also be referred to as a drive component.

[0086] For example, the actuator component (183) may perform motion control (e.g., mechanical motion control) for the camera module (180). The actuator component (183) may perform motion control for the camera module (180) using, for example, a voice coil motor (VCM) method, but is not limited thereto. For example, the actuator component (183) may perform motion control for the camera module (180) using a driving method such as an encode method or a piezo method.

[0087] According to one embodiment, the actuator component (183) can adjust (or shift) the position of the lens component (181) of the camera module (180) for motion control.

[0088] In one embodiment, the actuator component (183) can move the lens component (181) of the camera module (180) along an optical axis (e.g., the z-axis). In one embodiment, the actuator component (183) can move the image sensor component (182) of the camera module (180) along an optical axis (e.g., the z-axis). Movement of the lens component (181) and / or the image sensor component (182) along the optical axis (e.g., the z-axis) can, for example, provide a focusing adjustment function for at least a portion of the image(s) to be captured.

[0089] In one embodiment, the actuator component (183) can move the lens component (181) of the camera module (180) along one or more axes (e.g., the x-axis and / or the y-axis) substantially perpendicular to the optical axis (e.g., the z-axis). In one embodiment, the actuator component (183) can move the image sensor component (182) of the camera module (180) along one or more axes (e.g., the x-axis and / or the y-axis) substantially perpendicular to the optical axis (e.g., the z-axis). Movement in a direction (e.g., the x-axis or y-axis) substantially perpendicular to the optical axis (e.g., the z-axis) of the lens component (181) and / or the image sensor component (182) may provide, for example, an optical image stabilization (OIS) function, an image stabilization function, or another optical function (e.g., a function for adjusting the orientation of a camera (hereinafter, referred to as a camera orientation adjustment function)) for at least a portion of the image(s) to be captured. In the present disclosure, the orientation of a camera or the camera orientation may refer to the position and / or angle of the camera with respect to a subject being captured. In the present disclosure, the camera orientation may be abbreviated as composition.

[0090] In one embodiment, the actuator component (183) can adjust an angle (e.g., a tilting angle) with respect to the camera module (180) for motion control. For example, the actuator component (183) can adjust an angle (e.g., a tilting angle) of the lens component (181) with respect to the image sensor component (182) of the camera module (180). For example, the actuator component (183) can adjust an angle (e.g., a tilting angle) of the image sensor component (182) with respect to the lens component (181) of the camera module (180). For example, the actuator component (183) can adjust an angle (e.g., a tilting angle) of the camera module (180) itself or of both the lens component (181) and the image sensor component (182) of the camera module (180). Adjustment of the angle (or direction) of the lens component (181) and / or the image sensor component (182) (or the camera module (180) itself) may provide, for example, perspective, depth, OIS functionality, image stabilization functionality, or other optical functionality (e.g., camera framing functionality) for at least a portion of the image to be captured.

[0091] In one example, the actuator component (183) may perform optical control for the camera module (180). The actuator component (183) may perform optical control for the camera module (180) using, for example, an optical microelectromechanical system (MEMS), but is not limited thereto.

[0092] In one embodiment, the actuator component (183) can adjust the shape or optical properties of at least one lens (e.g., a flexible lens) included in the lens component (181) of the camera module (180) for optical control. Adjusting the shape or optical properties of the lens component (181) can provide, for example, a focusing adjustment function, an OIS function, an image stabilization function, a perspective adjustment function, a depth adjustment function, or other optical functions (e.g., a camera composition adjustment function) for at least a portion of an image to be captured. Through such optical control, all or part of the optical functions / effects that can be obtained through the motion control described above may also be obtained.

[0093] Hereinafter, for convenience of explanation, various embodiments of the present disclosure will be described by taking as an example that control of the camera module (180) corresponds to movement control (e.g., position and / or angle control). However, as described above, since all or part of the optical functions / effects (e.g., OIS function, image stabilization function, camera composition adjustment function, etc.) that can be obtained through movement control of the camera module (180) can be obtained through optical control of the camera module (180), it will be apparent to those skilled in the art that the description of various embodiments related to movement control of the camera module (180) below can be equally or similarly applied to optical control of the camera module (180).

[0094] FIG. 2b illustrates a method for controlling a camera module according to one embodiment of the present disclosure.

[0095] Referring to FIG. 2b, the camera module (180) can be controlled by the controller (210).

[0096] According to one embodiment, the controller (210) may provide control functions for the camera module (180). The controller (210) may be implemented by, for example, one or more processors (e.g., the processor (120) of FIG. 1) or at least a portion of one or more processors. The controller (210) or processor may include, but is not limited to, an AP, an image processor, a camera IC, an OIS IC, an OIS micro control unit (MCU), and / or an OIS driver IC.

[0097] In one embodiment, the controller (210) can direct the actuator component (183) of the camera module (180) to control the camera module (180). For example, the controller (210) can direct the actuator component (183) to perform motion control for the camera module (180). For example, the controller (210) can direct the actuator component (183) to perform optical control for the camera module (180).

[0098] According to one embodiment, to instruct the actuator component (183) to control the camera module (180), the controller (210) may use the input data to generate control output data and transmit the control output data to the actuator component (183).

[0099] According to one embodiment, the input data may include, for example, sensor data related to movement of an electronic device (e.g., electronic device (101) of FIG. 1) (e.g., inertial measurement unit (IMU) sensor data, accelerometer sensor data, gyroscope sensor data), user input data (e.g., user input data via a user interface (UI)), auto focus (AF)-related input data, input data related to camera adjustment (e.g., a request for camera adjustment), input data related to camera composition adjustment (e.g., a (remote) composition adjustment request, an automatic composition adjustment request)), image stabilization-related input data, and / or OIS-related input data.

[0100] According to one embodiment, the input data may be data generated by an electronic device (e.g., electronic device (101) of FIG. 1) including a controller (210) or data received from an external electronic device (e.g., electronic device (102) of FIG. 1).

[0101] In one embodiment, in a general OIS mode (e.g., an OIS mode that compensates for hand tremor), the input data may be sensor data related to the movement of the electronic device. In this case, the controller (210) may use the sensor data to generate control output data and transmit the control output data to the actuator component (183).

[0102] In one embodiment, in the automatic composition adjustment mode, the input data may be an automatic composition adjustment request generated by the electronic device. In this case, the controller (210) may generate control output data using the automatic composition adjustment request and transmit the control output data to the actuator component (183). For example, whenever an automatic composition adjustment request is acquired (or identified), the controller (210) may generate control output data corresponding to the automatic composition adjustment request and transmit the control output data to the actuator component (183).

[0103] According to one embodiment, in the remote framing adjustment mode, the input data may be a framing adjustment request (remote framing adjustment request) generated and received by an external electronic device, or generated by the electronic device. In this case, the controller (210) may generate control output data using the remote framing adjustment request and transmit the control output data to the actuator component (183). For example, whenever a remote framing adjustment request is acquired (or identified), the controller (210) may generate control output data corresponding to the remote framing adjustment request and transmit the control output data to the actuator component (183).

[0104] According to one embodiment, the controller (210) can receive data (image data) of image signal(s) acquired by the image sensor component (182) (e.g., data of a captured image, data of a preview image) from the image sensor component (182). The controller (210) can transmit the received image data to another configuration.

[0105] According to one embodiment, when there are multiple camera modules (180), the controller (210) may select one of the multiple camera modules to perform image stabilization control according to a preset method. For example, when a wide angle is required, the controller (210) may determine a camera module for a wide angle among the multiple camera modules as the camera module for image stabilization control.

[0106] In one embodiment, the image stabilization control may be hardware-based image stabilization control. For example, the image stabilization control may be hardware-based control (e.g., motion control or optical control) of a camera module associated with the image stabilization function.

[0107] In one embodiment, the image stabilization control may be a hardware-based image stabilization control utilizing physical / mechanical movements (e.g., position adjustment and / or angle adjustment). For example, the image stabilization control may be a movement control for a camera module associated with the image stabilization function. For example, the image stabilization control may be a control for position adjustment and / or angle adjustment of a camera module associated with the image stabilization function, a lens component included in the camera module, and / or an image sensor component included in the camera module.

[0108] In one embodiment, the image stabilization control may be OIS control. For example, the image stabilization control may be control for position adjustment and / or angle adjustment of a camera module associated with the OIS function. In one embodiment, the OIS control may be performed by the OIS module. In one embodiment, the OIS module may include an actuator component (183).

[0109] FIG. 3 illustrates a basic arrangement structure of a lens component and an image sensor component of a camera module according to one embodiment of the present disclosure.

[0110] Referring to FIG. 3, in a default arrangement structure, the lens component (181) (or lens) and the image sensor component (182) (or image sensor) of the camera module (180) may be arranged on a plane substantially perpendicular to the optical axis. For example, as illustrated, when the optical axis is the z-axis, a plane including the lens component (181) (hereinafter, lens plane) and a plane including the image sensor component (182) (hereinafter, image sensor plane) may be positioned on a plane parallel to the xy plane having different z values, respectively. In one embodiment, the optical axis may be defined by the lens component (181). In the present disclosure, the default arrangement structure may also be referred to as an initial arrangement structure.

[0111] According to one embodiment, in the basic arrangement structure, the center position of the lens component (181) and the center position of the image sensor component (182) can be located on the optical axis.

[0112] According to one embodiment, the positions and angles of the lens component (181) and the image sensor component (182) illustrated in FIG. 3 may correspond to initial positions and angles of the lens component (181) and the image sensor component (182). The initial positions and angles may be, for example, positions and angles to which the lens component (181) and the image sensor component (182) return after a procedure related to remote camera adjustment (or remote camera framing adjustment) or a procedure related to automatic camera adjustment (or automatic camera framing adjustment) is finally completed. In the present disclosure, remote camera framing adjustment may be referred to as remote framing adjustment, and automatic camera framing adjustment may be referred to as automatic framing adjustment.

[0113] Hereinafter, for convenience of explanation, embodiments of the present disclosure are described assuming that the optical axis is the z-axis.

[0114] FIG. 4a illustrates position adjustment of a lens component according to one embodiment of the present disclosure.

[0115] Referring to FIG. 4A, the position of the lens component (181) (or lens) of the camera module can be adjusted along at least one axis (e.g., the x-axis and / or the y-axis) that is substantially perpendicular to the optical axis (z-axis). For example, the position of the lens component (181) can be moved (or shifted) by a preset distance along the direction of the +x-axis, the -x-axis, the +y-axis and / or the -y-axis. Such position adjustment of the lens component (181) can provide, for example, an OIS function, an image stabilization function, or other optical functions (e.g., a camera composition adjustment function) for at least a portion of the image(s) to be captured. For example, when the position of the lens component (181) is adjusted while the position of the image sensor component (182) is fixed, the camera composition for capturing can be adjusted.

[0116] According to one embodiment, the movement of the lens component (181) may be controlled by an actuator component of the camera module (e.g., actuator component (183) of FIGS. 2a / b). For example, the movement of the lens component (181) may be controlled by the actuator component of the camera module under instructions from a controller (e.g., controller (210) of FIG. 2b).

[0117] In one embodiment, the controller can control the actuator component to adjust the position of the lens component (181) based on a request (e.g., a composition adjustment request or an automatic composition adjustment request) (e.g., a remote composition adjustment / automatic composition adjustment case). In one embodiment, the controller can control the actuator component to adjust the position of the lens component (181) based on sensor data related to the movement of the electronic device (a general OIS case).

[0118] In the present disclosure, a distance by which a position is adjusted (or moved) through a single control may be referred to as one step distance (or one step). In the present disclosure, shifting of a lens component may be referred to as lens shift, lens shift OIS, or lens shift IS.

[0119] FIG. 4b illustrates position adjustment of an image sensor component according to one embodiment of the present disclosure.

[0120] Referring to FIG. 4B, the position of the image sensor component (182) (or image sensor) of the camera module (180) can be adjusted along at least one axis (e.g., the x-axis and / or the y-axis) that is substantially perpendicular to the optical axis (z-axis). For example, the position of the image sensor component (182) can be moved (or shifted) by a preset distance along the direction of the +x-axis, the -x-axis, the +y-axis and / or the -y-axis. Such position adjustment of the lens component (181) can provide, for example, an OIS function, an image stabilization function, or other optical functions (e.g., a camera composition adjustment function) for at least a portion of the image(s) to be captured. For example, when the position of the image sensor component (182) is adjusted while the position of the lens component (181) is fixed, the camera composition for capturing can be adjusted.

[0121] According to one embodiment, the movement of the image sensor component (182) may be controlled by an actuator component of the camera module (e.g., actuator component (183) of FIGS. 2a / b). For example, the movement of the image sensor component (182) may be controlled by the actuator component of the camera module under instructions from a controller (e.g., controller (210) of FIG. 2b).

[0122] In one embodiment, the controller may control the actuator component to adjust the position of the image sensor component (182) based on a request (e.g., a framing adjustment request or an automatic framing adjustment request) (e.g., a remote framing adjustment / automatic framing adjustment case). In one embodiment, the controller may control the actuator component to adjust the position of the image sensor component (182) based on sensor data related to the movement of the electronic device (a general OIS case).

[0123] In the present disclosure, a distance by which a position is adjusted (or moved) through a single control may be referred to as a one-step distance (or one step). In the present disclosure, image sensor component shifting may be referred to as a sensor shift, sensor shift OIS, or sensor shift IS.

[0124] FIG. 5a illustrates angle adjustment of a lens component according to one embodiment of the present disclosure.

[0125] Referring to FIG. 5A, the angle of the lens component (181) (or lens) of the camera module can be adjusted around at least one axis substantially perpendicular to the optical axis (e.g., the z-axis), or a plane substantially perpendicular to the optical axis (e.g., the z-axis). For example, the lens component (181) can be tilted by a preset angle around the x-axis, the y-axis, or the xy-plane. Such angle adjustment of the lens component (181) can provide, for example, a perspective adjustment function, a depth adjustment function, an OIS function, an image stabilization function, or other optical functions (e.g., a camera composition adjustment function) for at least a portion of the image(s) to be captured. For example, when the angle of the lens component (181) is adjusted while the angle of the image sensor component (182) is fixed, the camera composition for capturing can be adjusted.

[0126] According to one embodiment, the tilting of the lens component (181) may be controlled by an actuator component of the camera module (e.g., actuator component (183) of FIGS. 2a / b). For example, the tilting of the lens component (181) may be controlled by the actuator component of the camera module under instructions from a controller (e.g., controller (210) of FIG. 2b).

[0127] In one embodiment, the controller may control the actuator component to adjust the angle of the lens component (181) based on a request (e.g., a request for framing adjustment or an automatic framing adjustment request) (e.g., a remote framing adjustment / automatic framing adjustment case). In one embodiment, the controller may control the actuator component to adjust the angle of the lens component (181) based on sensor data related to the movement of the electronic device (a general OIS case).

[0128] In the present disclosure, an angle adjusted (or tilted) through one control may be referred to as one step angle (or one step).

[0129] FIG. 5b illustrates angle adjustment of an image sensor component according to one embodiment of the present disclosure.

[0130] Referring to FIG. 5B, the angle of the image sensor component (182) (or image sensor) of the camera module can be adjusted about at least one axis substantially perpendicular to the optical axis (e.g., the z-axis) or a plane substantially perpendicular to the optical axis (e.g., the z-axis). For example, the image sensor component (182) can be tilted by a preset angle about the x-axis, the y-axis, or the xy-plane. Such angle adjustment of the image sensor component (182) can provide, for example, a perspective adjustment function, a depth adjustment function, an OIS function, an image stabilization function, or other optical functions (e.g., a camera composition adjustment function) for at least a portion of the image(s) to be captured. For example, when the angle of the image sensor component (182) is adjusted while the angle of the lens component (181) is fixed, the camera composition for capturing can be adjusted.

[0131] In one embodiment, the tilting of the image sensor component (182) may be controlled by an actuator component of the camera module (e.g., actuator component (183) of FIGS. 2a / b). For example, the tilting of the image sensor component (182) may be controlled by the actuator component of the camera module under instructions from a controller (e.g., controller (210) of FIG. 2b).

[0132] In one embodiment, the controller can control the actuator component to adjust the angle of the image sensor component (182) based on a request (e.g., a framing adjustment request or an automatic framing adjustment request) (e.g., a remote framing adjustment / automatic framing adjustment case). In one embodiment, the controller can control the actuator component to adjust the angle of the image sensor component (182) based on sensor data related to the movement of the electronic device (a general OIS case).

[0133] In the present disclosure, an angle adjusted (or tilted) through one control may be referred to as one step angle (or one step).

[0134] FIG. 5c illustrates angle adjustment of a camera module according to one embodiment of the present disclosure.

[0135] Referring to FIG. 5C, the angle of the camera module (180) can be adjusted around at least one axis substantially perpendicular to the optical axis (e.g., the z-axis) or a plane substantially perpendicular to the optical axis (e.g., the z-axis). For example, the camera module (180) can be tilted by a preset angle around the x-axis, the y-axis, or the xy-plane. Such angle adjustment of the camera module (180) can provide, for example, a perspective adjustment function, a depth adjustment function, an OIS function, an image stabilization function, or other optical functions (e.g., a camera composition adjustment function) for at least a portion of the image(s) to be captured. For example, when the angle of the camera module (180) itself is adjusted in the electronic device state, the camera composition for taking a picture can be adjusted.

[0136] According to one embodiment, tilting of the camera module (180) may be implemented by tilting the lens component (181) and the image sensor component (182) included in the camera module (180) at the same angle, but is not limited thereto.

[0137] According to one embodiment, the tilting of the camera module (180) may be controlled by an actuator component of the camera module (180) (e.g., actuator component (183) of FIGS. 2a / b). For example, the tilting of the camera module (180) may be controlled by the actuator component of the camera module (180) according to instructions from a controller (e.g., controller (210) of FIG. 2b).

[0138] In one embodiment, the controller can control the actuator component for adjusting the angle of the camera module (180) (e.g., the lens component (181) and the image sensor component (182)) based on a request (e.g., a framing adjustment request or an automatic framing adjustment request) (e.g., a remote framing adjustment / automatic framing adjustment case). In one embodiment, the controller can control the actuator component for adjusting the angle of the lens component (181) and the image sensor component (182) based on sensor data related to the movement of the electronic device (a general OIS case).

[0139] In the present disclosure, an angle adjusted (or tilted) through one control may be referred to as one step angle (or one step).

[0140] FIG. 6 illustrates exemplary compositions for shooting according to one embodiment of the present disclosure.

[0141] According to one embodiment, the composition may be associated with a visual effect determined by the position and / or angle of a camera module of the electronic device (101) (e.g., the camera module (180) of FIGS. 1 and 2a / b). According to one embodiment, the composition may be associated with at least one of, but not limited to, the position, size, arrangement, perspective, or ratio of at least one subject to the background within the image.

[0142] According to one embodiment, the adjustment of the composition may correspond to the adjustment of the position and / or angle of the camera module. Through such adjustment of the composition, for example, at least one of the position, size, perspective, or ratio of at least one subject within the image may be adjusted.

[0143] Part (a) of Fig. 6 shows an example of an image of a composition that a user wants to capture.

[0144] Referring to part (a) of FIG. 6, at least one subject (601) (e.g., a person) may be positioned in front of the camera module of the electronic device (101). In this case, the image of the desired composition may be, for example, an image of the same composition as that displayed on the first screen (610a) of the electronic device (101).

[0145] Part (b) of Fig. 6 shows an example of an image with a composition different from the desired composition.

[0146] Referring to part (b) of FIG. 6, for example, when the angle of the camera module of the electronic device (101) is higher than the angle for capturing an image of a desired composition, a preview having an image of the same composition as that displayed on the second screen (610b) of the electronic device (101) may be provided to the user. As illustrated, the position of the subject in the image of the composition displayed on the second screen (610b) is positioned lower than the position of the subject in the image of the desired composition displayed on the first screen (610a). In this case, the image is less stable than the image of the desired composition.

[0147] Part (c) of Fig. 6 shows an example of an image with a composition different from the desired composition.

[0148] Referring to part (c) of FIG. 6, for example, when the angle of the camera module of the electronic device (101) is directed to the right from the angle for capturing an image of a desired composition, a preview having an image of the same composition as that displayed on the third screen (610c) of the electronic device (101) may be provided to the user. As illustrated, the position of the subject in the image of the composition displayed on the third screen (610c) is shifted to the left compared to the position of the subject in the image of the desired composition displayed on the first screen (610a). In this case, the image is less stable than the image of the desired composition.

[0149] In this way, if the current composition is different from the desired composition, the user can adjust the current composition to the desired composition. For example, if the user is located at a close distance from the electronic device (101), the user can adjust the composition to the desired composition by directly manually adjusting the position and / or angle of the camera module of the electronic device (101). However, if the user is located at a long distance from the electronic device (101), it may be inconvenient for the user to move to the location where the electronic device (101) is placed in order to directly adjust the position and / or angle of the camera module of the electronic device (101). In addition, even if the user is located at a close distance from the electronic device (101) (e.g., the user is holding the electronic device (101) in his / her hand), it may be difficult to adjust the composition to the desired composition (or, an optimized composition) when manually adjusting the composition. For example, if the user is a person who does not have a good sense for taking pictures (e.g., a child), it may be difficult to accurately adjust the composition to the desired composition through manual composition adjustment.

[0150] Therefore, for the convenience of users, it is necessary to consider a method for adjusting the camera or composition remotely (remote control or remote composition adjustment) or automatically adjusting the camera or composition (auto adjustment or automatic composition adjustment).

[0151] FIG. 7 is a flowchart of a remote composition adjustment method according to one embodiment of the present disclosure.

[0152] According to one embodiment, a first electronic device (e.g., electronic device (101) of FIG. 1) may be, for example, an electronic device that provides a photographing function. According to one embodiment, a second electronic device (e.g., electronic device (102) of FIG. 1) may be, for example, an electronic device that provides a function for controlling a photographing function of the first electronic device. The second electronic device may be, for example, a wearable device such as a smartwatch or a smart phone, but is not limited thereto. In the present disclosure, the first electronic device may be referred to as an electronic device, and the second electronic device may be referred to as an external electronic device.

[0153] Referring to FIG. 7, in operation 7010, a first electronic device may transmit first preview data for a preview of an image to be captured to a second electronic device. In one embodiment, the first preview data may be a portion of a real-time preview video (or, live preview video).

[0154] In operation 7020, the second electronic device may provide a first preview (or a first preview image) based on the first preview data. For example, the second electronic device may display the first preview on the screen of the second electronic device. The first preview may be, for example, a preview such as that displayed on the first screen (1410a) of part (a) of FIG. 14, but is not limited thereto. The first preview may be used by the user to determine whether the current composition is a desired composition. The desired composition may be, for example, a composition having a position of a subject (1301) such as that displayed on the screen (1310) of FIG. 13a, but is not limited thereto.

[0155] In operation 7030, the second electronic device may identify a request (e.g., a request to adjust the composition for a photograph (hereinafter, referred to as a composition adjustment request)). In one embodiment, the request may correspond to a user input obtained (or received) while the first preview is displayed on the screen. In one embodiment, the user input may be associated with the first preview.

[0156] In one embodiment, the second electronic device may identify (or generate) a request (e.g., a composition adjustment request) generated based on a user input. For example, the second electronic device may identify (or generate) a composition adjustment request generated based on a user input via a UI (e.g., the UI of FIGS. 9A, 9B, 10, 11, and 12). The user input may be obtained (or received) while the first preview is displayed on the screen, for example.

[0157] For example, as illustrated in FIG. 9A, when a touch input is obtained for sliding a screen on which a preview (e.g., a first preview) is displayed in a first direction, the second electronic device may identify (or generate) a request (e.g., a composition adjustment request) corresponding to the first direction. According to one embodiment, the first direction may be, but is not limited to, an up, down, left, or right direction. The corresponding touch input may be obtained (or received) while, for example, the first preview is displayed on the screen.

[0158] For example, as illustrated in FIG. 9B, when a gesture input moving in a first direction (e.g., a gesture input moving in the first direction with an open palm) is obtained, the second electronic device may identify (or generate) a request (e.g., a composition adjustment request) corresponding to the first direction. According to one embodiment, the second electronic device may obtain a gesture input moving in the first direction through motion recognition. The gesture input may be obtained (or received) while the first preview is displayed on the screen, for example.

[0159] For example, as illustrated in part (a) of FIG. 10, when a touch input moving in a first rotational direction along a bezel (e.g., a touch bezel) is obtained, the second electronic device may identify (or generate) a request (e.g., a composition adjustment request) corresponding to the first rotational direction. According to one embodiment, the first rotational direction may be clockwise or counterclockwise, but is not limited thereto. The corresponding touch input may be obtained (or received) while the first preview is displayed on the screen, for example.

[0160] For example, as illustrated in part (b) of FIG. 10, when a user input is obtained for rotating a bezel (e.g., a rotating bezel) in a first rotational direction, the second electronic device may identify (or generate) a request (e.g., a composition adjustment request) corresponding to the first rotational direction. The user input may be obtained (or received) while the first preview is displayed on the screen, for example.

[0161] For example, as illustrated in parts (a) and (b) of FIG. 11, when a first user input for selecting a control direction (e.g., left-right direction or up-down direction) and a second user input for moving the bezel in the first rotational direction (or turning the bezel in the first rotational direction) are obtained, the second electronic device can identify a composition adjustment request corresponding to the control direction and the first rotational direction. The first user input and the second user input can be obtained (or received) while, for example, the first preview is displayed on the screen.

[0162] For example, as illustrated in FIG. 12, when a user input is obtained for selecting (e.g., touching) a preset area on a screen where a preview (e.g., a first preview) is displayed, the second electronic device may identify a composition adjustment request corresponding to the preset area. According to one embodiment, the preset area may be a first area (1210) located on the left side of the screen, a second area (1220) located on the upper side of the screen, a third area (1230) located on the right side of the screen, or a fourth area (1240) located on the lower side of the screen. The corresponding user input may be obtained (or received) while the first preview is displayed on the screen, for example.

[0163] At operation 7040, the second electronic device may transmit a request (e.g., a composition adjustment request) to the first electronic device.

[0164] In operation 7050, the first electronic device may perform image stabilization control based on a request (e.g., a composition adjustment request) to adjust the composition for shooting.

[0165] In one embodiment, the image stabilization control may be hardware-based image stabilization control. For example, the image stabilization control may be hardware-based control (e.g., motion control or optical control) of a camera module associated with the image stabilization function.

[0166] In one embodiment, the image stabilization control may be a hardware-based image stabilization control utilizing physical / mechanical movements (e.g., position adjustment and / or angle adjustment). For example, the image stabilization control may be a movement control for a camera module associated with the image stabilization function. For example, the image stabilization control may be a control for position adjustment and / or angle adjustment of a camera module associated with the image stabilization function, a lens component included in the camera module, and / or an image sensor component included in the camera module.

[0167] In one embodiment, the image stabilization control may be OIS control. For example, the image stabilization control may be control for position adjustment and / or angle adjustment of a camera module associated with the OIS function. In one embodiment, the OIS control may be performed by the OIS module. In one embodiment, the OIS module may include an actuator component.

[0168] For example, if the request is a request corresponding to a first direction (e.g., a request for framing adjustment), the first electronic device may perform control on a camera module associated with the first direction using image stabilization control.

[0169] For example, if the request is a request corresponding to a first rotation direction (e.g., a request for framing adjustment), the first electronic device may perform control on a camera module associated with the first rotation direction using image stabilization control.

[0170] For example, if the request is a request corresponding to a control direction (e.g., a first direction) and a first rotation direction (e.g., a composition adjustment request), the first electronic device may perform control on a camera module associated with the control direction and the first rotation direction using image stabilization control.

[0171] For example, if the request corresponds to a preset area (e.g., a request for framing adjustment), the first electronic device may perform control on a camera module associated with the preset area using image stabilization control.

[0172] In operation 7060, the first electronic device may transmit second preview data for a preview of an image associated with the adjusted composition to the second electronic device. In one embodiment, the second preview data may be a portion of a real-time preview video (or, live preview video).

[0173] In operation 7070, the second electronic device may provide a second preview (or a second preview image) based on the second preview data. For example, the second electronic device may display the second preview on the screen of the second electronic device. The second preview may be, for example, a preview such as that displayed on the second screen (1410b) of part (b) of FIG. 14, but is not limited thereto. The second preview may be used by the user to determine whether the adjusted composition is the desired composition.

[0174] In operation 7080, the second electronic device may identify a request (e.g., a request to adjust the composition or a request to take a picture). In one embodiment, the request may correspond to a user input obtained (or received) while the first preview was displayed on the screen. In one embodiment, the user input may be associated with the second preview.

[0175] For example, when a composition adjustment request is identified (A), operations 7040 to 7070 may be performed based on the identified composition adjustment request. For example, when the adjusted composition is not a desired composition, the user may provide user input for additional composition adjustment (e.g., user input via the UI of FIGS. 9A, 9B, 10, 11, and 12). In this case, the second electronic device may identify (or generate) a composition adjustment request (a second composition adjustment request) based on the user input for additional composition adjustment. Thereafter, the second electronic device may transmit the second composition adjustment request to the first electronic device. In this case, the first electronic device may perform image stabilization control based on the second composition adjustment request to adjust the composition for shooting. Thereafter, the first electronic device may transmit preview data (third preview data) for an image associated with the adjusted composition to the second electronic device. According to one embodiment, the third preview data may be a portion of a real-time preview video (or, live preview video). In this case, the second electronic device may provide a preview (third preview) based on the third preview data. The third preview may be, for example, a preview such as that displayed on the third screen (1410c) of part (c) of FIG. 14, but is not limited thereto. The third preview may be used by the user to determine whether the adjusted composition is a desired composition. Meanwhile, the operations related to the above-described composition adjustment (e.g., operations 7040 to 7070) may be repeatedly performed until the composition is adjusted to the desired composition.

[0176] For example, if a shooting request is identified (B), operation 7090 may be performed. In operation 7090, the second electronic device may transmit the shooting request to the first electronic device. In one embodiment, the second electronic device may identify (or generate) the shooting request based on a user input. For example, the second electronic device may identify (or generate) the shooting request when a user input (e.g., a touch input) is received for a shooting button (e.g., a touch button).

[0177] In operation 7100, the first electronic device can capture an image at an adjusted composition based on a shooting request.

[0178] In operation 7110, the first electronic device can transmit data of a captured image (captured image data) to the second electronic device.

[0179] FIG. 8 is a flowchart of a remote composition adjustment method according to one embodiment of the present disclosure.

[0180] According to one embodiment, a first electronic device (e.g., electronic device (101) of FIG. 1) may be, for example, an electronic device that provides a photographing function. According to one embodiment, a second electronic device (e.g., electronic device (102) of FIG. 1) may be, for example, an electronic device that provides a function for controlling a photographing function of the first electronic device. The second electronic device may be, for example, a wearable device such as a smartwatch or a smart phone, but is not limited thereto. In the present disclosure, the first electronic device may be referred to as an electronic device, and the second electronic device may be referred to as an external electronic device.

[0181] Unlike the embodiment of FIG. 7, in the embodiment of FIG. 8, even if the first electronic device does not receive a request (e.g., a composition adjustment request) from the second electronic device, the first electronic device can directly identify the request (e.g., a composition adjustment request). For example, if the user input for composition adjustment is a gesture input through the UI of FIG. 9b, for example, the first electronic device can obtain the gesture input through the motion recognition function of the first electronic device. In this case, the first electronic device can directly identify (or generate) a request (e.g., a composition adjustment request) corresponding to a first direction associated with the gesture input based on the gesture input.

[0182] Referring to FIG. 8, in operation 8010, a first electronic device may transmit first preview data for a preview of an image to be captured to a second electronic device. In one embodiment, the first preview data may be a portion of a real-time preview video (or, live preview video).

[0183] In operation 8020, the second electronic device may provide a first preview (or a first preview image) based on the first preview data. For example, the second electronic device may display the first preview on the screen of the second electronic device. The first preview may be, for example, a preview such as that displayed on the first screen (1410a) of part (a) of FIG. 14, but is not limited thereto. The first preview may be used by the user to determine whether the current composition is a desired composition. The desired composition may be, for example, a composition having a position of a subject (1301) such as that displayed on the screen (1310) of FIG. 13a, but is not limited thereto.

[0184] In operation 8030, the first electronic device may identify a request (e.g., a request to adjust the composition for a photograph (hereinafter, referred to as a composition adjustment request)). In one embodiment, the request may correspond to a user input obtained (or received) while the first preview is displayed on the screen. In one embodiment, the user input may be associated with the first preview.

[0185] For example, as illustrated in FIG. 9B, when a gesture input moving in a first direction (e.g., a gesture input moving in the first direction with an open palm) is obtained, the first electronic device may identify (or generate) a composition adjustment request corresponding to the first direction. According to one embodiment, the first direction may be, for example, an up, down, left, or right direction, but is not limited thereto. According to one embodiment, the first electronic device may obtain a gesture input moving in the first direction through motion recognition.

[0186] For example, a first electronic device may receive a request (e.g., a request for adjusting a composition) from a third electronic device that is different from the second electronic device. The third electronic device may be, for example, an electronic device that is linked to the first electronic device and / or the second electronic device. For example, the first electronic device may be an electronic device (e.g., a smart phone) that provides a photographing function, the second electronic device may be an electronic device (e.g., a smart phone) that provides a preview function, and the third electronic device may be an electronic device (e.g., a smart watch) that is linked to the first and second electronic devices and provides a photographing control function. In one embodiment, the request received from the third electronic device may correspond to a user input obtained (or received) while the first preview is displayed on the screen of the second electronic device and / or the third electronic device. In one embodiment, the user input may be associated with the first preview.

[0187] In operation 8040, the first electronic device may perform image stabilization control based on a request (e.g., a composition adjustment request) to adjust the composition for shooting.

[0188] In one embodiment, the image stabilization control may be hardware-based image stabilization control. For example, the image stabilization control may be hardware-based control (e.g., motion control or optical control) of a camera module associated with the image stabilization function.

[0189] In one embodiment, the image stabilization control may be a hardware-based image stabilization control utilizing physical / mechanical movements (e.g., position adjustment and / or angle adjustment). For example, the image stabilization control may be a movement control for a camera module associated with the image stabilization function. For example, the image stabilization control may be a control for position adjustment and / or angle adjustment of a camera module associated with the image stabilization function, a lens component included in the camera module, and / or an image sensor component included in the camera module.

[0190] In one embodiment, the image stabilization control may be OIS control. For example, the image stabilization control may be control for position adjustment and / or angle adjustment of a camera module associated with the OIS function. In one embodiment, the OIS control may be performed by the OIS module. In one embodiment, the OIS module may include an actuator component.

[0191] For example, if the request is a request corresponding to a first direction (e.g., a request for framing adjustment), the first electronic device may perform control on a camera module associated with the first direction using image stabilization control.

[0192] In operation 8050, the first electronic device may transmit second preview data for a preview of an image associated with the adjusted composition to the second electronic device. In one embodiment, the second preview data may be a portion of a real-time preview video (or, live preview video).

[0193] In operation 8060, the second electronic device may provide a second preview (or a second preview image) based on the second preview data. For example, the second electronic device may display the second preview on the screen of the second electronic device. The second preview may be, for example, a preview such as that displayed on the second screen (1410b) of part (b) of FIG. 14, but is not limited thereto. The second preview may be used by the user to determine whether the adjusted composition is the desired composition.

[0194] At operation 8070, the first electronic device may identify a request (e.g., a framing request or a shooting request).

[0195] For example, when a composition adjustment request is identified (A), operations 8040 to 8060 may be performed based on the identified composition adjustment request. For example, if the adjusted composition is not a desired composition, the user may provide user input for additional composition adjustment (e.g., user input via the UI of FIG. 9B). In this case, the first electronic device may identify (or generate) a composition adjustment request (second composition adjustment request) based on the user input for additional composition adjustment. Thereafter, the first electronic device may perform image stabilization control based on the second composition adjustment request to adjust the composition for shooting. Thereafter, the first electronic device may transmit preview data (third preview data) for an image associated with the adjusted composition to the second electronic device. In this case, the second electronic device may provide a preview (third preview) based on the third preview data. According to one embodiment, the third preview data may be a portion of a real-time preview video (or, live preview video). The third preview may be, for example, a preview such as that shown in the third screen (1410c) of part (c) of FIG. 14, but is not limited thereto. The third preview may be used by the user to determine whether the adjusted composition is the desired composition. Meanwhile, the operations related to the aforementioned composition adjustment (e.g., operations 8040 to 8060) may be repeatedly performed until the composition is adjusted to the desired composition.

[0196] For example, if a shooting request is identified (B), operation 8080 may be performed. In operation 8080, the first electronic device may capture an image at an adjusted angle based on the shooting request.

[0197] In operation 8090, the first electronic device can transmit data of the captured image (captured image data) to the second electronic device.

[0198] FIG. 9a illustrates an example of a UI for composition adjustment according to one embodiment of the present disclosure.

[0199] Referring to FIG. 9A, an electronic device (e.g., the second electronic device of FIGS. 7 and 8 ) may obtain user input via a UI for framing adjustment (or camera adjustment) and identify (or generate) a framing adjustment request based on the user input. In one embodiment, the user input may be obtained (or received) while a preview is displayed on the screen. In one embodiment, the user input may be associated with the preview.

[0200] According to one embodiment, an electronic device may obtain a touch input (hereinafter, referred to as a slide input) that slides a screen on which a preview (e.g., a first preview) is displayed in a first direction. According to one embodiment, when a slide input in the first direction is obtained, the electronic device may identify (or generate) a composition adjustment request corresponding to the first direction.

[0201] The first direction may be, for example, up, down, left, or right, as illustrated in FIG. 9A, but is not limited thereto. When the first direction is the up direction (910a), a composition adjustment request corresponding to the first direction may be, for example, a composition adjustment request for moving the position of the subject on the preview downward by a preset distance (e.g., 1 step). When the first direction is the down direction (920a), a composition adjustment request corresponding to the first direction may be, for example, a composition adjustment request for moving the position of the subject on the preview upward by a preset distance (e.g., 1 step). For example, through a composition adjustment request corresponding to the down direction (920a), the composition may be adjusted from the composition of part (a) of FIG. 14A to the composition of part (b) of FIG. 14A. When the first direction is the left direction (930a), the composition adjustment request corresponding to the first direction may be, for example, a composition adjustment request to move the position of the subject on the preview by a preset distance (e.g., 1 step) in the right direction. When the first direction is the right direction (940a), the composition adjustment request corresponding to the first direction may be, for example, a composition adjustment request to move the position of the subject on the preview by a preset distance (e.g., 1 step) in the left direction. Meanwhile, even if the composition adjustment request corresponding to the first direction is identified, if the driving range is exceeded, the electronic device may provide the user with information notifying that the driving range has been exceeded (e.g., a notification that 'further movement is difficult').

[0202] FIG. 9b illustrates an example of a UI for composition adjustment according to one embodiment of the present disclosure.

[0203] Referring to FIG. 9B, a first electronic device (e.g., the first electronic device of FIG. 8) or a second electronic device (e.g., the second electronic device of FIGS. 7 and 8) may obtain user input through a UI for framing adjustment (or camera adjustment) and identify a framing adjustment request based on the user input. In one embodiment, the user input may be obtained (or received) while a preview is displayed on the screen. In one embodiment, the user input may be associated with the preview.

[0204] According to one embodiment, the first electronic device or the second electronic device may obtain a gesture input moving in a first direction (e.g., a gesture input moving in the first direction with an open palm). According to one embodiment, when a gesture input moving in the first direction is obtained, the first electronic device or the second electronic device may identify a composition adjustment request corresponding to the first direction. According to one embodiment, the first electronic device or the second electronic device may obtain a gesture input moving in the first direction through motion recognition.

[0205] The first direction may be, for example, up, down, left, or right, as illustrated in FIG. 9b, but is not limited thereto. When the first direction is the up direction (910b), the composition adjustment request corresponding to the first direction may be, for example, a composition adjustment request for moving the position of the subject on the preview downward by a preset distance (e.g., 1 step). When the first direction is the down direction (920b), the composition adjustment request corresponding to the first direction may be, for example, a composition adjustment request for moving the position of the subject on the preview upward by a preset distance (e.g., 1 step). For example, through the composition adjustment request corresponding to the down direction (920b), the composition may be adjusted from the composition of part (a) of FIG. 14a to the composition of part (b) of FIG. 14a. When the first direction is the left direction (930b), the composition adjustment request corresponding to the first direction may be, for example, a composition adjustment request to move the position of the subject on the preview by a preset distance (e.g., 1 step) in the right direction. When the first direction is the right direction (940b), the composition adjustment request corresponding to the first direction may be, for example, a composition adjustment request to move the position of the subject on the preview by a preset distance (e.g., 1 step) in the left direction. Meanwhile, even if the composition adjustment request corresponding to the first direction is identified, if the driving range is exceeded, the electronic device may provide the user with information notifying that the driving range has been exceeded (e.g., a notification that 'further movement is difficult').

[0206] FIG. 10 illustrates an example of a UI for composition adjustment according to one embodiment of the present disclosure.

[0207] Referring to FIG. 10, an electronic device (e.g., the second electronic device of FIGS. 7 and 8) may obtain user input through a UI for framing adjustment (or camera adjustment) and identify a framing adjustment request based on the user input. In one embodiment, the user input may be obtained (or received) while a preview is displayed on the screen. In one embodiment, the user input may be associated with the preview.

[0208] According to one embodiment, the electronic device may acquire a touch input moving in a first rotational direction along a bezel (e.g., a touch bezel), as illustrated in part (a) of FIG. 10. According to one embodiment, when a touch input moving in the first rotational direction is acquired, the electronic device may identify a composition adjustment request corresponding to the first rotational direction.

[0209] According to one embodiment, the electronic device may obtain a user input for rotating a bezel (e.g., a rotating bezel) in a first rotational direction, as illustrated in part (b) of FIG. 10. According to one embodiment, when a user input for rotating in the first rotational direction is obtained, the electronic device may identify a composition adjustment request corresponding to the first rotational direction.

[0210] The first rotation direction may be, for example, clockwise or counterclockwise, but is not limited thereto. When the first rotation direction is counterclockwise, the composition adjustment request corresponding to the first rotation direction may be, for example, a composition adjustment request for moving the position of the subject on the preview in the left (and / or downward) direction by a preset distance (e.g., 1 step). For example, through the composition adjustment request corresponding to the counterclockwise direction, the composition may be adjusted from the composition of part (a) of FIG. 14A to the composition of part (b) of FIG. 14A. When the first rotation direction is clockwise, the composition adjustment request corresponding to the first rotation direction may be, for example, a composition adjustment request for moving the position of the subject on the preview in the right (and / or upward) direction by a preset distance (e.g., 1 step). Meanwhile, even if a request for angle adjustment corresponding to the first rotation direction is identified, if the driving range is exceeded, the electronic device may provide the user with information indicating that the driving range has been exceeded (e.g., a notification that 'further movement is difficult').

[0211] FIG. 11 illustrates an example of a UI for composition adjustment according to one embodiment of the present disclosure.

[0212] Referring to FIG. 11, an electronic device (e.g., the second electronic device of FIGS. 7 and 8) may obtain user input through a UI for framing adjustment (or camera adjustment) and identify a framing adjustment request based on the user input. In one embodiment, the user input may be obtained (or received) while a preview is displayed on the screen. In one embodiment, the user input may be associated with the preview.

[0213] According to one embodiment, the electronic device may obtain a first user input for selecting a control direction (e.g., left-right direction or up-down direction), and a second user input for moving the bezel in a first rotational direction (or rotating the bezel in the first rotational direction), as illustrated in parts (a) and (b) of FIG. 11. According to one embodiment, the second electronic device may identify a composition adjustment request corresponding to the control direction and the first rotational direction when the first user input for selecting the control direction and the second user input associated with the first rotational direction are obtained.

[0214] The first user input for selecting the control direction may be, for example, a touch input for selecting the left-right control direction (e.g., a touch input of FIG. 11 (a) for selecting the left-right control direction (e.g., a touch input for moving in the left-right direction while touching a preset area (1110a) on the screen)) or a touch input for selecting the up-down control direction (e.g., a touch input of FIG. 11 (b) for selecting the up-down control direction (e.g., a touch input for moving in the up-down direction while touching a preset area (1110b) on the screen)), but is not limited thereto. The first rotation direction may be, for example, a clockwise or counterclockwise direction as illustrated in parts (a) and (b) of FIG. 11, but is not limited thereto.

[0215] When the control direction is a left-right control direction and the first rotation direction is counterclockwise, the composition adjustment request corresponding to the control direction and the first rotation direction may be, for example, a composition adjustment request for moving the position of the subject on the preview to the left by a preset distance (e.g., 1 step). When the control direction is a left-right control direction and the first rotation direction is clockwise, the composition adjustment request corresponding to the control direction and the first rotation direction may be, for example, a composition adjustment request for moving the position of the subject on the preview to the right by a preset distance (e.g., 1 step). When the control direction is an up-down control direction and the first rotation direction is counterclockwise, the composition adjustment request corresponding to the control direction and the first rotation direction may be, for example, a composition adjustment request for moving the position of the subject on the preview to the upper direction by a preset distance (e.g., 1 step). For example, through a composition adjustment request corresponding to the up-down control direction and the counterclockwise direction, the composition can be adjusted from the composition of part (a) of FIG. 14A to the composition of part (b) of FIG. 14A. When the control direction is the up-down control direction and the first rotation direction is clockwise, the composition adjustment request corresponding to the control direction and the first rotation direction can be, for example, a composition adjustment request for moving the position of the subject on the preview downward by a preset distance (e.g., 1 step). Meanwhile, even if a composition adjustment request corresponding to the control direction and the first rotation direction is identified, if the driving range is exceeded, the electronic device can provide the user with information notifying that the driving range is exceeded (e.g., a notification that 'further movement is difficult').

[0216] FIG. 12 illustrates an example of a UI for composition adjustment according to one embodiment of the present disclosure.

[0217] Referring to FIG. 12, an electronic device (e.g., the second electronic device of FIGS. 7 and 8 ) may obtain user input through a UI for framing adjustment (or camera adjustment) and identify a framing adjustment request based on the user input. In one embodiment, the user input may be obtained (or received) while a preview is displayed on the screen. In one embodiment, the user input may be associated with the preview.

[0218] According to one embodiment, the second electronic device may obtain a user input for selecting (e.g., touching) a preset area on a screen where a preview (e.g., a first preview) is displayed, as illustrated in FIG. 12. According to one embodiment, when a user input for selecting the preset area is obtained, the electronic device may identify a composition adjustment request corresponding to the preset area.

[0219] According to one embodiment, the preset area may be, but is not limited to, a first area (1210) located on the left side of the screen, a second area (1220) located on the top side of the screen, a third area (1230) located on the right side of the screen, or a fourth area (1240) located on the bottom side of the screen.

[0220] If the preset area is the first area, the composition adjustment request corresponding to the first area (1210) may be, for example, a composition adjustment request to move the position of the subject on the preview to the left by a preset distance (e.g., 1 step). If the preset area is the second area (1220), the composition adjustment request corresponding to the first area may be, for example, a composition adjustment request to move the position of the subject on the preview upward by a preset distance (e.g., 1 step). If the preset area is the third area (1230), the composition adjustment request corresponding to the third area may be, for example, a composition adjustment request to move the position of the subject on the preview to the right by a preset distance (e.g., 1 step). If the preset area is the fourth area (1240), the composition adjustment request corresponding to the fourth area may be, for example, a composition adjustment request to move the position of the subject on the preview upward by a preset distance (e.g., 1 step). For example, through a composition adjustment request corresponding to the fourth area, the composition can be adjusted from the composition of part (a) of FIG. 14A to the composition of part (b) of FIG. 14A. Meanwhile, even if a composition adjustment request corresponding to a preset area is identified, if the driving range is exceeded, the electronic device can provide the user with information notifying that the driving range has been exceeded (e.g., a notification that 'further movement is difficult').

[0221] FIG. 13a illustrates an example of a preview of an image of a desired composition according to one embodiment of the present disclosure. FIG. 13b illustrates an example of a position and angle for a camera module for capturing a desired composition according to one embodiment of the present disclosure.

[0222] As illustrated in FIG. 13A, a preview of an image of a desired composition may be displayed on a screen (1310) of an electronic device (e.g., the electronic device (101) or the electronic device (102) of FIG. 1). For example, the desired composition may be a structure in which a subject (1301) (e.g., a person) or a specific part (e.g., a head) of the subject (1301) is positioned at a specific part or position within the image.

[0223] As illustrated in FIG. 13b, the positions and angles of the lens component (181) and the image sensor component (182) of the camera module (e.g., the camera module (180) of FIGS. 1, 2a, and 2b) for capturing an image of a desired composition may be at an initial position and an initial angle (e.g., the initial position and the initial angle of the basic arrangement structure of FIG. 3). In addition, as illustrated, the optical axis may pass through the first position (1310) of the subject (1301).

[0224] FIG. 14A illustrates an example of a preview of images with adjusted composition according to an embodiment of the present disclosure. FIG. 14B illustrates an example of composition adjustment through position adjustment of a lens component of a camera module according to an embodiment of the present disclosure. FIG. 14C illustrates an example of composition adjustment through position adjustment of an image sensor component of a camera module according to an embodiment of the present disclosure. FIG. 14D illustrates an example of composition adjustment through angle adjustment of a camera module according to an embodiment of the present disclosure.

[0225] The preview (1410a) illustrated in part (a) of FIG. 14a corresponds to a preview provided by a camera module having an angle (hereinafter, referred to as a first angle) of the camera module illustrated in part (a) of FIG. 14b, part (a) of FIG. 14c, or part (a) of FIG. 14d, for example. Due to the angular difference between the first angle of the camera module and the angle of the camera module for a desired composition illustrated in FIG. 13b, for example, a preview of a different composition from the desired composition is provided.

[0226] The preview (1410b) illustrated in part (b) of FIG. 14a corresponds to a preview provided by a camera module having the position and / or angle of the camera module illustrated in part (b) of FIG. 14b, part (b) of FIG. 14c, or part (b) of FIG. 14d, for example.

[0227] The preview (1410c) shown in part (c) of FIG. 14a corresponds to a preview provided by a camera module having the position and / or angle of the camera module shown in part (c) of FIG. 14b, part (c) of FIG. 14c, or part (c) of FIG. 14d, for example.

[0228] In one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) may control a camera module associated with an image stabilization function to adjust the composition for shooting to a desired composition. Through such control, the composition may be adjusted remotely.

[0229] For example, as illustrated in FIG. 14b, the electronic device can adjust the position of the lens component (181) of the camera module using image stabilization control. For example, when a request (e.g., a composition adjustment request) corresponding to a first direction (e.g., a downward direction) is identified at the position of the lens component (181) illustrated in part (a) of FIG. 14b, the electronic device can move the position of the lens component (181) in a second direction (e.g., an upward direction) associated with the first direction by a first preset distance along an axis (e.g., an x-axis or a y-axis) substantially perpendicular to the optical axis (z-axis), as illustrated in part (b) of FIG. 14b. For example, when a request (e.g., a composition adjustment request) corresponding to a first direction (e.g., a downward direction) is identified at the position of the lens component (181) as shown in part (b) of FIG. 14b, the electronic device may move the position of the lens component (181) in a second direction (e.g., an upward direction) associated with the first direction by a first preset distance along an axis (e.g., an x-axis or a y-axis) substantially perpendicular to the optical axis (z-axis) as shown in part (c) of FIG. 14b. In this way, each time the request (e.g., a composition adjustment request) is identified, the composition may be adjusted (e.g., the position of the subject within the image may be adjusted by a distance corresponding to a 1-step distance, as in FIG. 14a) by adjusting the position of the lens component (181) (e.g., adjusting by a 1-step distance).

[0230] For example, as illustrated in FIG. 14c, the electronic device can adjust the position of the image sensor component (182) of the camera module using image stabilization control. For example, when a request (e.g., a composition adjustment request) corresponding to a first direction (e.g., downward direction) is identified at the position of the image sensor component (182) illustrated in part (a) of FIG. 14c, the electronic device can move the position of the image sensor component (182) in a third direction (e.g., downward direction) associated with the first direction by a preset second distance along an axis (e.g., x-axis or y-axis) substantially perpendicular to the optical axis (z-axis), as illustrated in part (b) of FIG. 14c. For example, when a request (e.g., a composition adjustment request) corresponding to a first direction (e.g., a downward direction) is identified at the position of the image sensor component (182) as shown in part (b) of FIG. 14c, the electronic device may move the position of the image sensor component (182) in a third direction (e.g., a downward direction) associated with the first direction by a preset second distance along an axis (e.g., an x-axis or a y-axis) substantially perpendicular to the optical axis (z-axis), as shown in part (c) of FIG. 14c. In this way, whenever the corresponding request (e.g., a composition adjustment request) is identified, the composition may be adjusted by adjusting the position of the image sensor component (182) (e.g., adjusting by a distance of one step). Through this, the position of a subject within an image may be adjusted by a distance corresponding to a one-step angle, as shown in FIG. 14a, for example.

[0231] For example, as illustrated in FIG. 14d, the electronic device can adjust the angle (or direction) of the camera module using image stabilization control. For example, when a request (e.g., a composition adjustment request) corresponding to a first direction (e.g., downward direction) is identified in the angle of the camera module including the lens component (181) and the image sensor component (182) illustrated in part (a) of FIG. 14d, the electronic device can tilt the angle of the camera module including the lens component (181) and the image sensor component (182) in a fourth direction associated with the first direction by a preset first angle, as illustrated in part (b) of FIG. 14d. For example, when a request (e.g., a composition adjustment request) corresponding to a first direction (e.g., downward direction) is identified in the angle of the camera module including the lens component (181) and the image sensor component (182) as shown in part (b) of FIG. 14d, the electronic device may tilt the angle of the camera module including the lens component (181) and the image sensor component (182) in a fourth direction associated with the first direction by a preset first angle, as shown in part (c) of FIG. 14d. In this way, whenever the request (e.g., a composition adjustment request) is identified, the composition may be adjusted by adjusting the angle of the lens component (181), the image sensor component (182), or the camera module (180) (e.g., adjusting by a one-step angle). Through this, the position of the subject within the image may be adjusted by a distance corresponding to a one-step angle, as shown in FIG. 14a, for example.

[0232] FIG. 15 is a flowchart of an automatic composition adjustment method according to one embodiment of the present disclosure.

[0233] Referring to FIG. 15, in operation 15010, an electronic device (e.g., electronic device (101) of FIG. 1) may identify a request to automatically adjust the composition for shooting (hereinafter, an automatic composition adjustment request).

[0234] In one embodiment, the electronic device may identify (or generate) a composition auto-adjustment request corresponding to the composition auto-adjustment mode when a user input for selecting the composition auto-adjustment mode is obtained. In one embodiment, the electronic device may identify (or generate) a composition auto-adjustment request when a preset condition is satisfied.

[0235] In operation 15020, the electronic device can automatically adjust the composition for shooting based on the automatic composition adjustment request.

[0236] According to one embodiment, the electronic device can automatically adjust the composition for shooting by performing image stabilization control based on an automatic composition adjustment request.

[0237] In one embodiment, the image stabilization control may be hardware-based image stabilization control. For example, the image stabilization control may be hardware-based control (e.g., motion control or optical control) of a camera module associated with the image stabilization function.

[0238] In one embodiment, the image stabilization control may be a hardware-based image stabilization control utilizing physical / mechanical movements (e.g., position adjustment and / or angle adjustment). For example, the image stabilization control may be a movement control for a camera module associated with the image stabilization function. For example, the image stabilization control may be a control for position adjustment and / or angle adjustment of a camera module associated with the image stabilization function, a lens component included in the camera module, and / or an image sensor component included in the camera module.

[0239] In one embodiment, the image stabilization control may be OIS control. For example, the image stabilization control may be control for position adjustment and / or angle adjustment of a camera module associated with the OIS function. In one embodiment, the OIS control may be performed by the OIS module. In one embodiment, the OIS module may include an actuator component.

[0240] In one embodiment, the electronic device may adjust the composition through control of a camera module associated with an image stabilization function (e.g., camera module (180) of FIGS. 1, 2a, and 2b) based on an automatic correction request.

[0241] In one embodiment, the electronic device can adjust the composition to one of adjustable compositions through control over a camera module associated with an image stabilization function.

[0242] According to one embodiment, the electronic device can adjust the composition to an optimal composition among adjustable compositions through control of a camera module associated with an image stabilization function.

[0243] According to one embodiment, the electronic device can adjust the composition to one of a plurality of optimal composition candidates among correctable compositions through control of a camera module associated with an image stabilization function.

[0244] In operation 15030, the electronic device can capture an image at an adjusted angle.

[0245] At operation 15040, the electronic device may determine whether further framing adjustments are required. If further framing adjustments are required, operations 15020 and 15030 may be performed again.

[0246] In one embodiment, the electronic device may determine that additional composition adjustments are necessary if there are still adjustable compositions remaining.

[0247] In one embodiment, the electronic device may determine that additional composition adjustments are necessary if optimal composition candidates remain.

[0248] FIG. 16 is a flowchart of an automatic composition adjustment method according to one embodiment of the present disclosure.

[0249] The automatic composition adjustment method of FIG. 16 may be an example of an automatic composition adjustment method following a first automatic composition adjustment mode. The first automatic composition adjustment mode may be a mode for adjusting the composition to all adjustable compositions through, for example, control of a camera module associated with an image stabilization function (e.g., control of an OIS module of the camera). In one embodiment, the OIS module may include an actuator component (e.g., actuator component (183) of FIGS. 2a / b).

[0250] Referring to FIG. 16, in operation 16010, an electronic device (e.g., electronic device (101) of FIG. 1) may identify (or generate) a request to automatically adjust the composition for shooting (hereinafter, an automatic composition adjustment request).

[0251] In one embodiment, the electronic device may identify (or generate) a composition auto-adjustment request corresponding to the first composition auto-adjustment mode when a user input selecting the first composition auto-adjustment mode is obtained.

[0252] In operation 16020, the electronic device may adjust the composition for shooting to one of the adjustable compositions based on the automatic composition adjustment request. For example, if the automatic composition adjustment request is identified while in the composition (first composition) illustrated in part (a) of FIG. 19, the electronic device may adjust the composition to a composition (second composition) moved upward by a preset distance (e.g., a distance corresponding to a 1-step distance) as illustrated in part (b) of FIG. 19.

[0253] According to one embodiment, the number of adjustable angles may be determined by a driving range (e.g., a position adjustment range and / or an angle adjustment range) of the camera module, a lens component within the camera module, or an image component within the camera module, and / or a distance from the camera module to the subject.

[0254] In one embodiment, the electronic device may adjust the composition based on a composition auto-adjustment request through control of a camera module associated with an image stabilization function (e.g., control of an OIS module of the camera) (e.g., control of the camera module (180) of FIGS. 1, 2a, and 2b).

[0255] In operation 16030, the electronic device can capture an image at an adjusted angle.

[0256] At operation 16040, the electronic device may determine whether additional composition adjustments are required. In one embodiment, the electronic device may determine that additional composition adjustments are required if there are still adjustable compositions remaining.

[0257] If further composition adjustment is required, operations 16020 and 16030 may be performed again. For example, if it is determined that further composition adjustment is required while in the composition (second composition) illustrated in part (b) of FIG. 19, the electronic device may adjust the composition to a composition (third composition) moved upward by a preset distance (e.g., a distance corresponding to a 1-step distance) as illustrated in part (c) of FIG. 19, and capture an image in the adjusted composition. For example, if it is determined that further composition adjustment is required while in the composition (third composition) illustrated in part (c) of FIG. 19, the electronic device may adjust the composition to a composition (fourth composition) moved upward by a preset distance (e.g., a distance corresponding to a 1-step distance) as illustrated in part (d) of FIG. 19, and capture an image in the adjusted composition. In this manner, images can be sequentially captured in all adjustable compositions.

[0258] FIG. 17 is a flowchart of an automatic composition adjustment method according to one embodiment of the present disclosure.

[0259] The automatic composition adjustment method of FIG. 17 may be an example of an automatic composition adjustment method following a second automatic composition adjustment mode. The second automatic composition adjustment mode may be a mode that automatically adjusts the composition to a plurality of optimal composition candidates among all adjustable compositions through control of a camera module using an image stabilization function (e.g., control of an OIS module of a camera). In one embodiment, the OIS module may include an actuator component (e.g., actuator component (183) of FIG. 2a / b).

[0260] Referring to FIG. 17, in operation 17010, an electronic device (e.g., electronic device (101) of FIG. 1) may identify (or generate) a request to automatically adjust the composition for shooting (hereinafter, an automatic composition adjustment request).

[0261] In one embodiment, the electronic device may identify (or generate) a composition auto-adjustment request corresponding to the second composition auto-adjustment mode when a user input selecting the second composition auto-adjustment mode is obtained.

[0262] In operation 17020, the electronic device can adjust the composition for shooting to one of the compositions determined as a plurality of optimal composition candidates based on the automatic composition adjustment request.

[0263] According to one embodiment, the electronic device can determine a plurality of optimal composition candidates from among adjustable compositions according to a preset manner.

[0264] For example, the electronic device may determine a plurality of optimal composition candidates by using at least one of a user input related to setting an optimal composition, a preset guide function, a pre-trained artificial intelligence model, or information about the ratio of a background and a subject within a preset image.

[0265] For example, when using the preset guide function, the electronic device can activate the horizontal and vertical guide functions and determine multiple optimal composition candidates by using the points where the horizontal and vertical guide lines intersect or the composition divided by the horizontal and vertical guide lines (e.g., a rule-of-three composition). For example, as illustrated in FIG. 20, when the subject is a person, a composition in which the person's face is located at the second point among the points where the horizontal and vertical guide lines intersect (e.g., a composition illustrated in part (a) of FIG. 20) and a composition in which the person's face is located at the first point (e.g., a composition illustrated in part (b) of FIG. 20) can be determined as optimal composition candidates. Meanwhile, when using the preset guide function, the number of horizontal and vertical guide lines used can be set in various ways.

[0266] For example, user input related to setting the optimal composition may include, but is not limited to, user input for the shooting mode (e.g., selfie mode, panorama mode, etc.), user input for the number of subjects, user input for selecting a specific subject (e.g., oneself) among multiple subjects, and / or user input for the ratio of the subject to the background.

[0267] For example, when using a pre-learned artificial intelligence model, the electronic device can input the image data of the current composition as input data of the artificial intelligence model, and obtain data of optimal composition candidates as output data of the artificial intelligence model.

[0268] According to one embodiment, the electronic device can adjust the composition by controlling a camera module (e.g., a camera module (180) of FIGS. 1, 2a, and 2b) using an image stabilization function (e.g., controlling an OIS module of the camera) based on a request for automatic composition adjustment.

[0269] At operation 17030, the electronic device can capture an image from the adjusted optimal composition candidate.

[0270] In operation 17040, the electronic device may determine whether further composition adjustments are required. In one embodiment, the electronic device may determine that further composition adjustments are required if an optimal composition candidate remains.

[0271] If further composition adjustment is required, operations 17020 and 17030 may be performed again. For example, if it is determined that further composition adjustment is required while in the optimal composition candidate illustrated in part (a) of FIG. 20, the electronic device may adjust the composition to the optimal composition candidate illustrated in part (b) of FIG. 20 and capture an image from the adjusted optimal composition candidate. In this manner, images can be captured sequentially from all optimal composition candidates.

[0272] FIG. 18 is a flowchart of an automatic composition adjustment method according to one embodiment of the present disclosure.

[0273] The automatic composition adjustment method of FIG. 18 is an example of an automatic composition adjustment method following a third automatic composition adjustment mode. The third automatic composition adjustment mode may be a mode that automatically adjusts the composition to an optimal composition among all adjustable compositions through, for example, control of a camera module using an image stabilization function (e.g., control of an OIS module of a camera). In one embodiment, the OIS module may include an actuator component (e.g., actuator component (183) of FIGS. 2a / b).

[0274] Referring to FIG. 18, in operation 18010, an electronic device (e.g., electronic device (101) of FIG. 1) may identify a request to automatically adjust the composition (hereinafter, an automatic composition adjustment request).

[0275] According to one embodiment, the electronic device may identify a composition auto-adjustment request corresponding to the third composition auto-adjustment mode when a user input selecting the third composition auto-adjustment mode is obtained.

[0276] In operation 18020, the electronic device can adjust the composition for shooting to one optimal composition based on the automatic composition adjustment request.

[0277] According to one embodiment, the electronic device can determine an optimal composition from among compositions capable of adjusting the composition for shooting according to a preset method.

[0278] For example, the electronic device may determine the optimal composition using at least one of user input, a preset guide function, a pre-trained artificial intelligence model, or information about the ratio of the background and subject within a preset image.

[0279] For example, when using the preset guide function, the electronic device can activate the horizontal and vertical guide functions and determine the optimal composition by using the point where the horizontal guide line and the vertical guide line intersect or the composition divided by the horizontal guide line and the vertical guide line (e.g., a rule-of-three composition). For example, as illustrated in FIG. 20, when the subject is a person, the electronic device can determine as the optimal composition either the composition in which the person's face is located at the second point among the points where the horizontal guide line and the vertical guide line intersect (e.g., the composition illustrated in part (a) of FIG. 20) or the composition in which the person's face is located at the first point (e.g., the composition illustrated in part (b) of FIG. 20).

[0280] Meanwhile, when using the preset guide function, the number of horizontal and vertical guide lines used can be set in various ways.

[0281] For example, user input related to setting the optimal composition may include, but is not limited to, user input for the shooting mode (e.g., selfie mode, panorama mode, etc.), user input for the number of subjects, and / or user input for the ratio of subjects to background.

[0282] For example, when using a pre-learned artificial intelligence model, the electronic device can input the image data of the current composition as input data of the artificial intelligence model, and obtain data of optimal composition candidates as output data of the artificial intelligence model.

[0283] According to one embodiment, the electronic device can adjust the composition by controlling a camera module (e.g., a camera module (180) of FIGS. 1, 2a, and 2b) using an image stabilization function (e.g., controlling an OIS module of the camera) based on a request for automatic composition adjustment.

[0284] In operation 18030, the electronic device can sequentially capture images at an adjusted optimal composition.

[0285] Meanwhile, the auto-composition mode can be used not only when the electronic device is stationary, but also when the electronic device is not stationary (e.g., when the user is holding the electronic device in their hand). When the auto-composition mode is used when the electronic device is stationary, the captured image may be unstable due to unwanted motion (e.g., hand tremors). Therefore, when using the auto-composition mode, a method of performing image stabilization by simultaneously using the standard OIS function, which compensates for unwanted motion based on the motion data of the electronic device, may be considered.

[0286] Below, with reference to FIG. 21, a method for using the automatic composition adjustment function and the general OIS function together is described.

[0287] FIG. 21 is a flowchart of an operation of capturing an image in an adjusted composition of an automatic composition adjustment method according to one embodiment of the present disclosure.

[0288] In the embodiment of FIG. 21, the operation of capturing an image in an adjusted composition may be, for example, an example of operation 15030 of FIG. 15, operation 16030 of FIG. 16, operation 17030 of FIG. 17, or operation 18030 of FIG. 18. The adjusted composition may be, for example, one of the optimal composition candidates, the optimal composition, or one of the adjustable compositions.

[0289] Referring to FIG. 21, in operation 21010, the electronic device may adjust the position and / or angle of the camera module from a reference position and / or reference angle corresponding to the adjusted composition to compensate for the movement of the electronic device by using sensor data related to the movement of the electronic device. The sensor data related to the movement may include, but is not limited to, IMU sensor data, accelerometer sensor data, and / or gyroscope sensor data.

[0290] In one embodiment, the electronic device may use sensor data related to the movement of the electronic device to adjust the position and / or angle of a lens component and / or an image sensor component of a camera module from a reference position and / or reference angle corresponding to the adjusted composition. This allows the general OIS function for compensating for unwanted motion of the electronic device to be provided in the adjusted composition.

[0291] In one embodiment, when the composition is adjusted, the position and / or angle of the camera module associated with the adjusted composition may be set as a reference position (or default position) and / or a reference angle (or default angle) for the camera module for performing a general OIS function (mode). For example, when the composition is adjusted, the position and / or angle of the lens component and / or the image sensor component included in the camera module associated with the adjusted composition may be set as a reference position (or default position) and / or a reference angle (or default angle) of the lens component and / or the image sensor component included in the camera module for performing a general OIS function (mode).

[0292] According to one embodiment, the degree of position and / or angle adjustment of a camera module for compensating for the motion of an electronic device may be lower than the degree of position and / or angle adjustment of a camera module for adjusting the composition. For example, the degree of position adjustment of a lens component or an image sensor component of a camera module for compensating for the motion of an electronic device may be lower than the degree of position adjustment of a lens component or an image sensor component of a camera module for adjusting the composition. For example, the degree of angle adjustment of a lens component and / or an image sensor component of a camera module for compensating for the motion of an electronic device may be lower than the degree of angle adjustment of a lens component and / or an image sensor component of a camera module for adjusting the composition. In this way, the composition may be prevented from being changed through motion compensation.

[0293] In operation 21020, the electronic device can capture an image using a camera module whose position and / or angle are adjusted from a reference position and / or reference angle corresponding to the adjusted composition. By providing this general OIS function in conjunction with an automatic composition adjustment function, an image (stabilized image) in which unwanted motion of the electronic device is corrected in the automatically adjusted composition(s) can be provided to the user.

[0294] FIG. 22 illustrates a flowchart of a method for remotely adjusting the composition of an electronic device according to one embodiment of the present disclosure.

[0295] Referring to FIG. 22, an electronic device (e.g., electronic device (101) of FIG. 1) can transmit first preview data for a preview of an image to be captured to an external electronic device (e.g., electronic device (102) of FIG. 1) (22010).

[0296] The electronic device can receive a composition adjustment request from an external electronic device to adjust the composition for shooting (22020).

[0297] The electronic device can adjust the composition by controlling a camera module (e.g., camera module (180) of FIGS. 1, 2a, and 2b) associated with an image stabilization function based on a composition adjustment request (22030).

[0298] The electronic device can transmit second preview data to an external electronic device for previewing an image associated with the adjusted composition (22040).

[0299] According to one embodiment, a camera module of an electronic device may include a lens component including at least one lens (e.g., lens component (181) of FIGS. 2A and 2B), an image sensor component including at least one image sensor (e.g., image sensor component (182) of FIGS. 2A and 2B), and an actuator component (183) for performing control for the camera module (e.g., actuator component (183) of FIGS. 2A and 2B).

[0300] In one embodiment, performing control on a camera module associated with an image stabilization function based on a composition adjustment request may include adjusting a position of a lens component or an image sensor component along one or more axes substantially perpendicular to an optical axis based on the composition adjustment request.

[0301] In one embodiment, performing control on a camera module associated with an image stabilization function based on a composition adjustment request may include adjusting an angle of a lens component, an image sensor component, or a camera module based on the composition adjustment request.

[0302] In one embodiment, the image stabilization function may correspond to an OIS function.

[0303] According to one embodiment, the composition may be defined by at least one of the position, size, placement, perspective, or ratio of at least one subject to a background within the image.

[0304] In one embodiment, the electronic device, in response to receiving a request for framing adjustment, can identify whether the electronic device is stationary based on sensor data associated with the movement of the electronic device. In one embodiment, the electronic device, in response to identifying that the electronic device is stationary, can perform control on a camera module associated with an image stabilization function to adjust the framing.

[0305] According to one embodiment, the electronic device can capture an image at an adjusted angle.

[0306] In one embodiment, the electronic device can maintain the positions and angles of the lens component and the image sensor component associated with the adjusted composition until movement of the electronic device is identified. In one embodiment, the electronic device can adjust the positions and / or angles of the lens component and the image sensor component to their initial positions and / or initial angles when movement of the electronic device is identified.

[0307] In one embodiment, a composition adjustment request may be generated by the external electronic device based on user input associated with a preview provided by the external electronic device based on the first preview data.

[0308] FIG. 23 illustrates a flowchart of an automatic composition adjustment method of an electronic device according to one embodiment of the present disclosure.

[0309] Referring to FIG. 23, an electronic device (e.g., electronic device (101) of FIG. 1) can identify an automatic composition adjustment request for automatically adjusting the composition for shooting (23010).

[0310] The electronic device can control a camera module of the electronic device (e.g., camera module (180) of FIGS. 1, 2a, and 2b) associated with an image stabilization function based on an automatic composition adjustment request, thereby adjusting the composition to at least one composition (23020).

[0311] According to one embodiment, the electronic device can capture an image through the camera module in each of at least one adjusted frame.

[0312] According to one embodiment, a camera module of an electronic device may include a lens component including at least one lens (e.g., lens component (181) of FIGS. 2A and 2B), an image sensor component including at least one image sensor (e.g., image sensor component (182) of FIGS. 2A and 2B), and an actuator component (183) for performing control for the camera module (e.g., actuator component (183) of FIGS. 2A and 2B).

[0313] In one embodiment, performing control on a camera module associated with an image stabilization function based on an automatic framing request may include adjusting a position of a lens component or an image sensor component along one or more axes substantially perpendicular to an optical axis based on the automatic framing request.

[0314] In one embodiment, performing control on a camera module associated with an image stabilization function based on an automatic framing adjustment request may include adjusting an angle of a lens component, an image sensor component, or a camera module based on the automatic framing adjustment request.

[0315] According to one embodiment, an electronic device may determine an optimal composition among adjustable compositions through control of a camera module according to a preset method. According to one embodiment, the electronic device may correct the composition to the optimal composition to adjust the composition to at least one composition.

[0316] According to one embodiment, an electronic device may determine a plurality of optimal composition candidates from among adjustable compositions through control of a camera module according to a preset method. According to one embodiment, the electronic device may adjust the composition to at least one of the plurality of optimal composition candidates.

[0317] According to one embodiment, the electronic device may use at least one of a user input related to setting an optimal composition, a preset guide function, a pre-learned artificial intelligence model, or information on the ratio of a background and a subject within a preset image to determine an optimal composition or a plurality of optimal composition candidates.

[0318] In one embodiment, the electronic device can adjust the composition to all adjustable compositions through control over the camera module to adjust the composition to at least one composition.

[0319] FIG. 24 illustrates a remote control procedure according to one embodiment of the present disclosure.

[0320] Referring to FIG. 24, in operation 24010, a first electronic device (e.g., the electronic device (101) of FIG. 1) may transmit first data corresponding to a live preview image acquired by a camera of the first electronic device to a second electronic device (e.g., the electronic device (102) of FIG. 2). In the present disclosure, the second electronic device may also be referred to as an external electronic device.

[0321] In operation 24020, the second electronic device may display a live preview image based on the first data. For example, the second electronic device may display a live preview image corresponding to the first data through the display of the second electronic device.

[0322] In operation 24030, the second electronic device may identify a user input. For example, the second electronic device may receive the user input while a live preview image is being displayed on the second electronic device.

[0323] In one embodiment, the user input may be a user input associated with a live preview image. For example, the user input may be a user input obtained through a UI (e.g., the UI of FIGS. 9a, 9b, 10, 11, and 12).

[0324] In operation 24040, the second electronic device may transmit a request corresponding to a user input to the first electronic device. The request corresponding to the user input may be, for example, a request corresponding to a user input obtained (or received) while the live preview image is being displayed on the second electronic device.

[0325] In one embodiment, a request corresponding to a user input may be associated with a camera orientation. The camera orientation may be associated with, for example, at least one of the position, size, arrangement, perspective, or ratio of one or more subjects within an image captured by the camera, or the ratio of the at least one subject to the background.

[0326] In operation 24050, the first electronic device can control the OIS module of the camera module to change a relative position of the lens with respect to the image sensor in one or more axes perpendicular to an optical axis of the lens based on the received request. In one example, to change the relative position of the lens with respect to the image sensor in one or more axes perpendicular to the optical axis of the lens, the first electronic device can adjust the position of the lens of the camera module in one or more axes by controlling the OIS module of the camera module. In one example, to change the relative position of the lens with respect to the image sensor in one or more axes perpendicular to the optical axis of the lens, the first electronic device can adjust the position of the image sensor of the camera module in the one or more axes by controlling the OIS module of the camera module.

[0327] In one embodiment, the first electronic device may control the OIS module of the camera module to adjust the tilt angle of the lens relative to the image sensor based on the received request. The tilt angle adjustment of the lens relative to the image sensor may be performed together with, or instead of, changing the relative position of the lens relative to the image sensor as described above.

[0328] According to one embodiment, the OIS module may include an actuator component (e.g., actuator component (183) of FIGS. 2a / b).

[0329] At operation 24060, the first electronic device can transmit second data corresponding to a live preview image after the relative position of the lens with respect to the image sensor has been adjusted.

[0330] In operation 24070, the second electronic device may display a live preview image based on the second data. For example, the second electronic device may display a live preview image corresponding to the second data through the display of the second electronic device.

[0331] FIG. 25 is a flowchart of a method of an electronic device according to one embodiment of the present disclosure.

[0332] In an embodiment of FIG. 25, an electronic device (e.g., electronic device (101) of FIG. 1) may include a display (e.g., display module (160) of FIG. 1), a processor (e.g., processor (120) of FIG. 1), a camera (e.g., camera (180) of FIG. 1) including a lens, an image sensor, and an OIS module), communication circuitry (e.g., communication module (190) of FIG. 1), and / or memory (e.g., memory (130) of FIG. 1). In one embodiment, the OIS module may include an actuator component (e.g., actuator component (183) of FIGS. 2a / b).

[0333] According to one embodiment, the memory stores instructions that, when executed by the processor, cause the electronic device to perform at least one of the following operations:

[0334] The electronic device can transmit first data corresponding to a live preview image acquired by the camera to an external electronic device (e.g., electronic device (102) of FIG. 2) via a communication circuit of the electronic device (25010).

[0335] An electronic device may receive a request transmitted by an external electronic device via a communication circuit of the electronic device (25020). In one embodiment, the request may correspond to a user input received while a live preview image is displayed on the external electronic device.

[0336] The electronic device can control the OIS module of the camera to change the relative position of the lens to the image sensor in one or more axes perpendicular to the optical axis of the lens, based on a request (25030).

[0337] The electronic device can transmit, via the communication circuitry of the electronic device, second data corresponding to the live preview image after the relative position of the lens with respect to the image sensor has been adjusted to an external electronic device (25040).

[0338] In one embodiment, the OIS module can be controlled to adjust the tilt angle of the lens relative to the image sensor based on a request.

[0339] In one embodiment, the request is associated with a camera composition, which may be associated with at least one of, for example, a position, size, placement, perspective, or ratio of at least one subject within the image to a background.

[0340] In one embodiment, the electronic device can, in response to receiving a request, identify whether the electronic device is stationary based on sensor data associated with movement of the electronic device, and control the OIS module to change the relative position of the lens with respect to the image sensor in response to identifying that the electronic device is stationary.

[0341] In one embodiment, the electronic device can capture an image at the changed relative position of the lens with respect to the image sensor or at the adjusted tilt angle of the lens with respect to the image sensor.

[0342] In one embodiment, the electronic device maintains the changed relative position of the lens with respect to the image sensor or the adjusted tilt angle of the lens with respect to the image sensor until movement of the electronic device is identified, and when movement of the electronic device is identified, the electronic device can adjust the lens and the image sensor to an initial position and an initial angle.

[0343] FIG. 26 is a flowchart of a method of an electronic device according to one embodiment of the present disclosure.

[0344] In an embodiment of FIG. 26, an electronic device (e.g., electronic device (101) of FIG. 1) may include a display (e.g., display module (160) of FIG. 1), a processor (e.g., processor (120) of FIG. 1), a camera (e.g., camera (180) of FIG. 1) including a lens, an image sensor, and an OIS module), communication circuitry (e.g., communication module (190) of FIG. 1), and / or memory (e.g., memory (130) of FIG. 1). In one embodiment, the OIS module may include an actuator component (e.g., actuator component (183) of FIGS. 2a / b).

[0345] According to one embodiment, the memory stores instructions that, when executed by the processor, cause the electronic device to perform at least one of the following operations:

[0346] The electronic device can identify whether a preset condition for adjusting the orientation of the camera is met (26010). For example, if a user input is received for selecting an automatic composition adjustment mode (e.g., the first automatic composition adjustment mode of FIG. 16 , the second automatic composition adjustment mode of FIG. 17 , or the third automatic composition adjustment mode of FIG. 18 ), the electronic device can identify whether a preset condition for automatically adjusting the orientation of the camera according to the automatic composition adjustment mode is met.

[0347] The electronic device can control the OIS module of the camera based on the identification to adjust the composition of the camera to at least one composition (26020).

[0348] The electronic device can capture images through the camera in each of at least one adjusted frame (26030). In one embodiment, the electronic device can sequentially automatically adjust the frame of the camera in at least one frame, and sequentially capture images through the camera in each of the automatically adjusted frames.

[0349] According to one embodiment, an electronic device (101) may include a display (160); a processor (120); a camera (180) including a lens, an image sensor, and an optical image stabilization (OIS) module; a communication circuit (190); and a memory (130) storing instructions. The instructions, when executed by the processor, may cause the electronic device to perform at least one operation. The at least one operation may include transmitting, via the communication circuit, first data corresponding to a live preview image acquired by the camera to an external electronic device. The at least one operation may include receiving a request transmitted by the external electronic device. The request may correspond to a user input received while the live preview image is displayed on the external electronic device. At least one operation may include controlling the OIS module of the camera to change a relative position of the lens with respect to the image sensor in one or more axes perpendicular to an optical axis of the lens based on the request. At least one operation may include transmitting, to the external electronic device, second data corresponding to the live preview image after the relative position of the lens with respect to the image sensor has been adjusted.

[0350] In one embodiment, based on the request, the OIS module can be controlled to adjust a tilt angle of the lens with respect to the image sensor.

[0351] In one embodiment, the request is associated with an orientation of the camera, and the orientation of the camera may be associated with at least one of a position, size, arrangement, perspective, or ratio of at least one subject to a background within an image acquired by the camera.

[0352] In one embodiment, the instructions, when executed by the processor, may cause the electronic device to: in response to receiving the request, identify whether the electronic device is stationary based on sensor data associated with movement of the electronic device; and in response to identifying that the electronic device is stationary, control the OIS model to change the relative position of the lens with respect to the image sensor.

[0353] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to: capture an image at the changed relative position of the lens with respect to the image sensor or at the adjusted tilt angle of the lens with respect to the image sensor.

[0354] In one embodiment, the instructions, when executed by the processor, may cause the electronic device to: maintain the changed relative position of the lens with respect to the image sensor or the adjusted tilt angle of the lens with respect to the image sensor until movement of the electronic device is identified, and, if movement of the electronic device is identified, adjust the lens and the image sensor to an initial position and an initial angle.

[0355] According to one embodiment, a method of an electronic device (101) may include transmitting, through a communication circuit of the electronic device, first data corresponding to a live preview image acquired by a camera of the electronic device to an external electronic device. The camera may include a lens, an image sensor, and an optical image stabilization (OIS) module. The method of the electronic device (101) may include receiving a request transmitted by the external electronic device. The request may correspond to a user input received while the live preview image is displayed on the external electronic device. The method of the electronic device (101) may include controlling, based on the request, the OIS module of the camera to change a relative position of the lens with respect to the image sensor in one or more axes perpendicular to an optical axis of the lens. The method of the electronic device (101) may include transmitting, to the external electronic device, second data corresponding to the live preview image after the relative position of the lens with respect to the image sensor has been adjusted.

[0356] According to one embodiment, based on the composition adjustment request, the OIS module can be controlled to adjust a tilt angle of the lens with respect to the image sensor.

[0357] In one embodiment, the request is associated with an orientation of the camera, and the orientation of the camera may be associated with at least one of a position, size, arrangement, perspective, or ratio of at least one subject to a background within an image acquired by the camera.

[0358] According to one embodiment, the method of the electronic device (101) may include, in response to receiving the request, an operation of identifying whether the electronic device is stationary based on sensor data associated with movement of the electronic device. The method of the electronic device (101) may include, in response to identifying that the electronic device is stationary, an operation of controlling the OIS model to change the relative position of the lens with respect to the image sensor.

[0359] According to one embodiment, the method of the electronic device (101) may include capturing an image at the changed relative position of the lens with respect to the image sensor or at the adjusted tilt angle of the lens with respect to the image sensor.

[0360] According to one embodiment, the method of the electronic device (101) may include an operation of maintaining the changed relative position of the lens with respect to the image sensor or the adjusted tilt angle of the lens with respect to the image sensor until movement of the electronic device is identified. The method of the electronic device (101) may include an operation of adjusting the lens and the image sensor to an initial position and an initial angle when movement of the electronic device is identified.

[0361] According to one embodiment, an electronic device (101) may include a display (160); a processor (120); a camera (180) including a lens, an image sensor, and an optical image stabilization (OIS) module; and a memory (130) for storing instructions. The instructions, when executed by the processor, may cause the electronic device to perform at least one operation. The at least one operation may include an operation of identifying whether a preset condition for adjusting the orientation of the camera is met. The at least one operation may include an operation of controlling the OIS module of the camera based on the identification to adjust the orientation of the camera to at least one orientation. The at least one operation may include an operation of capturing an image through the camera in each of the at least one adjusted orientation.

[0362] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to: identify that a preset condition is met for automatically adjusting the composition of the camera according to the auto-composition adjustment mode when a user input selecting the auto-composition adjustment mode is obtained.

[0363] In one embodiment, based on the identification, the OIS module can be controlled to change the relative position of the lens with respect to the image sensor in one or more axes perpendicular to the optical axis of the lens.

[0364] According to one embodiment, based on the identification, the OIS module can be controlled to adjust a tilt angle of the lens relative to the image sensor.

[0365] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to: determine an optimal composition among adjustable compositions by controlling the OIS module of the camera according to a preset method. Adjusting the composition to at least one composition may include adjusting the composition to the optimal composition.

[0366] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to determine a plurality of optimal composition candidates among adjustable compositions through control of the OIS module of the camera according to a preset manner. Adjusting the composition to at least one composition may include sequentially adjusting the composition to the plurality of optimal composition candidates.

[0367] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to use at least one of a user input related to setting an optimal composition, a preset guide function, a pre-learned artificial intelligence model, or information about the ratio of a background and a subject within a preset image to determine the optimal composition or the plurality of optimal composition candidates.

[0368] In one embodiment, adjusting the composition to at least one composition may include sequentially adjusting the composition to all compositions that are adjustable via control of the OIS module of the camera.

[0369] According to one embodiment, the system may include an external electronic device including the electronic device (101) and a smart watch having a display configured to display a live preview image acquired by a camera of the electronic device (101).

[0370] In the specific embodiments of the present disclosure described above, components included in the present disclosure are expressed in the singular or plural form, depending on the specific embodiment presented. However, the singular or plural expressions are selected to suit the presented situation for convenience of explanation, and the present disclosure is not limited to singular or plural components. Components expressed in the plural form may be composed of singular elements, or components expressed in the singular form may be composed of plural elements.

[0371] While the detailed description of this disclosure has described specific embodiments, it should be understood that various modifications are possible without departing from the scope of this disclosure. Therefore, the scope of this disclosure should not be limited to the described embodiments, but should be defined not only by the scope of the claims described below, but also by equivalents thereof.

[0372] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs from the description below.

Claims

1. In an electronic device (101), display (160); Processor (120); A camera (180) including a lens, an image sensor and an optical image stabilization (OIS) module; Communication circuit (190); and A memory (130) for storing instructions, said instructions causing said electronic device to: Through the above communication circuit, first data corresponding to a live preview image acquired by the camera is transmitted to an external electronic device, receiving a request transmitted by said external electronic device, said request corresponding to a user input received while said live preview image is displayed on said external electronic device; An electronic device, based on said request, for controlling said OIS module of said camera to change a relative position of said lens with respect to said image sensor in one or more axes perpendicular to an optical axis of said lens, and causing said electronic device to transmit, to said external electronic device, second data corresponding to said live preview image after the relative position of said lens with respect to said image sensor is adjusted.

2. In paragraph 1, An electronic device, wherein based on the request, the OIS module is controlled to adjust the tilt angle of the lens with respect to the image sensor.

3. In paragraph 1 or 2, An electronic device wherein the request is associated with an orientation of the camera, wherein the orientation of the camera is associated with at least one of a position, a size, a placement, a perspective, or a ratio of the at least one subject to a background within an image acquired by the camera.

4. In claims 1 to 3, the instructions, when executed by the processor, cause the electronic device to: In response to receiving said request, identifying whether said electronic device is stationary based on sensor data associated with movement of said electronic device; An electronic device, responsive to identifying that said electronic device is in said stationary state, causing said electronic device to control said OIS model to change said relative position of said lens with respect to said image sensor.

5. In claims 1 to 4, the instructions, when executed by the processor, cause the electronic device to: An electronic device causing an image to be captured at said changed relative position of said lens with respect to said image sensor or at said adjusted tilt angle of said lens with respect to said image sensor.

6. In claims 1 to 5, the instructions, when executed by the processor, cause the electronic device to: maintaining said changed relative position of said lens with respect to said image sensor or said adjusted tilt angle of said lens with respect to said image sensor until movement of said electronic device is identified; An electronic device that causes the lens and the image sensor to be adjusted to an initial position and an initial angle when movement of the electronic device is identified.

7. In the method of electronic device (101), An operation of transmitting first data corresponding to a live preview image acquired by a camera of the electronic device to an external electronic device through a communication circuit of the electronic device, the camera including a lens, an image sensor and an optical image stabilization (OIS) module; An action of receiving a request transmitted by said external electronic device, said request corresponding to a user input received while said live preview image is displayed on said external electronic device; An operation of controlling the OIS module of the camera to change a relative position of the lens with respect to the image sensor in one or more axes perpendicular to the optical axis of the lens based on the request; and A method of an electronic device, comprising the action of transmitting second data corresponding to the live preview image after the relative position of the lens with respect to the image sensor is adjusted to the external electronic device.

8. In the electronic device (101), display (160); Processor (120); A camera (180) including a lens, an image sensor and an optical image stabilization (OIS) module; and A memory (130) for storing instructions, said instructions causing said electronic device to: Identify whether a preset condition for adjusting the orientation of the above camera is met, Based on the above identification, the OIS module of the camera is controlled to adjust the composition of the camera to at least one composition, An electronic device causing said camera to capture an image in each of said at least one adjusted frame.

9. In the 8th paragraph, the instructions, when executed by the processor, cause the electronic device to: An electronic device, which causes a preset condition to be met to automatically adjust the composition of a camera according to said auto-composition adjustment mode when a user input for selecting an auto-composition adjustment mode is obtained.

10. In clause 8 or 9, An electronic device wherein, based on said identification, said OIS module is controlled to change the relative position of said lens with respect to said image sensor in one or more axes perpendicular to an optical axis of said lens.

11. In clause 8 or 9, An electronic device, wherein, based on the identification, the OIS module is controlled to adjust the tilt angle of the lens with respect to the image sensor.

12. In claims 8 to 11, the instructions, when executed by the processor, cause the electronic device to: According to a preset method, the OIS module of the camera is caused to determine an optimal composition among the adjustable compositions, An electronic device wherein adjusting the composition to at least one composition comprises adjusting the composition to the optimal composition.

13. In clauses 8 to 12, the instructions, when executed by the processor, cause the electronic device to: According to a preset method, causing a plurality of optimal composition candidates to be determined among the adjustable compositions through control of the OIS module of the camera, An electronic device wherein adjusting the composition to at least one composition comprises sequentially adjusting the composition to the plurality of optimal composition candidates.

14. In clauses 8 to 13, the instructions, when executed by the processor, cause the electronic device to: An electronic device that causes at least one of a user input related to setting an optimal composition, a preset guideline function, a pre-learned artificial intelligence model, or information on the ratio of a background and a subject within a preset image to be used to determine the optimal composition or a plurality of optimal composition candidates.

15. In clauses 8 to 14, An electronic device wherein adjusting the composition to at least one composition comprises sequentially adjusting the composition to all compositions that are adjustable through control of the OIS module of the camera.

Images