Electronic device for performing image stabilization and operation method therefor

Abstract

An electronic device according to various embodiments comprises a sensor, a camera module, one or more processors, and a memory for storing one or more instructions, wherein the camera module may comprise a lens assembly, an image sensor, and a driving unit for transferring the lens assembly or the image sensor. The one or more instructions, when executed by the one or more processors individually or collectively, may instruct the electronic device to: control the driving unit to move the position of the lens assembly or the image sensor according to movement information on the basis of a first operation mode; control the driving unit such that the lens assembly or the image sensor is positioned at a reference position on the basis of a second operation mode; perform an image stabilization process; and when operating on the basis of the first operation mode, enable the driving unit to control the position of the lens assembly or the image sensor on the basis of a first gain value, and when operating on the basis of the second operation mode, enable the driving unit to control the position of the lens assembly or the image sensor on the basis of a second gain value.

Description

Electronic device for performing image stabilization and method of operation thereof

[0001] The present disclosure relates to an electronic device for performing image stabilization and a method of operating the same.

[0002] As the functionality of mobile devices has diversified, demand for improved photography and video recording capabilities has also increased. Consequently, technology to compensate for camera shake during video recording is advancing.

[0003] The electronic device may include a camera module capable of performing an OIS (optical image stabilization) function. The OIS function is a function to compensate for shaking by moving an OIS driving unit (e.g., a lens assembly or an image sensor) included in the camera module within the OIS driving range. The camera module may move the lens assembly or the image sensor in a direction that offsets the movement of the electronic device for OIS driving.

[0004] An electronic device can perform video digital image stabilization (VDIS) and / or electronic image stabilization (EIS) on multiple image frames. VDIS is a method of reducing shaking through digital processing in a mobile device, and the processor can compensate for multiple image frames through VDIS.

[0005] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art in connection with the present disclosure.

[0006] An electronic device according to an embodiment may include a sensor for detecting a movement of the electronic device, a camera module, one or more processors, and a memory for storing one or more instructions. The camera module may include a lens assembly, an image sensor, and a driving unit for moving the lens assembly or the image sensor within a driving range. The one or more instructions may cause the electronic device, when individually or collectively executed by one or more processors, to control the driving unit to move a position of the lens assembly or the image sensor based on movement information acquired from the sensor, based on a first operating mode among a plurality of operating modes associated with an operation of the driving unit. The one or more instructions may cause the electronic device, when individually or collectively executed by one or more processors, to control the driving unit to position the lens assembly or the image sensor at a reference position within the driving range, based on a second operating mode different from the first operating mode among the plurality of operating modes. The one or more instructions may cause the electronic device, when individually or collectively executed by one or more processors, to perform image processing on image frames acquired through the camera module while controlling the driving unit to perform image stabilization processing to reduce image shake. One or more instructions, when executed individually or collectively by one or more processors, may cause the electronic device to: when operating based on a first operating mode, cause the driver to control the position of the lens assembly or the image sensor based on a first gain value, and when operating based on a second operating mode, cause the driver to control the position of the lens assembly or the image sensor based on a second gain value different from the first gain value.

[0007] In one embodiment, a method of operating an electronic device including a camera module may include an operation of controlling a driving unit to move a position of a lens assembly or an image sensor based on movement information acquired from a sensor that detects movement of the electronic device, based on a first operation mode among a plurality of operation modes associated with an operation of a driving unit that moves a lens assembly or an image sensor within a driving range. The method of operating an electronic device may include an operation of controlling a driving unit to position the lens assembly or the image sensor at a reference position within the driving range, based on a second operation mode different from the first operation mode among the plurality of operation modes. The method of operating an electronic device may include an operation of performing image stabilization processing to reduce shaking of an image by performing image processing on image frames acquired through the camera module while controlling the driving unit. The method of operating an electronic device may include an operation of controlling a position of the lens assembly or the image sensor based on a first gain value when operating based on the first operation mode, and an operation of controlling a position of the lens assembly or the image sensor based on a second gain value different from the first gain value when operating based on the second operation mode.

[0008] A non-transitory computer-readable recording medium may cause an electronic device including a camera module, when executed, to perform a method including: controlling a driving unit to move a position of a lens assembly or an image sensor based on motion information acquired from a sensor that detects motion of the electronic device, based on a first operating mode among a plurality of operating modes associated with an operation of a driving unit that moves a lens assembly or an image sensor within a driving range. The non-transitory computer-readable recording medium may cause an electronic device including a camera module, when executed, to perform a method including: controlling a driving unit to position a lens assembly or an image sensor at a reference position within the driving range, based on a second operating mode different from the first operating mode among a plurality of operating modes. The non-transitory computer-readable recording medium may cause an electronic device including a camera module, when executed, to perform a method including: performing image processing on an image frame acquired through the camera module while controlling the driving unit to perform image stabilization processing to reduce shaking of the image. A computer-readable non-transitory recording medium can cause an electronic device including a camera module to perform a method, the method comprising: when the electronic device operates based on a first operating mode, the driving unit controls the position of the lens assembly or the image sensor based on a first gain value; and when the electronic device operates based on a second operating mode, the driving unit controls the position of the lens assembly or the image sensor based on a second gain value different from the first gain value.

[0009] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0010] FIG. 2 is a block diagram illustrating a camera module according to various embodiments.

[0011] Figure 3 is a block diagram showing the configuration of an electronic device according to one embodiment.

[0012] Figure 4 is an exploded perspective view of a camera module according to one embodiment.

[0013] Figure 5 is a perspective view of a camera module according to one embodiment.

[0014] FIG. 6 is a block diagram illustrating a structure in which an electronic device performs image stabilization processing according to one embodiment.

[0015] FIG. 7 is a drawing for explaining an operation of controlling a driving unit based on a first operation mode according to one embodiment.

[0016] FIG. 8 is a drawing for explaining an operation of controlling a driving unit based on a second operation mode according to one embodiment.

[0017] FIG. 9 is a drawing for explaining an operation of controlling the position of a lens assembly or an image sensor based on a gain value according to one embodiment.

[0018] FIG. 10 is a flowchart illustrating a process in which an electronic device performs a first operation mode or a second operation mode based on movement information according to an embodiment.

[0019] FIG. 11 is a flowchart illustrating a process in which an electronic device according to one embodiment changes a first operation mode to a second operation mode and performs the second operation mode.

[0020] FIG. 12 is a diagram for explaining an operation for determining whether the frequency of motion information satisfies a specified condition according to one embodiment.

[0021] FIG. 13 is a diagram for explaining an operation for determining whether the frequency of motion information satisfies a specified condition according to one embodiment.

[0022] Figure 14 is a flowchart illustrating a process of a second operation mode according to one embodiment.

[0023] FIG. 15 is a diagram for explaining an operation of controlling the position of a lens assembly or an image sensor based on a second gain value different from a first gain value according to one embodiment.

[0024] Figure 16 is a flowchart illustrating a process for fixing a target value to a specified value according to one embodiment.

[0025] FIG. 17 is a diagram for explaining the effect of improving the performance of image stabilization processing by fixing the target value to a specified value according to one embodiment.

[0026] FIG. 18 is a flowchart illustrating a process for performing an operation mode based on user input according to one embodiment.

[0027] FIG. 19 is a flowchart illustrating a process for performing an operation mode based on detection of an external device according to one embodiment.

[0028] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention. However, the disclosed embodiments may be implemented in various different forms and are not limited to the embodiments described herein.

[0029] When the movement of the electronic device shakes above a certain frequency, there was a problem of difficulty in controlling the driving unit that moves the lens or image sensor within the driving range and a problem of reduced performance of VDIS (and / or EIS).

[0030] In one embodiment, an electronic device and an operating method thereof may be provided that allow the position of a lens or an image sensor to be stably maintained even when the movement of the electronic device shakes above a specific frequency. For example, a camera module that controls the position of a lens or an image sensor, an electronic device including the camera module, and an operating method thereof may be provided so that the position of the lens or the position of the image sensor can be stably maintained even when the electronic device shakes above a specific frequency. Accordingly, even when shooting a video or a preview video in a situation where the electronic device shakes significantly, the performance of image stabilization may be improved, and a user may obtain a video or a preview video with improved quality. The user may obtain a preview video or video with shake-compensated even in a situation where the electronic device shakes or is shaken by various impacts while shooting a preview video or video through the electronic device.

[0031] The technical problems to be achieved in this document are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field pertaining to the present disclosure from the description of the present disclosure.

[0032] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0033] The processor (120) may control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing, for example, software (e.g., a program (140)), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (120) may store a command or data received from another component (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the command or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0034] The auxiliary processor (123) may control at least a part of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0035] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0036] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0037] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0038] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0039] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0040] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0041] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0042] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0043] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0044] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0045] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0046] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0047] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0048] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0049] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) may support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0050] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. According to some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0051] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0052] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0053] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service by itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0054] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments disclosed in this document are not limited to the aforementioned devices.

[0055] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0056] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0057] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more commands stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one command among the one or more commands stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one command called. The one or more commands may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0058] According to one embodiment, the method according to the various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0059] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0060] FIG. 2 is a block diagram (200) illustrating a camera module (180) according to various embodiments. Referring to FIG. 2, the camera module (180) may include a lens assembly (210), a flash (220), an image sensor (230), an image stabilizer (240), a memory (250) (e.g., a buffer memory), or an image signal processor (260). The lens assembly (210) may collect light emitted from a subject that is a target of image capturing. The lens assembly (210) may include one or more lenses. According to one embodiment, the camera module (180) may include a plurality of lens assemblies (210). In this case, the camera module (180) may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies (210) may have the same lens properties (e.g., angle of view, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties that are different from the lens properties of the other lens assemblies. A lens assembly (210) may include, for example, a wide-angle lens or a telephoto lens.

[0061] The flash (220) can emit light used to enhance light emitted or reflected from a subject. According to one embodiment, the flash (220) can include one or more light-emitting diodes (e.g., red-green-blue (RGB) LED, white LED, infrared LED, or ultraviolet LED), or a xenon lamp. The image sensor (230) can acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through the lens assembly (210) into an electrical signal. According to one embodiment, the image sensor (230) can include one image sensor selected from among image sensors having different properties, such as an RGB sensor, a black and white (BW) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same property, or a plurality of image sensors having different properties. Each image sensor included in the image sensor (230) can be implemented using, for example, a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor.

[0062] The image stabilizer (240) can move at least one lens or image sensor (230) included in the lens assembly (210) in a specific direction or control the operating characteristics of the image sensor (230) (e.g., adjusting the read-out timing, etc.) in response to the movement of the camera module (180) or the electronic device (101) including the same. This allows compensating for at least some of the negative effects of the movement on the captured image. In one embodiment, the image stabilizer (240) can detect such movement of the camera module (180) or the electronic device (101) using a gyro sensor (not shown) or an acceleration sensor (not shown) disposed inside or outside the camera module (180). In one embodiment, the image stabilizer (240) can be implemented as, for example, an optical image stabilizer. The memory (250) can temporarily store at least a portion of the image acquired through the image sensor (230) for the next image processing task. For example, when image acquisition is delayed due to the shutter, or when multiple images are acquired at high speed, the acquired original image (e.g., a Bayer-patterned image or a high-resolution image) is stored in the memory (250), and a corresponding copy image (e.g., a low-resolution image) can be previewed through the display module (160). Thereafter, when a specified condition is satisfied (e.g., a user input or a system command), at least a portion of the original image stored in the memory (250) can be acquired and processed, for example, by the image signal processor (260). According to one embodiment, the memory (250) may be configured as at least a portion of the memory (130), or as a separate memory that operates independently therefrom.

[0063] The image signal processor (260) can perform one or more image processing operations on an image acquired through an image sensor (230) or an image stored in a memory (250). The one or more image processing operations may include, for example, depth map generation, 3D modeling, panorama generation, feature extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, the image signal processor (260) may perform control (e.g., exposure time control, read-out timing control, etc.) on at least one of the components included in the camera module (180) (e.g., the image sensor (230)). The image processed by the image signal processor (260) may be stored back in the memory (250) for further processing or may be provided to an external component of the camera module (180) (e.g., the memory (130), the display module (160), the electronic device (102), the electronic device (104), or the server (108)). In one embodiment, the image signal processor (260) may be at least a part of the processor (120). It may be configured as a separate processor that is configured or operates independently from the processor (120). If the image signal processor (260) is configured as a separate processor from the processor (120), at least one image processed by the image signal processor (260) may be displayed through the display module (160) as is or after undergoing additional image processing by the processor (120).

[0064] According to one embodiment, the electronic device (101) may include a plurality of camera modules (180), each having different properties or functions. In this case, for example, at least one of the plurality of camera modules (180) may be a wide-angle camera, and at least another may be a telephoto camera. Similarly, at least one of the plurality of camera modules (180) may be a front camera, and at least another may be a rear camera.

[0065] Figure 3 is a block diagram showing the configuration of an electronic device (300) according to one embodiment.

[0066] The electronic device (300) of FIG. 3 may be referenced by the electronic device (101) of FIG. 1. With respect to FIG. 3, the configuration described in FIGS. 1 and 2 may be briefly described or the description may be omitted.

[0067] In FIG. 3, the electronic device (300) is illustrated as including a sensor (310), a camera module (320), a processor (330), and a memory (340), but the configuration of the electronic device (300) is not limited thereto. For example, the electronic device (300) may omit at least one of the above-described configurations, or may further include at least one other configuration.

[0068] According to one embodiment, the electronic device (300) may include a sensor (310). The sensor (310) of FIG. 3 may be included in the sensor module (176) of FIG. 1. According to one embodiment, the sensor (310) may detect movement of the electronic device (300). For example, the electronic device (300) may obtain movement information of the electronic device (300) through the sensor (310). In one example, the sensor (310) may include at least one of a gyro sensor and an acceleration sensor. For example, the electronic device (300) may obtain angular velocity information of the electronic device (300) through the gyro sensor. For example, the electronic device (300) may obtain acceleration information of the electronic device (300) through the acceleration sensor. However, the present invention is not limited thereto. For example, the electronic device (300) may include various sensors capable of detecting movement of the electronic device (300).

[0069] According to one embodiment, the sensor (310) may be formed separately from the camera module (320) or may be included in the camera module (320). In one example, the sensor (310) may be included in a printed circuit board (not shown) on which the processor (330) is mounted.

[0070] According to one embodiment, the electronic device (300) may include a camera module (320). According to one embodiment, the camera module (320) may include a lens assembly (321), an image sensor (322), and a driving unit (333). However, the configuration of the camera module (320) is not limited thereto. For example, the camera module (320) may omit at least one of the above-described configurations, or may further include at least one other configuration.

[0071] According to one embodiment, the driving unit (333) can move the lens assembly (321) or the image sensor (322) within the driving range. For example, the driving unit (333) can move the lens assembly (321) or the image sensor (322) within the optical image stabilization (OIS) driving range. For example, the driving unit (333) can move the lens assembly (321) or the image sensor (322) within the auto focus (AF) driving range. For example, the driving unit (333) can move the lens assembly (321) or the image sensor (322) within the OIS driving range and the AF driving range.

[0072] According to one embodiment, the electronic device (300) may perform an OIS function for image stabilization. The OIS may include a lens shift method in which the lens assembly (321) moves or tilts in a direction substantially perpendicular to the optical axis, and / or a sensor shift method in which the image sensor (322) moves or tilts in a direction substantially perpendicular to the optical axis. In one example, the OIS may include a method in which a reflective member (e.g., a prism or a mirror) tilts, or a reflective member shift method. The reflective member may be disposed on the lens, and / or disposed between the lens assembly and the image sensor. However, the present invention is not limited thereto. For example, the OIS may include a module tilt method in which a camera module tilts.

[0073] According to one embodiment, in a lens shift type OIS, the driving unit (333) may include a lens assembly (321). For example, the camera module (320) may include a movably arranged lens assembly (321), and an OIS carrier that accommodates the lens assembly (321) and moves or tilts in a direction substantially perpendicular to the optical axis together with the lens assembly (321).

[0074] According to one embodiment, in a sensor shift type OIS, the driving unit (333) may include an image sensor (322). For example, the camera module (320) may include an image sensor (322) arranged to be movable.

[0075] The present disclosure may be applied to at least one of an electronic device using a lens shift type OIS, an electronic device using a sensor shift type OIS, an electronic device using a module tilt type OIS, or an electronic device using an OIS using a reflective member tilting type.

[0076] The present disclosure may also be applied when an electronic device (300) includes two or more camera modules (320). According to one embodiment, the electronic device (300) may include two or more camera modules (320). For example, the electronic device (300) may include a first camera module including a telephoto lens, and a second camera module including a wide-angle lens. For example, the electronic device (300) may also include a first camera module including a telephoto lens, a second camera module including a wide-angle lens, and a third camera module including an ultra-wide lens.

[0077] According to one embodiment, at least some of the camera modules (320) included in the electronic device (300) may support the OIS function. According to one embodiment, at least some of the camera modules (320) included in the electronic device (300) may not support the OIS function. For example, the first camera module and the second camera module may support the OIS function, and the third camera module may not support the OIS function. According to one embodiment, when the electronic device (300) includes two or more camera modules (320), the electronic device (300) may include two or more OIS processing units that control the respective OIS driving units. For example, the electronic device (300) may include a first camera module and a second camera module that support the OIS function, and the first OIS processing unit may control the OIS driving unit included in the first camera module, and the second OIS processing unit may control the OIS driving unit included in the second camera module.

[0078] According to one embodiment, when the electronic device (300) includes two or more camera modules (320), the electronic device (300) may include one OIS processing unit that controls each of the OIS driving units. For example, the electronic device (300) may include a first camera module and a second camera module that support the OIS function, and may control both the OIS driving unit included in the first camera module and the OIS driving unit included in the second camera module through a single OIS processing unit. Although FIGS. 3 to 17 of the present disclosure describe one camera module included in the electronic device, this is one example and does not limit the technical idea of ​​the present disclosure.

[0079] According to one embodiment, the electronic device (300) may include a processor (330) and a memory (340). In one example, the electronic device (300) may include one or more processors (330) and a memory (340) storing one or more instructions. In the present disclosure, it may be understood that the electronic device (300) is controlled by the one or more processors (330) executing instructions stored in the memory (340). For example, it may be understood that the operation of at least one of the sensor (310) or the camera module (320) is controlled by the one or more processors (330) executing instructions stored in the memory (340). For example, one or more commands stored in the memory (340) may be executed by one or more processors (330) so that the electronic device (300) may control the driving unit (333) to move the position of the lens assembly (321) or the image sensor (322) based on the movement information obtained from the sensor (310) based on the first operation mode. For example, one or more commands stored in the memory (340) may be executed by one or more processors (330) so that the electronic device (300) may control the driving unit (333) so that the lens assembly (321) or the image sensor (322) is positioned at a reference position within the operation range based on the second operation mode.

[0080] Fig. 4 is an exploded perspective view of a camera module (400) according to one embodiment. Fig. 5 is a perspective view of a camera module (400) according to one embodiment.

[0081] The camera module (400) of FIGS. 4 and 5 may be referenced by the camera module (180) of FIGS. 1 and 2 and / or the camera module (320) of FIG. 3.

[0082] Referring to FIGS. 4 and 5, a camera module (400) according to an embodiment may include a lens assembly (410), an optical image stabilization (OIS) driving unit for driving the optical image stabilization (OIS) of the camera module (400), an auto focus (AF) driving unit for driving the auto focus (AF) of the camera module (400), a housing (450), and an image sensor (460). However, the configuration of the camera module (400) is not limited thereto. For example, the camera module (400) may omit at least one component among the above-described components, or may further include at least one other component. For example, the camera module (400) may further include at least one of a base on which the housing (450) is placed, a cover covering at least a portion of the housing (450), or a printed circuit board connected to the image sensor (460).

[0083] According to one embodiment, the lens assembly (410) may include at least one lens arranged such that an image of a subject is formed on the image sensor (460). The camera module (400) may receive light through the lens assembly (410). For example, the light may enter the interior of the camera module (400) through at least a portion of the at least one lens. In one example, the light may enter the interior of the camera module (400) in a direction (40) substantially parallel to an optical axis (e.g., in the -z-axis direction).

[0084] According to one embodiment, at least a portion of the lens assembly (410) may be exposed to the exterior of the camera module (400). For example, at least a portion of the lens assembly (410) may protrude from an upper surface of the housing (450) (e.g., a surface facing the +z direction).

[0085] According to one embodiment, at least a portion of the lens assembly (410) may be disposed inside the housing (450). In one example, the lens assembly (210) may be disposed movably relative to the housing (450). For example, the lens assembly (210) may be movable in at least one of a first direction (e.g., the x-axis direction or the -x-axis direction) and / or a second direction (e.g., the y-axis direction or the -y-axis direction) relative to the housing (450) by an OIS driving unit (e.g., the first carrier (420) and / or the second carrier (430)) for driving the OIS of the camera module (400). For example, the lens assembly (210) may be movable in a third direction (e.g., the z-axis direction or the -z-axis direction) relative to the housing (450) by an AF driving unit (e.g., the third carrier (440)) for driving the AF of the camera module (400).

[0086] According to one embodiment, the camera module (400) may include an OIS driving unit for optical image stabilization. The OIS driving unit may move the lens assembly (410) in a first direction (e.g., in the x-axis direction or the -x-axis direction) and / or a second direction (e.g., in the y-axis direction or the -y-axis direction). The first direction may be different from the second direction. According to one embodiment, the OIS driving unit may include an OIS actuator configured to move the lens assembly (410) for optical image stabilization. For example, the OIS driving unit may move in a direction substantially perpendicular to a third direction (e.g., in the z-axis direction or the -z-axis direction) in which a distance between the lens assembly (410) and the image sensor (460) is changed for optical image stabilization.

[0087] According to one embodiment, the OIS drive unit may include a first carrier (420) that transports the lens assembly (410) in a first direction (e.g., in the x-axis direction or the -x-axis direction) and / or a second direction (e.g., in the y-axis direction or the -y-axis direction). In one example, the lens assembly (410) may be coupled to the first carrier (420). For example, the lens assembly (410) may be fixed to the first carrier (420). For example, the lens assembly (410) may be at least partially surrounded by the first carrier (420).

[0088] According to one embodiment, a first magnet (421) may be disposed on a first side (e.g., a side facing the -y direction) of a first carrier (420). The first magnet (421) may be disposed to face a first coil (451). The first coil (451) may be disposed inside a housing (450) facing the first side of the first carrier (420) on which the first magnet (421) is disposed. The first carrier (420) may be moved relative to the housing (450) by a force in a second direction (e.g., in the y-axis direction or the -y-axis direction) applied to the first magnet (421) by a magnetic field generated by a current flowing in the first coil (451). As the first carrier (420) moves, the lens assembly (410) may be moved in the second direction.

[0089] According to one embodiment, a second magnet (422) may be disposed on a second side (e.g., a side facing the -x direction) of the first carrier (420). The second magnet (422) may be disposed to face the second coil (452). The second coil (452) may be disposed inside the housing (450) facing the second side of the first carrier (420) on which the second magnet (422) is disposed. The first carrier (420) may be moved relative to the housing (450) by a force in a first direction (e.g., in the x-axis direction or the -x-axis direction) applied to the second magnet (422) by a magnetic field generated by a current flowing in the second coil (452). As the first carrier (420) moves, the lens assembly (410) may be moved in the first direction.

[0090] According to one embodiment, the first side of the first carrier (420) may be substantially perpendicular to the second side of the first carrier (420). The first magnet (421) may be arranged to be substantially perpendicular to the second magnet (422). The first coil (451) may be arranged to be substantially perpendicular to the second coil (452).

[0091] According to one embodiment, the third magnet (445) may be positioned substantially perpendicular to the first magnet (421). The second magnet (422) may be positioned opposite the third magnet (445). The second magnet (422) may be positioned substantially parallel to the third magnet (445). However, this is not limited thereto.

[0092] According to one embodiment, the OIS driving unit may include a first carrier (420) coupled with a lens assembly (410), and a second carrier (430) configured to transport the first carrier (420) in a second direction (e.g., in the y-axis direction or the -y-axis direction). For example, the first carrier (420) may be disposed on the second carrier (430), such that as the second carrier (430) moves in the second direction, the first carrier (420) may be transported in the second direction. For example, the first carrier (420) and / or the second carrier (430) may be moved in the first direction and / or the second direction by a sphere.

[0093] According to one embodiment, when the OIS driving unit further includes a second carrier (430), a first sphere (432) may be disposed between the first carrier (420) and the second carrier (430). The first carrier (420) may move on the second carrier (430) by the first sphere (432). For example, the first carrier (420) may move in a first direction (e.g., in the x-axis direction or the -x-axis direction) with respect to the housing (450) on the second carrier (430) by the first sphere (432). However, the present invention is not limited thereto. For example, the second carrier (430) and the first sphere (432) may be omitted in the OIS driving unit.

[0094] According to one embodiment, when the OIS driving unit further includes a second carrier (430), at least one of the first carrier (420) or the second carrier (430) may include a first guide portion (431) that guides the first sphere (432) to move in a first direction. The first sphere (432) may be disposed on the first guide portion (431). As the first sphere (432) is guided along the first guide portion (431), the first carrier (420) may move in the first direction. For example, as the first sphere (432) moves, rolls, rotates, or tilts along the first guide portion (431), the first carrier (420) may move in the first direction. When the first carrier (420) moves in the first direction, the lens assembly (410) accommodated in the first carrier (420) can move together in the first direction.

[0095] According to one embodiment, a second sphere (442) may be disposed between the first carrier (420) and the third carrier (440). In one example, when the OIS driving unit further includes a second carrier (430), the second sphere (442) may be disposed between the second carrier (430) and the third carrier (440). The first carrier (420) may move on the third carrier (440) by the second sphere (442). For example, the first carrier (420) may move in a first direction and / or a second direction with respect to the housing (450) by the second sphere (442).

[0096] According to one embodiment, at least one of the first carrier (420) or the third carrier (440) may include a second guide portion (441) that guides the second sphere (442) to move in the first direction and / or the second direction. In one example, when the OIS driving unit further includes the second carrier (430), at least one of the second carrier (430) or the third carrier (440) may include a second guide portion (441) that guides the second sphere (442) to move in the second direction. The second sphere (442) may be disposed on the second guide portion (441). The second sphere (442) may be disposed on the second guide portion (441). By guiding the second sphere (442) along the second guide portion (441), the first carrier (420) may move in the first direction and / or the second direction. For example, the first carrier (420) can move in the first direction and / or the second direction by the second sphere (442) moving, rolling, rotating, or tilting along the second guide portion (441). When the first carrier (420) moves in the first direction and / or the second direction, the lens assembly (410) accommodated in the first carrier (420) can move together in the first direction and / or the second direction.

[0097] According to one embodiment, at least one of the first guide portion (431) or the second guide portion (441) may have a guide groove or rail shape, but is not limited thereto.

[0098] According to one embodiment, the OIS driving unit may include at least one of a plurality of first spheres (432), a plurality of first guide portions (431), a plurality of second spheres (442), or a plurality of second guide portions (441).

[0099] According to one embodiment, the camera module (400) may include an AF driving unit for an auto-focus function. The AF driving unit may move the lens assembly (410) in a third direction (e.g., in the z-axis direction or the -z-axis direction). According to one embodiment, the AF driving unit may include an AF actuator configured to move the lens assembly (410) for the auto-focus function. For example, the AF driving unit may move the lens assembly (410) in a third direction in which a distance between the image sensor (460) and the lens assembly (410) changes for the auto-focus function.

[0100] According to one embodiment, the AF driving unit may include a third carrier (440) and a third sphere (444). However, the components of the AF driving unit are not limited thereto. For example, the AF driving unit may omit at least one of the above-described components, or may further include at least one other component. For example, the AF driving unit may further include a third magnet (445) and a third coil (453). In one example, the AF driving unit may also include a plurality of third spheres (444).

[0101] In one embodiment, the third carrier (440) can transport at least a portion of the OIS drive unit that accommodates the lens assembly (410) in a third direction. For example, the third carrier (440) can transport at least one of the first carrier (420) that accommodates the lens assembly (410) or the second carrier (430) in the third direction. In one example, the third carrier (440) can surround at least a portion of a side surface of the second carrier (430).

[0102] According to one embodiment, the third sphere (444) may be positioned between the third carrier (440) and the housing (450). The third sphere (444) may be positioned to move in a third direction with respect to the housing (450). For example, a plurality of third spheres (444) may be positioned side by side along the third direction.

[0103] According to one embodiment, a third magnet (445) may be disposed on one side (e.g., a side facing the +x direction) of a third carrier (440). A third coil (453) may be disposed to face the third magnet (445). For example, the third coil (453) may be disposed on an inner side of a housing (450) facing the side of the third carrier on which the third magnet (445) is disposed. The third carrier (440) may be moved relative to the housing (450) by a third direction force acting on the third magnet (445) due to a magnetic field generated by a current flowing in the third coil (453). As the third carrier (440) moves, the lens assembly (410) may be moved in the third direction. For example, as the third carrier (440) moves, the distance from the image sensor (460) to the lens assembly (410) may be changed.

[0104] According to one embodiment, the third carrier (440) may include a third guide portion (443) that guides the third sphere (444) to move in a third direction. The third guide portion (443) may be formed to allow the third carrier (440) to move in the third direction. For example, the third guide portion (443) may be formed so that the third direction is a longitudinal direction. For example, the third guide portion (443) may have a guide groove or rail shape, but is not limited thereto. In one example, the third carrier (440) may include a plurality of third guide portions (443). For example, the third carrier (440) may include a plurality of third guide portions (443) arranged with a third magnet (445) therebetween. Although not shown in FIGS. 4 and 5, the housing (450) may include a guide portion formed at a position corresponding to the third guide portion (443) of the third carrier (440).

[0105] According to one embodiment, the third sphere (444) may be placed on the third guide portion (443). The third guide portion (443) may guide the third sphere (444) to move in a third direction. For example, the third sphere (444) may move, roll, rotate, or tilt on the third guide portion (443). However, the present invention is not limited thereto.

[0106] According to one embodiment, the third sphere (444) is guided along the third guide portion (443), so that the third carrier (440) can be transported in a third direction. When the third carrier (440) is transported in the third direction, at least some of the OIS driving units arranged on the third carrier (440) can be transported in the third direction. The lens assembly (410) can be transported in the third direction by the movement of at least some of the OIS driving units.

[0107] According to one embodiment, the housing (450) may form the exterior of the camera module (400). For example, the housing (450) may provide a framework for supporting other components. The shape of the housing (450) may be modified depending on the shape of the camera module (400) or the components included in the camera module (400). For example, the housing (450) may have a box shape. However, the present invention is not limited thereto.

[0108] According to one embodiment, the housing (450) may provide a space for accommodating other components. Components included in the camera module (400) may be arranged inside the housing (450). For example, at least one of a portion of the lens assembly (410), an OIS driving unit for OIS driving of the camera module (400), or an AF driving unit for AF driving of the camera module (400) may be arranged in the space. However, the present invention is not limited thereto.

[0109] According to one embodiment, the image sensor (460) can detect light incident in the z-axis direction through the lens assembly (410). The electronic device can control an OIS driving unit (e.g., the first carrier (420) and / or the second carrier (430)) to move the position of the image detected through the image sensor (460).

[0110] FIGS. 4 and 5 are merely examples for explaining the configuration of the camera module (400), and the configuration of the camera module (400) can be applied in various ways. For example, the OIS driving unit of the camera module (400) may be configured to move the position of the image sensor (460) rather than the lens assembly (410). Alternatively, the camera module (400) may include a curved camera in which the path (40) of light incident on the image sensor (460) is curved one or more times.

[0111] The method described in FIGS. 4 and 5 is described on the premise of a VCM (voice coil motor), but this is only one example, and the technical idea of ​​the present disclosure described in FIGS. 4 and 5 can also be applied to camera modules using motors of other types. Among the VCM type camera modules, depending on the form of the electromagnetic force (e.g., Lorenz, solenoid), the shape of the magnet and the coil (e.g., the first coil (451) and the second coil (452)) and / or the driving method of the OIS driving unit may differ from the contents described in FIGS. 4 and 5.

[0112] FIG. 6 is a block diagram illustrating a structure in which an electronic device performs image stabilization processing according to one embodiment.

[0113] The contents of FIG. 6 may be referenced by the contents of FIG. 1, FIG. 2, FIG. 3, FIG. 4, and / or FIG. 5 described above. The same terms are used for configurations that are identical or substantially identical to those described above.

[0114] Referring to FIG. 6, an electronic device (500) according to an embodiment may include a sensor (510), an OIS MCU (520), a processor (530), and a position control unit (540). However, the configuration of the electronic device (500) is not limited thereto. For example, the electronic device (500) may omit at least one of the above-described components, or may further include at least one other component. For example, the electronic device (500) may further include an AF control unit (531), a lens assembly (610), one or more magnets (611), one or more coils (612), and / or a position sensor (613) included in the processor (530).

[0115] According to one embodiment, the sensor (510) may be a separate component from the camera module (e.g., the camera module (320) of FIG. 3) and the processor (530). However, the present invention is not limited thereto. For example, the sensor (510) may be included in the camera module or may be included in the processor (530). For example, the sensor (510) may be included in a printed circuit board on which the processor (530) is mounted. In one example, the sensor (510) and the OIS MCU (520) may be a single component.

[0116] According to one embodiment, the OIS MCU (micro controller unit) (520) may be a separate configuration from the camera module (e.g., the camera module (320) of FIG. 3) and the processor (530). For example, the OIS MCU (520) may be a separate integrated circuit (IC). For example, the OIS MCU (520) may be included in a processor different from the processor (530). However, the present invention is not limited thereto. For example, the OIS MCU (520) may be included in the camera module or the processor (530). For example, the OIS MCU (520) may be included in a printed circuit board on which the processor (530) is mounted.

[0117] According to one embodiment, the position control unit (540) may be a separate component from the OIS MCU (520) and the processor (530). However, this is not limited thereto. For example, the position control unit (540) may be included in the OIS MCU (520) or may be included in the processor (530). In one example, the position control unit (540) may be included in a processor different from the processor (530).

[0118] According to one embodiment, in the present disclosure, it may be understood that the operation of at least one of the sensor (510) or the camera module (e.g., the camera module (320) of FIG. 3) is controlled by at least one of the processor (530) or the OIS MCU (520) executing commands stored in a memory (e.g., the memory (130) of FIG. 1). For example, it may be understood that the operation of the position control unit (540) is controlled by at least one of the processor (530) or the OIS MCU (520) executing commands stored in a memory.

[0119] According to one embodiment, the sensor (510) can detect the movement of the electronic device (500). For example, the sensor (510) can detect at least one of the angular velocity or acceleration of the electronic device (500). The electronic device (500) can obtain movement information from the sensor (510). For example, the electronic device (500) can obtain at least one of the angular velocity information or the acceleration information from the sensor (510). However, the present invention is not limited thereto. For example, the electronic device (500) can obtain various information related to the movement of the electronic device (500) from the sensor (510).

[0120] According to one embodiment, the OIS MCU (520) may determine a target value related to OIS operation based on movement information of the electronic device (500) obtained from the sensor (510). For example, the OIS MCU (520) may determine at least one of an angular value or an angular velocity value based on the movement information. For example, the OIS MCU (520) may calculate at least one of an angular value or an angular velocity value related to OIS operation.

[0121] According to one embodiment, the OIS MCU (520) may include a first filter. In one example, the first filter may include a phase compensation filter (PCF). The first filter may compensate for a delay of angular velocity information acquired from the sensor (510). According to one embodiment, the OIS MCU (520) may include a second filter. In one example, the second filter may remove a direct current (DC) component of the angular velocity information acquired from the sensor (510). For example, the second filter may remove a DC component of the angular velocity information compensated by the first filter. According to one embodiment, the OIS MCU (520) may include a first operation unit. The first operation unit may convert the angular velocity information acquired from the sensor (510) into angle information. For example, the first operation unit may integrate the angular velocity information acquired from the sensor (510) and convert it into angle information. For example, the first operation unit can integrate the angular velocity value from which the DC component has been removed by the second filter and convert it into an angular value. According to one embodiment, the OIS MCU (520) can include a second operation unit. The second operation unit can adjust the suppression ratio (SR) of the angular velocity information acquired from the sensor (510). For example, the second operation unit can adjust the suppression ratio of the angular value acquired from the first operation unit. According to one embodiment, the OIS MCU (520) can include a variable attenuator. The variable attenuator can secure a margin for moving the lens assembly or the image sensor to a reference position.

[0122] According to one embodiment, the OIS MCU (520) can determine a target value related to OIS operation through the above-described configurations. However, the configuration of the OIS MCU (520) is not limited to the above-described configurations. For example, the OIS MCU (520) may omit at least one of the above-described configurations or may further include at least one other configuration.

[0123] According to one embodiment, the processor (530) may include a VDIS driving unit, an EIS driving unit, and an AF control unit (531). However, the configuration of the processor (530) is not limited thereto. For example, the processor (530) may omit at least one of the above-described configurations, or may further include at least one other configuration. In one example, the processor (530) may include a sensor (510), an OIS MCU (520), and / or a position control unit (540).

[0124] According to one embodiment, the electronic device (500) may include a position sensor (613) that obtains position information of the lens assembly (610) or position information of the image sensor. For example, the camera module may include the position sensor (613). In one example, the camera module may include a plurality of position sensors (613). In one example, the position sensors (613) may include Hall sensors. However, the present invention is not limited thereto. For example, the camera module may include at least one of a tunnel magnetoresistance (TMR) sensor, an anisotropic magneto-resistance (AMR) sensor, or a giant magneto-resistance (GMR) sensor. The camera module can detect the movement (rotation) of a magnetic body based on a resistance value that changes based on the relative angle of a plurality of magnetic bodies using a tunnel magneto-resistance sensor, or can detect the movement (rotation) of a magnetic body using at least one of an anisotropic magneto-resistance (AMR) sensor or a giant magneto-resistance (GMR) sensor.

[0125] According to one embodiment, the position control unit (540) can obtain a target value related to the determined OIS operation from the OIS MCU (520). For example, the position control unit (540) can obtain at least one of the determined angle value or angular velocity value.

[0126] According to one embodiment, the position control unit (540) can obtain a sensing value related to the position information of the lens assembly (610) (or, image sensor) from the position sensor (613). For example, the position control unit (540) can obtain a sensing value related to at least one of the velocity information or the acceleration information of the lens assembly (610) (or, image sensor) from the position sensor (613).

[0127] According to one embodiment, the position control unit (540) may determine an output value for controlling the driving unit based on the determined target value and sensed value. For example, the position control unit (540) may determine an output value related to the intensity of the current applied to the coil (612) based on the determined target value and sensed value. The electronic device (500) may control the driving unit to move the position of the lens assembly (610) (e.g., the lens assembly (321) of FIG. 3) or the image sensor (e.g., the image sensor (322) of FIG. 3) based on the determined output value.

[0128] According to one embodiment, the target value determined by the OIS MCU (520) may be transmitted to the processor (530). For example, at least one of the determined angle value or angular velocity value may be transmitted to the processor (530). For example, the target value determined by the OIS MCU (520) may be transmitted to the VDIS driving unit and / or the EIS driving unit. The processor (530) may perform image stabilization processing to reduce image shaking by performing image processing on image frames acquired through the camera module based on the determined target value. For example, the processor (530) may perform image stabilization processing to reduce image shaking by performing image processing on image frames acquired through the camera module based on at least one of the determined angle value or angular velocity value. For example, the processor (530) may perform at least one of VDIS (video digital image stabilization) or EIS (electronic image stabilization) on the image frames. For example, the processor (530) can compensate for shaking of the electronic device (500) by performing at least one of video digital image stabilization (VDIS) or electronic image stabilization (EIS) on image frames.

[0129] According to one embodiment, the processor (530) may obtain a sensing value related to the position information of the lens assembly (610) (or the image sensor) from the position sensor (613). For example, the processor (530) may obtain a sensing value related to the position information of the lens assembly (610) (or the image sensor) directly from the position sensor (613). For example, the processor (530) may obtain a sensing value related to at least one of the velocity information or the acceleration information of the lens assembly (610) (or the image sensor) from the position sensor (613). For example, the sensing value obtained from the position sensor (613) may be transmitted to the VDIS driving unit and / or the EIS driving unit. The processor (530) may perform image stabilization processing to reduce shaking of the image by performing image processing on image frames obtained through the camera module based on the sensing value. For example, the processor (530) may perform image stabilization processing to reduce image shaking by performing image processing on image frames acquired through the camera module based on at least one of an angular value or an angular velocity value. For example, the processor (530) may perform at least one of video digital image stabilization (VDIS) or electronic image stabilization (EIS) on the image frames. For example, the processor (530) may perform at least one of video digital image stabilization (VDIS) or electronic image stabilization (EIS) on the image frames to compensate for shaking of the electronic device (500).

[0130] According to one embodiment, the electronic device (500) may determine the camera operation mode as one of multiple operation modes associated with the operation of the driving unit. For example, the electronic device (500) may support multiple operation modes. For example, the electronic device (500) may support a first operation mode and a second operation mode. However, the present invention is not limited thereto. For example, the electronic device (500) may further support a third operation mode.

[0131] According to one embodiment, the second operating mode may include an operating mode that is performed when the movement of the electronic device shakes more than a certain frequency than the first operating mode. The third operating mode may include an operating mode that is performed when the movement of the electronic device shakes more than a certain frequency than the first and second operating modes.

[0132] According to one embodiment, the electronic device (500) may determine a camera operation mode based on movement information acquired from the sensor (510). For example, the electronic device (500) may perform at least one of a first operation mode, a second operation mode, or a third operation mode based on whether a frequency of the movement information satisfies a specified condition. For example, the electronic device (500) may perform the first operation mode based on detection of a frequency component lower than a specified frequency value from the movement information. For example, the electronic device (500) may perform the second operation mode based on detection of a frequency component higher than a specified frequency value from the movement information. However, the present invention is not limited thereto. For example, the electronic device (500) may perform at least one of the first operation mode, the second operation mode, or the third operation mode based on a user input. For example, the electronic device (500) may also perform at least one of the first operation mode, the second operation mode, or the third operation mode based on whether an external device is detected. In one example, at least two of the first operation mode, the second operation mode, and the third operation mode may be different operation modes. However, this is not limited thereto. In one example, the first operation mode, the second operation mode, and the third operation mode may be performed independently. For example, the first operation mode, the second operation mode, and the third operation mode may be performed selectively.

[0133] According to one embodiment, when the electronic device (500) operates based on the first operation mode, the driving unit may control the position of the lens assembly (610) or the image sensor based on the first gain value. For example, when the electronic device (500) operates based on the first operation mode, the position control unit (540) may determine an output value for controlling the driving unit based on the first gain value.

[0134] According to one embodiment, when the electronic device (500) operates based on the second operation mode, the driving unit may control the position of the lens assembly (610) or the image sensor based on the second gain value. For example, when the electronic device (500) operates based on the second operation mode, the position control unit (540) may determine an output value for controlling the driving unit based on the second gain value. The second gain value may be a value greater than the first gain value.

[0135] According to one embodiment, when the electronic device (500) operates based on the third operation mode, the driving unit may control the position of the lens assembly (610) or the image sensor based on the third gain value. For example, when the electronic device (500) operates based on the third operation mode, the position control unit (540) may determine an output value for controlling the driving unit based on the third gain value. The third gain value may be a value greater than the second gain value. The third gain value may be a value greater than the first gain value.

[0136] According to one embodiment, the electronic device (500) may set a target position to which the lens assembly (610) or the image sensor is moved based on a target value determined based on movement information acquired from the sensor (510) based on the first operation mode. The electronic device (500) may control a driving unit to move the position of the lens assembly (610) or the image sensor to the target position based on the movement information based on the first operation mode.

[0137] According to one embodiment, the electronic device (500) may set a reference position within the driving range as a target position to which the lens assembly (610) or the image sensor is moved based on the second operation mode. The electronic device (500) may control the driving unit so that the lens assembly (610) or the image sensor is positioned at the reference position within the driving range based on the second operation mode. For example, the electronic device (500) may control the driving unit so that the lens assembly (610) or the image sensor is moved to the reference position within the driving range based on the second operation mode.

[0138] According to one embodiment, the electronic device (500) may set a reference position within the driving range as a target position to which the lens assembly (610) or the image sensor is moved based on the third operation mode. The electronic device (500) may control the driving unit so that the lens assembly (610) or the image sensor is positioned at the reference position within the driving range based on the third operation mode. For example, the electronic device (500) may control the driving unit so that the lens assembly (610) or the image sensor is moved to the reference position within the driving range based on the third operation mode. In the present disclosure, the content regarding the third operation mode may be referenced by the content regarding the second operation mode.

[0139] FIG. 7 is a drawing for explaining an operation of controlling a driving unit based on a first operation mode according to one embodiment.

[0140] The camera module (700) of FIG. 7 can be referenced by the camera module described above.

[0141] Referring to FIG. 7, a camera module (700) according to one embodiment may include a lens assembly (710) and a housing (750). However, the present invention is not limited thereto. In the content of FIG. 7, the lens assembly (710) may be replaced with an image sensor.

[0142] According to one embodiment, the lens assembly (710) (or, image sensor) may be movably arranged in a first axis (720) direction and a second axis (730) direction substantially perpendicular to the optical axis of the lens assembly (710) (e.g., the optical axis (40) of FIG. 5) within the housing (750). The first axis (720) may be substantially perpendicular to the second axis (730). However, the present invention is not limited thereto. For example, the lens assembly (710) (or, image sensor) may also be movably arranged in a third axis direction substantially parallel to the optical axis of the lens assembly (710) within the housing (750).

[0143] According to one embodiment, the electronic device may control the driving unit (e.g., the driving unit (333) of FIG. 3) based on the first operation mode to move the position of the lens assembly (710) (or the image sensor). For example, the electronic device may control the OIS driving unit based on the first operation mode to move the position of the lens assembly (710) on the first and second axis planes. For example, the electronic device may move the position of the lens assembly (710) on the xy axis plane. However, the present invention is not limited thereto. For example, the electronic device may control the AF driving unit based on the first operation mode to move the position of the lens assembly (710) (or the image sensor) on the third axis that is substantially perpendicular to the first and second axis planes. The lens assembly (710) (or the image sensor) may move within a driving range. Referring to FIG. 7, the driving range may include the interior of the camera module (700) and / or the interior of the housing (750). However, it is not limited to this.

[0144] According to one embodiment, the electronic device may control the driving unit based on the movement information obtained from the sensor (e.g., the sensor (310) of FIG. 3) based on the first operation mode. For example, the electronic device may control the OIS driving unit (e.g., the OIS driving unit of FIG. 4) to control the position of the lens assembly (710) in the direction of the first axis (720). The electronic device may control the OIS driving unit to control the position of the lens assembly (710) in the direction of the second axis (730). The electronic device may move the lens assembly (710) (or the image sensor) in the target movement direction on the xy-axis plane through the position movement in the direction of the first axis (720) and the position movement in the direction of the second axis (730).

[0145] According to one embodiment, by controlling the OIS driving unit, the shape of the subject can appear at a first position in an image acquired through the camera without applying an offset to the lens assembly (710). When the lens assembly (710) moves in the target movement direction by controlling the OIS driving unit, the shape of the subject can appear at a second position in the image that has moved in the opposite direction.

[0146] In one embodiment, when operating based on the first operation mode, the electronic device may control the driving unit to control the position of the lens assembly (710) (or image sensor) based on the first gain value. For example, the electronic device may control the driving unit to fix the position of the lens assembly (710) (or image sensor) based on the first gain value.

[0147] FIG. 8 is a diagram illustrating an operation for controlling a driving unit based on a second operation mode according to one embodiment. In the present disclosure, the operation for controlling a driving unit based on a third operation mode may be referred to as the operation for controlling a driving unit based on a second operation mode.

[0148] The camera module (700) of FIG. 8 can be referenced by the camera module described above.

[0149] According to one embodiment, the electronic device may control the driving unit (e.g., the driving unit (333) of FIG. 3) based on the second operation mode to move the lens assembly (710) (or the image sensor) to a reference position (740) within the driving range. For example, the electronic device may control the OIS driving unit based on the second operation mode to move the lens assembly (710) to a reference position (740). For example, the electronic device may move the lens assembly (710) to a reference position (740) on an xy-axis plane. However, the present invention is not limited thereto. For example, the electronic device may control the AF driving unit based on the second operation mode to move the lens assembly (710) (or the image sensor) to a reference position (not shown) on a third axis that is substantially perpendicular to the first and second axis planes.

[0150] According to one embodiment, the electronic device can control the OIS driving unit (e.g., the OIS driving unit of FIG. 4) to move the lens assembly (710) to a reference position (740) in the direction of the first axis (720). The electronic device can control the OIS driving unit to move the lens assembly (710) to a reference position (740) in the direction of the second axis (730). The electronic device can move the lens assembly (710) (or the image sensor) in a direction toward the reference position (740) on the xy-axis plane through the position movement in the direction of the first axis (720) and the position movement in the direction of the second axis (730).

[0151] According to one embodiment, the reference position (740) may include at least one of the center of the driving range on the first axis (720) or the center of the driving range on the second axis (730), but is not limited thereto. For example, the reference position (740) may include at least one of the center of the camera module (700) or the center of the housing (750). In one example, the reference position with respect to the AF driving unit may include at least one of the infinity focus position or the 1.5 to 3 M focus position on the third axis, but is not limited thereto.

[0152] According to one embodiment, when operating based on the second operation mode, the electronic device may control the driving unit to control the position of the lens assembly (710) (or the image sensor) based on the second gain value. For example, the electronic device may control the driving unit to maintain (e.g., fix) the position of the lens assembly (710) (or the image sensor) based on the second gain value. For example, the electronic device may control the OIS driving unit based on the second gain value such that the position of the lens assembly (710) (or the image sensor) is maintained at a reference position (740) with respect to a first direction (e.g., the first axis (720) direction or the x-axis direction) and a second direction (e.g., the second axis (730) direction or the y-axis direction). For example, the electronic device may control the AF driving unit based on the second gain value such that the position of the lens assembly (710) (or the image sensor) is maintained at a reference position with respect to a third direction (e.g., the third axis direction or the z-axis direction).

[0153] In one embodiment, the second gain value may be a different value than the first gain value. For example, the second gain value may be a greater value than the first gain value.

[0154] According to one embodiment, the electronic device controls the driving unit based on a second gain value that is greater than the first gain value, so that the position of the lens assembly (710) (or the image sensor) can be more strongly maintained in the second operation mode than in the first operation mode. For example, the position of the lens assembly (710) (or the image sensor) can be more strongly maintained at the reference position (740). In one example, being 'strongly maintained' can include that the displacement that occurs when the same disturbance is applied can be less.

[0155] FIG. 9 is a drawing for explaining an operation of controlling the position of a lens assembly or an image sensor based on a gain value according to one embodiment.

[0156] The contents of Figure 9 may be referenced by the contents of other drawings.

[0157] According to one embodiment, the position control unit (840) can obtain a target value from the OIS MCU (820). For example, the position control unit (840) can obtain at least one of an angular velocity value or an acceleration value determined by the OIS MCU (820) from the OIS MCU (820). The target value can be determined based on movement information obtained from a sensor. For example, the target value can be determined based on at least one of angular velocity information obtained from a gyro sensor or acceleration information obtained from an acceleration sensor.

[0158] According to one embodiment, the electronic device may include a position sensor (913) that obtains position information of the lens assembly (910) or position information of the image sensor. For example, the camera module may include the position sensor (913). In one example, the electronic device may include a plurality of position sensors (913). In one example, the position sensor (913) may include a Hall sensor. However, the present invention is not limited thereto. For example, the camera module may include at least one of a tunnel magnetoresistance (TMR) sensor, an anisotropic magneto-resistance (AMR) sensor, or a giant magneto-resistance (GMR) sensor. The camera module may use the tunnel magnetoresistance sensor to determine movement (rotation) of a plurality of magnetic bodies based on a resistance value that changes based on a relative angle of the magnetic bodies, or may use at least one of the anisotropic magneto-resistance (AMR) sensor or the giant magneto-resistance (GMR) sensor to determine movement (rotation) of the magnetic bodies.

[0159] According to one embodiment, the driving unit may include one or more magnets (911) arranged in the driving unit and one or more coils (912) for controlling the position of the lens assembly (910) or the position of the image sensor. The magnets (911) may be arranged to face the coils (912). The magnets (911) may be referenced by at least one of the first magnets (421), the second magnets (422), or the third magnets (445) of FIGS. 4 and 5 described above. The coils (912) may be referenced by at least one of the first coils (451), the second coils (452), or the third coils (453) of FIGS. 4 and 5 described above.

[0160] According to one embodiment, the position control unit (840) can obtain sensing values ​​related to position information of the lens assembly (910) or the image sensor from the position sensor (913). The position sensor (913) can detect the position of the lens assembly (910) or the image sensor. The position sensor (913) can detect the position of the driving unit that moves the lens assembly (910) or the image sensor.

[0161] According to one embodiment, the position control unit (840) may determine an output value for controlling the intensity of a current applied to one or more coils (912) based on a target value obtained from the OIS MCU (820) and a sensing value obtained from the position sensor (913). For example, the position control unit (840) may determine an output value based on a difference value between the target value obtained from the OIS MCU (820) and the sensing value obtained from the position sensor (913). For example, the position control unit (840) may determine an output value by adjusting a difference value between the target value obtained from the OIS MCU (820) and the sensing value obtained from the position sensor (913). For example, the position control unit (840) can determine an output value by multiplying a difference value between a target value obtained from the OIS MCU (820) and a sensing value obtained from the position sensor (913) by a specific value (first control (841)), integrating the difference value (second control (842)), and / or differentiating the difference value (third control (843)).

[0162] According to one embodiment, the position control unit (840) may determine the output value based on the gain value. For example, the position control unit (840) may determine the output value by adjusting the gain value. For example, the position control unit (840) may determine the output value by adjusting the gain value associated with the difference value and / or the coefficient associated with the difference value. The operation of adjusting the gain value may include the operation of adjusting the gain value associated with the difference value and / or the coefficient associated with the difference value. For example, the operation of adjusting the gain value may include the operation of adjusting the coefficient associated with the first control (841), the second control (842), and / or the third control (843). For example, the operation of adjusting the gain value may include the operation of adjusting at least one of a coefficient multiplied by a value obtained by integrating the difference value or a coefficient multiplied by a value obtained by differentiating the difference value.

[0163] According to one embodiment, when operating based on the first operation mode, the electronic device may control the driving unit to control the position of the lens assembly (910) or the image sensor based on the first gain value. When operating based on the second operation mode, the electronic device may control the driving unit to control the position of the lens assembly (910) or the image sensor based on the second gain value. In one example, when operating based on the first operation mode, the position control unit (840) may determine the output value based on the first gain value. When operating based on the second operation mode, the position control unit (840) may determine the output value based on the second gain value. The second gain value may be a value greater than the first gain value. For example, when operating based on the second operation mode, the position control unit (840) may determine the output value based on the second gain value greater than the first gain value.

[0164] According to one embodiment, based on the determined output value, the electronic device can control the current flowing in the coil (912). For example, when operating based on the first operation mode, the electronic device can control the intensity of the current flowing in the coil (912) based on the first output value determined based on the first gain value. For example, when operating based on the second operation mode, the electronic device can control the intensity of the current flowing in the coil (912) based on the second output value determined based on the second gain value. According to one embodiment, since the second gain value is greater than the first gain value, the position of the lens assembly (910) (or the image sensor) can be maintained more strongly in the second operation mode than in the first operation mode. For example, the position of the lens assembly (910) (or the image sensor) can be maintained more strongly at a reference position (e.g., the reference position (740) of FIG. 8).

[0165] FIG. 10 is a flowchart illustrating a process (1000) in which an electronic device performs a first operation mode or a second operation mode based on movement information according to an embodiment.

[0166] In the present disclosure, the operation of the electronic device may be understood as being performed by one or more processors (e.g., the processor (120) of FIG. 1) executing instructions stored in a memory (e.g., the memory (130) of FIG. 1) to perform operations or control components of the electronic device.

[0167] According to one embodiment, in operation 1001, the electronic device can check movement information. For example, the electronic device can check movement information acquired from a sensor. For example, the electronic device can check the frequency of the movement information. For example, the electronic device can check whether a frequency component higher than a specified frequency value is detected from the movement information. For example, the electronic device can check whether a frequency component lower than a specified frequency value is detected from the movement information. However, the present invention is not limited thereto. For example, the electronic device can check the time for which the frequency of the movement information is maintained. For example, the electronic device can check the time for which the detection of a frequency component higher than a specified frequency value is maintained from the movement information. For example, the electronic device can check the time for which the detection of a frequency component lower than a specified frequency value is maintained from the movement information.

[0168] According to one embodiment, based on the movement information confirmed in operation 1001, in operation 1002, the electronic device can determine the type of camera operation mode. For example, the electronic device can determine which operation mode the currently performed camera operation mode (hereinafter referred to as “current operation mode”) is among a plurality of operation modes. For example, the electronic device can determine whether the current operation mode is a second operation mode. For example, the electronic device can determine whether the current operation mode is a first operation mode. However, the present invention is not limited thereto. For example, the electronic device can determine whether the current operation mode is a third operation mode.

[0169] According to one embodiment, in response to determining that the camera operation mode (e.g., the current operation mode) is the first operation mode in operation 1002, in operation 1003, the electronic device may determine whether the motion information satisfies a specified condition. In one example, the electronic device may determine whether the frequency of the motion information satisfies the specified condition. For example, the electronic device may determine whether a frequency component higher than a specified frequency value is detected from motion information acquired from a sensor. For example, the electronic device may determine whether a frequency component lower than a specified frequency value is detected from motion information acquired from a sensor. For example, the electronic device may detect a frequency component higher than a specified frequency value based on at least one of a periodicity of the motion information or a derivative of the motion information. For example, the electronic device may detect a frequency component lower than a specified frequency value based on at least one of a periodicity of the motion information or a derivative of the motion information. In one example, the specified frequency value may include a frequency value higher than a band of interest of the OIS operation. For example, the specified frequency value may be greater than or equal to 10 Hz. However, this is not limited thereto. In one example, the specified frequency value may include the frequency value of motion information acquired during drone filming. In one example, the specified frequency value may also include the frequency value of motion information acquired during filming while the electronic device is mounted on a motorcycle, cart, or bicycle. In one example, the electronic device may determine whether the time for which the frequency of the motion information is maintained satisfies a specified condition. For example, the electronic device may determine whether the time for which the detection of a frequency component higher than the specified frequency value from the motion information is maintained is longer than the specified time. For example, the electronic device may determine whether the time for which the detection of a frequency component higher than the specified frequency value from the motion information is maintained is shorter than the specified time.For example, it can be determined whether the time for which detection of a frequency component lower than a specified frequency value from motion information is maintained is longer than a specified time. For example, it can be determined whether the time for which detection of a frequency component lower than a specified frequency value from motion information is maintained is shorter than a specified time.

[0170] According to one embodiment, if it is determined in operation 1003 that the motion information satisfies a specified condition, in operation 1004, the electronic device may change the first operation mode to the second operation mode. For example, the electronic device may change the operation mode from the first operation mode, which is the current operation mode, to the second operation mode. For example, if it is determined that the frequency of the motion information satisfies a specified condition, the electronic device may change to the second operation mode. For example, if it is determined that a frequency component higher than a specified frequency value is detected from the motion information, the electronic device may change the camera operation mode so that the current operation mode becomes the second operation mode from the first operation mode. For example, if it is determined that a frequency value higher than 10 Hz is detected from the motion information, the electronic device may change to the second operation mode. However, the present invention is not limited thereto. For example, if it is determined that the time for which the frequency of the motion information is maintained satisfies a specified condition, the electronic device may change to the second operation mode. For example, if it is determined that the detection of a frequency component higher than a specified frequency value from the motion information is maintained for a specified period of time, the electronic device can change the camera operation mode so that the current operation mode is the first operation mode to the second operation mode.

[0171] According to one embodiment, based on the change to the second operation mode in operation 1004, in operation 1005, the electronic device can perform the second operation mode. For example, based on the second operation mode, the electronic device can control the driving unit so that the lens assembly or the image sensor is positioned at a reference position within the driving range. The electronic device can control the position of the lens assembly or the image sensor based on the second gain value. The electronic device can fix the target value output from the position sensor to a designated value. While controlling the driving unit, the electronic device can perform image stabilization processing to reduce shaking of the image by performing image processing on image frames acquired through the camera module.

[0172] According to one embodiment, in operation 1007, the electronic device can determine whether the operation of capturing an image through the camera has ended. If the operation of capturing an image has ended, the electronic device can end the process for capturing the image. If the operation of capturing an image has not ended, the electronic device can check the movement information in operation 1001.

[0173] According to one embodiment, if it is determined in operation 1003 that the motion information does not satisfy a specified condition, in operation 1006, the electronic device may perform a first operation mode. For example, the electronic device may maintain the first operation mode, which is the current operation mode. For example, if it is determined that the frequency of the motion information does not satisfy the specified condition, the electronic device may perform the first operation mode. For example, if it is determined that a frequency component higher than a specified frequency value is not detected from the motion information, the electronic device may perform the first operation mode, which is the current operation mode. For example, if it is determined that a frequency component lower than a specified frequency value is detected from the motion information, the electronic device may perform the first operation mode, which is the current operation mode. For example, if it is determined that a frequency value lower than 10 Hz is detected from the motion information, the electronic device may perform the first operation mode. However, the present invention is not limited thereto. For example, if it is determined that the time for which the frequency of the motion information is maintained does not satisfy a specified condition, the electronic device may perform the first operation mode. For example, if it is determined that the time for which detection of a frequency component lower than a specified frequency value from the motion information is maintained is longer than a specified time, the electronic device can perform the first operation mode.

[0174] According to one embodiment, in operation 1006, the electronic device may control the driving unit to move the position of the lens assembly or the position of the image sensor based on the motion information acquired from the sensor based on the first operation mode. The electronic device may control the position of the lens assembly or the image sensor based on the first gain value. For example, the electronic device may control the driving unit to move the lens assembly or the image sensor in a target motion direction based on the acquired motion information. The lens assembly or the image sensor may be moved to the target position. While controlling the driving unit, the electronic device may perform image processing on image frames acquired through the camera module to perform image stabilization processing to reduce shaking of the image.

[0175] According to one embodiment, in operation 1007, the electronic device can determine whether the operation of capturing an image through the camera has ended. If the operation of capturing an image has ended, the electronic device can end the process for capturing the image. If the operation of capturing an image has not ended, the electronic device can check the movement information in operation 1001.

[0176] According to one embodiment, in response to determining in operation 1002 that the camera operation mode (e.g., the current operation mode) is not the first operation mode, in operation 1008, the electronic device may determine whether the motion information satisfies a specified condition. For example, in response to determining that the camera operation mode is the second operation mode, the electronic device may determine whether the motion information satisfies the specified condition. In one example, the electronic device may determine whether the frequency of the motion information satisfies the specified condition. For example, the electronic device may determine whether a frequency component higher than a specified frequency value is detected from motion information acquired from a sensor. For example, the electronic device may determine whether a frequency component lower than a specified frequency value is detected from motion information acquired from a sensor. For example, the electronic device may detect a frequency component higher than a specified frequency value based on at least one of a periodicity of the motion information or a derivative of the motion information. For example, the electronic device may detect a frequency component lower than a specified frequency value based on at least one of a periodicity of the motion information or a derivative of the motion information. In one example, the specified frequency value may include a frequency value higher than the band of interest of the OIS operation. For example, the specified frequency value may be greater than or equal to 10 Hz, but is not limited thereto. In one example, the specified frequency value may include a frequency value of motion information acquired during drone filming. In one example, the specified frequency value may include a frequency value of motion information acquired during filming while the electronic device is mounted on a motorcycle, cart, or bicycle. In one example, the electronic device may determine whether the time for which the frequency of the motion information is maintained satisfies a specified condition. For example, the electronic device may determine whether the time for which the detection of a frequency component higher than the specified frequency value from the motion information is maintained is longer than a specified time.For example, the electronic device can determine whether the duration for which detection of a frequency component above a specified frequency value from motion information is maintained is less than a specified duration. For example, the electronic device can determine whether the duration for which detection of a frequency component below a specified frequency value from motion information is maintained is greater than a specified duration. For example, the electronic device can determine whether the duration for which detection of a frequency component below a specified frequency value from motion information is maintained is less than a specified duration.

[0177] According to one embodiment, if it is determined in operation 1008 that the motion information satisfies a specified condition, in operation 1009, the electronic device may perform a second operation mode. For example, the electronic device may maintain the second operation mode, which is the current operation mode. For example, if it is determined that the frequency of the motion information satisfies a specified condition, the electronic device may perform the second operation mode. For example, if it is determined that a frequency component higher than a specified frequency value is detected from the motion information, the electronic device may perform the second operation mode, which is the current operation mode. For example, if it is determined that a frequency component higher than a specified frequency value is detected from the motion information, the electronic device may perform the second operation mode, which is the current operation mode. For example, if it is determined that a frequency value higher than 10 Hz is detected from the motion information, the electronic device may perform the second operation mode. However, the present invention is not limited thereto. For example, if it is determined that the time for which the frequency of the motion information is maintained satisfies a specified condition, the electronic device may perform the second operation mode. For example, if it is determined that the detection of a frequency component higher than a specified frequency value from the motion information is maintained for a specified period of time or longer, the electronic device may perform a second operation mode.

[0178] According to one embodiment, in operation 1009, the electronic device may control the driving unit so that the lens assembly or the image sensor is positioned at a reference position within the driving range based on the second operation mode. The electronic device may control the position of the lens assembly or the image sensor based on the second gain value. The electronic device may fix the target value output from the position sensor to a designated value. While controlling the driving unit, the electronic device may perform image processing on image frames acquired through the camera module to perform image stabilization processing to reduce shaking of the image.

[0179] According to one embodiment, in operation 1007, the electronic device can determine whether the operation of capturing an image through the camera has ended. If the operation of capturing an image has ended, the electronic device can end the process for capturing the image. If the operation of capturing an image has not ended, the electronic device can check the movement information in operation 1001.

[0180] According to one embodiment, if it is determined in operation 1008 that the motion information does not satisfy a specified condition, in operation 1010, the electronic device may change the second operation mode to the first operation mode. For example, the electronic device may change the operation mode from the second operation mode, which is the current operation mode, to the first operation mode. For example, if it is determined that the frequency of the motion information does not satisfy the specified condition, the electronic device may change to the first operation mode. For example, if it is determined that a frequency component lower than a specified frequency value is detected from the motion information, the electronic device may change the camera operation mode so that the current operation mode becomes the first operation mode from the second operation mode. For example, if it is determined that a frequency value lower than 10 Hz is detected from the motion information, the electronic device may change to the first operation mode. However, the present invention is not limited thereto. For example, if it is determined that the time for which the frequency of the motion information is maintained satisfies a specified condition, the electronic device may change to the first operation mode. For example, if it is determined that the detection of a frequency component lower than a specified frequency value from the motion information is maintained for a specified period of time or longer, the electronic device may change the camera operation mode so that the current operation mode is changed from the second operation mode to the first operation mode.

[0181] According to one embodiment, based on the change to the first operation mode in operation 1010, in operation 1011, the electronic device may perform the first operation mode. For example, the electronic device may control the driving unit to move the position of the lens assembly or the position of the image sensor based on motion information acquired from the sensor based on the first operation mode. The electronic device may control the position of the lens assembly or the image sensor based on the first gain value. For example, the electronic device may control the driving unit to move the lens assembly or the image sensor in a target movement direction based on the acquired motion information. The lens assembly or the image sensor may be moved to the target position. While controlling the driving unit, the electronic device may perform image stabilization processing to reduce shaking of the image by performing image processing on image frames acquired through the camera module.

[0182] According to one embodiment, in operation 1007, the electronic device can determine whether the operation of capturing an image through the camera has ended. If the operation of capturing an image has ended, the electronic device can end the process for capturing the image. If the operation of capturing an image has not ended, the electronic device can check the movement information in operation 1001.

[0183] The operation of the electronic device based on the second operation mode is described in detail in Fig. 14.

[0184] FIG. 11 is a flowchart illustrating a process (1100) in which an electronic device according to one embodiment changes a first operation mode to a second operation mode and performs the second operation mode.

[0185] FIG. 11 is a flowchart illustrating a process when it is determined that the frequency of movement information acquired from a sensor satisfies a specified condition, as an example of the embodiment of FIG. 10, when the current operation mode is the first operation mode.

[0186] According to one embodiment, in operation 1110, the electronic device may control the driving unit based on the first operation mode. For example, if the frequency of the motion information acquired from the sensor does not satisfy a specified condition, the electronic device may control the driving unit based on the first operation mode. For example, the electronic device may control the driving unit to move the position of the lens assembly or the image sensor based on the motion information acquired from the sensor.

[0187] In one embodiment, in operation 1110, when operating based on the first operation mode, the electronic device can control the driving unit to control the position of the lens assembly or the image sensor based on the first gain value.

[0188] According to one embodiment, in operation 1120, the electronic device may determine whether to change the first operation mode to the second operation mode based on the frequency of the motion information. For example, the electronic device may determine whether to change the first operation mode to the second operation mode based on whether the frequency of the motion information satisfies a specified condition. For example, the electronic device may determine whether to change the first operation mode to the second operation mode based on whether a frequency component higher than a specified frequency value is detected from the motion information. For example, the electronic device may change the first operation mode to the second operation mode if the frequency of the motion information satisfies a specified condition. For example, the electronic device may change the first operation mode to the second operation mode if a frequency component higher than a specified frequency value is detected from the motion information. For example, the electronic device may maintain the first operation mode if the frequency of the motion information does not satisfy the specified condition. For example, the electronic device may maintain the first operation mode if a frequency component higher than a specified frequency value is not detected from the motion information.

[0189] According to one embodiment, in response to changing the first operation mode to the second operation mode in operation 1120, in operation 1130, the electronic device may position the lens assembly or the image sensor at a reference position based on the second operation mode. For example, the electronic device may control the driving unit to position the lens assembly or the image sensor at the reference position based on the second operation mode. For example, the electronic device may control the OIS driving unit to position the lens assembly or the image sensor at at least one of the center of the driving range on the first axis or the center of the driving range on the second axis. For example, the electronic device may control the AF driving unit to position the lens assembly or the image sensor at a reference position on the third axis.

[0190] In one embodiment, in operation 1130, when operating based on the second operation mode, the electronic device may control the driving unit to control the position of the lens assembly or the image sensor based on the second gain value. In one example, the second gain value may be a different value from the first gain value. For example, the second gain value may be greater than the first gain value. However, the present invention is not limited thereto.

[0191] According to one embodiment, in operation 1140, the electronic device may perform image processing on image frames acquired through the camera module while controlling the driving unit based on the second operation mode. The electronic device may perform image stabilization processing to reduce shaking of the image by performing image processing on the image frames. For example, the electronic device may perform at least one of video digital image stabilization (VDIS) or electronic image stabilization (EIS) on the image frames. For example, the electronic device may perform at least one of video digital image stabilization (VDIS) or electronic image stabilization (EIS) on the image frames to compensate for shaking of the electronic device (500). The electronic device (500) may output a video or preview image on which image stabilization is performed through VDIS (and / or EIS) through a display.

[0192] According to one embodiment, in response to not changing the first operation mode to the second operation mode in operation 1120, the electronic device may perform the first operation mode. For example, the electronic device may maintain the first operation mode, which is the current operation mode. The electronic device may perform image processing on image frames acquired through the camera module while controlling the driving unit based on the first operation mode. The electronic device may perform image stabilization processing to reduce shaking of the image by performing image processing on the image frames. For example, the electronic device may perform at least one of video digital image stabilization (VDIS) or electronic image stabilization (EIS) on the image frames. For example, the electronic device may compensate for shaking of the electronic device (500) by performing at least one of video digital image stabilization (VDIS) or electronic image stabilization (EIS) on the image frames. The electronic device (500) may output a video or preview image on which image stabilization is performed through VDIS (and / or EIS) through a display.

[0193] FIG. 12 is a diagram illustrating an operation for determining whether a frequency of motion information satisfies a specified condition according to one embodiment. FIG. 13 is a diagram illustrating an operation for determining whether a frequency of motion information satisfies a specified condition according to one embodiment.

[0194] FIGS. 12 and 13 described below may be examples of operations for determining whether the frequency of motion information satisfies a specified condition. For example, an electronic device may detect a frequency component greater than a specified frequency value based on at least one of the periodicity of the motion information (see FIG. 12) or the differential value of the motion information (see FIG. 13). However, the present invention is not limited thereto. For example, the electronic device may detect a frequency component greater than a specified frequency value based on a Fourier transform value of the motion information.

[0195] The contents of Fig. 12 may be referenced by the contents of other drawings. The contents of Fig. 13 may be referenced by the contents of other drawings.

[0196] Operation 1010 of FIG. 10 can be referenced by the contents of FIG. 12.

[0197] Referring to FIG. 12, according to one embodiment, an electronic device may determine whether a frequency of motion information satisfies a specified condition based on motion information acquired from a sensor. For example, the electronic device may detect a frequency component higher than a specified frequency value from the motion information. In one example, the specified frequency value may include a frequency value higher than a band of interest of an OIS operation. For example, the specified frequency value may be greater than or equal to 10 Hz. However, the present invention is not limited thereto. In one example, the specified frequency value may include a frequency value of motion information acquired during drone filming. In one example, the specified frequency value may also include a frequency value of motion information acquired during filming while the electronic device is mounted on a cart or bicycle.

[0198] According to one embodiment, the electronic device may detect a frequency component greater than or equal to a specified frequency value based on the periodicity of the motion information. For example, referring to FIG. 12, the electronic device may detect a frequency component greater than or equal to a specified frequency value based on a period in which an angular velocity signal (1210) acquired from a sensor has a value of 0. For example, if the interval (1211) of the period of the angular velocity signal (1210) satisfies a specified condition, the electronic device may determine that the frequency of the motion information has a frequency component greater than or equal to the specified frequency value. For example, if the interval (1211) of the period of the angular velocity signal (1210) becomes smaller than the specified interval, the electronic device may determine that the frequency of the motion information has a frequency component greater than or equal to the specified frequency value. In one example, if the interval (1211) of the period of the angular velocity signal (1210) remains smaller than the specified interval, the electronic device may determine that the frequency of the motion information has a frequency component greater than or equal to the specified frequency value.

[0199] According to one embodiment, the amplitude of the angular velocity signal (1210) may be affected by the determination of a specified condition. For example, if the amplitude is not large but is a high frequency signal greater than a specified frequency interval (1211), it may be a general noise signal and does not affect the lens shake, thus not affecting the clarity of the image. However, if it is a high frequency signal with a large amplitude, it may affect the movement of the lens. Therefore, the electronic device may determine a given condition by considering the amplitude of the angular velocity signal (1210).

[0200] Operation 1010 of FIG. 10 can be referenced by the contents of FIG. 13.

[0201] Referring to FIG. 13, according to one embodiment, an electronic device may determine whether a frequency of motion information satisfies a specified condition based on motion information acquired from a sensor. For example, the electronic device may detect a frequency component higher than a specified frequency value from the motion information. In one example, the specified frequency value may include a frequency value higher than a band of interest of the OIS operation. For example, the specified frequency value may be greater than or equal to 10 Hz. However, the present invention is not limited thereto. In one example, the specified frequency value may include a frequency value of motion information acquired during drone filming. In one example, the specified frequency value may also include a frequency value of motion information acquired during filming while the electronic device is mounted on a cart or bicycle.

[0202] According to one embodiment, the electronic device may detect a frequency component higher than a specified frequency value based on a differential value of motion information. For example, referring to FIG. 13, a first graph (1310) may be a graph representing an angular velocity value of motion information over time, and a second graph (1320) may be a graph representing a differential value of the angular velocity of the motion information over time. A first line (1311) of the first graph (1310) and a second line (1321) of the second graph (1320) may indicate whether a frequency of the motion information has a frequency component higher than a specified frequency value.

[0203] When referring to the first line (1311) and the second line (1312), it may be easier to use the second graph (1320) than the first graph (1310) to determine whether the frequency of the motion information has a frequency component greater than a specified frequency value.

[0204] FIG. 14 is a flowchart illustrating a process (1070) of a second operation mode according to one embodiment.

[0205] The process (1070) of FIG. 14 may be referenced by the operation 1070 of FIG. 10. The contents of FIG. 14 may be referenced by the contents of other drawings.

[0206] According to one embodiment, in operation 1071, the electronic device may position the lens assembly or the image sensor at a reference position. For example, the electronic device may control a driving unit so that the lens assembly or the image sensor is positioned at the reference position. For example, the electronic device may control an OIS driving unit so that the lens assembly or the image sensor is positioned at the reference position. For example, the electronic device may control an AF driving unit so that the lens assembly or the image sensor is positioned at the reference position.

[0207] According to one embodiment, in operation 1073, the electronic device may control the position of the lens assembly or the image sensor based on the second gain value. For example, the electronic device may maintain the position of the lens assembly or the image sensor based on the second gain value. For example, the electronic device may maintain the position of the lens assembly or the image sensor at a reference position based on the second gain value. For example, the electronic device may control the OIS driver based on the second gain value so that the lens assembly or the image sensor is maintained at the reference position with respect to the first direction (e.g., the x-axis direction) and the second direction (e.g., the y-axis direction). For example, the electronic device may control the AF driver based on the second gain value so that the lens assembly or the image sensor is maintained at the reference position with respect to the third direction (e.g., the z-axis direction). Therefore, in a mode (e.g., the second operating mode) in which image stabilization is performed using VDIS (and / or EIS), the position of the lens assembly or the image sensor can be stably maintained even if the electronic device shakes more than a specific frequency.

[0208] In one embodiment, the second gain value may be different from the first gain value when operating based on the first operating mode. For example, the second gain value may be greater than the first gain value. In one embodiment, the first gain value and the second gain value may include gain values ​​for controlling a current flowing in one or more coils (e.g., the first coil (451), the second coil (452), the third coil (453) of FIGS. 4 and 5 and / or the coil (612) of FIG. 6) based on position information acquired by the driving unit from the position sensor.

[0209] According to one embodiment, in operation 1075, the electronic device may perform image stabilization processing on image frames acquired through the camera module while controlling the driving unit based on the second operation mode. For example, the electronic device may perform image stabilization processing on image frames acquired through the camera module while maintaining the position of the lens assembly or the position of the image sensor based on the second operation mode. For example, the electronic device may perform image stabilization processing using VDIS (and / or EIS) on image frames acquired through the camera module while maintaining the position of the lens assembly or the position of the image sensor based on the second operation mode. Accordingly, even when the electronic device captures a video and / or a preview video in a situation where it shakes significantly, the position of the lens assembly or the position of the image sensor is maintained, so that the performance of VDIS (and / or EIS) may be improved and a video and / or a preview video with improved quality may be provided.

[0210] FIG. 15 is a diagram for explaining an operation of controlling the position of a lens assembly or an image sensor based on a second gain value different from a first gain value according to one embodiment.

[0211] The contents of Figure 15 may be referenced by the contents of other drawings.

[0212] The first graph (1510) of FIG. 15 may be a Bode plot illustrating a first gain value (1511) and a first phase (1512) in a first operation mode according to an embodiment, and the second graph (1520) may be a Bode plot illustrating a second gain value (1521) and a second phase (1522) in a second operation mode according to an embodiment. Referring to FIG. 15, the second gain value (1521) of the second operation mode may be greater than the first gain value (1511) of the first operation mode.

[0213] FIG. 16 is a flowchart illustrating a process (1600) for fixing a target value to a specified value according to one embodiment.

[0214] In one embodiment, operation 1610 may be a subsequent operation to operation 1073 of FIG. 10 , but is not limited thereto. In one embodiment, operation 1075 of FIG. 10 may be a subsequent operation to operation 1620 , but is not limited thereto.

[0215] In one embodiment, at operation 1610, the electronic device may fix the target value to a specified value. For example, the electronic device may set the target value to a value of 0. In one example, the target value may include a value output from a position sensor (e.g., position sensor 612 of FIG. 6). In one example, the target value may include an output value transmitted from the position sensor to the VDIS (and / or EIS) driver. In one example, setting the target value to a value of 0 may include maintaining the lens assembly or the image sensor positioned at a reference position.

[0216] According to one embodiment, in operation 1620, the electronic device may control the intensity of a current applied to one or more coils (e.g., the first coil (451), the second coil (452), the third coil (453) of FIGS. 4 and 5 and / or the coil (612) of FIG. 6) based on a sensing value of a position sensor (e.g., the position sensor (612) of FIG. 6). For example, the electronic device may control the intensity of a current applied to one or more coils based on a sensing value based on a position of a lens assembly or a position of an image sensor.

[0217] According to one embodiment, in operation 1075, the electronic device may perform image stabilization processing to reduce image shake by performing image processing on image frames based on at least one of a target value, a sensed value, or motion information. For example, the electronic device may perform image stabilization processing to reduce image shake by performing image processing on image frames based on at least one of a target value of 0, a sensed value based on a position of a lens assembly or a position of an image sensor, or motion information.

[0218] According to one embodiment, by fixing the target value to a specified value (e.g., a value of 0), the movement of the driving unit can be reduced or prevented from acting as noise due to the frequency of the movement information having a frequency component higher than the specified frequency value.

[0219] However, the operation of preventing the movement of the driving unit from acting as noise due to the frequency of the motion information having a frequency component higher than the specified frequency value is not limited thereto. For example, the electronic device may not fix the target value to the specified value and may not use the target value. For example, the electronic device may perform image stabilization processing to reduce image shaking by performing image processing on image frames based on motion information acquired from a sensor without using the target value.

[0220] FIG. 17 is a diagram for explaining the effect of improving the performance of image stabilization processing by fixing the target value to a specified value according to one embodiment.

[0221] The contents of Fig. 17 may be referenced by the contents of other drawings. For example, the target values ​​of Fig. 17 may be referenced by the target values ​​of Fig. 16.

[0222] The first graph (1710) of FIG. 17 may be a graph representing a first angular velocity motion vector (1711) and a first image motion vector (1712) when the target value is not fixed to a specified value (e.g., a value of 0). The second graph (1720) of FIG. 17 may be a graph representing a second angular velocity motion vector (1711) and a second image motion vector (1712) when the target value is fixed to a specified value (e.g., a value of 0). In one example, the angular velocity motion vector may include a motion vector based on motion information. The image motion vector may include an amount of pixels moved compared to a previous image. The electronic device may compare the image motion vector and the angular velocity motion vector. For example, the electronic device may determine whether and / or the degree of similarity between the angular velocity motion vector and the image motion vector. Based on the comparison result, the electronic device can determine whether the video digital image stabilization (VDIS) (and / or electronic image stabilization (EIS)) has been performed stably. For example, the electronic device can determine that the video digital image stabilization (VDIS) (and / or electronic image stabilization (EIS)) has been performed stably based on the comparison result that the angular velocity motion vector and the image motion vector are determined to be similar.

[0223] According to one embodiment, when the target value is fixed to a specified value (e.g., a value of 0), the angular velocity motion vector and the image motion vector may be more similar than when the target value is not fixed to a specified value (e.g., a value of 0). For example, when referring to the first graph (1710) and the second graph (1720), when comparing the degree of similarity between the first angular velocity motion vector (1711) and the first image motion vector (1712), and the degree of similarity between the second angular velocity motion vector (1721) and the second image motion vector (1722), the degree of similarity between the second angular velocity motion vector (1721) and the second image motion vector (1722) may be greater. When the target value is fixed to a specified value (e.g., 0), video digital image stabilization (VDIS) (and / or electronic image stabilization (EIS)) can be performed more stably compared to when the target value is not fixed to a specified value (e.g., 0).

[0224] FIG. 18 is a flowchart illustrating a process (1800) for performing an operation mode based on user input according to one embodiment.

[0225] The contents of Figure 18 may be referenced by the contents of other drawings.

[0226] According to one embodiment, a process (1800) for performing an operation mode based on a user input may include an operation (1810) for obtaining a user input and an operation (1820) for performing an operation mode based on the user input. However, the operations of the process (1800) are not limited thereto. For example, the process (1800) may omit at least one of the above-described operations or may further include at least one other operation.

[0227] According to one embodiment, in operation 1810, the electronic device may obtain a user input. For example, the electronic device may obtain a user input to change an operation mode (e.g., a current operation mode) of a camera module. In one example, the electronic device may obtain the user input through a display (e.g., the display module (160) of FIG. 1). For example, the electronic device may obtain the user input when a user touches a visual object displayed on the display (e.g., a button and / or icon related to changing the operation mode). However, the present invention is not limited thereto. According to various embodiments, the user input may be obtained through various methods. For example, the electronic device may obtain the user input when a user touches a physical button.

[0228] In one embodiment, the user input may include a user input that causes the camera module to change its operating mode (e.g., the current operating mode). For example, the user input may include a user input that causes the camera module to change to a first operating mode. For example, the user input may include a user input that causes the camera module to change to a second operating mode. For example, the user input may include a user input that causes the camera module to change to a third operating mode.

[0229] According to one embodiment, in operation 1820, the electronic device may perform an operation mode based on a user input. For example, the electronic device may perform one of a plurality of operation modes based on the user input. For example, the electronic device may perform an operation mode corresponding to the user input. According to one embodiment, the electronic device may change to one of the plurality of operation modes based on the user input. For example, the electronic device may change the current operation mode to another of the plurality of operation modes based on the user input. For example, the electronic device may change to the first operation mode based on a user input to change to the first operation mode. For example, the electronic device may change the current operation mode from a second operation mode or a third operation mode to the first operation mode. In response to the change to the first operation mode, the electronic device may perform the first operation mode. For example, the electronic device may change to the second operation mode based on a user input to change to the second operation mode. For example, the electronic device may change the current operation mode from the first operation mode or the third operation mode to the second operation mode. The electronic device may perform the second operating mode in response to a change to the second operating mode. For example, the electronic device may change to the third operating mode based on a user input that causes the electronic device to change to the third operating mode. For example, the electronic device may change the current operating mode from the first operating mode or the second operating mode to the third operating mode. The electronic device may perform the third operating mode in response to a change to the third operating mode.

[0230] FIG. 19 is a flowchart illustrating a process (1900) for performing an operation mode based on detection of an external device according to one embodiment.

[0231] The contents of Figure 19 may be referenced by the contents of other drawings.

[0232] According to one embodiment, in operation 1910, the electronic device may detect an external device. For example, the electronic device may detect that the electronic device is connected to the external device. For example, the electronic device may detect that the electronic device is physically connected to the external device. For example, the electronic device may detect that the electronic device is in physical contact with the external device. For example, the electronic device may detect that the electronic device is placed on the external device. For example, the electronic device may detect that the electronic device is accommodated in the external device. For example, the electronic device may detect that the electronic device is seated on the external device. In one example, the external device may include an electronic device and / or an accessory. For example, the external device may include a drone, a motorcycle, a bicycle, a cart, and / or a mount. For example, the electronic device may detect that the electronic device is mounted on a drone. For example, the electronic device may detect that the electronic device is placed on a mount. For example, the electronic device may detect that the electronic device is placed on a mount on a motorcycle.

[0233] In one embodiment, in response to the electronic device not detecting an external device in operation 1910, in operation 1920, the electronic device may perform a first mode of operation. For example, the electronic device may maintain the first mode of operation.

[0234] According to one embodiment, in response to the electronic device detecting an external device in operation 1910, in operation 1930, the electronic device may perform a designated operation mode among a plurality of operation modes. For example, the electronic device may perform the designated operation mode based on the external device. For example, the electronic device may perform the designated operation mode based on the type of the external device.

[0235] According to one embodiment, in response to detecting an external device, the electronic device may control the driving unit to position the lens assembly or image sensor at a reference position within the driving range based on a specified operating mode. For example, the electronic device may control the driving unit based on a second operating mode. For example, the electronic device may control the driving unit based on a third operating mode. For example, the electronic device may perform the second operating mode based on the electronic device being placed on a motorcycle (e.g., on a motorcycle mount). For example, the electronic device may change to the second operating mode or maintain the second operating mode based on the electronic device being placed on a motorcycle (e.g., on a motorcycle mount). For example, the electronic device may perform the third operating mode based on the electronic device being mounted on a drone. For example, the electronic device may change to the third operating mode or maintain the third operating mode based on the electronic device being mounted on a drone.

[0236] As described above, an electronic device according to an embodiment (e.g., the electronic device (101) of FIG. 1) may include a sensor for detecting a movement of the electronic device, a camera module, one or more processors, and a memory for storing one or more instructions. The camera module may include a lens assembly, an image sensor, and a driving unit for moving the lens assembly or the image sensor within a driving range. The one or more instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to: control the driving unit to move a position of the lens assembly or the image sensor according to movement information acquired from the sensor, based on a first operating mode among a plurality of operating modes associated with an operation of the driving unit. The one or more instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to: control the driving unit to position the lens assembly or the image sensor at a reference position within the driving range, based on a second operating mode different from the first operating mode among the plurality of operating modes. The one or more commands, when individually or collectively executed by the one or more processors, may cause the electronic device to perform image stabilization processing to reduce image shake by performing image processing on image frames acquired through the camera module while controlling the driving unit.The one or more instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to: when operating based on the first operating mode, control the position of the lens assembly or the image sensor based on a first gain value; and when operating based on the second operating mode, control the position of the lens assembly or the image sensor based on a second gain value different from the first gain value.

[0237] In one embodiment, the one or more instructions may cause the one or more processors, when individually or collectively executed, to determine whether the electronic device will change to one of the plurality of operating modes based on a frequency of the motion information. The one or more instructions may cause the one or more processors, when individually or collectively executed, to control the driving unit so that the lens assembly or the image sensor is positioned at a reference position within the driving range based on a third operating mode different from the first operating mode and the second operating mode among the plurality of operating modes. The one or more instructions may cause the one or more processors, when individually or collectively executed, to control the driving unit so that the lens assembly or the image sensor is positioned at a reference position within the driving range when the electronic device operates based on the third operating mode.

[0238] According to one embodiment, the one or more instructions, when executed individually or collectively by the one or more processors, may cause the electronic device to change to at least one of the second operating mode or the third operating mode based on detection of a frequency component greater than or equal to a specified frequency value from the motion information.

[0239] According to one embodiment, the one or more instructions, when executed individually or collectively by the one or more processors, may cause the electronic device to detect a frequency component greater than or equal to the specified frequency value based on at least one of a periodicity of the motion information or a derivative of the motion information.

[0240] In one embodiment, the third gain value may be greater than the second gain value.

[0241] In one embodiment, the one or more instructions, when executed by the one or more processors individually or collectively, may cause the electronic device to obtain a user input that causes the electronic device to change to one of the plurality of operating modes. The one or more instructions, when executed by the one or more processors individually or collectively, may cause the electronic device to change to one of the plurality of operating modes based on the user input.

[0242] In one embodiment, the one or more instructions, when executed by the one or more processors individually or collectively, may cause the electronic device to determine whether the electronic device has detected an external device. The one or more instructions, when executed by the one or more processors individually or collectively, may cause the electronic device, in response to detecting the external device, to control the driving unit so that the lens assembly or the image sensor is positioned at a reference position within the driving range based on a designated operating mode among the plurality of operating modes.

[0243] According to one embodiment, the electronic device may include a housing that accommodates the lens assembly and the driving unit. The driving unit may be arranged to move in a first axial direction substantially perpendicular to an optical axis of the lens assembly and a second axial direction substantially perpendicular to the first axis within the housing. The reference position may include at least one of a center of the driving range on the first axis or a center of the driving range on the second axis.

[0244] According to one embodiment, the driving unit may include at least one of an OIS driving unit that moves the lens assembly or the image sensor in at least one of a first direction or a second direction different from the first direction, or an AF driving unit that moves the lens assembly or the image sensor in a third direction substantially perpendicular to the first direction and the second direction according to focus-related information for an image captured through the image sensor.

[0245] According to one embodiment, the one or more instructions, when the one or more processors are individually or collectively executed, cause the electronic device to operate based on the second operating mode: control the OIS driving unit based on the second gain value so that the position of the lens assembly or the position of the image sensor is maintained at the reference position with respect to the first direction and the second direction, and control the AF driving unit based on the second gain value so that the position of the lens assembly or the position of the image sensor is maintained at the reference position with respect to the third direction.

[0246] According to one embodiment, the camera module may include a position sensor that obtains position information of the lens assembly or position information of the image sensor. The driving unit may include one or more coils for controlling the position of the lens assembly or the position of the image sensor. The first gain value and the second gain value may include gain values ​​for the driving unit to control a current flowing in the one or more coils based on the position information.

[0247] According to one embodiment, the camera module may include a position sensor that obtains position information of the lens assembly or position information of the image sensor. The driving unit may include one or more coils for controlling the position of the lens assembly or the position of the image sensor. The one or more commands, when individually or collectively executed by the one or more processors, may cause the electronic device to fix a target value output from the position sensor to a designated value. The one or more commands, when individually or collectively executed by the one or more processors, may cause the electronic device to control the intensity of a current applied to the one or more coils based on a sensing value obtained from the position sensor.

[0248] According to one embodiment, the sensor for detecting movement of the electronic device may include at least one of a gyro sensor or an acceleration sensor.

[0249] In one embodiment, the second gain value may be greater than the first gain value.

[0250] As described above, the operating method of an electronic device including a camera module according to an embodiment (e.g., the electronic device (101) of FIG. 1) may include an operation of controlling the driving unit to move a position of the lens assembly or the image sensor based on motion information acquired from a sensor that detects motion of the electronic device, based on a first operating mode among a plurality of operating modes associated with an operation of the driving unit that moves the lens assembly or the image sensor within a driving range. The operating method of the electronic device may include an operation of controlling the driving unit to position the lens assembly or the image sensor at a reference position within the driving range, based on a second operating mode different from the first operating mode among the plurality of operating modes. The operating method of the electronic device may include an operation of performing image stabilization processing to reduce shaking of the image by performing image processing on image frames acquired through the camera module while controlling the driving unit. The operating method of the electronic device may include an operation in which, when operating based on the first operating mode, the driving unit controls the position of the lens assembly or the image sensor based on a first gain value, and when operating based on the second operating mode, the driving unit controls the position of the lens assembly or the image sensor based on a second gain value different from the first gain value.

[0251] According to one embodiment, the operating method of the electronic device may include an operation of determining whether to change to one of the plurality of operating modes based on a frequency of the movement information. The operating method of the electronic device may include an operation of controlling the driving unit so that the lens assembly or the image sensor is positioned at a reference position within the driving range based on a third operating mode different from the first operating mode and the second operating mode among the plurality of operating modes. The operating method of the electronic device may include an operation of controlling the position of the lens assembly or the image sensor based on a third gain value different from the first gain value and the second gain value when the electronic device operates based on the third operating mode.

[0252] According to one embodiment, the operating method of the electronic device may include an operation of changing to at least one of the second operating mode or the third operating mode based on detection of a frequency component higher than a specified frequency value from the movement information or detection of a frequency component higher than a certain value. The operating method of the electronic device may include an operation of detecting a frequency component higher than the specified frequency value based on at least one of a periodicity of the movement information or a differential value of the movement information.

[0253] According to one embodiment, the driving unit may include at least one of an OIS driving unit that moves the lens assembly or the image sensor in at least one of a first direction or a second direction different from the first direction, or an AF driving unit that moves the lens assembly or the image sensor in a third direction substantially perpendicular to the first direction and the second direction according to focus-related information for an image captured through the image sensor.

[0254] According to one embodiment, when operating based on the second operation mode, the operation of controlling the position of the lens assembly or the image sensor based on a second gain value different from the first gain value by the driving unit may include an operation of controlling the OIS driving unit based on the second gain value so that the position of the lens assembly or the position of the image sensor is maintained at the reference position with respect to the first direction and the second direction, and an operation of controlling the AF driving unit based on the second gain value so that the position of the lens assembly or the position of the image sensor is maintained at the reference position with respect to the third direction.

[0255] According to one embodiment, the method of operating the electronic device may include an operation of fixing a target value output from the position sensor to a specified value. The method of operating the electronic device may include an operation of controlling the intensity of a current applied to one or more coils for controlling the position of the lens assembly or the position of the image sensor based on the sensing value obtained from the position sensor.

[0256] According to one embodiment, a computer-readable, non-transitory recording medium may cause an electronic device including a camera module to perform a method, including an operation of controlling a driving unit to move a position of the lens assembly or the image sensor based on motion information acquired from a sensor that detects motion of the electronic device, based on a first operation mode among a plurality of operation modes associated with an operation of a driving unit that moves a lens assembly or an image sensor within a driving range when the electronic device is running. The computer-readable, non-transitory recording medium may cause an electronic device including a camera module to perform a method, including an operation of controlling the driving unit to position the lens assembly or the image sensor at a reference position within the driving range, based on a second operation mode different from the first operation mode among the plurality of operation modes when the electronic device including the camera module is running. The computer-readable, non-transitory recording medium may cause an electronic device including a camera module to perform a method, including an operation of performing image processing on an image frame acquired through the camera module while controlling the driving unit when the electronic device including the camera module is running to perform image stabilization processing to reduce shaking of the image. The above computer-readable non-transitory recording medium can perform a method including an operation in which, when the electronic device including the camera module operates based on the first operation mode during execution, the driving unit controls the position of the lens assembly or the image sensor based on a first gain value, and when the electronic device operates based on the second operation mode, the driving unit controls the position of the lens assembly or the image sensor based on a second gain value different from the first gain value.

[0257] According to one embodiment, the recording medium may cause the electronic device, when running, to execute an operation for determining whether to change to one of the plurality of operation modes based on a frequency of the motion information. The recording medium may cause the electronic device, when running, to execute an operation for controlling the driving unit so that the lens assembly or the image sensor is positioned at a reference position within the driving range based on a third operation mode different from the first and second operation modes among the plurality of operation modes. The recording medium may cause the driving unit to execute an operation for controlling the position of the lens assembly or the image sensor based on a third gain value different from the first and second gain values ​​when the electronic device operates based on the third operation mode when running.

[0258] According to one embodiment, the recording medium may cause the electronic device to perform an operation of detecting a frequency component greater than or equal to the specified frequency value based on at least one of a periodicity of the motion information or a differential value of the motion information when the electronic device is executed.

[0259] According to one embodiment, the recording medium may cause the electronic device to execute an operation of fixing a target value output from a position sensor that acquires position information of the lens assembly or position information of the image sensor to a designated value when the electronic device is executed. The recording medium may cause the electronic device to execute an operation of controlling the intensity of a current applied to one or more coils for controlling the position of the lens assembly or the position of the image sensor based on the sensing value acquired from the position sensor when the electronic device is executed.

[0260] The methods according to the embodiments described in the claims or specification of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software.

[0261] When implemented in software, a computer-readable storage medium storing one or more programs (software modules) may be provided. The one or more programs stored in the computer-readable storage medium are configured for execution by one or more processors within an electronic device. The one or more programs include instructions that cause the electronic device to execute methods according to embodiments described in the claims or specification of the present disclosure.

[0262] In the present disclosure, the functions or operations performed by the electronic device may be performed by one or more processors executing one or more instructions stored in a memory. The functions or operations of the electronic device mentioned in the present disclosure may be performed by one processor executing one or more instructions, or may be performed by a combination of multiple processors executing one or more instructions. The processor mentioned in the present disclosure may be understood to include circuitry for performing calculations or controlling other components of the electronic device. For example, the one or more processors may include a central processing unit (CPU), a microprocessor unit (MPU), an application processor (AP), a communication processor (CP), a neural processing unit (NPU), a system on a chip (SoC), or an integrated circuit (IC) configured to execute one or more instructions. The one or more processors may be configured to perform the operations of the electronic device described above.

[0263] In the present disclosure, a program (software module, software) may be stored in a non-volatile memory including a random access memory (RAM), a flash memory, a read only memory (ROM), an electrically erasable programmable read only memory (EEPROM), a magnetic disc storage device, a compact disc ROM (CD-ROM), digital versatile discs (DVDs) or other forms of optical storage devices, a magnetic cassette. Or, it may be stored in a memory formed by a combination of some or all of these. The memory may be formed by a single storage medium, or may be formed by a combination of a plurality of storage media. The one or more commands may be stored in a single storage medium, or may be distributed and stored in a plurality of storage media.

[0264] Additionally, the program may be stored on an attachable storage device that is accessible via a communication network such as the Internet, an intranet, a local area network (LAN), a wide LAN (WLAN), or a storage area network (SAN), or a combination thereof. Such a storage device may be connected to a device performing an embodiment of the present disclosure via an external port. Additionally, a separate storage device on the communication network may be connected to a device performing an embodiment of the present disclosure.

[0265] In the specific embodiments of the present disclosure described above, components included in the disclosure are expressed in the singular or plural form, depending on the specific embodiment presented. However, the singular or plural expressions are selected to suit the presented situation for convenience of explanation, and the present disclosure is not limited to singular or plural components. Components expressed in the plural form may be composed of singular elements, or components expressed in the singular form may be composed of plural elements.

[0266] Additionally, in the present disclosure, terms such as “part”, “module”, etc. may refer to a hardware component such as a processor or circuit, and / or a software component executed by a hardware component such as a processor.

[0267] A "component" or "module" may be implemented by a program stored in an addressable storage medium and executed by a processor. For example, a "component" or "module" may be implemented by components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.

[0268] The specific implementations described in this disclosure are merely exemplary and do not limit the scope of the present disclosure in any way. For the sake of brevity, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted.

[0269] Additionally, in the present disclosure, “comprising at least one of a, b, or c” may mean “comprising only a, including only b, including only c, or including a combination of two or more (including a and b, including b and c, including a and c, or including all of a, b, and c).

[0270] While the detailed description of this disclosure has described specific embodiments, it should be understood that various modifications are possible without departing from the scope of this disclosure. Therefore, the scope of this disclosure should not be limited to the described embodiments, but should be defined not only by the scope of the claims described below, but also by equivalents thereof.

Claims

1. In electronic devices, A sensor for detecting movement of the electronic device; A camera module comprising a lens assembly, an image sensor, and a driving unit for moving the lens assembly or the image sensor within a driving range; one or more processors; and Contains a memory that stores one or more instructions, The one or more instructions, when individually or collectively executed by the one or more processors, cause the electronic device to: Controlling the driving unit to move the position of the lens assembly or the image sensor based on the movement information acquired from the sensor based on a first operation mode among a plurality of operation modes associated with the operation of the driving unit, Based on a second operation mode different from the first operation mode among the plurality of operation modes, the driving unit is controlled so that the lens assembly or the image sensor is positioned at a reference position within the driving range, While controlling the above driving unit, image stabilization processing is performed to reduce image shaking by performing image processing on image frames acquired through the camera module. When operating based on the first operation mode, the driving unit controls the position of the lens assembly or the image sensor based on the first gain value, and when operating based on the second operation mode, the driving unit controls the position of the lens assembly or the image sensor based on a second gain value different from the first gain value. Electronic devices.

2. In claim 1, The one or more instructions, when executed individually or collectively by the one or more processors, cause the electronic device to: Based on the frequency of the above movement information, it is determined whether to change to one of the above multiple operation modes, Based on a third operation mode different from the first operation mode and the second operation mode among the plurality of operation modes, the driving unit is controlled so that the lens assembly or the image sensor is positioned at a reference position within the driving range, When operating based on the third operation mode, the driving unit controls the position of the lens assembly or the image sensor based on a third gain value different from the first gain value and the second gain value. Electronic devices.

3. In claim 2, The one or more instructions, when executed individually or collectively by the one or more processors, cause the electronic device to: To change to at least one of the second operation mode or the third operation mode based on the detection of a frequency component higher than a specified frequency value from the movement information or the detection of a frequency component higher than a certain value. Electronic devices.

4. In claim 3, The one or more instructions, when executed individually or collectively by the one or more processors, cause the electronic device to: Based on at least one of the periodicity of the motion information or the differential value of the motion information, detecting a frequency component higher than the specified frequency value. Electronic devices.

5. In claim 2, The third gain value is greater than the second gain value, Electronic devices.

6. In claim 1, The one or more instructions, when executed individually or collectively by the one or more processors, cause the electronic device to: Obtaining a user input to change one of the above multiple operation modes, Based on the above user input, changing to one of the above multiple operation modes, Electronic devices.

7. In claim 1, The one or more instructions, when executed individually or collectively by the one or more processors, cause the electronic device to: Determine whether an external device has been detected, In response to detecting the external device, the driving unit is controlled to position the lens assembly or the image sensor at a reference position within the driving range based on a designated operating mode among the plurality of operating modes. Electronic devices.

8. In claim 1, Further comprising a housing accommodating the lens assembly and the driving unit, The driving unit is arranged to be movable in a first axis direction substantially perpendicular to the optical axis of the lens assembly and a second axis direction substantially perpendicular to the first axis within the housing, An electronic device, wherein the reference position includes at least one of the center of the driving range on the first axis or the center of the driving range on the second axis.

9. In claim 1, An electronic device, wherein the driving unit includes at least one of an OIS driving unit that moves the lens assembly or the image sensor in at least one of a first direction or a second direction different from the first direction, or an AF driving unit that moves the lens assembly or the image sensor in a third direction substantially perpendicular to the first direction and the second direction according to focus-related information for an image captured through the image sensor.

10. In claim 9, The one or more instructions, when executed individually or collectively by the one or more processors, cause the electronic device to: When operating based on the above second operation mode: Controlling the OIS driving unit based on the second gain value so that the position of the lens assembly or the position of the image sensor is maintained at the reference position with respect to the first direction and the second direction, Controlling the AF driving unit based on the second gain value so that the position of the lens assembly or the position of the image sensor is maintained at the reference position with respect to the third direction. Electronic devices.

11. In claim 1, The above camera module further includes a position sensor that obtains position information of the lens assembly or position information of the image sensor, The above driving unit includes one or more coils, The one or more instructions, when executed individually or collectively by the one or more processors, cause the electronic device to: The target value output from the above position sensor is fixed to a specified value, Controlling the intensity of the current applied to one or more coils based on the sensing value obtained from the position sensor. Electronic devices.

12. In claim 1, The above sensor comprises at least one of a gyro sensor or an acceleration sensor. Electronic devices.

13. In claim 1, The second gain value is greater than the first gain value, Electronic devices.

14. A method of operating an electronic device including a camera module, An operation of controlling the driving unit to move the position of the lens assembly or the image sensor based on movement information acquired from a sensor detecting movement of the electronic device, based on a first operation mode among a plurality of operation modes associated with the operation of the driving unit that moves the lens assembly or the image sensor within the driving range; An operation of controlling the driving unit so that the lens assembly or the image sensor is positioned at a reference position within the driving range based on a second operation mode different from the first operation mode among the plurality of operation modes; An operation of performing image stabilization processing to reduce image shaking by performing image processing on image frames acquired through the camera module while controlling the driving unit; and A method comprising: an operation in which, when operating based on the first operation mode, the driving unit controls the position of the lens assembly or the image sensor based on a first gain value, and when operating based on the second operation mode, the driving unit controls the position of the lens assembly or the image sensor based on a second gain value different from the first gain value; 15. In a non-transitory computer-readable recording medium, when an electronic device including a camera module is executed: An operation of controlling the driving unit to move the position of the lens assembly or the image sensor based on movement information acquired from a sensor detecting movement of the electronic device, based on a first operation mode among a plurality of operation modes associated with the operation of the driving unit that moves the lens assembly or the image sensor within the driving range; An operation of controlling the driving unit so that the lens assembly or the image sensor is positioned at a reference position within the driving range based on a second operation mode different from the first operation mode among the plurality of operation modes; An operation of performing image stabilization processing to reduce image shaking by performing image processing on image frames acquired through the camera module while controlling the driving unit; and A computer program for causing a method to be performed, comprising: an operation in which, when operating based on the first operation mode, the driving unit controls the position of the lens assembly or the image sensor based on a first gain value; and when operating based on the second operation mode, the driving unit controls the position of the lens assembly or the image sensor based on a second gain value different from the first gain value; A recording medium having recorded thereon a computer program for causing the method to be performed.

Images