Electronic device including reflector and lens assembly

Abstract

An electronic device according to one embodiment of the present disclosure may include a lens assembly, a reflector that changes the angle of view by adjusting light incident toward the lens assembly, at least one Hall sensor that determines the position of the reflector, a camera module including a first memory that stores a first correction value for correcting shading by the lens assembly, a second memory that stores a reflector shading correction value for correcting shading by the reflector, and at least one processor electrically connected to the camera module and the second memory. The at least one processor may drive the camera module to acquire a first image frame, and while the first image frame is acquired through the camera module, acquire first Hall data corresponding to the position of the reflector through the at least one Hall sensor, acquire the first correction value from the first memory, acquire a second correction value corresponding to the first Hall data among the reflector shading correction values ​​from the second memory, and perform shading correction for the first image frame based on the first correction value and the second correction value.

Description

Technology Field

[0001] Embodiments of the present disclosure relate to an electronic device comprising a reflector and a lens assembly. Background Technology

[0002] Electronic devices such as smartphones may include a camera module that is a miniaturized version of the digital camera mentioned above. As users prefer thin smartphones, camera modules mounted on electronic devices such as smartphones are being miniaturized to satisfy these user demands. However, since there are limitations to miniaturizing the camera module while maintaining its performance, a periscope camera may be adopted. A periscope camera may include a reflector, such as a prism, capable of changing the direction of light. The problem to be solved

[0003] Conventionally, the movement range of the prism included in the camera module was limited to a certain range, which restricted scanning functions such as changing the angle of view using the camera module. Therefore, a single image was obtained by photographing a light source at a fixed position of the prism, and a single shading correction value was extracted from this image and applied. On the other hand, recently, scanning functions are being utilized by actively rotating the prism. When the prism needs to move over a wide range (e.g., about 20˚ to 25˚) through the scanning function, there is a problem in that it is difficult to achieve accurate shading correction if a single shading correction value is applied to images obtained at various positions of the prism. means of solving the problem

[0004] An electronic device according to one embodiment may include a lens assembly, a reflector that changes the angle of view by adjusting light incident toward the lens assembly, at least one Hall sensor that determines the position of the reflector, a camera module including a first memory that stores a first correction value for correcting shading by the lens assembly, a second memory that stores a reflector shading correction value and instructions for correcting shading by the reflector, wherein the reflector shading correction value has different values ​​depending on the position of the reflector, and at least one processor electrically connected to the camera module and the second memory. The above at least one processor, when executing the above instructions, drives the camera module to acquire a first image frame, and while the first image frame is acquired through the camera module, acquires first Hall data corresponding to the position of the reflector through the at least one Hall sensor, acquires the first correction value from the first memory, acquires a second correction value corresponding to the first Hall data among the reflector shading correction values ​​from the second memory, and can perform shading correction for the first image frame based on the first correction value and the second correction value.

[0005] A method of operating an electronic device according to one embodiment may include: driving a camera module included in the electronic device to acquire a first image frame; acquiring first Hall data corresponding to the position of a reflector through at least one Hall sensor capable of verifying the position of a reflector included in the camera module while the first image frame is acquired through the camera module; acquiring a first correction value for correcting shading by a lens assembly from a first memory included in the camera module; acquiring a second correction value corresponding to the first Hall data among reflector shading correction values ​​for correcting shading by the reflector from a second memory included in the electronic device; and performing shading correction for the first image frame based on the first correction value and the second correction value.

[0006] An electronic device according to one embodiment may include a lens assembly, an actuator capable of moving the lens assembly to perform optical image stabilization (OIS) or optical zoom, at least one Hall sensor for determining the position of the lens assembly, a camera module including a first memory for storing a first correction value for correcting shading by the lens assembly, a second memory for storing a shading movement correction value and instructions for correcting shading that changes as the lens assembly moves, wherein the shading movement correction value has a different value depending on the position of the lens assembly, and at least one processor electrically connected to the camera module and the second memory. The above at least one processor, when executing the above instructions, drives the camera module to acquire a first image frame, and while the first image frame is acquired through the camera module, acquires first Hall data corresponding to the position of the lens assembly through the at least one Hall sensor, acquires the first correction value from the first memory, acquires a second correction value corresponding to the first Hall data among the shading shift correction values ​​from the second memory, and can perform shading correction for the first image frame based on the first correction value and the second correction value. Effects of the invention

[0007] According to various embodiments of the present disclosure, an electronic device can apply a shading correction value corresponding to the position of the prism. Additionally, when the electronic device rotates the prism to acquire images of various viewing angles, the brightness of the image can be made uniform across the entire area.

[0008] According to various embodiments of the present disclosure, the electronic device may store shading correction values ​​in the flash memory of the electronic device along with the non-volatile memory of the camera module, thereby reducing the amount of data stored in the non-volatile memory. In addition, production costs may be reduced by reducing the steps required in the production process of the electronic device.

[0009] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects can be clearly understood by those skilled in the art to which the present disclosure belongs from the description below. Brief explanation of the drawing

[0010] FIG. 1 shows an electronic device according to one embodiment. FIG. 2 shows a camera module according to one embodiment. FIG. 3 shows an exploded perspective view of a camera module according to one embodiment. FIG. 4 illustrates an example in which an electronic device according to one embodiment controls incident light through a reflector to change the angle of view. FIG. 5 illustrates an example of a shading profile obtained differently depending on the position of the viewing angle in an electronic device according to one embodiment. FIG. 6 is a flowchart illustrating the operation of an electronic device according to one embodiment performing shading correction for an image frame. FIG. 7 illustrates an example of a first correction value stored in an electronic device according to one embodiment. FIG. 8 illustrates an example of shading by a reflector in an electronic device according to one embodiment. FIG. 9 illustrates an example of performing shading correction for an image frame based on a first correction value and a second correction value in an electronic device according to one embodiment. FIG. 10 illustrates examples of shading profiles before and after performing shading correction in an electronic device according to one embodiment. FIG. 11 illustrates an example of shading profile data after shading correction is performed according to one embodiment. FIG. 12 shows an electronic device according to one embodiment. FIG. 13 is a flowchart illustrating the operation of an electronic device according to one embodiment performing shading correction for an image frame. FIG. 14 illustrates examples of a shading profile and shading correction according to optical image stabilization (OIS) in an electronic device according to one embodiment. FIG. 15 is a block diagram of an electronic device in a network environment according to various embodiments. FIG. 16 is a block diagram illustrating a camera module according to various embodiments. Specific details for implementing the invention

[0011] FIG. 1 shows the structure of an electronic device (100) and a camera module (180) according to one embodiment.

[0012] FIG. 1 is a schematic drawing showing the exterior of an electronic device (100) equipped with a camera module (180) according to one embodiment and the camera module (180). Although the embodiment of FIG. 1 is illustrated and described on the premise of a mobile device, particularly a smartphone, it will be clearly understood by those skilled in the art that it can be applied to various electronic devices or mobile devices equipped with a camera.

[0013] Referring to FIG. 1, a display (110) may be disposed on the front of an electronic device (100) according to one embodiment. In one embodiment, the display (110) may occupy most of the front of the electronic device (100). On the front of the electronic device (100), a display (110) and a bezel (190) area surrounding at least some edges of the display (110) may be disposed. The display (110) may include a flat area and a curved area extending from the flat area toward the side of the electronic device (100). The electronic device (100) illustrated in FIG. 1 is one example, and various embodiments are possible. For example, the display (110) of the electronic device (100) may include only a flat area without a curved area, or may have a curved area only on one edge rather than both sides. In addition, in one embodiment, the curved area extends to the rear of the electronic device, so that the electronic device (100) may have an additional flat area.

[0014] In one embodiment, the electronic device (100) may additionally include a speaker, a receiver, a front camera, a proximity sensor, or a home key. In one embodiment, the electronic device (100) may be provided with a rear cover (150) integrated with the main body of the electronic device. In another embodiment, the rear cover (150) may be detachable from the main body of the electronic device (100) and may have a form that allows the battery to be replaced. The rear cover (150) may be referred to as a battery cover or a back cover.

[0015] In one embodiment, a fingerprint sensor (171) for user fingerprint recognition may be included in a portion (170) of the display (110). The fingerprint sensor (171) may be placed on the lower layer of the display (110) so that it is not visible to the user or is difficult to see. Additionally, in addition to the fingerprint sensor (171), a sensor for additional user / biometric authentication may be placed in a portion of the display (110). In another embodiment, the sensor for user / biometric authentication may be placed in a portion of the bezel (190). For example, an IR (infrared) sensor for iris authentication may be exposed through a portion of the display (110) or through a portion of the bezel (190).

[0016] In one embodiment, a front camera (161) may be disposed in a portion (160) of the front of the electronic device (100). In the embodiment of FIG. 1, the front camera (161) is shown as being exposed through a portion of the display (110), but in other embodiments, the front camera (161) may be exposed through a bezel (190). The electronic device (100) may include one or more front cameras (161). For example, the electronic device (100) may include two front cameras, such as a first front camera and a second front camera. In one embodiment, the first front camera and the second front camera may be cameras of the same type having equivalent specifications (e.g., pixels), but the first front camera and the second front camera may be implemented as cameras of different specifications. The electronic device (100) may support dual camera-related functions (e.g., 3D shooting, auto focus, etc.) through the two front cameras. The description of the front camera mentioned above may be applied in the same or similar way to the rear camera of the electronic device (100).

[0017] In one embodiment, a front camera (161) may be positioned on the back surface of a region (160) of a display (110) so as to face the region (160). For example, the front camera (161) may be a hidden under-display camera (UDC) that is not visually exposed to the region (160).

[0018] According to one embodiment, a portion (160) of a display (110) facing at least a portion of a front camera (161) may be formed as a transparent area having a specified transmittance as part of an area for displaying content. According to one embodiment, the transparent area may be formed to have a transmittance in the range of about 5% to about 20%. For example, the transparent area may include an area that overlaps with an effective area (e.g., field of view (FOV)) of the front camera (161) through which light passes to form an image by an image sensor to generate an image. For example, the transparent area of ​​the display (110) may include an area with a lower pixel density and / or wiring density than the surrounding area.

[0019] In one embodiment, the electronic device (100) may additionally be equipped with various hardware or sensors (163) that assist in shooting, such as a flash. For example, a distance sensor (e.g., TOF sensor) for detecting the distance between the subject and the electronic device (100) may be additionally included. The distance sensor may be applied to both the front camera (161) and / or the rear camera. The distance sensor may be placed separately from or included with the front camera (161) and / or the rear camera.

[0020] In one embodiment, at least one physical key may be disposed on the side of the electronic device (100). For example, a first function key (151) for turning the display (110) ON / OFF or turning the power of the electronic device (100) ON / OFF may be disposed on the right edge relative to the front of the electronic device (100). In one embodiment, a second function key (152) for controlling the volume of the electronic device (100) or controlling screen brightness, etc., may be disposed on the left edge relative to the front of the electronic device (100). In addition, additional buttons or keys may be disposed on the front or rear of the electronic device (100). For example, a physical button or touch button mapped to a specific function may be disposed in the lower area of ​​the front bezel (190).

[0021] Referring to FIG. 1, an electronic device (100) according to one embodiment may include at least one sensor, such as a camera module, a flash, and a distance sensor, on the rear surface of the electronic device (100) (e.g., a surface facing the +z direction). The camera module (180) may include a lens assembly (182), an AF / OIS carrier assembly (183), an infrared cut filter (184), and an image sensor (185). The camera module (180) may further include a first memory (not shown). The first memory will be described later with reference to FIG. 2 and FIG. 3. In one embodiment, the processor (220) may include an image signal processor (ISP) (187) electrically connected to the image sensor (185). In embodiments of the present disclosure, the image signal processor (187) may be referred to as being included in the processor (120).

[0022] In one embodiment, the lens assembly (182), AF / OIS carrier assembly (183), and image sensor (185) may not be arranged parallel to the direction of light incident on the electronic device (100), but may be arranged substantially perpendicular to the direction of light incident on the electronic device (100). For example, the lens assembly (182), AF / OIS carrier assembly (183), and image sensor (185) may be arranged substantially perpendicular to the direction of light (101) (e.g., -z direction) incident on the electronic device (100), and the thickness of the electronic device (100) may be reduced. A reflector (181) included in the camera module (180) may operate to change the direction of the incident light (101) to the direction of reflection (102) so that light passes through the lens assembly (182), AF / OIS carrier assembly (183), and image sensor (185). For example, the reflector (181) can change the direction of light to the +x direction by reflecting light incident in the -z direction. Due to these structural characteristics, the electronic device (100) can perform shake correction by controlling the movement of the reflector (181) and can change the angle of view by adjusting the light incident on the lens assembly (182). In one embodiment, the reflector (181) may include at least one of a prism or a mirror.

[0023] In one embodiment, the lens assembly (182) may differ in the number, arrangement, and / or type of lenses depending on the front camera (161) and the rear camera. Depending on the type of lens assembly (182), the front camera (161) and the rear camera may have different characteristics (e.g., focal length, maximum magnification, etc.). The lens may move back and forth along the optical axis (e.g., 102) and may operate to change the focal length so that the target object can be clearly captured.

[0024] In one embodiment, the camera module (180) may include a lens assembly (182) that mounts at least one lens aligned on an optical axis, and an AF / OIS carrier assembly (183) that surrounds at least a portion of the perimeter of the lens assembly (182) around the optical axis. The AF / OIS carrier assembly (183) may mount at least one coil. The camera module (180) may control the at least one coil to move the lens assembly (182) in a direction perpendicular to and / or parallel to the optical axis (e.g., 102).

[0025] In one embodiment, an infrared blocking filter (184) may be placed on the upper surface (e.g., -x direction) of the image sensor (185). An image of a subject passing through the lens may be detected by the image sensor (185) after being partially filtered by the infrared blocking filter (184).

[0026] In one embodiment, the image sensor (185) may be a CMOS (complementary metal oxide semiconductor) sensor or a CCD (charged coupled device) sensor. A plurality of individual pixels are integrated in the image sensor (185), and each individual pixel may include a micro lens, a color filter, and a photodiode. Each individual pixel may convert input light into an electrical signal as a type of photodetector. The photodetector may include a photodiode.

[0027] In one embodiment, the image sensor (185) may be electrically connected to an image signal processor (187) connected to a printed circuit board (188) by a connector (186). The connector (186) may be a flexible printed circuit board (FPCB) or a cable, etc. Light information of an object incident through the lens assembly (182) may be converted into an electrical signal by the image sensor (185) and input to the image signal processor (187). In one embodiment, the image signal processor (187) may be placed independently of the processor (120) within the electronic device (100) or may be driven as part of the processor (120).

[0028] In one embodiment, the camera module (180) may be positioned on the front as well as the rear of the electronic device (100). Additionally, the electronic device (100) may include multiple camera modules in addition to one camera module (180) to improve camera performance. For example, the electronic device (100) may further include a front camera (161) for video calls or taking selfies. The front camera (161) may support a relatively lower number of pixels compared to the rear camera module. The front camera (161) may be relatively smaller than the rear camera module.

[0029] In one embodiment, the electronic device (100) may include a processor (120) electrically connected to a camera module (180). In one embodiment, the processor (120) may be understood to mean at least one processor. For example, the processor (220) may be understood to include an application processor (AP), an image signal processor (187), and a communication processor (CP).

[0030] In one embodiment, the electronic device (100) may include a second memory (130) electrically connected to a processor (220). The second memory (130) may be a flash memory. The second memory (130) may store instructions that can be executed by the processor (120).

[0031] The electronic device (100) illustrated in FIG. 1 is an example and does not limit the form of the device to which the technical concept disclosed in this document applies. For example, the technical concept disclosed in this document may be applied to a foldable electronic device that can be folded in a horizontal or vertical direction by employing a flexible display and a hinge structure, or to a tablet or a laptop. Although the electronic device (100) in FIG. 1 is illustrated with a bar-type or plate-type appearance, various embodiments of this disclosure are not limited thereto. For example, the illustrated electronic device (100) may be part of a rollable electronic device. A rollable electronic device may refer to an electronic device in which the display (110) is capable of bending deformation so that at least a part of the display (110) can be wound or rolled or stored inside the electronic device (100). The rollable electronic device can be used to expand the screen display area by unfolding the display (110) or by exposing a larger area of ​​the display (110) to the outside, depending on the user's needs. The display (110) may also be referred to as a slide-out display or an expandable display.

[0033] FIG. 2 shows a camera module (180) according to one embodiment. FIG. 3 shows an exploded perspective view of a camera module (180) according to one embodiment.

[0034] Referring to FIGS. 2 and FIGS. 3, the camera module (180) may include a cover (310), a housing assembly (320), a prism module (210), a lens assembly (330), and an image sensor (360). The lens assembly (330) of FIG. 3 may correspond to the lens assembly (182) of FIG. 1, and the image sensor (360) of FIG. 3 may correspond to the image sensor (185) of FIG. 1. Additionally, the prism (211) of FIG. 2 may correspond to the reflector (181) of FIG. 1, and the printed circuit board (370) of FIG. 3 may correspond to the printed circuit board (188) of FIG. 1. Among the configurations shown in FIG. 2 or FIG. 3, the configuration described in FIG. 1 may be omitted from description or briefly described.

[0035] In one embodiment, the prism module (210) may include a prism (211) (e.g., a reflector (181) in FIG. 1), a prism connecting part (213), a prism holder (215), a first ball set (217), and a second ball set (219). The prism connecting part (213) may connect the prism (211) and the prism holder (215). The prism holder (215) may be 'U' shaped, and the space between the prism holder (215) may contain the prism (211) and the prism connecting part (213). The prism holder (215) is physically coupled to the prism (211) and can move integrally with the prism (211). The prism holder (215) may protect the prism (211) from external impact.

[0036] In one embodiment, the prism (211) may rotate along a panning axis (e.g., z-axis) and a tilting axis (e.g., y-axis). For example, the prism (211) may rotate along a tilting axis (e.g., y-axis) or a panning axis (e.g., z-axis) based on a first ball set (217). Rotation along the tilting axis (e.g., y-axis) may be understood as rotating in a first direction (e.g., pitch direction), and rotation along the panning axis (e.g., z-axis) may be understood as rotating in a second direction (e.g., yaw direction). The prism module (210) may be referred to as a prism assembly and / or a prism structure.

[0037] In one embodiment, the prism (211) included in the prism module (210) is positioned in front of the lens assembly (330) (e.g., in the -x axis direction), and the prism (211) can reflect light incident on one axis (e.g., in the -z direction) toward the lens assembly (330). For example, the prism (211) can divert light entering from the rear of the electronic device (100) (e.g., in the -z direction) by about 90˚ so that it is directed toward the lens assembly (330). The camera module (180) may include a prism holder (215) that encloses the prism (211) and the prism connecting part (213).

[0038] In one embodiment, the camera module (180) may include at least two magnetic bodies (e.g., the first magnet (216) of FIG. 2 and / or the second magnet (331) of FIG. 3) fixed to the prism module (210) and / or lens assembly (330). The camera module (180) may include at least two driving coils (350) interacting with the at least two magnetic bodies. The at least two magnetic bodies move integrally with the prism (211) and / or lens assembly (330) and can transmit electromagnetic force by the at least two driving coils (350) to the prism (211) and / or lens assembly (330).

[0039] In one embodiment, the camera module (180) may include at least one Hall sensor (390). The processor (120) can determine the position of the prism (211) through at least one Hall sensor (390).

[0040] In one embodiment, the camera module (180) may include a housing assembly (320) capable of mounting a prism module (210) and a lens assembly (330). The housing assembly (320) may be covered by a cover (310). The camera module (180) may further include a plurality of bearings capable of supporting rotational and / or linear movement of the lens assembly (330).

[0041] In one embodiment, the image sensor (360) may be connected to and positioned with a printed circuit board (370) (e.g., a printed circuit board (PCB), a printed board assembly (PBA), a flexible PCB (FPCB), or a rigid-flexible PCB (RFPCB)). The image sensor (360) is positioned behind the lens assembly (330) (e.g., in the +x direction) and can collect light that has passed through the lens assembly (330) through the image sensor (360).

[0042] In one embodiment, the printed circuit board (370) may be electrically connected to an autofocus (AF) driver and an optical image stabilization (OIS) driver. When the electronic device (100) shakes, at least one processor included in the electronic device (100) (e.g., processor (120) of FIG. 1) may generate an OIS control value to compensate for the shaking of the electronic device (100), and the electronic device (100) may perform shake correction by transmitting an electrical signal corresponding to the OIS control value to at least one coil of the OIS driver. When taking a picture with a camera, at least one processor included in the electronic device (100) (e.g., processor (120) of FIG. 1) may generate an AF control value to adjust the focal distance between the subject and the camera, and the electronic device (100) may implement AF by transmitting an electrical signal corresponding to the AF control value to at least one coil of the AF driver.

[0043] In one embodiment, the camera module (180) may include a first memory (380). In one embodiment, the first memory (380) may be a non-volatile memory. For example, the first memory (380) may be an EEPROM (electrically erasable and programmable read-only memory). In one embodiment, the first memory (380) may be configured as separate hardware separated from the second memory (130), which is the main memory of the electronic device (100). For example, the second memory (130) may be configured as separate hardware not included in the camera module (180), and the first memory (380) may be included in the camera module (180) and configured as a single module with the camera module (180). In one embodiment, the first memory (380) may be placed on a part of the camera module (180). For example, the first memory (380) may be attached to one side of the housing assembly (320). The first memory (380) may be placed outside and / or inside the housing assembly (320). In one embodiment, the first memory (380) may be electrically and / or operatively connected to the processor (120).

[0044] In one embodiment, the first memory (380) may be manufactured together in the module process of the camera module (180). The first memory (380) may store optical characteristics of the lens assembly (330). For example, the first memory (380) may store a first correction value for correcting shading by the lens assembly (330).

[0046] FIG. 4 illustrates an example in which an electronic device (100) according to one embodiment changes the angle of view by controlling incident light through a reflector (181).

[0047] FIG. 4 illustrates an example of the angle of view of images that can be obtained by controlling a reflector (181) by a processor (120). According to one embodiment, the electronic device (100) can change the angle of view by controlling the light incident toward the lens assembly (182) through the reflector (181). For example, the processor (120) can change the angle of view by rotating the reflector (181) in the pitch direction and / or the yaw direction.

[0048] According to one embodiment, the processor (120) can acquire an image frame corresponding to the angle of view of the ROI (region of interest) 1 (401) when the position of the reflector (181) is at a first position. For example, the first position of the reflector (181) can be understood as the position of the reflector (181) that directs light toward the center of the lens assembly (182). As another example, the first position may mean the position of the reflector (181) that has not rotated in the pitch direction or the yaw direction.

[0049] In one embodiment, the image frame may have a size equal to the angle of view, or a rectangular size smaller than the angle of view (e.g., excluding some areas adjacent to the edges of the angle of view). In the following description, the expressions “image frame corresponding to the angle of view” or “image frame having the angle of view” may be understood as an image frame acquired while the camera is facing the corresponding angle of view.

[0050] According to one embodiment, when the processor (120) changes the position of the reflector (181) from the first position to the second position, it can obtain an image frame having the angle of view of ROI 2 (402). For example, ROI 2 (402) may mean an angle of view moved a certain distance downward from ROI 1 (401). Likewise, the processor (120) can obtain an image frame having the angle of view of ROI 3 (403) when the position of the reflector (181) is at the third position, an image frame having the angle of view of ROI 4 (404) when the position of the reflector (181) is at the fourth position, an image frame having the angle of view of ROI 5 (405) when the position of the reflector (181) is at the fifth position, and an image frame having the angle of view of ROI 6 (406) when the position of the reflector (181) is at the sixth position. Additionally, the processor (120) can acquire image frames having angles of view of ROI 7 (407) to ROI 15 (415), respectively, when the position of the reflector is at the 7th to 15th position. The angles of view shown in FIG. 4 (e.g., ROI 1 (401) to ROI 15 (415)) are merely examples, and various embodiments that can be implemented by a person skilled in the art are possible. For example, the angles of view may be divided into 10 or fewer or 16 or more depending on the movement of the reflector (181), and the range in which the angles of view can be moved may be expanded if the range in which the reflector (181) can rotate in the pitch direction and / or yaw direction is expanded. However, for convenience of explanation, embodiments of the present disclosure are described below based on the shading profiles of image frames having angles of view of ROI 1 (401) to ROI 15 (415).

[0052] FIG. 5 illustrates an example of a shading profile obtained differently depending on the position of the viewing angle in an electronic device (100) according to one embodiment.

[0053] FIG. 5 illustrates an example of a shading profile of an image frame obtained by a processor (120) from a camera module (180). In FIG. 5, each shading profile (501 to 515) represents the ratio of the brightness of a center point of each image frame to the brightness of that point. For example, reference numeral 50a represents an area where the brightness of that point is 95% to 100% of the brightness of the center point, reference numeral 50b represents an area where the brightness of that point is 90% to 95% of the brightness of the center point, reference numeral 50c represents an area where the brightness of that point is 85% to 90% of the brightness of the center point, and reference numeral 50d represents an area where the brightness of that point is less than 85% of the brightness of the center point.

[0054] Referring to FIG. 5, the image frame acquired by the camera module (180) may have a shading profile in which brightness decreases as it moves away from the center point. For example, in the first shading profile (501), the area close to the center of the image frame has 95% to 100% of the brightness compared to the center point, but as it moves away from the center point, the brightness may decrease to less than 85% compared to the center point. In the second shading profile (502) to the 15th shading profile (515), it can be seen that brightness decreases as it moves away from the center point. In one embodiment, since at least one lens included in the lens assembly (182) has a curvature, the amount of light reaching the image sensor (185) is greater as it gets closer to the center, and the amount of light reaching the image sensor (185) may decrease as it moves away from the center according to the curvature.

[0055] According to one embodiment, the shading profile may appear differently depending on the position of the angle of view of the image frame. For example, an image frame having the angle of view of ROI 1 (401) in FIG. 4 may have a first shading profile (501). Similarly, image frames having the angles of view of ROI 2 (402) to ROI 15 (415) in FIG. 4 may each have a second shading profile (502) to a fifth shading profile (515). The angles of view in FIG. 4 (e.g., ROI 1 (401) to ROI 15 (415)) and the shading profiles in FIG. 5 (e.g., first shading profile (501) to fifth shading profile (515)) may correspond to each other. According to one embodiment, the shading profile may appear differently depending on the position of the reflector (181). When the processor (120) changes the position of the reflector (181) to adjust the position of the angle of view, the distribution of the amount of light incident on the lens assembly (182) may vary depending on the position of the reflector (181).

[0056] According to one embodiment, the shading profile of image frames acquired by the processor (120) may include both shading by a lens assembly (182) and shading by a reflector (181). For example, the first shading profile (501) to the 15th shading profile (515) may include shading by a constant lens assembly (182) regardless of the position of the reflector (181). Additionally, the first shading profile (501) to the 15th shading profile (515) may include shading by different reflectors (181) depending on the position of the reflector (181).

[0057] According to one embodiment, the electronic device (100) according to the present disclosure may perform the operations described in FIG. 6 to correct a shading profile (e.g., a first shading profile (501) to a fifth shading profile (515)) according to the position of the reflector (181) while performing a scanning function by adjusting the position of the reflector (181).

[0059] FIG. 6 is a flowchart illustrating the operation of an electronic device (100) according to one embodiment performing shading correction for an image frame. The operations described in FIG. 6 can be performed by the processor (120) shown in FIG. 1. For example, the processor (120) can perform the operations of FIG. 6 when executing an instruction stored in the second memory (130).

[0060] According to one embodiment, in operation 601, the processor (120) can drive the camera module (180) to acquire a first image frame. For example, the first image frame may be an image frame having a brightness distribution of any one of the first shading profile (501) to the 15th shading profile (515) shown in FIG. 5.

[0061] According to one embodiment, in operation 603, the processor (120) may acquire first Hall data corresponding to the position of the reflector (181) through the Hall sensor (390) while the first image frame is acquired through the camera module (180). For example, the first Hall data may correspond to the angle at which the reflector (181) is rotated in the pitch direction and / or yaw direction relative to the first position. The processor (120) may determine the degree to which the reflector (181) has moved (e.g., rotated) based on the first Hall data.

[0062] According to one embodiment, the first Hall data may be Hall data corresponding to the first image frame. For example, the camera module (180) may acquire image data corresponding to the first image frame by exposing the image sensor (185) while acquiring the first image frame, and the processor (120) may acquire the first Hall data corresponding to the position of the reflector (181) during the exposure time of the image sensor (185) through the Hall sensor (390). According to the method described above, each image frame may be synchronized with each Hall data.

[0063] According to one embodiment, in operation 605, the processor (120) may obtain a first correction value from the first memory (380). The first correction value may mean a correction value for correcting shading by the lens assembly (182). The first correction value will be described later with reference to FIG. 7.

[0064] According to one embodiment, in operation 607, the processor (120) can obtain a second correction value corresponding to the first hole data among the reflector shading correction values ​​from the second memory (130). The reflector shading correction value is a correction value for correcting shading by the reflector (181), and may have different values ​​depending on the position of the reflector (181). The processor (120) can obtain a second correction value which is a value corresponding to the first hole data among the reflector shading correction values. The second correction value will be described later with reference to FIG. 8.

[0065] According to one embodiment, in operation 609, the processor (120) can perform shading correction for the first image frame based on the first correction value and the second correction value.

[0066] According to one embodiment, the processor (120) can correct shading caused by the lens assembly (182) through a first correction value and correct shading caused by the reflector that varies depending on the position of the reflector (181) through a second correction value.

[0067] According to one embodiment, the processor (120) may display a first image frame with shading correction performed on it as a preview on the display (110). For example, the processor (120) may output the first image frame with shading correction performed on it as a preview on the display (110) while running an application (e.g., a camera application) using the camera module (180).

[0068] According to one embodiment, the processor (120) may store the first image frame, on which shading correction has been performed, in the second memory (130). For example, the processor (120) may store the first image frame, on which shading correction has been performed, in the second memory (130) in response to receiving a shooting command from a user.

[0069] According to one embodiment, the processor (120) can drive the camera module (180) after operation 609 to acquire a second image frame that follows the first image frame. While acquiring the second image frame through the camera module (180), the processor (120) can acquire second Hall data corresponding to the position of the reflector (181) through the Hall sensor (390). The processor (120) can acquire a third correction value corresponding to the second Hall data among the reflector shading correction values ​​from the second memory (130). The processor (120) can perform shading correction for the second image frame based on the first correction value and the third correction value.

[0070] According to one embodiment, even if the angle of view changes while the processor (120) acquires the first image frame and the second image frame, the shading by the lens assembly (182) may remain constant. Therefore, when the processor (120) performs shading correction for the first image frame and the second image frame, it may continue to use the first correction value acquired from the first memory (380). According to one embodiment, when the angles of view of the first image frame and the second image frame are different, the shading by the reflector (181) may change because the position of the reflector (181) has changed while the camera module (180) acquires the first image frame and the second image frame. Therefore, the processor (120) may again acquire a third correction value from the second memory (130) that is distinguished from the second correction value and corresponds to the second hole data.

[0072] FIG. 7 illustrates an example of a first correction value (710) stored in an electronic device (100) according to one embodiment.

[0073] According to one embodiment, the first memory (380) may store a first correction value (710) for correcting shading by the lens assembly (182). The processor (120) may obtain the first correction value (710) from the first memory (380) and perform shading correction for the first image frame based on the first correction value (710).

[0074] According to one embodiment, the first correction value (710) can be calculated based on shading profile data (700) corresponding to the brightness distribution of the image obtained when the position of the reflector (181) is the first position. For example, the shading profile data (700) may be used to calculate the first correction value (710) during the production process of the electronic device (100). The shading profile data (700) is a brightness distribution by region of an image frame and may include brightness values ​​according to the pixel position of the captured image. In one embodiment, the first position of the reflector may mean the position when the angle of view of the image being captured by the electronic device (100) is ROI 1 (401) of FIG. 4. When the reflector (181) is in the first position, shading by the reflector does not occur or occurs in a negligible amount, so a first correction value (710) can be calculated based on the shading profile data (700) obtained when the position of the reflector (181) is in the first position.

[0075] According to one embodiment, shading profile data (700) may be obtained by dividing an image frame into multiple regions and based on the ratio of brightness between the divided regions. For example, FIG. 7 illustrates shading profile data (700) obtained by dividing an image frame into 7x7 regions. According to one embodiment, the shading value of each region included in the shading profile data (700) may be obtained based on the ratio between the brightness of the corresponding region and the brightness of the center region. For example, if the brightness of the upper-left region (1, 1) of the image frame is 76.4% compared to the brightness of the center region (4, 4), the shading value of the (1, 1) region may be 76.4.

[0076] According to one embodiment, a first correction value (710) can be obtained based on shading profile data (700). For example, the first correction value (710) can be calculated based on the value obtained by dividing the shading value of the (4, 4) area by the brightness value of each area included in the shading profile data (700). For example, in the case of the (1, 1) area, the correction value of 1.31 can be obtained by dividing the shading value of the (4, 4) area, 100, by the shading value of the (1, 1) area, 76.4.

[0077] According to one embodiment, the shading profile data (700) and the first correction value (710) illustrated in FIG. 7 are examples, and various embodiments that can be implemented by a person skilled in the art are possible. For example, the first correction value may be calculated by dividing the image frame into 8x8 or more, and the specific shading values ​​included in the shading profile data (700) may vary depending on the configuration of the lens assembly (182).

[0078] According to one embodiment, the shading profile data (700) may be obtained by capturing a light source image during the production process of the electronic device (100). The light source image may be an image of a light source. Since at least one lens included in the lens assembly (182) has a curvature, shading may occur due to the lens assembly (182), and the shading profile data (700) may vary depending on the curvature of the at least one lens, a process of calculating a first correction value (710) for each electronic device (100) may be included during the production process of the electronic device (100). For example, the first correction value (710) may have a slight difference for each electronic device (100).

[0079] According to one embodiment, a first correction value (710) calculated based on shading profile data (700) can be stored in a first memory (380). For example, the first correction value (710) can be stored in an EEPROM, which is a non-volatile memory included in the camera module (180). Since the first correction value (710) does not have a large data capacity and is data acquired during the production process of the camera module (180), it can be stored in the first memory (380) rather than the main memory of the electronic device (100).

[0080] According to one embodiment, the processor (120) may obtain a first correction value (710) from a first memory (380) included in the camera module (180) and perform lens shading correction for the first image frame. For example, the processor (120) may multiply the first correction value (710) by the first image frame. When the processor (120) performs lens shading correction through the first correction value (710) for the first image frame, shading by the lens assembly (182) included in the first image frame may be corrected.

[0082] FIG. 8 illustrates an example of shading (800) by a reflector (181) in an electronic device (100) according to one embodiment.

[0083] According to one embodiment, the second memory (130) may store a reflector shading correction value for correcting shading by a reflector (181). The processor (120) may obtain a second correction value corresponding to the first hole data among the reflector shading correction values ​​from the second memory (130), and may perform shading correction for the first image frame based on the second correction value.

[0084] Referring to FIG. 8, shading (800) by the reflector may appear differently depending on the position of the reflector (181). The shading (800) by the reflector in FIG. 8 may refer to shading profile data after the shading by the lens assembly (182) included in image frames having different angles of view has been corrected. For example, when the processor (120) corrects the shading by the lens assembly (182) through the first correction value (710) of FIG. 7 for image frames with angles of view ROI 1 (401), ROI 2 (402), ROI 11 (411), ROI 12 (412), and ROI 14 (414), the image frames in which the shading by the lens assembly (182) has been corrected may have shading profile data corresponding to the shading (800) by the reflector in FIG. 8.

[0085] According to one embodiment, the electronic device (100) may store a reflector shading correction value calculated using a design value (810) rather than an actual value (820) of the shading (800) by the reflector in a second memory (130). For example, even if the electronic device (100) stores a reflector shading correction value calculated based on an actual value (820) of the shading (800) by the reflector, the processor (120) may correct the shading by the reflector (181) of the image frame based on the reflector shading correction value. However, in order to store a reflector shading correction value obtained using an actual value (820), a process is added to obtain shading profile data (e.g., actual value (820) of FIG. 8) according to the position of the reflector (181) during the production process of the electronic device (100) and to calculate the reflector shading correction value. In addition, since the data capacity of the reflector shading correction value calculated for each position of the reflector (181) is large, it may occupy a large amount of storage space in the first memory (380).

[0086] According to one embodiment, if there is no significant difference between shading correction based on a design value (810) and shading correction based on an actual value (820), it may be advantageous to perform reflector shading correction using the design value (810) in terms of process and / or storage space. For example, in FIG. 8, it can be seen that there is no significant difference between shading (800) by a reflector according to the design value (810) and shading (800) by a reflector according to the actual value (820). For example, when comparing the design value (812) and the actual value (822) of ROI 2 (402), there is a commonality that the bottom of the image is brighter than the top, and the difference in shading values ​​of each area included in the shading profile data is not large. As another example, when comparing the design value (813) and the actual value (823) of ROI 11 (411), there is a commonality that the upper left corner of the image is darker than the center area. Therefore, when the processor (120) performs shading correction for the first image frame based on a reflector shading correction value corresponding to the actual value (820), and when it performs shading correction for the first image frame based on a reflector shading correction value corresponding to the design value (810), there may not be a significant difference in the quality of the shading correction for the first image frame. In the electronic device (100) according to an embodiment of the present disclosure, instead of using the actual value (820), a reflector shading correction value calculated based on the design value (810) may be stored in the second memory (130). The processor (120) can obtain a second correction value corresponding to the first hole data among the reflector shading correction values ​​calculated based on the design value (810) from the second memory (130), and can perform shading correction for the first image frame based on the second correction value.

[0087] According to one embodiment, the reflector shading correction value stored in the second memory (130) may be a design value defined by the manufacturer of the electronic device (100). The reflector shading correction value may be understood as a value specified or designed by the manufacturer of the electronic device (100). For example, the reflector shading correction value may be a value calculated based on the design value (810) of FIG. 8. According to one embodiment, the reflector shading correction value may be stored in the form of at least one of a shading correction function or a lookup table corresponding to the position of the reflector (181).

[0088] According to one embodiment, the electronic device (100) stores shading correction values ​​in the second memory (130), which is a flash memory of the electronic device (100), along with the first memory (380), which is a non-volatile memory of the camera module (180), thereby reducing the amount of data stored in the first memory (380). Additionally, since the steps required in the production process of the electronic device (100) can be reduced, production costs can be reduced.

[0090] FIG. 9 illustrates an example of performing shading correction on an image frame based on a first correction value (710) and a second correction value (920) in an electronic device (100) according to one embodiment.

[0091] According to one embodiment, the processor (120) can perform shading correction for a first image frame based on a first correction value (710) and a second correction value (920) in operation 609 of FIG. 6. FIG. 9 illustrates an example of shading correction performed on shading profile data (910) of a first image frame based on a first correction value (710) and a second correction value (920).

[0092] According to one embodiment, the shading profile data (910) represents the brightness distribution of the first image frame. For example, the first image frame is an image frame obtained from ROI 2 (402), and the shading profile data (910) of the first image frame may include both shading by the lens assembly (182) and shading by the reflector (181).

[0093] According to one embodiment, the processor (120) can correct the shading by the lens assembly (182) for the first image frame based on the first correction value (710). For example, the processor (120) can multiply the first image frame by the first correction value (710). FIG. 9 illustrates an example of shading profile data (915) in which the shading by the lens assembly (182) is corrected when the shading profile data (910) of the first image frame is multiplied by the first correction value (710).

[0094] According to one embodiment, the processor (120) can correct shading by the reflector (181) based on the second correction value (920) for the first image frame. For example, the processor (120) can multiply the first image frame by the second correction value (920). FIG. 9 illustrates an example of shading profile data (925) in which shading correction is performed by multiplying the second correction value (920) by the shading profile data (915) in which shading by the lens assembly (182) is corrected.

[0095] According to one embodiment, the shading profile data (925) on which shading correction has been performed represents the brightness distribution of a first image frame in which shading by the lens assembly (182) and shading by the reflector (181) have been corrected. For example, when shading correction is performed on the first image frame, the brightness of each region in which the image frame is divided into a plurality of regions may not differ significantly from the brightness of the central region (e.g., the brightness of the (4, 4) region). Referring to FIG. 9, the shading value of each region in the shading profile data (925) on which shading correction has been performed may be 98 to 103.

[0096] According to one embodiment, FIG. 9 illustrates that a first correction value (710) is multiplied by a second correction value (920) after multiplying the first image frame, but this is merely one example and various embodiments that can be implemented by a person skilled in the art are possible. For example, the processor (120) may multiply the first correction value (710) after multiplying the first image frame by the second correction value (920), or may multiply the shading correction value obtained by multiplying the first correction value (710) and the second correction value (920) by the first image frame.

[0098] FIG. 10 illustrates examples of shading profiles before and after performing shading correction in an electronic device (100) according to one embodiment.

[0099] Referring to FIG. 10, the shading profile (1010) before shading correction may represent the brightness distribution of an image frame in which shading by the lens assembly (182) and shading by the reflector (181) are not corrected. The shading profile (1020) after shading correction may represent the brightness distribution of an image frame in which shading by the lens assembly (182) and shading by the reflector (181) are corrected.

[0100] According to one embodiment, the brightness distribution of the first image frame obtained by the processor (120) in operation 601 of FIG. 6 may correspond to a shading profile (1010) before shading correction. For example, in the shading profile (1010) before shading correction, the brightness may decrease as it moves away from the center point of the first image frame.

[0101] According to one embodiment, the brightness distribution of the first image frame, for which the processor (120) performs shading correction in operation 609 of FIG. 6, may correspond to the shading profile (1020) after shading correction. For example, in the shading profile (1020) after shading correction, the brightness may be constant even as it moves away from the center point of the first image frame. Cases where the brightness is constant may include cases where the brightness is the same or where the difference in brightness is negligibly small.

[0102] According to one embodiment, the electronic device (100) may apply a shading correction value (e.g., a second correction value) corresponding to the position of the reflector (181) (e.g., the prism (211) of FIG. 2). Additionally, according to an embodiment of the present disclosure, when the electronic device (100) rotates the reflector (181) to acquire images of various viewing angles, the brightness of the image may be made uniform across the entire area.

[0104] FIG. 11 illustrates an example of shading profile data (1100) after shading correction is performed according to one embodiment.

[0105] According to one embodiment, when the processor (120) performs shading correction based on a first correction value (710) and a second correction value (920) for image frames having different viewing angles depending on the position of the reflector (181), the shading profile data (1100) after shading correction may have a constant brightness distribution.

[0106] Referring to FIG. 11, the shading profile data (1101) after shading correction of an image having the field of view of ROI 1 (401) may have a constant brightness in all areas, so the shading value of each area may be 100. The shading profile data (1102, 1103, 1104, and 1105) after shading correction of an image having the field of view of ROI 2 (402) to ROI 14 (414) may have the same or similar brightness in all areas, so the shading value of each area may be 95 to 104. That is, the electronic device (100) can obtain an image frame having a constant brightness in all areas of the image through shading correction according to the embodiments of the present disclosure. At this time, constant brightness may include cases where the brightness is the same in all areas of the image or there is no significant difference to the extent that it can be considered the same. For example, when an electronic device (100) captures a subject (e.g., a light source) having a specific brightness, the processor (120) can acquire an image having the same or similar brightness within a certain range in all areas of the image.

[0108] FIG. 12 shows an electronic device (100) according to one embodiment.

[0109] Referring to FIG. 12, the electronic device (100) may include a rear cover (150), a first function key (151), and a camera module (180). The electronic device (100) of FIG. 12 may correspond to the electronic device (100) of FIG. 1, and the camera module (180) of FIG. 12 may correspond to the camera module (180) of FIG. 1. Among the configurations shown in FIG. 12, the configuration described in FIG. 1 may be omitted or briefly described. For example, the description of the configuration shown in FIG. 1 may also apply to FIG. 12, except for the description of the arrangement.

[0110] Referring to FIG. 12, the camera module (180) may include a lens assembly (182), an AF / OIS carrier assembly (183), an infrared cut-off filter (184), and an image sensor (185). The electronic device (100) may include an image signal processor (187) electrically connected to the camera module (180). The image signal processor (187) may be referred to as being included in the processor (120).

[0111] In one embodiment, the lens assembly (182), AF / OIS carrier assembly (183), and image sensor (185) may be arranged parallel to the direction of light incident on the electronic device (100), unlike as shown in FIG. 1. For example, the lens assembly (182), AF / OIS carrier assembly (183), and image sensor (185) may be arranged substantially parallel to the direction of light incident on the electronic device (100) (e.g., the -z direction). In one embodiment, the camera module (180) may not include a reflector (181) (e.g., the prism (211) of FIG. 2), unlike as shown in FIG. 1.

[0112] According to one embodiment, the electronic device (100) can move the lens assembly (182) to support optical image stabilization (OIS) or optical zoom. For example, the camera module (180) may include an actuator capable of moving the lens assembly (182) to perform OIS or optical zoom. The camera module (180) may control the actuator to move the lens assembly (182) perpendicular to the optical axis (e.g., z-axis) to perform OIS, or to move the lens assembly (182) parallel to the optical axis (e.g., z-axis) to perform optical zoom.

[0113] According to one embodiment, the camera module (180) may further include at least one Hall sensor for determining the position of the lens assembly. The Hall sensor for determining the position of the lens assembly may be distinguished from the Hall sensor (390) shown in FIG. 3. Also, in one embodiment, the camera module (180) may further include a first memory (380) for storing a first correction value for correcting shading by the lens assembly (182). For example, the first memory (380) may be an EEPROM. The first memory (380) may correspond to the first memory (380) shown in FIG. 3.

[0114] According to one embodiment, the electronic device (100) may include a second memory (130) electrically connected to a processor (120). For example, the second memory (130) may be a flash memory. According to one embodiment, the second memory (130) may store a shading shift correction value for correcting shading that changes as the lens assembly (182) moves. For example, the shading shift correction value may be a value for correcting shading that changes due to the lens assembly (182) as the lens assembly (182) moves perpendicular or parallel to the optical axis.

[0116] FIG. 13 is a flowchart illustrating the operation of an electronic device (100) according to one embodiment performing shading correction for an image frame. The operations described in FIG. 13 can be performed by the processor (120) illustrated in FIG. 1. For example, the processor (120) can perform the operations of FIG. 13 when executing an instruction stored in the second memory (130).

[0117] According to one embodiment, in operation 1301, the processor (120) can drive the camera module (180) to acquire a first image frame. Operation 1301 may correspond to operation 601 of FIG. 6.

[0118] According to one embodiment, in operation 1303, the processor (120) may acquire first Hall data corresponding to the position of the lens assembly (182) through a Hall sensor while a first image frame is acquired through the camera module (180). For example, the first Hall data may correspond to the distance the lens assembly (182) has moved from a reference position in a direction perpendicular to and / or parallel to the optical axis. The reference position may refer to the position of the lens assembly (182) where light is incident on the center of the lens assembly (182) and optical zoom is not performed. The processor (120) may determine the position of the lens assembly (182) based on the first Hall data.

[0119] According to one embodiment, the first hole data may be hole data synchronized with the first image frame. For example, the camera module (180) may acquire image data included in the first image frame by exposing the image sensor (185) while acquiring the first image frame, and the processor (120) may acquire the first hole data corresponding to the position of the lens assembly (182) during the exposure time of the image sensor (185) through the hole sensor (390).

[0120] According to one embodiment, in operation 1305, the processor (120) may obtain a first correction value from the first memory (380). Operation 1305 may correspond to operation 605 of FIG. 6. The first correction value may correspond to the first correction value (710) described in relation to FIG. 7.

[0121] According to one embodiment, in operation 1307, the processor (120) may obtain a second correction value corresponding to the first hole data among the shading shift correction values ​​from the second memory (130). The shading shift correction values ​​may have different values ​​depending on the position of the lens assembly (182). The processor (120) may obtain a second correction value that is a value corresponding to the first hole data among the shading shift correction values. The second correction value may be distinguished from the second correction value (920) described in relation to FIGS. 8 and 9.

[0122] According to one embodiment, in operation 1309, the processor (120) can perform shading correction for the first image frame based on the first correction value and the second correction value.

[0124] FIG. 14 illustrates an example of a shading profile and shading correction according to OIS in an electronic device (100) according to one embodiment.

[0125] Referring to FIG. 14, the shading profile (1410) represents the brightness distribution of an image frame acquired when the position of the lens assembly (182) is at a reference position. The reference position may be a position where light is incident on the center of the lens assembly (182). The shading profile (1421), shading profile (1422), shading profile (1423), and shading profile (1424) represent the brightness distribution that changes as the actuator moves the lens assembly (182) to perform OIS. The shading profile (1431) is a combination of the shading profile (1410) and the shading profile (1421), representing the brightness distribution of an image frame acquired by the processor (120) performing OIS. Likewise, shading profile (1432) can be understood as a combination of shading profile (1410) and shading profile (1422), shading profile (1433) as a combination of shading profile (1410) and shading profile (1423), and shading profile (1434) as a combination of shading profile (1410) and shading profile (1424).

[0126] According to one embodiment, the processor (120) can correct the shading caused by the lens assembly (182) corresponding to the shading profile (1410) through a first correction value. Additionally, the processor (120) can correct the shading that changes as the lens assembly (182) moves, corresponding to the shading profile (1421) to the shading profile (1424), through a second correction value. That is, the processor (120) can correct the shading caused by the lens assembly (182) and the shading that changes as the lens assembly (182) moves, based on the first correction value and the second correction value.

[0127] FIG. 14 describes an example in which the processor (120) controls the actuator to perform OIS, but the embodiment of the present disclosure may also be applied in cases where the lens assembly (182) is moved parallel to the optical axis through the actuator to perform optical zoom.

[0129] An electronic device according to one embodiment may include a lens assembly, a reflector that changes the angle of view by adjusting light incident toward the lens assembly, at least one Hall sensor that determines the position of the reflector, a camera module including a first memory that stores a first correction value for correcting shading by the lens assembly, a second memory that stores a reflector shading correction value and instructions for correcting shading by the reflector, wherein the reflector shading correction value has a different value depending on the position of the reflector, and at least one processor electrically connected to the camera module and the second memory. The above at least one processor, when executing the above instructions, drives the camera module to acquire a first image frame, and while the first image frame is acquired through the camera module, acquires first Hall data corresponding to the position of the reflector through the at least one Hall sensor, acquires the first correction value from the first memory, acquires a second correction value corresponding to the first Hall data among the reflector shading correction values ​​from the second memory, and can perform shading correction for the first image frame based on the first correction value and the second correction value.

[0130] In an electronic device according to one embodiment, the first correction value may be calculated based on shading profile data corresponding to the brightness distribution of an image obtained when the position of the reflector is the first position.

[0131] In an electronic device according to one embodiment, the reflector shading correction value may be stored in at least one form of a shading correction function or a lookup table corresponding to the position of the reflector.

[0132] In an electronic device according to one embodiment, the reflector shading correction value may be a design value defined by the manufacturer of the electronic device.

[0133] In an electronic device according to one embodiment, the at least one processor, when executing the instructions, drives the camera module to acquire a second image frame following the first image frame, and while the second image frame is acquired through the camera module, acquires second Hall data corresponding to the position of the reflector through the at least one Hall sensor, acquires a third correction value corresponding to the second Hall data among the reflector shading correction values ​​from the second memory, and can perform shading correction for the second image frame based on the first correction value and the third correction value.

[0134] In an electronic device according to one embodiment, the reflector may include at least one of a prism or a mirror.

[0135] An electronic device according to one embodiment includes a display electrically connected to the at least one processor, and the at least one processor can display the first image frame, on which shading correction has been performed, as a preview on the display when executing the instructions.

[0136] In an electronic device according to one embodiment, the at least one processor may store the first image frame, on which shading correction has been performed, in the second memory when executing the instructions.

[0137] In an electronic device according to one embodiment, the first memory may be an EEPROM (electrically erasable programmable read-only memory).

[0138] In an electronic device according to one embodiment, the second memory may be a flash memory.

[0139] A method of operating an electronic device according to one embodiment may include: driving a camera module included in the electronic device to acquire a first image frame; acquiring first Hall data corresponding to the position of a reflector through at least one Hall sensor capable of verifying the position of a reflector included in the camera module while the first image frame is acquired through the camera module; acquiring a first correction value for correcting shading by a lens assembly from a first memory included in the camera module; acquiring a second correction value corresponding to the first Hall data among reflector shading correction values ​​for correcting shading by the reflector from a second memory included in the electronic device; and performing shading correction for the first image frame based on the first correction value and the second correction value.

[0140] In a method of operating an electronic device according to one embodiment, the first correction value may be calculated based on shading profile data corresponding to the brightness distribution of an image obtained when the position of the reflector is the first position.

[0141] In a method of operating an electronic device according to one embodiment, the reflector shading correction value may be stored in at least one form of a shading correction function or a lookup table corresponding to the position of the reflector.

[0142] A method of operation of an electronic device according to one embodiment may include: driving the camera module to acquire a second image frame that follows the first image frame; acquiring second Hall data corresponding to the position of the reflector through the at least one Hall sensor while the second image frame is acquired through the camera module; acquiring a third correction value corresponding to the second Hall data among the reflector shading correction values ​​from the second memory; and performing shading correction for the second image frame based on the first correction value and the third correction value.

[0143] In a method of operating an electronic device according to one embodiment, the operation of storing the first image frame, on which shading correction has been performed, in the second memory may be included.

[0144] An electronic device according to one embodiment may include a lens assembly, an actuator capable of moving the lens assembly to perform optical image stabilization (OIS) or optical zoom, at least one Hall sensor for determining the position of the lens assembly, a camera module including a first memory for storing a first correction value for correcting shading by the lens assembly, a second memory for storing a shading movement correction value and instructions for correcting shading that changes as the lens assembly moves, wherein the shading movement correction value has a different value depending on the position of the lens assembly, and at least one processor electrically connected to the camera module and the second memory. The above at least one processor, when executing the above instructions, drives the camera module to acquire a first image frame, and while the first image frame is acquired through the camera module, acquires first Hall data corresponding to the position of the lens assembly through the at least one Hall sensor, acquires the first correction value from the first memory, acquires a second correction value corresponding to the first Hall data among the shading shift correction values ​​from the second memory, and can perform shading correction for the first image frame based on the first correction value and the second correction value.

[0145] In an electronic device according to one embodiment, the first correction value may be calculated based on shading profile data corresponding to the brightness distribution of an image obtained when the position of the lens assembly is at the first position.

[0146] In an electronic device according to one embodiment, the first memory may be an EEPROM.

[0147] In an electronic device according to one embodiment, the second memory may be a flash memory.

[0148] In an electronic device according to one embodiment, the at least one processor, when executing the instructions, drives the camera module to acquire a second image frame following the first image frame, and while the second image frame is acquired through the camera module, acquires second Hall data corresponding to the position of the lens assembly through the at least one Hall sensor, acquires a third correction value corresponding to the second Hall data among the shading movement correction values ​​from the second memory, and can perform shading correction for the second image frame based on the first correction value and the third correction value.

[0150] FIG. 15 is a block diagram of an electronic device (1501) in a network environment (1500) according to various embodiments. Referring to FIG. 15, in the network environment (1500), the electronic device (1501) may communicate with an electronic device (1502) through a first network (1598) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (1504) or a server (1508) through a second network (1599) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (1501) may communicate with the electronic device (1504) through a server (1508). According to one embodiment, the electronic device (1501) may include a processor (1520), memory (1530), input module (1550), sound output module (1555), display module (1560), audio module (1570), sensor module (1576), interface (1577), connection terminal (1578), haptic module (1579), camera module (1580), power management module (1588), battery (1589), communication module (1590), subscriber identification module (1596), or antenna module (1597). In some embodiments, at least one of these components (e.g., connection terminal (1578)) may be omitted from the electronic device (1501), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (1576), camera module (1580), or antenna module (1597)) may be integrated into a single component (e.g., display module (1560)).

[0151] The processor (1520) can, for example, execute software (e.g., program (1540)) to control at least one other component (e.g., hardware or software component) of the electronic device (1501) connected to the processor (1520) and perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (1520) can store commands or data received from other components (e.g., sensor module (1576) or communication module (1590)) in volatile memory (1532), process the commands or data stored in volatile memory (1532), and store the resulting data in non-volatile memory (1534). According to one embodiment, the processor (1520) may include a main processor (1521) (e.g., a central processing unit or an application processor) or an auxiliary processor (1523) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (1501) includes a main processor (1521) and an auxiliary processor (1523), the auxiliary processor (1523) may be configured to use less power than the main processor (1521) or to be specialized for a specified function. The auxiliary processor (1523) may be implemented separately from the main processor (1521) or as part thereof.

[0152] The auxiliary processor (1523) may control at least some of the functions or states associated with at least one component of the electronic device (1501) (e.g., display module (1560), sensor module (1576), or communication module (1590)) on behalf of the main processor (1521) while the main processor (1521) is in an inactive (e.g., sleep) state, or together with the main processor (1521) while the main processor (1521) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (1523) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (1580) or communication module (1590)). According to one embodiment, the auxiliary processor (1523) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (1501) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (1508)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0153] The memory (1530) can store various data used by at least one component of the electronic device (1501) (e.g., processor (1520) or sensor module (1576)). The data may include, for example, software (e.g., program (1540)) and input or output data for related commands. The memory (1530) may include volatile memory (1532) or non-volatile memory (1534).

[0154] The program (1540) may be stored as software in memory (1530) and may include, for example, an operating system (1542), middleware (1544), or an application (1546).

[0155] The input module (1550) can receive commands or data to be used for a component of the electronic device (1501) (e.g., processor (1520)) from outside the electronic device (1501) (e.g., user). The input module (1550) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0156] The sound output module (1555) can output a sound signal to the outside of the electronic device (1501). The sound output module (1555) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0157] The display module (1560) can visually provide information to an external (e.g., user) of the electronic device (1501). The display module (1560) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (1560) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0158] The audio module (1570) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (1570) can acquire sound through an input module (1550) or output sound through an audio output module (1555) or an external electronic device (e.g., electronic device (1502)) (e.g., speaker or headphones) that is directly or wirelessly connected to the electronic device (1501).

[0159] The sensor module (1576) can detect the operating state of the electronic device (1501) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (1576) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0160] The interface (1577) may support one or more specified protocols that can be used for the electronic device (1501) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (1502)). According to one embodiment, the interface (1577) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0161] The connection terminal (1578) may include a connector through which the electronic device (1501) can be physically connected to an external electronic device (e.g., electronic device (1502)). According to one embodiment, the connection terminal (1578) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0162] The haptic module (1579) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (1579) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0163] The camera module (1580) can capture still images and video. According to one embodiment, the camera module (1580) may include one or more lenses, image sensors, image signal processors, or flashes.

[0164] The power management module (1588) can manage power supplied to the electronic device (1501). According to one embodiment, the power management module (1588) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0165] The battery (1589) can supply power to at least one component of the electronic device (1501). According to one embodiment, the battery (1589) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0166] The communication module (1590) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (1501) and an external electronic device (e.g., electronic device (1502), electronic device (1504), or server (1508)), and the performance of communication through the established communication channel. The communication module (1590) may include one or more communication processors that operate independently of the processor (1520) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1590) may include a wireless communication module (1592) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (1594) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (1504) via a first network (1598) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (1599) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1592) can identify or authenticate the electronic device (1501) within a communication network such as the first network (1598) or the second network (1599) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (1596).

[0167] The wireless communication module (1592) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (1592) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (1592) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (1592) can support various requirements specified in the electronic device (1501), external electronic device (e.g., electronic device (1504)), or network system (e.g., second network (1599)). According to one embodiment, the wireless communication module (1592) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0168] An antenna module (1597) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (1597) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (1597) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (1598) or a second network (1599), may be selected from the plurality of antennas, for example, by a communication module (1590). A signal or power may be transmitted or received between the communication module (1590) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (1597).

[0169] According to various embodiments, the antenna module (1597) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0170] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0171] According to one embodiment, commands or data may be transmitted or received between an electronic device (1501) and an external electronic device (1504) through a server (1508) connected to a second network (1599). Each of the external electronic devices (1502, or 1504) may be the same or a different type of device as the electronic device (1501). According to one embodiment, all or part of the operations performed on the electronic device (1501) may be performed on one or more of the external electronic devices (1502, 1504, or 1508). For example, if the electronic device (1501) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (1501) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (1501). The electronic device (1501) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (1501) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (1504) may include an Internet of Things (IoT) device. The server (1508) may be an intelligent server using machine learning and / or neural networks.According to one embodiment, an external electronic device (1504) or server (1508) may be included within the second network (1599). The electronic device (1501) may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0172] The electronic device according to the various embodiments disclosed in this document may be a device of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0173] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, each of phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as “first,” “second,” or “first” or “second” may be used simply to distinguish a component from another component and do not limit the components in any other aspect (e.g., importance or order). Where any (e.g., first) component is referred to as “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicationally,” it means that said component may be connected to said other component directly (e.g., wired), wirelessly, or through a third component.

[0174] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0175] Various embodiments of the present document may be implemented as software (e.g., program (1540)) comprising one or more instructions stored in a storage medium (e.g., internal memory (1536) or external memory (1538)) readable by a machine (e.g., electronic device (1501)). For example, a processor (e.g., processor (1520)) of the machine (e.g., electronic device (1501)) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0176] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0177] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations among the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0179] FIG. 16 is a block diagram (1600) illustrating a camera module (1580) according to various embodiments. Referring to FIG. 16, the camera module (1580) may include a lens assembly (1610), a flash (1620), an image sensor (1630), an image stabilizer (1640), a memory (1650) (e.g., a buffer memory), or an image signal processor (1660). The lens assembly (1610) may collect light emitted from a subject that is the target of image capture. The lens assembly (1610) may include one or more lenses. According to one embodiment, the camera module (1580) may include a plurality of lens assemblies (1610). In this case, the camera module (1580) may form, for example, a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies (1610) may have the same lens properties (e.g., angle of view, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties different from the lens properties of other lens assemblies. The lens assemblies (1610) may include, for example, wide-angle lenses or telephoto lenses.

[0180] A flash (1620) may emit light used to enhance light emitted or reflected from a subject. According to one embodiment, the flash (1620) may include one or more light-emitting diodes (e.g., RGB (red-green-blue) LED, white LED, infrared LED, or ultraviolet LED), or a xenon lamp. An image sensor (1630) may acquire an image corresponding to the subject by converting light emitted or reflected from the subject and transmitted through a lens assembly (1610) into an electrical signal. According to one embodiment, the image sensor (1630) may include, for example, one image sensor selected from image sensors with different properties such as an RGB sensor, a BW (black and white) sensor, an IR sensor, or a UV sensor, a plurality of image sensors having the same properties, or a plurality of image sensors having different properties. Each image sensor included in the image sensor (1630) can be implemented using, for example, a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor.

[0181] The image stabilizer (1640) may move at least one lens or image sensor (1630) included in the lens assembly (1610) in a specific direction or control the operational characteristics of the image sensor (1630) (e.g., adjusting read-out timing, etc.) in response to the movement of the camera module (1580) or the electronic device (1501) containing it. This allows for compensating for at least some of the negative effects caused by the movement on the image being captured. According to one embodiment, the image stabilizer (1640) may detect such movement of the camera module (1580) or the electronic device (1501) using a gyroscope sensor (not shown) or an accelerometer sensor (not shown) placed inside or outside the camera module (1580). According to one embodiment, the image stabilizer (1640) may be implemented, for example, as an optical image stabilizer. The memory (1650) may temporarily store at least a portion of the image acquired through the image sensor (1630) for the next image processing operation. For example, if image acquisition by the shutter is delayed or multiple images are acquired at high speed, the acquired original image (e.g., a Bayer-patterned image or a high-resolution image) is stored in the memory (1650), and the corresponding copy image (e.g., a low-resolution image) can be previewed through the display module (1560). Subsequently, when a specified condition is satisfied (e.g., user input or system command), at least a portion of the original image stored in the memory (1650) may be acquired and processed, for example, by an image signal processor (1660). According to one embodiment, the memory (1650) may be configured as at least a portion of the memory (1530) or as a separate memory that operates independently thereof.

[0182] The image signal processor (1660) can perform one or more image processing operations on an image acquired through the image sensor (1630) or an image stored in memory (1650). The above one or more image processing methods may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softing). Additionally or generally, the image signal processor (1660) may perform control (e.g., exposure time control, or readout timing control, etc.) over at least one of the components included in the camera module (1580) (e.g., image sensor (1630)). The image processed by the image signal processor (1660) may be stored back in memory (1650) for further processing or provided to an external component of the camera module (1580) (e.g., memory (1530), display module (1560), electronic device (1502), electronic device (1504), or server (1508)). According to one embodiment, the image signal processor (1660) It may be configured as at least part of the processor (1520) or as a separate processor operating independently of the processor (1520). If the image signal processor (1660) is configured as a separate processor from the processor (1520), at least one image processed by the image signal processor (1660) may be displayed through the display module (1560) as is or after additional image processing by the processor (1520).

[0183] According to one embodiment, the electronic device (1501) may include a plurality of camera modules (1580), each having different attributes or functions. In this case, for example, at least one of the plurality of camera modules (1580) may be a wide-angle camera and at least another may be a telephoto camera. Similarly, at least one of the plurality of camera modules (1580) may be a front camera and at least another may be a rear camera.

Claims

Claim 1 In an electronic device, a camera module comprising: a lens assembly; a reflector that changes the angle of view by adjusting light incident toward the lens assembly; at least one Hall sensor for determining the position of the reflector; and a first memory for storing a first correction value for correcting shading by the lens assembly; a second memory for storing a plurality of reflector shading correction values ​​and instructions corresponding to a plurality of positions of the reflector to correct shading by the reflector, wherein the reflector shading correction values ​​have different values ​​depending on the position of the reflector; The electronic device comprises at least one processor, wherein the instructions are executed individually or collectively by the at least one processor, and wherein the electronic device: drives the camera module to acquire a first image frame; while the first image frame is acquired through the camera module, acquires first Hall data corresponding to the position of the reflector through the at least one Hall sensor; identifies a first position which is the position of the reflector based on the first Hall data; acquires a first correction value corresponding to the brightness distribution of the image acquired when the position of the reflector is the first position from the first memory; acquires a second correction value corresponding to the first position among the plurality of reflector shading correction values ​​from the second memory; and performs shading correction for the first image frame based on the first correction value and the second correction value. Claim 2 An electronic device according to claim 1, wherein the first correction value is calculated based on shading profile data corresponding to the brightness distribution of an image obtained when the position of the reflector is the first position. Claim 3 An electronic device according to claim 1, wherein the reflector shading correction value is stored in at least one form of a shading correction function or a lookup table corresponding to the position of the reflector. Claim 4 An electronic device according to claim 1, wherein the reflector shading correction values ​​are design values ​​defined by the manufacturer of the electronic device. Claim 5 An electronic device according to claim 1, wherein the instructions are executed individually or collectively by the at least one processor, and the electronic device: drives the camera module to acquire a second image frame following the first image frame; while the second image frame is acquired through the camera module, acquires second Hall data corresponding to the position of the reflector through the at least one Hall sensor; identifies a second position which is the position of the reflector based on the second Hall data; acquires a third correction value corresponding to the brightness distribution of the image acquired when the position of the reflector is the second position among the reflector shading correction values ​​from the second memory; and performs shading correction for the second image frame based on the first correction value and the third correction value. Claim 6 An electronic device according to claim 1, wherein the reflector comprises at least one of a prism or a mirror. Claim 7 An electronic device according to claim 1, comprising a display, wherein the instructions are executed individually or collectively by the at least one processor so that the electronic device displays the first image frame on which the shading correction has been performed as a preview on the display. Claim 8 An electronic device according to claim 1, wherein the instructions are executed individually or collectively by the at least one processor so that the electronic device stores the first image frame on which the shading correction has been performed in the second memory. Claim 9 An electronic device according to claim 1, wherein the first memory is an EEPROM (electrically erasable programmable read-only memory). Claim 10 An electronic device according to claim 1, wherein the second memory is a flash memory. Claim 11 A method of operating an electronic device comprising: driving a camera module included in the electronic device to acquire a first image frame; acquiring first Hall data corresponding to the position of a reflector through at least one Hall sensor capable of verifying the position of a reflector included in the camera module while the first image frame is acquired through the camera module; identifying a first position, which is the position of the reflector, based on the first Hall data, and acquiring a first correction value for correcting shading by a lens assembly from a first memory included in the camera module, wherein the first correction value is a value corresponding to the brightness distribution of an image acquired when the position of the reflector is the first position; acquiring a second correction value corresponding to the first position among a plurality of reflector shading correction values ​​for correcting shading by the reflector from a second memory included in the electronic device; and performing shading correction for the first image frame based on the first correction value and the second correction value. Claim 12 A method of operation of an electronic device according to claim 11, wherein the first correction value is calculated based on shading profile data corresponding to the brightness distribution of an image obtained when the position of the reflector is the first position. Claim 13 A method of operating an electronic device according to claim 11, wherein the reflector shading correction values ​​are stored in at least one form of a shading correction function or a lookup table corresponding to the position of the reflector. Claim 14 A method of operation of an electronic device according to claim 11, comprising: driving the camera module to acquire a second image frame following the first image frame; acquiring second Hall data corresponding to the position of the reflector through the at least one Hall sensor while the second image frame is acquired through the camera module; identifying a second position which is the position of the reflector based on the second Hall data; acquiring a third correction value corresponding to the brightness distribution of the image acquired when the position of the reflector is the second position among the plurality of reflector shading correction values ​​from the second memory; and performing shading correction for the second image frame based on the first correction value and the third correction value. Claim 15 A method of operation of an electronic device according to claim 11, comprising the operation of storing the first image frame on which shading correction has been performed in the second memory. Claim 16 In an electronic device, a camera module comprising: a lens assembly; an actuator capable of moving the lens assembly to perform optical image stabilization (OIS) or optical zoom; at least one Hall sensor for determining the position of the lens assembly; and a first memory for storing a first correction value for correcting shading by the lens assembly; a second memory for storing a plurality of shading movement correction values ​​and instructions for correcting shading that changes as the lens assembly moves, wherein the shading movement correction values ​​have different values ​​depending on the position of the lens assembly; and at least one processor, wherein the instructions are executed individually or collectively by the at least one processor, and the electronic device: drives the camera module to acquire a first image frame, and while the first image frame is acquired through the camera module, acquires first Hall data corresponding to the position of the lens assembly through the at least one Hall sensor, and based on the first Hall data, identifies a first position which is the position of the lens assembly, and An electronic device that obtains a first correction value corresponding to the brightness distribution of an image obtained when the position of the lens assembly is the first position from a first memory, obtains a second correction value corresponding to the first hole data among the plurality of shading shift correction values ​​from a second memory, and performs shading correction for the first image frame based on the first correction value and the second correction value. Claim 17 An electronic device according to claim 16, wherein the first correction value is calculated based on shading profile data corresponding to the brightness distribution of an image obtained when the position of the lens assembly is the first position. Claim 18 An electronic device according to claim 16, wherein the first memory is an EEPROM. Claim 19 An electronic device according to claim 16, wherein the second memory is a flash memory. Claim 20 An electronic device according to claim 16, wherein the instructions are executed individually or collectively by the at least one processor, so that the electronic device: drives the camera module to acquire a second image frame following the first image frame; while the second image frame is acquired through the camera module, acquires second Hall data corresponding to the position of the lens assembly through the at least one Hall sensor; identifies a second position which is the position of the lens assembly based on the second Hall data; acquires a third correction value corresponding to the brightness distribution of the image acquired when the position of the lens assembly is the second position among the plurality of shading shift correction values ​​from the second memory; and performs shading correction for the second image frame based on the first correction value and the third correction value.

Images