Electronic device performing automatic exposure control, control method therefor, and recording medium recording

Abstract

A method for controlling an electronic device including a camera is provided. The method for controlling the electronic device includes a step of obtaining an image frame from a camera. Further, the method for controlling the electronic device includes a step of identifying whether a current frame rate is lower than a reference frame rate. Further, the method for controlling the electronic device includes a step of identifying whether a luminance value of an image frame obtained from the image sensor is lower than a reference luminance value based on a determination that the current frame rate is lower than a reference frame rate. Further, the method for controlling the electronic device includes a step of determining whether the camera is in a first state in which the camera is occluded by an obstacle, based on a determination that the brightness value of the obtained image frame is lower than a reference brightness value. Further, the method for controlling the electronic device includes a step of changing an exposure setting of the camera to a first exposure setting based on determining that the camera is in the first state.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to an electronic device for performing automatic exposure control, a method of controlling the electronic device, and a recording medium having recorded thereon a computer program for executing the method of controlling the electronic device. Background Art

[0002] Through automatic exposure control, the camera determines the exposure setting when capturing an image. The camera uses the exposure setting determined by the automatic exposure control to set the operating parameters of components such as the aperture, image sensor, and shutter included in the camera, and performs image capture. Since the automatic exposure control is performed in response to the illumination changing in real time, the camera performs the automatic exposure control based on the image frame input in real time. Therefore, the automatic exposure control is performed in real time in response to the change of the shooting environment, and the camera will continuously change the exposure setting as the shooting environment changes. Summary of the invention

[0003] According to one aspect of the present disclosure, a method for controlling an electronic device including a camera is provided. The method for controlling an electronic device includes obtaining an image frame from a camera. In addition, the method for controlling an electronic device includes identifying whether a current frame rate is lower than a reference frame rate. In addition, the method for controlling an electronic device includes: based on determining that the current frame rate is lower than the reference frame rate, identifying whether a brightness value of an image frame obtained from an image sensor is lower than a reference brightness value. In addition, the method for controlling an electronic device includes: based on determining that the brightness value of the image frame is lower than the reference brightness value, determining whether the camera corresponds to a first state in which the camera is blocked by an obstacle. In addition, the method for controlling an electronic device includes: based on determining that the camera corresponds to the first state, changing the exposure setting of the camera to the first exposure setting.

[0004] According to one aspect of the present disclosure, an electronic device is provided. The electronic device includes a camera having an image sensor. In addition, the electronic device includes a memory storing at least one instruction. In addition, the electronic device includes at least one processor connected to the memory. The at least one processor is configured to execute the at least one instruction to obtain an image frame from the camera. In addition, the at least one processor is configured to identify whether the current frame rate is lower than the reference frame rate. In addition, the at least one processor is configured to: based on determining that the current frame rate is lower than the reference frame rate, identify whether the brightness value of the image frame obtained from the image sensor is lower than the reference brightness value. In addition, the at least one processor is configured to: based on determining that the brightness value of the image frame is lower than the reference brightness value, determine whether the camera corresponds to a first state in which the camera is blocked by an obstacle. In addition, the at least one processor is configured to: based on determining that the camera corresponds to the first state, change the exposure setting of the camera to the first exposure setting.

[0005] According to an aspect of the present disclosure, there is provided a computer-readable recording medium having recorded thereon a computer program for executing a method of controlling an electronic device including a camera on a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 is a diagram illustrating the operation of an electronic device according to an embodiment of the present disclosure.

[0007] Figure 2 is a diagram showing a structure of an electronic device according to an embodiment of the present disclosure.

[0008] Figure 3 is a flowchart of a method of controlling an electronic device according to an embodiment of the present disclosure.

[0009] Figure 4 is a flowchart of a method of controlling an electronic device according to an embodiment of the present disclosure.

[0010] Figure 5 is a diagram illustrating a process of determining whether an exposure setting for an image frame is appropriate according to an embodiment of the present disclosure.

[0011] Figure 6 is a diagram illustrating an operation of storing exposure settings and brightness values ​​of an image frame in a buffer according to an embodiment of the present disclosure.

[0012] Figure 7 is a diagram illustrating a process of determining whether a state of an electronic device corresponds to a first state according to an embodiment of the present disclosure.

[0013] Figure 8 is a diagram illustrating a process of determining whether buffer data is valid according to an embodiment of the present disclosure.

[0014] Fig. 9 is a diagram illustrating an example of valid buffer data and an example of invalid buffer data according to an embodiment of the present disclosure.

[0015] Fig.10 is a diagram illustrating a process of determining a first exposure setting according to an embodiment of the present disclosure.

[0016] Fig.11 is a diagram illustrating a process of determining a first exposure setting according to an embodiment of the present disclosure.

[0017] Fig.12 The effect of controlling exposure settings according to an embodiment of the present disclosure is shown.

[0018] Fig.13 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present disclosure.

[0019] Fig.14is a diagram illustrating an arrangement of a camera, a proximity sensor, and a time-of-flight (ToF) camera in an electronic device according to an embodiment of the present disclosure.

[0020] Fig.15 is a diagram illustrating a protective case of an electronic device according to an embodiment of the present disclosure.

[0021] Fig.16 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present disclosure.

[0022] Fig.17 is a diagram illustrating a configuration for determining a first state of an electronic device based on image frame data according to an embodiment of the present disclosure.

[0023] Fig.18 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present disclosure.

[0024] Fig.19 is a diagram showing a structure of an electronic device according to an embodiment of the present disclosure.

[0025] Fig. 20 is a diagram showing a structure of an electronic device according to an embodiment of the present disclosure.

[0026] Fig.21 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present disclosure.

[0027] Fig. 22 is a diagram showing a result of automatic exposure control according to an embodiment of the present disclosure.

[0028] Fig.23 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present disclosure.

[0029] Fig.24 is a flowchart illustrating a process of determining a first exposure setting based on an illuminance value according to an embodiment of the present disclosure.

[0030] Fig.25 is a flowchart of a method of controlling an electronic device according to an embodiment of the present disclosure.

[0031] Fig.26 is a graph showing changes in ambient illuminance, International Organization for Standardization (ISO), and image frame brightness values ​​according to a comparative example of the present disclosure.

[0032] Fig. 27 is a diagram illustrating changes in ambient illumination, ISO, and image frame brightness values ​​according to an embodiment of the present disclosure.

[0033] Fig.28 is a diagram illustrating an image frame according to a comparative example and an embodiment of the present disclosure.

[0034] Fig.29 is a block diagram illustrating an electronic device in a network environment according to an embodiment of the present disclosure.

[0035] Fig.30 is a block diagram illustrating a camera module according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] This specification describes and discloses the principles of the embodiments to clarify the scope of the claims and enable those skilled in the art to implement the embodiments set forth in the claims. The embodiments of the present disclosure can be implemented in various forms.

[0037] Throughout the drawings, the same reference numerals represent the same elements. This specification does not describe all components in the embodiments, and the common knowledge in the art or the repeated description of the embodiments will be omitted below. Terms such as "module" and "unit" used herein can be implemented using one or a combination of two or more of software, hardware and firmware, and according to the embodiment, multiple "modules" or "units" can be implemented as a single unit or element, or a single "module" or "unit" can include multiple elements.

[0038] When describing the embodiments, when it is determined that the detailed description of the related known technology may unnecessarily obscure the essence of the present disclosure, its detailed description will be omitted. In addition, the numbers (e.g., first, second, etc.) used in the description of the specification are merely identification symbols for distinguishing one element from another element.

[0039] Furthermore, throughout the specification, unless otherwise specifically described to the contrary, when a component is referred to as being “connected” or “coupled” to another component, it should be understood that the component may be directly connected or coupled to the other component, but may also be connected or coupled to the other component through another intermediate component therebetween.

[0040] Hereinafter, the operating principle and various embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0041] Figure 1 is a diagram illustrating the operation of an electronic device according to an embodiment of the present disclosure.

[0042] The embodiments of the present disclosure relate to an electronic device 100 having a camera 110. When the electronic device 100 is equipped with the camera 110, the camera 110 is located at a predetermined position on the electronic device 100. When the electronic device 100 operates in a photographing mode, the camera 110 continuously generates image frames at a predetermined frame rate.

[0043] When operating in the shooting mode, it may happen that the camera 110 located at a predetermined position is blocked by an obstacle. For example, when the user holds the electronic device 100 and blocks the camera 110 with a finger or the like, the camera 110 is blocked by the obstacle. In another example, when the electronic device 100 operates in the shooting mode, when the electronic device 100 is placed on a table or the like and the camera 110 faces the floor, the camera 110 is blocked by the obstacle.

[0044] When the camera 110 is blocked by an obstacle, an image frame is generated while the camera 110 is blocked by the obstacle. In this case, the electronic device 100 performs automatic exposure control based on the image frame generated when the camera 110 is blocked by the obstacle. Automatic exposure control is an operation of changing the exposure setting of the camera 110 based on the captured image frame. The exposure setting includes, for example, a shutter speed, an aperture value, and an ISO value.

[0045] When the camera 110 is blocked by an obstacle, the brightness value of the image frame changes, and the exposure setting is determined based on the changed brightness value. However, the image frame captured when the camera 110 is blocked by an obstacle does not reflect the actual surrounding environment. In addition, when the electronic device 100 adjusts the exposure setting by performing automatic exposure control while the camera 110 is blocked by an obstacle, and then the obstacle is removed, the electronic device 100 does not immediately reflect the surrounding environment. For this reason, a delay occurs when the electronic device 100 performs automatic exposure control by reflecting the changed illumination due to the removal of the obstacle. This results in a decrease in the quality of the image captured after the obstacle is removed, and causes a delay in resuming image capture after the automatic exposure control is completed.

[0046] According to an embodiment of the present disclosure, the electronic device 100 provides improved automatic exposure control by quickly compensating the exposure setting after the camera is blocked by an obstacle and then the obstacle is removed. According to an embodiment, by using the exposure setting for the image frame before the camera 110 is blocked by the obstacle, the electronic device 100 can quickly operate with the exposure setting adjusted according to the surrounding environment after the obstacle is removed.

[0047] like Figure 1 As shown, it is assumed and described that the camera 110 of the electronic device 100 is blocked by the user's finger.

[0048] In operation 130, the user holds the electronic device 100 in his hand and takes a photo. In operation 130, since the camera 110 is not blocked by obstacles, the image frame 140 is generated in a normal state.

[0049] Next, in operation 132, the user blocks the camera 110 of the electronic device 100 with a finger 120. The finger 120 corresponds to an obstacle. Since the camera 110 is blocked by the finger 120, an image frame 142 captured when the camera 110 is blocked is generated. According to an embodiment of the present disclosure, the electronic device 100 changes the exposure setting to a first exposure setting in a first state in which the camera 110 is blocked by an obstacle. The first exposure setting may be determined by a predetermined rule. The first exposure setting may be determined based on the exposure setting before the electronic device 100 operates in the first state. Therefore, the first exposure setting reflects the environmental conditions before the electronic device 100 operates in the first state.

[0050] Next, in operation 134, the user removes the finger 120 from the camera 110 of the electronic device 100 so that the camera 110 operates without obstruction. According to an embodiment of the present disclosure, at the time point when the obstacle is removed, the electronic device 100 operates with a first exposure setting that reflects the environmental conditions before the electronic device 100 operates in the first state. Therefore, when the environmental conditions do not change significantly before and after entering the first state, the first exposure setting reflects the environmental conditions immediately after the obstacle is removed. Therefore, according to an embodiment of the present disclosure, the electronic device 100 can operate by quickly reflecting the environmental conditions after the camera 110 is blocked by an obstacle and then the obstacle is removed.

[0051] Figure 2 is a diagram showing a structure of an electronic device according to an embodiment of the present disclosure.

[0052] According to an embodiment of the present disclosure, the electronic device 100 may be implemented in various forms. For example, the electronic device 100 may include at least one of a smart phone, a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, a wearable device, or a flexible device. According to various embodiments, the wearable device may include at least one of the following: an accessory type device (e.g., a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device (HMD)), a fabric or clothing integrated type device (e.g., electronic clothing), a body attached type device (e.g., a skin patch or a tattoo), or a bio-implantable type device (e.g., a implantable circuit).

[0053] In addition, according to an embodiment of the present disclosure, the electronic device 100 may be a home appliance. The home appliance may include at least one of the following: for example, a television (TV), a digital video disk (DVD) player, an audio player, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a TV box, a game console, an electronic dictionary, an electronic key, a camera, or an electronic photo frame.

[0054] According to an embodiment of the present disclosure, the electronic device 100 includes a camera 110 , a processor 210 , and a memory 220 .

[0055] The camera 110 may be configured as one or more cameras. According to an embodiment, the camera 110 is located at one side of the electronic device 100. In addition, according to an embodiment, the electronic device 100 includes the camera 110 located at the front side and the camera 110 located at the rear side.

[0056] The camera 110 includes an image sensor 230. The image sensor 230 performs photoelectric conversion on incident light to generate image frame data corresponding to an electric signal. The image sensor 230 may correspond to, for example, a charge coupled device (CCD) image sensor.

[0057] The camera 110 also includes at least one lens, a lens driver, an aperture, an aperture driver, a shutter or a shutter driver, or a combination thereof.

[0058] In addition, the camera 110 may be a combination of a plurality of cameras arranged on one side. For example, the camera 110 may include at least one of an ultra wide-angle camera, a wide-angle camera, a telephoto camera, or a zoom camera, or a combination thereof.

[0059] The processor 210 controls all operations of the electronic device 100. The processor 210 may be implemented as one or more processors. The processor 210 may execute program code or one or more instructions stored in the memory 220. The processor 210 may be composed of hardware components that perform arithmetic, logic and input / output (I / O) operations and signal processing. The processor 210 may include at least one of the following: for example, a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU), an application processor (AP), an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), and a field programmable gate array (FPGA), but is not limited thereto. In addition, the processor 210 may include a neural processing unit (NPU) that performs operations of an artificial intelligence (AI) model.

[0060] The memory 220 may store data and instructions required for the operation of the electronic device 100. The memory 220 may be implemented as at least one of a volatile storage medium or a non-volatile storage medium or a combination thereof. For example, the memory 220 may be configured as a non-volatile memory including at least one of a flash memory, a hard disk, a multimedia card micro memory, a read-only memory (ROM), an electrically erasable programmable ROM (EEPROM), or a PROM. In an embodiment, the memory 220 may be an embedded multimedia card (eMMC).

[0061] When the electronic device 100 operates in the shooting mode, the camera 110 generates image frames at a predetermined frame rate. The processor 210 may determine the frame rate based on the automatic exposure control. The processor 210 may set the frame rate to be higher as the illumination increases. The processor 210 stores the image frames generated by the camera 110 in the memory 220. For example, the memory 220 includes a buffer, and the processor 210 stores the image frames in the buffer.

[0062] The processor 210 identifies whether the current frame rate is lower than the reference frame rate. The current frame rate is a value corresponding to the frame rate at which the camera 110 generates image frames from the image sensor 230 and refers to a currently set frame rate value. The reference frame rate is a preset reference value.

[0063] The processor 210 determines whether the current frame rate is lower than the reference frame rate. Based on the determination that the current frame rate is lower than the reference frame rate, the processor 210 determines whether the brightness value of the image frame obtained from the image sensor 230 is lower than the reference brightness value. The processor 210 obtains the image frame from the image sensor 230 or the buffer and calculates the brightness value of the image frame. The brightness value of the image frame may correspond to an average value of the brightness values ​​of the pixels in the image frame. The reference brightness value corresponds to a preset reference value. The processor 210 determines whether the brightness value of the image frame is lower than the reference brightness value by comparing the brightness value of the image frame with the reference brightness value.

[0064] Based on determining that the brightness value of the image frame is lower than the reference brightness value, the processor 210 determines whether the state of the electronic device 100 corresponds to the first state in which the camera 110 is blocked by an obstacle. The processor 210 may use various methods to determine whether the state of the electronic device 100 corresponds to the first state. According to an embodiment, the processor 210 may determine whether the state of the electronic device 100 corresponds to the first state based on the detection value of the proximity sensor. According to an embodiment, the processor 210 may use a depth map obtained by a time-of-flight (ToF) camera to determine whether the state of the electronic device 100 corresponds to the first state. According to an embodiment, the processor 210 may use state information obtained from a protective case coupled to the electronic device 100 to determine whether the state of the electronic device 100 corresponds to the first state. According to an embodiment, the processor 210 may determine whether the state of the electronic device 100 corresponds to the first state based on the focus state of the camera 110. According to an embodiment, the processor 210 may determine whether the state of the electronic device 100 corresponds to the first state based on the image data of the obtained image frame. According to an embodiment, the processor 210 may determine whether the state of the electronic device 100 corresponds to the first state based on the detection value of the acceleration sensor. According to an embodiment, the processor 210 may determine whether the state of the electronic device 100 corresponds to the first state based on the folding angle of the foldable device. Various methods of determining whether the state of the electronic device 100 is the first state will be described in more detail below.

[0065] Based on determining that the state of the electronic device 100 corresponds to the first state, the processor 210 changes the exposure setting of the camera 110 to the first exposure setting. When it is determined that the state of the electronic device 100 corresponds to the first state, the camera 110 is blocked by an obstacle. Therefore, the image frame obtained from the camera 110 does not reflect the surrounding environment. In the first state, the processor 210 sets the first exposure setting for exposure instead of calculating the exposure setting based on the obtained image frame. According to an embodiment, the first exposure setting may be determined based on the exposure setting before the state of the electronic device 100 is determined to be the first state. Each time the processor 210 obtains an image frame from the image sensor 230, the processor 210 stores the brightness value and exposure setting corresponding to each image frame in a buffer. The buffer stores the brightness value and exposure setting for a predetermined number of the latest image frames. According to an embodiment, the exposure setting for the image frame corresponding to the highest brightness value among the values ​​stored in the buffer may be determined as the first exposure setting. According to an embodiment, the processor 210 may calculate the average value of the brightness values ​​of the image frames stored in the buffer, and determine the exposure setting for the following image frame as the first exposure setting, the brightness value of the image frame being closest to the brightness value average value. According to an embodiment, the processor 210 may obtain an exposure setting corresponding to the ambient brightness value from the first database, and determine the obtained exposure setting as the first exposure setting. Various methods of determining the first exposure setting will be described in more detail below.

[0066] Figure 3 is a flowchart of a method of controlling an electronic device according to an embodiment of the present disclosure.

[0067] The various operations in the electronic device control method disclosed in the present invention can be performed by various forms of electronic devices including a processor and a camera. The present disclosure mainly describes an embodiment of the method of controlling an electronic device performed by an electronic device 100 according to an embodiment of the present disclosure. Therefore, the embodiment described for the electronic device 100 may be applicable to the embodiment of the method of controlling an electronic device, and conversely, the embodiment described for the method of controlling an electronic device may be applicable to the embodiment of the electronic device 100. The method of controlling an electronic device according to the disclosed embodiment is not limited to being performed by the electronic device 100 disclosed herein, but may be performed by various forms of electronic devices.

[0068] In operation S302, the electronic device 100 obtains an image frame from the camera 110. The electronic device 100 stores the image frame, a brightness value of the image frame, and an exposure setting for the image frame in a buffer.

[0069] Next, in operation S304, the electronic device 100 determines whether the current frame rate is lower than the reference frame rate.

[0070] When the current frame rate is higher than or equal to the reference frame rate, the electronic device 100 performs an operation of obtaining a next image frame in operation S302.

[0071] When the current frame rate is lower than the reference frame rate, the electronic device 100 determines whether the brightness value of the image frame is lower than the reference brightness value in operation S306. The brightness value of the image frame is an average value of the brightness values ​​of the pixels in the image frame.

[0072] When the brightness value of the image frame is higher than or equal to the reference brightness value, the electronic device 100 performs process A. Figure 4 Describe process A.

[0073] When the brightness value of the image frame is lower than the reference brightness value, in operation S308, the electronic device 100 determines whether the state of the electronic device 100 corresponds to the first state in which the camera 110 is blocked by an obstacle. According to an embodiment, the electronic device 100 may determine whether its state corresponds to the first state based on the detection value of the proximity sensor. According to an embodiment, the electronic device 100 may determine whether its state corresponds to the first state by using a depth map obtained by a ToF camera. According to an embodiment, the electronic device 100 may determine whether its state corresponds to the first state using state information obtained from a protective case coupled to the electronic device 100. According to an embodiment, the electronic device 100 may determine whether its state corresponds to the first state based on the focus state of the camera 110. According to an embodiment, the electronic device 100 may determine whether its state corresponds to the first state based on the image data of the obtained image frame. According to an embodiment, the electronic device 100 may determine whether its state corresponds to the first state based on the detection value of the acceleration sensor. According to an embodiment, the electronic device 100 may determine whether its state corresponds to the first state based on the folding angle of the foldable device. Various methods for determining whether the state of the electronic device 100 is the first state will be described in more detail below.

[0074] The electronic device 100 may use the above-mentioned various methods to determine whether its state corresponds to the first state. The electronic device 100 may use at least one of the above-mentioned various methods, or a combination thereof. In the case where the electronic device 100 uses multiple methods to determine whether its state corresponds to the first state, when it is determined by at least one of the multiple methods that its state corresponds to the first state, the electronic device 100 determines that its state corresponds to the first state.

[0075] When the state of the electronic device 100 does not correspond to the first state, the electronic device 100 performs process A. Figure 4 Describe process A.

[0076] When the state of the electronic device 100 corresponds to the first state, in operation S310, the electronic device 100 changes the exposure setting of the camera 110 to the first exposure setting. According to an embodiment, the first exposure setting may be determined based on the exposure setting applied before the state of the electronic device 100 is determined to be the first state. Each time the electronic device 100 obtains an image frame from the image sensor 230, the electronic device 100 stores the brightness value and exposure setting corresponding to the corresponding image frame in a buffer. The buffer stores the brightness value and exposure setting for a predetermined number of the latest image frames. According to an embodiment, the exposure setting for the image frame corresponding to the highest brightness value among the values ​​stored in the buffer may be determined as the first exposure setting. According to an embodiment, the electronic device 100 may calculate the average value of the brightness values ​​of the image frames stored in the buffer, and determine the exposure setting for the following image frame, whose brightness value is closest to the average brightness value, as the first exposure setting. According to an embodiment, the processor 210 may obtain the exposure setting corresponding to the ambient brightness value from the first database, and determine the obtained exposure setting as the first exposure setting. Various methods for determining the first exposure setting will be described in more detail below.

[0077] When the electronic device 100 changes the exposure setting of the camera 110 to the first exposure setting in operation S310, the electronic device 100 obtains a next image frame in operation S302 and repeats subsequent operations.

[0078] Figure 4 is a flowchart of a method of controlling an electronic device according to an embodiment of the present disclosure.

[0079] Figure 4 Shown is a reference Figure 3 The operation of the described process A. When the brightness value of the image frame is higher than or equal to the reference brightness value, the electronic device 100 performs the process A. In addition, when the state of the electronic device 100 does not correspond to the first state, the electronic device 100 performs the process A.

[0080] In process A, the electronic device 100 changes the frame rate to a second frame rate in operation S402. According to an embodiment, the second frame rate is a preset value and is a frame rate higher than the reference frame rate. The electronic device 100 obtains an image frame by increasing the frame rate to the second frame rate in operation S402. When the electronic device 100 changes the frame rate, it transmits information indicating the change in the frame rate to the application layer.

[0081] Next, in operation S404, the electronic device 100 obtains image frames at a second frame rate. The electronic device 100 obtains image frames from the image sensor 230 at the second frame rate.

[0082] Next, in operation S406, the electronic device 100 determines whether the exposure setting of the obtained image frame is appropriate. The electronic device 100 may determine whether the exposure setting of the obtained image frame is appropriate based on the pixel brightness value distribution of the obtained image frame.

[0083] Based on determining that the exposure setting is not appropriate, in operation S408, the electronic device 100 calculates the exposure setting based on the image frame obtained at the second frame rate. The electronic device 100 may calculate the exposure setting based on the automatic exposure control algorithm. When the electronic device 100 determines that underexposure occurs, the electronic device 100 may perform at least one or a combination of reducing the shutter speed, increasing the ISO value, or increasing the aperture value. When the electronic device determines that overexposure occurs, the electronic device 100 may perform at least one or a combination of increasing the shutter speed, reducing the ISO value, or reducing the aperture value.

[0084] The electronic device 100 changes the current exposure setting to the exposure setting calculated based on the image frame obtained at the second frame rate. When the exposure setting is changed, the electronic device 100 obtains the next image frame in operation S302 and repeats subsequent operations.

[0085] When it is determined that the exposure setting is appropriate, the electronic device 100 obtains a next image frame in operation S302 and repeats subsequent operations.

[0086] Figure 5 is a diagram illustrating a process of determining whether an exposure setting for an image frame is appropriate according to an embodiment of the present disclosure.

[0087] Figure 5 The process of determining whether the exposure setting for the image frame is appropriate may correspond to Figure 4 Operation S406.

[0088] Graph 510 is a histogram representing multiple pixel brightness values ​​of an image frame. The horizontal axis of graph 510 represents pixel brightness values. The horizontal axis corresponds to increasingly higher brightness values ​​from left to right. The vertical axis of graph 510 represents the number of pixels. The horizontal axis of graph 510 represents 256 brightness values ​​from 0 to 255.

[0089] According to an embodiment of the present disclosure, the processor 210 may determine whether the exposure setting for the image frame is appropriate based on the histogram of the pixel brightness values ​​of the image frame. The processor 210 identifies the distribution of the pixel brightness values ​​from the histogram of the pixel brightness values, and determines whether the exposure is appropriate based on the distribution of the pixel brightness values. The processor 210 sets multiple intervals of the pixel brightness values ​​in the histogram of the pixel brightness values. For example, the multiple intervals of the brightness values ​​may include shadows, midtones, and highlights. The shadows represent low brightness intervals, the midtones represent medium brightness intervals, and the highlights represent high brightness intervals.

[0090] According to an embodiment, the processor 210 determines whether the exposure setting for the image frame is appropriate based on the ratio of the pixel brightness values ​​of each interval in the plurality of intervals. The pixel brightness value ratio is the ratio of the number of pixels in the corresponding interval to the total number of pixels. When the pixel brightness value distribution ratio of the shadow exceeds the reference brightness distribution value, the processor 210 determines that underexposure occurs. When the pixel brightness value distribution ratio of the highlight exceeds the reference brightness distribution value, the processor 210 determines that overexposure occurs. When both the pixel brightness value distribution ratio of the shadow and the pixel brightness value distribution ratio of the highlight are less than or equal to the reference brightness distribution value, the processor 210 determines that appropriate exposure occurs.

[0091] According to an embodiment, the processor 210 determines whether the exposure setting for the image frame is appropriate based on whether the number of pixels having the lowest brightness value or the highest brightness value exceeds the reference value. When the number of pixels having the lowest brightness value exceeds the reference value, the processor 210 determines that underexposure occurs, as shown in the graph 520. When the number of pixels having the highest brightness value exceeds the reference value, the processor 210 determines that overexposure occurs, as shown in the graph 522. When both the number of pixels having the lowest brightness value and the number of pixels having the highest brightness value are less than or equal to the reference value, the processor 210 determines that appropriate exposure occurs, as shown in the graph 524.

[0092] Figure 6 is a diagram illustrating an operation of storing brightness values ​​and exposure settings of an image frame in a buffer according to an embodiment of the present disclosure.

[0093] According to an embodiment of the present disclosure, each time an image frame is generated, the processor 210 stores the brightness value and exposure setting of each image frame in the buffer 650. The buffer 650 may be included in the memory 220.

[0094] When the processor 210 starts a shooting mode (e.g., a camera session), it generates a shooting information table 660 in the buffer 650. The shooting information table 660 may include a record identifier (ID), a shutter speed, an aperture value, an ISO value, and a brightness value. Each time the processor 210 obtains an image frame from the image sensor 230, the processor 210 may obtain an image frame brightness value and an exposure setting of the obtained image frame and add them to the shooting information table 660. When new data is generated, the new data is added to the last row of the shooting information table 660. The shooting information table 660 maintains a predetermined number of rows. When the number of rows of the shooting information table 660 reaches a predetermined number, the buffer 650 may delete the first row when adding a new row.

[0095] The image sensor 230 includes a plurality of sub-pixels 612a, 612b, and 612c. The plurality of sub-pixels 612a, 612b, and 612c may each correspond to a predetermined color component. The pixel 610 of the image sensor 230 includes a plurality of sub-pixels 612a, 612b, and 612c having a predetermined color pattern. The plurality of sub-pixels 612a, 612b, and 612c include filter layers 620a, 620b, and 620c and sensor arrays 622a, 622b, and 622c, respectively. The filter layers 620a, 620b, and 620c correspond to color filters corresponding to the color components of the corresponding sub-pixels 612a, 612b, and 612c, respectively. The sensor arrays 622a, 622b, and 622c photoelectrically convert the filtered light signals 632a, 632b, and 632c incident through the color filters into electrical signals, respectively, to generate electrical raw data. The color filter filters the incident light 630 and transmits light signals 632a, 632b, 632c of color components corresponding to the color filter to the sensor arrays 622a, 622b, and 622c, respectively.

[0096] The image sensor 230 generates raw data 642a, 642b, and 642c of each color component. The processor 210 generates raw data 640 corresponding to the image frame by synthesizing the raw data 642a, 642b, and 642c of each color component. The processor 210 generates the raw data 640 corresponding to each image frame. The processor 210 calculates the image frame brightness value based on the raw data 640 of the image frame or the data obtained by post-processing the raw data 640. In addition, the processor 210 obtains the exposure setting set when each image frame is photographed. The processor 210 adds the image frame brightness value and the set exposure setting of each image frame to the shooting information table 660.

[0097] Automatic exposure control is a closed-loop control. Each time an image frame is generated, automatic exposure control is repeatedly performed. Therefore, when an image frame is generated, automatic exposure control of the next image frame can be performed based on the previous image frame in a closed-loop manner. For example, the processor 210 performs automatic exposure control of the next image frame based on the value in the last row 662 of the shooting information table 660.

[0098] Figure 7 is a diagram illustrating a process of determining whether a state of an electronic device corresponds to a first state according to an embodiment of the present disclosure.

[0099] According to an embodiment of the present disclosure, when the state of the electronic device 100 corresponds to the first state, the electronic device 100 also determines whether the buffer data is valid. The buffer data corresponds to the above-mentioned shooting information table 660. According to an embodiment, when the buffer data is valid, the electronic device 100 increases the exposure by setting the exposure setting to the first exposure setting in the first state. On the other hand, when the buffer data is invalid, the electronic device 100 increases the frame rate instead of increasing the exposure in the first state. According to an embodiment of the present disclosure, when the buffer data is invalid, the electronic device 100 increases the frame rate instead of increasing the exposure in the first state, thereby providing an effect of preventing overexposure.

[0100] When the electronic device 100 is in the above Figure 3 When it is determined in operation S306 that the brightness value of the image frame is less than the reference brightness value, in operation S308, the electronic device 100 determines whether its state corresponds to the first state in which the camera 110 is blocked by an obstacle. When it is determined in operation S308 that the state of the electronic device 100 corresponds to the first state, in operation S702, the electronic device 100 determines whether the buffer data is valid. According to an embodiment, the electronic device 100 may determine whether the buffer data is valid based on the brightness values ​​of a plurality of image frames stored in the buffer 650. The electronic device 100 calculates the distribution of the brightness values ​​of the plurality of image frames stored in the buffer 650, and determines whether the brightness values ​​of the plurality of image frames stored in the buffer 650 are stable based on the distribution of the brightness values ​​of the plurality of image frames. In addition, according to an embodiment, the electronic device 100 determines whether the brightness values ​​of the plurality of image frames all exceed the minimum brightness value. Based on determining that the brightness values ​​of the plurality of image frames stored in the buffer 650 are stable and the brightness values ​​of the plurality of image frames exceed the minimum brightness value, the electronic device 100 determines that the brightness values ​​of the image frames and the exposure settings are valid. Reference is made to Figure 8 and Fig. 9 The operation of determining whether the buffer data is valid is described in more detail.

[0101] When the electronic device 100 determines that the buffer data is valid, the electronic device 100 changes the exposure setting to a preset first exposure setting in operation S310.

[0102] When the electronic device 100 determines in operation S308 that the state of the electronic device 100 does not correspond to the first state, the electronic device 100 increases the frame rate to the second frame rate. In addition, when the electronic device 100 determines in operation S702 that the buffer data is invalid, the electronic device 100 increases the frame rate to the second frame rate.

[0103] Figure 8 is a diagram illustrating a process of determining whether buffer data is valid according to an embodiment of the present disclosure.

[0104] Fig. 9 is a diagram illustrating an example of valid buffer data and an example of invalid buffer data according to an embodiment of the present disclosure.

[0105] As mentioned above Figure 7 As described above, when the electronic device 100 determines in operation S308 that its state corresponds to the first state in which the camera 110 is blocked by an obstacle, the electronic device 100 determines in operation 702 whether the buffer data is valid. Figure 8 Operations S802, S804, S806, and S808 may be included in operation S702.

[0106] The electronic device 100 determines whether the buffer data is valid based on the buffer data currently stored in the buffer 650. The currently stored buffer data is data obtained before the time point when the state of the electronic device 100 is determined to correspond to the first state in operation S308 based on the frame rate and the image frame brightness value.

[0107] First, in operation S802, the electronic device 100 determines whether the distribution of the image frame brightness value is stable. Based on the image frame brightness value 910 in the photographing information table 660 stored in the buffer 650, the electronic device 100 determines whether the distribution of the image frame brightness value is stable.

[0108] Fig. 9 Graphs 920, 922, and 930 show image frame brightness values ​​that change over time. The electronic device 100 generates graphs 920, 922, and 930 based on the image frame brightness values ​​910. The electronic device 100 determines the distribution of the image frame brightness values ​​910 based on the graphs 920, 922, and 930.

[0109] When the change value of the image frame brightness value is greater than the reference value, as shown in the curve graph 920, the electronic device 100 may determine that the distribution of the image frame brightness value 910 is unstable. Based on the image frame brightness value before the time point T1 at which it is determined that the state of the electronic device 100 corresponds to the first state, the electronic device 100 determines whether the distribution of the image frame brightness value 910 is stable. According to an embodiment, the electronic device 100 may use the first-order derivative or the second-order derivative on the curve graph 920 to determine whether the change value of the image frame brightness value is greater than the reference value. According to an embodiment, the electronic device 100 uses interpolation to determine whether the change value of the image frame brightness value is greater than the reference value. For example, the electronic device 100 measures the change of the image frame brightness value based on the standard deviation of the interpolated value of the image frame brightness value, and compares the change value with the reference value. According to an embodiment, the electronic device 100 uses clustering to determine whether the change value of the image frame brightness value is greater than the reference value. As a result of clustering, the electronic device 100 determines whether the deviation of the image frame brightness value is greater than the reference value. When the change value of the image frame brightness value is less than or equal to the reference value, the electronic device 100 determines that the distribution of the image frame brightness value 910 is stable.

[0110] When the electronic device 100 determines that the distribution of the image frame brightness values ​​is unstable, the electronic device 100 determines that the buffer data is invalid in operation S808.

[0111] When the electronic device 100 determines that the distribution of the image frame brightness values ​​is stable, in operation S804, the electronic device 100 determines whether the image frame brightness values ​​exceed the minimum brightness value. Based on the image frame brightness values ​​before the time point T1 at which it is determined that the state of the electronic device 100 corresponds to the first state, the electronic device 100 determines whether all the image frame brightness values ​​910 exceed the minimum brightness value. The minimum brightness value corresponds to the threshold value in the curve graph 922.

[0112] When the image frame brightness value is less than the threshold, the electronic device 100 determines in operation S808 that the buffer data is invalid. The threshold may correspond to, for example, a brightness value of 20. Fig. 9 As shown in the curve graph 922, when the image frame brightness value before the time point T1 is less than the threshold value and the image frame brightness value after the time point T2 increases compared to before the time point T1, the electronic device 100 may be in an overexposed state after the time point T2. Therefore, the electronic device 100 increases the frame rate instead of increasing the exposure. On the other hand, when the image frame brightness value before the time point T1 is high and the image frame brightness value after the time point T2 is low, overexposure does not occur. Therefore, for the case where the image frame brightness value before the time point T1 is high and the image frame brightness value after the time point T2 is low, the electronic device 100 sets the exposure setting to the first exposure setting to increase the exposure.

[0113] When the distribution of the image frame brightness value is stable and the image frame brightness value exceeds the minimum brightness value, the electronic device 100 determines that the buffer data is valid in operation S806. Fig. 9 In the curve graph 930, the change value of the image frame brightness value before the time point T1 when the state of the electronic device 100 is determined to correspond to the first state is less than or equal to the reference value, and the image frame brightness value exceeds the minimum brightness value. In this case, the electronic device 100 determines that the buffer data is valid.

[0114] When it is determined in operation S806 that the buffer data is valid, the electronic device 100 changes the camera exposure setting to the first exposure setting in operation S310.

[0115] When it is determined in operation S808 that the buffer data is invalid, the electronic device 100 increases the current frame rate to a second frame rate in operation S704.

[0116] Fig.10 is a diagram illustrating a process of determining a first exposure setting according to an embodiment of the present disclosure.

[0117] According to an embodiment of the present disclosure, the electronic device 100 may determine the exposure setting for the image frame corresponding to the highest image frame brightness value in the buffer data as the first exposure setting. The electronic device 100 identifies the image frame 1010 having the highest image frame brightness value in the buffer data. Fig.10 In the example of FIG. 1 , the image frame 1010 having the record ID value of 2 corresponds to the image frame having the highest image frame brightness value. The electronic device 100 sets the exposure setting for the image frame 1010 having the highest image frame brightness value as the first exposure setting.

[0118] Fig.11 is a diagram illustrating a process of determining a first exposure setting according to an embodiment of the present disclosure.

[0119] According to an embodiment of the present disclosure, the electronic device 100 sets the exposure setting for an image frame whose image frame brightness value is closest to the image frame brightness value average value as the first exposure setting.

[0120] In operation S1102, the electronic device 100 calculates the average value of the image frame brightness values ​​910 stored in the buffer data. Fig.11 In the case of the buffer data, the average value corresponds to 182.5.

[0121] Next, in operation S1104, the electronic device 100 identifies an image frame whose image frame brightness value is closest to the average value of the image frame brightness value 910. Fig.11In the case of the buffer data, the electronic device 100 recognizes that the brightness value of the image frame corresponding to the record ID 3 is 180, which is closest to the average value 182.5.

[0122] Next, in operation S1106, the electronic device 100 sets the exposure setting for the image frame whose image frame brightness value is determined to be closest to the average value as the first exposure setting. Fig.11 In the case of buffer data, the electronic device 100 sets the exposure setting for the image frame corresponding to the record ID 3 to the first exposure setting.

[0123] Fig.12 The effect of controlling exposure settings according to an embodiment of the present disclosure is shown.

[0124] According to an embodiment of the present disclosure, in the first state where the camera 110 is blocked by an obstacle, the electronic device 100 sets the first exposure setting based on the exposure setting applied before entering the first state. Fig.12 , at point B corresponding to the first state, the exposure setting at point A before the first state is applied. In the first state, when the obstacle is removed from the camera 110, the surrounding environment at point C is unpredictable. However, the surrounding environment at point C is likely to be similar to the surrounding environment at point A. In an embodiment of the present disclosure, determination of the first state is performed during the shooting mode. Since the user captures the image without blocking the camera 110 as much as possible, the first state is likely to correspond to a short period of time of several seconds or tens of seconds. After a short period of several seconds or tens of seconds after entering the first state, the environment is likely to be similar to the surrounding environment before entering the first state. Therefore, according to an embodiment of the present disclosure, by applying during the first state a first exposure setting determined based on the exposure setting applied before the first state, an effect of increasing the possibility of quickly reaching an appropriate exposure after the end of the first state can be achieved.

[0125] Fig.13 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present disclosure.

[0126] According to an embodiment of the present disclosure, based on the detection value of a predetermined sensor or the image data of an image frame, the electronic device 100 can determine whether the state of the electronic device 100 corresponds to the first state. The electronic device 100 determines whether the state of the electronic device 100 corresponds to the first state only when it is determined based on the frame rate and the image frame brightness value that a predetermined condition is satisfied. According to an embodiment of the present disclosure, by determining whether the state of the electronic device 100 corresponds to the first state only when the predetermined condition is satisfied as described above, the electronic device 100 can reduce the load of obtaining and processing sensor data. In addition, according to an embodiment of the present disclosure, by determining whether the state of the electronic device 100 corresponds to the first state only when the predetermined condition is satisfied, the electronic device 100 can reduce the processing load of determining whether the state of the electronic device 100 corresponds to the first state based on the data of the image frame.

[0127] According to an embodiment of the present disclosure, the electronic device 100 includes a camera 110, a processor 210, a memory 220, a proximity sensor 1310, and a ToF camera 1320. The electronic device 100 may include only one of the proximity sensor 1310 and the ToF camera 1320, or may include both. Fig.13 In the Figure 2 The same parts of the configuration of the electronic device 100 are described above, and the configurations of the proximity sensor 1310 and the ToF camera 1320 as additional components and components related thereto are described in detail.

[0128] The proximity sensor 1310 is a sensor provided outside the electronic device 100 and can detect the presence of a nearby object. The proximity sensor 1310 emits an electromagnetic field or electromagnetic waves (eg, infrared rays) and detects the presence of a nearby object by detecting a return signal. The proximity sensor 1310 includes, for example, an infrared sensor.

[0129] According to an embodiment of the present disclosure, the proximity sensor 1310 may be located around the camera 110. For example, the proximity sensor 1310 may be disposed within a range of about 1 centimeter (cm) from the camera 110.

[0130] The processor 210 may determine whether the camera 110 corresponds to a first state in which the camera 110 is blocked by an obstacle based on the detection value of the proximity sensor 1310. The processor 210 receives input from the proximity sensor 1310, and when the proximity sensor 1310 is activated, the processor 210 may determine that the camera 110 corresponds to a first state in which the camera 110 is blocked by an obstacle.

[0131] The ToF camera 1320 is disposed outside the electronic device 100 and is used to identify an object in front and measure the distance to the object. The ToF camera 1320 may be located around the camera 110. For example, the ToF camera 1320 may be disposed within a range of about 1 cm from the camera 110. The ToF camera 1320 emits infrared light and measures the time required for the infrared light to be reflected back from the object in front, thereby measuring the distance to the object in front. The processor 210 may identify a three-dimensional (3D) structure in front of the ToF camera 1320 based on the distance to the object measured by the ToF camera 1320 and generate a depth map.

[0132] The depth map includes a plurality of pixels arranged in a two-dimensional (2D) array. Each pixel in the depth map represents the distance between the corresponding pixel and the detected object. The smaller the pixel value in the depth map, the closer the object is, and the larger the pixel value, the farther the object is. The field of view (FOV) of the ToF camera 1320 may correspond to the FOV of the camera 110, or may have a predetermined alignment condition. Therefore, the processor 210 may match the coordinates of each point on the depth map with the coordinates of the corresponding point on the image frame captured by the camera 110.

[0133] The processor 210 may determine whether the state of the electronic device 100 corresponds to the first state in which the camera 110 is blocked by an obstacle based on the depth map. When the minimum value of the depth map is less than the threshold, the processor 210 may determine that the state of the electronic device 100 corresponds to the first state.

[0134] According to an embodiment, when the camera 110 corresponds to the first state, the processor 210 changes the exposure setting of the camera 110 to the first exposure setting.

[0135] According to an embodiment, when the camera 110 corresponds to the first state and the buffer data is valid, the processor 210 changes the exposure setting of the camera 110 to the first exposure setting. When the processor 210 determines that the camera 110 corresponds to the first state and the buffer data is invalid, the processor 210 increases the frame rate to the first frame rate.

[0136] Fig.14 is a diagram illustrating an arrangement of a camera, a proximity sensor, and a ToF camera in an electronic device according to an embodiment of the present disclosure.

[0137] According to an embodiment of the present disclosure, the camera 110 includes a plurality of cameras. For example, the camera 110 may include a front camera 110a and rear cameras 110b, 110c, and 110d. In a shooting mode, the electronic device 100 may use the cameras 110a, 110b, 110c, and 110d selected from the front camera 110a and the rear cameras 110b, 110c, and 110d to perform an image capture operation and generate an image frame.

[0138] The electronic device 100 may include a proximity sensor 1310 around the front camera 110a. When the electronic device 100 uses the front camera 110a to capture an image in a shooting mode, when the front proximity sensor 1310 detects the approach of an object, the electronic device 100 may determine that its state corresponds to the first state.

[0139] The electronic device 100 may include a plurality of rear cameras 110b, 110c, and 110d. The plurality of rear cameras 110b, 110c, and 110d may include an ultra-wide-angle camera 110b, a main camera 110c, and a telephoto camera 110d. When the processor 210 uses the rear cameras 110b, 110c, and 110d for image capture, the processor 210 may obtain image frames from the plurality of rear cameras 110b, 110c, and 110d, respectively, and synthesize the image frames from the rear cameras 110b, 110c, and 110d to generate a final image frame.

[0140] The electronic device 100 may include a ToF camera 1410 arranged around the plurality of rear cameras 110b, 110c, and 110d. The electronic device 100 may determine whether the electronic device 100 corresponds to the first state based on a depth map obtained from the ToF camera 1410 when capturing images using the rear cameras 110b, 110c, and 110d.

[0141] When the electronic device 100 includes a plurality of cameras 110a, 110b, 110c, and 110d located at different positions, the electronic device 100 performs image capture by selecting some of the plurality of cameras 110a, 110b, 110c, and 110d in the shooting mode. The selection of the cameras 110a, 110b, 110c, and 110d may be performed automatically, or may be performed according to user input. When the cameras 110a, 110b, 110c, and 110d are selected for image capture, the electronic device 100 determines whether the electronic device 100 corresponds to the first state using the detection value of the sensor (e.g., the proximity sensor 1310 or the ToF camera 1410) near the selected cameras 110a, 110b, 110c, and 110d. For example, when the front camera 110a is selected in the shooting mode, the processor 210 activates the proximity sensor 1310 around the front camera 110a to obtain the sensor detection value of the proximity sensor 1310, and determines whether the electronic device 100 corresponds to the first state based on the sensor detection value. When the rear cameras 110b, 110c, and 110d are selected in the shooting mode, the processor 210 activates the ToF camera 1410 around the rear cameras 110b, 110c, and 110d to obtain a depth map from the ToF camera 1410, and determines whether the electronic device 100 corresponds to the first state using the depth map.

[0142] Fig.15 is a diagram illustrating a protective case of an electronic device according to an embodiment of the present disclosure.

[0143] According to an embodiment of the present disclosure, the electronic device 100 may be coupled with a protective case 1510. The protective case 1510 may include a body housing the electronic device 100 and a cover covering a display of the electronic device 100. According to an embodiment of the present disclosure, the protective case 1510 may have a folded state 1520, in which the cover covers the front side (the side where the display is provided) of the electronic device 100, and an open state 1522, in which the cover is unfolded to open the front side of the electronic device 100.

[0144] The protective case 1510 is detachably coupled to the electronic device 100. The protective case 1510 is configured so that the cover can be opened and closed by the user. The protective case 1510 may include a magnet at a predetermined position of the cover. By identifying the magnet on the protective case 1510, the electronic device 100 can identify the folded state 1520 and the opened state 1522 of the protective case 1510.

[0145] According to an embodiment, the protective case 1510 may have a display 1512 at the front side. The display 1512 may include, for example, a light emitting diode (LED) transmissive display.

[0146] Fig.16 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present disclosure.

[0147] According to an embodiment of the present disclosure, the electronic device 100 includes a camera 110, a processor 210, a memory 220, and a Hall sensor 1610. Fig.16 In the Figure 2 The same parts as in the configuration of the electronic device 100 are described above, and the description focuses on the configuration of the Hall sensor 1610 as an additional component and components related thereto.

[0148] The Hall sensor 1610 detects the direction and magnitude of a magnetic field. The Hall sensor 1610 is a sensor using the Hall effect, and detects a magnetic field around it.

[0149] The electronic device 100 can be used with Fig.15 The protective case 1510 is coupled to the protective case 1510 of the electronic device 100. The electronic device 100 includes a Hall sensor 1610 at a predetermined position. The Hall sensor 1610 may be located on the front side where the display of the electronic device 100 is located. The protective case 1510 may include a magnet at a position corresponding to the position of the Hall sensor 1610 of the electronic device 100. When the protective case 1510 is in the folded state 1520, the magnet of the protective case 1510 may be located around the Hall sensor 1610 of the electronic device 100.

[0150] The processor 210 detects the folded state 1520 and the opened state 1522 of the protective case 1510 based on the detection value of the Hall sensor 1610. The processor 210 can detect the engagement state with the protective case 1510 by detecting the coupling with the protective case 1510. In a state where the electronic device 100 is coupled with the protective case, the processor 210 detects the folded state 1520 and the opened state 1522 of the protective case 1510 based on the detection value of the Hall sensor 1610.

[0151] When the protective case 1510 is in the folded state 1520, the processor 210 determines that the front camera 110 is in the first state where the front camera 110 is blocked by an obstacle. When the current frame rate is lower than the reference frame rate and the image frame brightness value is lower than the reference brightness value, the processor 210 obtains a detection value from the Hall sensor 1610. The processor 210 determines whether the protective case 1510 is in the folded state 1520 or the open state 1522 based on the detection value of the Hall sensor 1610. When the protective case 1510 is in the folded state 1520, the processor 210 determines that the front camera 110 is in the first state where it is blocked by an obstacle.

[0152] According to an embodiment, when the camera 110 corresponds to the first state, the processor 210 changes the exposure setting of the camera 110 to the first exposure setting.

[0153] According to an embodiment, when the camera 110 corresponds to the first state and the buffer data is valid, the processor 210 changes the exposure setting of the camera 110 to the first exposure setting. When the processor 210 determines that the camera 110 corresponds to the first state and the buffer data is invalid, the processor 210 increases the frame rate to the first frame rate.

[0154] Fig.17 is a diagram illustrating a configuration for determining a first state of an electronic device based on image frame data according to an embodiment of the present disclosure.

[0155] According to an embodiment of the present disclosure, based on the image data of the image frame 1710 , the electronic device 100 may determine whether the state of the electronic device 100 corresponds to the first state.

[0156] According to an embodiment of the present disclosure, in operation 1720, when the current frame rate is lower than the reference frame rate and the image frame brightness value is lower than the reference brightness value, the electronic device 100 may determine whether its state corresponds to the first state based on the focus information of the image frame. When the image frame corresponds to an out-of-focus state, the electronic device 100 may determine that its state corresponds to the first state. According to an embodiment, the electronic device 100 may further determine whether the image frame brightness value is lower than the reference first state brightness value. The reference first state brightness value may correspond to a value lower than the above-mentioned value. Figure 3 The luminance value of the reference luminance value used in operation S306.

[0157] According to an embodiment of the present disclosure, in operation 1730, when the current frame rate is lower than the reference frame rate and the image frame brightness value is less than the reference brightness value, the electronic device 100 may determine whether its state corresponds to the first state based on the image data of the image frame. The electronic device 100 may perform scene recognition based on the image data of the image frame. When, as a result of scene recognition, the electronic device 100 determines that the image frame is captured when the camera 110 is blocked by an obstacle, the electronic device 100 determines that its state corresponds to the first state.

[0158] According to an embodiment, when the camera 110 corresponds to the first state, the processor 210 changes the exposure setting of the camera 110 to the first exposure setting.

[0159] According to an embodiment, when the camera 110 corresponds to the first state and the buffer data is valid, the processor 210 changes the exposure setting of the camera 110 to the first exposure setting. When the processor 210 determines that the camera 110 corresponds to the first state and the buffer data is invalid, the processor 210 increases the frame rate to the first frame rate.

[0160] Fig.18 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present disclosure.

[0161] According to an embodiment of the present disclosure, the electronic device 100 includes a camera 110, a processor 210, a memory 220, and an acceleration sensor 1810. Fig.18 In the Figure 2 The same parts as in the configuration of the electronic device 100 are described herein, and the description focuses on the configuration of the acceleration sensor 1810 as an additional component and components related thereto.

[0162] The acceleration sensor 1810 detects acceleration of the electronic device 100. The acceleration sensor 1810 may include a gyro sensor.

[0163] The processor 210 may detect the movement of the electronic device 100 based on the detection value of the acceleration sensor 1810. The processor 210 may detect a state 1820 in which the electronic device 100 is placed on the floor based on the detection value of the acceleration sensor 1810.

[0164] When the state of the electronic device 100 corresponds to the state 1820 of being placed on the floor, the processor 210 determines that the electronic device 100 is in the first state. When the current frame rate is lower than the reference frame rate and the image frame brightness value is less than the reference brightness value, the processor 210 obtains a detection value from the acceleration sensor 1810. The processor 210 can determine whether the electronic device is in the state 1820 of the electronic device being placed on the floor based on the detection value of the acceleration sensor 1810. When the electronic device 100 is in the state 1820 of being placed on the floor, the processor 210 determines that the electronic device 100 corresponds to the first state.

[0165] According to an embodiment, the electronic device 100 may include a front camera and a rear camera. The electronic device 100 may determine that a camera whose image frame brightness value is less than or equal to a reference brightness value among the front camera and the rear camera corresponds to the first state.

[0166] According to an embodiment, the processor 210 may detect the state in which the electronic device 100 is placed on the floor and the direction in which it is placed based on the detection value of the acceleration sensor 1810. When the front camera is positioned to face the floor, the processor 210 may determine that the front camera corresponds to the first state in which the front camera is blocked by an obstacle. When the rear camera is positioned to face the floor, the processor 210 may determine that the rear camera corresponds to the first state in which the rear camera is blocked by an obstacle.

[0167] When a camera currently performing image capturing among the front camera and the rear camera in the photographing mode corresponds to the first state, the processor 210 may change the exposure setting of the camera performing image capturing to the first exposure setting.

[0168] According to an embodiment, when the camera 110 corresponds to the first state and the buffer data is valid, the processor 210 changes the exposure setting of the camera 110 to the first exposure setting. When the processor 210 determines that the camera 110 corresponds to the first state and the buffer data is invalid, the processor 210 increases the frame rate to the first frame rate.

[0169] Fig.19 is a diagram showing a structure of an electronic device according to an embodiment of the present disclosure.

[0170] According to an embodiment of the present disclosure, the electronic device 100 may have Fig.19 The electronic device 100 includes a folding structure and can be folded or opened by the folding structure. The electronic device 100 can have an opened state and a folded state.

[0171] The electronic device 100 may detect a folding angle θ. The electronic device 100 includes an angle sensor and detects the folding angle θ using the angle sensor. The folding angle θ refers to an angle formed by two bodies of the electronic device 100. When the electronic device 100 is in a folded state, the folding angle θ may correspond to 0 degrees.

[0172] The electronic device 100 may include a front camera disposed at the front side, and the front camera is blocked when the electronic device 100 is in the folded state. The front camera may be blocked by a facing body of the electronic device 100 in the folded state.

[0173] Fig. 20 is a diagram showing a structure of an electronic device according to an embodiment of the present disclosure.

[0174] According to an embodiment of the present disclosure, the electronic device 100 may have Fig. 20 The electronic device 100 includes a folding structure and can be folded or opened by the folding structure. The electronic device 100 can have an opened state and a folded state.

[0175] The electronic device 100 may detect a folding angle θ. The electronic device 100 includes an angle sensor and detects the folding angle θ using the angle sensor. The folding angle θ refers to an angle formed by two bodies of the electronic device 100. When the electronic device 100 is in a folded state, the folding angle θ may correspond to 0 degrees.

[0176] The electronic device 100 may include a front camera disposed at the front side, and the front camera is blocked when the electronic device 100 is in the folded state. The front camera may be blocked by a facing body of the electronic device 100 in the folded state.

[0177] Fig.21 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present disclosure.

[0178] According to an embodiment of the present disclosure, the electronic device 100 includes a camera 110, a processor 210, a memory 220, and an angle sensor 2110. Fig.21 In the Figure 2 The same parts as in the configuration of the electronic device 100 are described above, and the description focuses on the configuration of the angle sensor 2110 as an additional component and components related thereto.

[0179] The angle sensor 2110 detects an angle of the electronic device 100. According to an embodiment, the angle sensor 2110 includes an acceleration sensor. The angle sensor 2110 detects movement of two bodies of the electronic device 100, and the processor 210 detects a folding angle θ formed by the two bodies based on the movement of the two bodies.

[0180] The processor 210 may detect the folding angle θ of the electronic device 100 based on the detection value of the angle sensor 2110. The processor 210 controls the operation of the electronic device 100 based on the folding angle θ.

[0181] When the folding angle θ is less than the reference angle value, the processor 210 determines that the electronic device 100 is in the first state. When the current frame rate is lower than the reference frame rate and the image frame brightness value is less than the reference brightness value, the processor 210 obtains a detection value from the angle sensor 2110. The processor 210 calculates the folding angle θ of the electronic device 100 based on the detection value of the angle sensor 2110. When the folding angle θ is less than the reference angle value, the processor 210 determines that the state of the electronic device 100 corresponds to the first state.

[0182] According to an embodiment, the electronic device 100 may include a front camera and a rear camera. The front camera is arranged at the front side that retracts inward when the electronic device 100 is in a folded state. In the folded state, the electronic device 100 determines that the front camera corresponds to a first state in which the front camera is blocked by an obstacle.

[0183] When capturing an image with the front camera, when the current frame rate is less than the reference frame rate and the image frame brightness value is less than the reference brightness value, the processor 210 determines whether the front camera corresponds to the first state. In this case, when the folding angle θ is less than the reference angle value, the processor 210 determines that the front camera corresponds to the first state.

[0184] When a camera currently performing image capturing among the front camera and the rear camera in the photographing mode corresponds to the first state, the processor 210 may change the exposure setting of the camera performing image capturing to the first exposure setting.

[0185] According to an embodiment, when the camera 110 corresponds to the first state and the buffer data is valid, the processor 210 changes the exposure setting of the camera 110 to the first exposure setting. When the processor 210 determines that the camera 110 corresponds to the first state and the buffer data is invalid, the processor 210 increases the frame rate to the first frame rate.

[0186] Fig. 22 is a diagram showing a result of automatic exposure control according to an embodiment of the present disclosure.

[0187] As mentioned above Figures 19 to 21 As described, according to an embodiment of the present disclosure, the electronic device 100 determines whether it is in the first state based on the folded state or the folded angle and controls the exposure setting. According to an embodiment of the present disclosure, even immediately after the electronic device 100 changes from the folded state to the opened state, the overexposure phenomenon can be prevented and the appropriate exposure setting can be obtained with a high probability.

[0188] like Fig. 22 As shown, it is assumed that the electronic device 100 is in an open state in operation 2202 and then changes to a folded state in operation 2204. Operation 2206 is a comparative example in which an embodiment of the present disclosure is not applied, and operation 2208 is a case in which an embodiment of the present disclosure is applied. In this case, according to the comparative example in which an embodiment of the present disclosure is not applied, overexposure occurs immediately after the electronic device 100 changes back to the open state from the folded state, as in operation 2206. This causes a delay in returning to proper exposure from the overexposed state. On the other hand, in operation 2208 in which an embodiment of the present disclosure is applied, overexposure does not occur immediately after the electronic device 100 changes back to the open state from the folded state due to the above-mentioned exposure setting control or frame rate control. Therefore, according to an embodiment of the present disclosure, overexposure of the front camera does not occur immediately after changing from the folded state to the open state, and the delay caused by the automatic exposure control is short or almost non-existent.

[0189] Fig.23 is a block diagram illustrating a structure of an electronic device according to an embodiment of the present disclosure.

[0190] According to an embodiment of the present disclosure, the electronic device 100 includes a camera 110, a processor 210, a memory 220, an illumination sensor 2310, and a communication module 2320. Fig.23 In the Figure 2 The description of the same parts in the configuration of the electronic device 100 is omitted, and the configuration of the illuminance sensor 2310 and the communication module 2320 as additional components and the components related thereto are described in detail.

[0191] The illuminance sensor 2310 detects the illuminance of its surroundings. The illuminance sensor 2310 may include a light sensor having a photoelectric effect. The illuminance sensor 2310 generates a sensor detection value and outputs it to the processor 210. The processor 210 obtains the illuminance value of the surrounding environment based on the sensor detection value input from the illuminance sensor 2310.

[0192] Based on the illuminance value, the processor 210 obtains an exposure setting from an exposure setting lookup table 2340 stored in a database. The exposure setting lookup table 2340 includes exposure settings according to the illuminance value. The processor 210 may obtain an exposure setting corresponding to the detected illuminance value from the exposure setting lookup table 2340. The processor 210 sets the exposure setting corresponding to the illuminance value as a first exposure setting.

[0193] According to an embodiment, the exposure setting lookup table 2340 may be stored in the memory 220. The processor 210 obtains an exposure setting corresponding to the illuminance value from the exposure setting lookup table 2340 stored in the memory 220 and determines a first exposure setting.

[0194] According to an embodiment, the exposure setting lookup table 2340 may be stored in the server 2330. The electronic device 100 may communicate with the server 2330 via the communication module 2320 to obtain an exposure setting corresponding to the illuminance value from the exposure setting lookup table 2340. When the processor 210 obtains the illuminance value, the processor 210 generates an exposure setting request including the illuminance value. When receiving the exposure setting request, the server 2330 identifies the exposure setting corresponding to the illuminance value included in the exposure setting request from the exposure setting lookup table 2340. The server 2330 sends the identified exposure setting to the electronic device 100. The electronic device 100 sets the exposure setting received from the server 2330 as the first exposure setting.

[0195] Fig.24 is a flowchart illustrating a process of determining a first exposure setting based on an illuminance value according to an embodiment of the present disclosure.

[0196] According to an embodiment of the present disclosure, when it is determined in operation S308 that the electronic device 100 corresponds to the first state, the electronic device 100 obtains an illuminance value of the surrounding environment in operation S2402.

[0197] In operation S2404, the electronic device 100 obtains the exposure setting corresponding to the illuminance value from the exposure setting lookup table based on the illuminance value of the surrounding environment. The electronic device 100 obtains the exposure setting corresponding to the illuminance value from the memory 220 or the server 2330.

[0198] The electronic device 100 sets the acquired exposure setting as the first exposure setting in operation S2406. When the first exposure setting is acquired, the electronic device 100 changes the current exposure setting to the first exposure setting in operation S310.

[0199] Fig.25 is a flowchart of a method of controlling an electronic device according to an embodiment of the present disclosure.

[0200] According to an embodiment of the present disclosure, the electronic device 100 determines that its state corresponds to the first state and changes the camera exposure setting to the first exposure setting in operation S310, and then determines whether the camera 110 is revealed or clear again in operation S2502. The electronic device 100 can determine whether the camera 110 is revealed using a method of determining whether the camera 110 corresponds to the first state.

[0201] According to an embodiment, the electronic device 100 determines whether the camera 110 is exposed again based on the detection value of the proximity sensor. When the electronic device 100 determines that there is no obstacle nearby based on the detection value of the proximity sensor around the camera 110 blocked by the obstacle, the electronic device 100 determines that the camera 110 is exposed again.

[0202] According to an embodiment, the electronic device 100 determines whether the camera 110 is revealed again using the depth map obtained by the ToF camera. When the minimum depth value in the depth map exceeds the reference value, the electronic device 100 determines that the camera 110 around the ToF camera is revealed again.

[0203] According to an embodiment, the electronic device 100 determines whether the camera 110 is exposed again using state information obtained from the protective case coupled to the electronic device 100. When the cover of the protective case is in an open state, the electronic device 100 determines that the camera 110 is exposed again.

[0204] According to an embodiment, the electronic device 100 determines whether the camera 110 is revealed again based on the focus state of the camera 110. When the camera 110 is focused, the electronic device 100 determines that the camera 110 is revealed again.

[0205] According to an embodiment, the electronic device 100 determines whether the camera 110 is exposed again based on the image data of the obtained image frame. When the electronic device 100 determines that the camera 110 is not blocked by an obstacle based on the scene recognition result of the obtained image frame, the electronic device 100 determines that the camera 110 is exposed again.

[0206] According to an embodiment, the electronic device 100 determines whether the camera 110 is exposed again based on the detection value of the acceleration sensor. When the electronic device 100 determines that it is not placed on the floor based on the detection value of the acceleration sensor, the electronic device 100 determines that the camera 110 is exposed again.

[0207] According to an embodiment, the electronic device 100 determines whether the camera 110 is exposed again based on the folding angle of the foldable device. When the folding angle is greater than or equal to the reference angle value, the electronic device 100 determines that the camera 110 is exposed again.

[0208] The electronic device 100 may use the above-mentioned various methods to determine whether its state corresponds to the first state. The electronic device 100 may use at least one of the above-mentioned various methods, or a combination thereof. In the case where the electronic device 100 uses multiple methods to determine whether its state corresponds to the first state, when it is determined by at least one of the multiple methods that its state corresponds to the first state, the electronic device 100 determines that its state corresponds to the first state. In operation S2502, the electronic device 100 determines whether the camera 110 is revealed again based on the method by which the electronic device 100 determines that its state corresponds to the first state.

[0209] For example, in operation S308, when it is determined based on the detection value of the proximity sensor that the front camera corresponds to the first state in which the front camera is blocked by an obstacle, the electronic device 100 determines based on the detection value of the proximity sensor whether the obstacle has been removed. When the electronic device 100 determines based on the detection value of the proximity sensor that the obstacle has been removed, the electronic device 100 determines that the front camera is revealed again.

[0210] In addition, for example, in operation S308, the electronic device 100 may determine that the front camera corresponds to the first state based on the folding angle of the foldable device. In this case, when the folding angle becomes a reference angle value or greater, the electronic device 100 determines that the front camera is exposed again.

[0211] Fig.26 is a graph showing changes in ambient illumination, ISO, and image frame brightness values ​​according to a comparative example of the present disclosure.

[0212] Fig.26 The comparative example of shows the changes in the ambient illumination 2610, ISO, and image frame brightness value in the process of blocking the camera of the electronic device 100 with an obstacle and then revealing the camera again without applying the exposure setting control according to the embodiment of the present disclosure. Fig.26 In the figure, time point A is the time point when the camera is blocked by an obstacle, and time point C is the time point when the obstacle moves away from the camera.

[0213] Before the camera is blocked by an obstacle at time point A, the ISO corresponds to 55 and the ambient illumination corresponds to 270 lux. After time point A, the camera is blocked by an obstacle and the ambient illumination decreases to 1.5 lux. Through automatic exposure control, the ISO is increased to 1800 or higher at time point B. After the obstacle is removed at time point C, the ambient illumination increases rapidly to 260 lux. However, from time point C to time point D, the ISO decreases very slowly to 55. This results in overexposure between time point C and time point D due to high ambient illumination and high ISO.

[0214] Fig. 27 is a diagram illustrating changes in ambient illumination, ISO, and image frame brightness values ​​according to an embodiment of the present disclosure.

[0215] Fig. 27 1 shows changes in ambient illumination, ISO, and image frame brightness values ​​in a process of blocking a camera of the electronic device 100 with an obstacle and then revealing the camera again by applying exposure setting control according to an embodiment of the present disclosure. Fig. 27 In the figure, time point A is the time point when the camera is blocked by an obstacle, and time point C is the time point when the obstacle moves away from the camera.

[0216] According to an embodiment of the present disclosure, when the electronic device 100 determines that its state corresponds to the first state, the electronic device 100 controls the exposure setting at time point B to be changed to the first exposure setting or increases the current frame rate. For example, the electronic device 100 controls the ISO value to be reduced to 55 at time point B based on the exposure setting or frame rate. Therefore, according to an embodiment of the present disclosure, an effect of no overexposure between time point C and time point D immediately after the obstacle is removed from the camera 110 can be achieved. Therefore, according to an embodiment of the present disclosure, the time to reach proper exposure immediately after the obstacle is removed from the camera 110 is significantly shortened, thereby significantly reducing the delay of automatic exposure control.

[0217] Fig.28 is a diagram illustrating an image frame according to a comparative example and an embodiment of the present disclosure.

[0218] The comparative example of the present disclosure shows a process of shielding a camera of the electronic device 100 with an obstacle and then revealing the camera again, without applying the exposure setting control according to the embodiment of the present disclosure. Fig.28 The image frames in are all image frames captured by an electronic device that performs automatic exposure control.

[0219] Image frame 2810 is an image frame captured before the camera is blocked by an obstacle. Image frame 2820 is an image frame captured when the camera is blocked by an obstacle.

[0220] Image frame 2832 is an image frame captured according to the comparative example when the obstacle blocking the camera is removed and the camera is clear again. According to the comparative example, the image is captured with overexposure immediately after the obstacle is removed, as shown in image frame 2832.

[0221] Image frame 2834 is an image frame captured according to an embodiment of the present disclosure when an obstacle blocking the camera is removed and the camera is revealed again. According to an embodiment of the present disclosure, an image is captured with appropriate exposure immediately after the obstacle is removed, as shown in image frame 2834.

[0222] Fig.29 is a block diagram illustrating an electronic device in a network environment according to an embodiment of the present disclosure.

[0223] Fig.29 The electronic device 2901 of the embodiment may correspond to the electronic device 100 described above. In addition, the server 2908 may correspond to the reference Fig.23 Described server 2330. In addition, Fig.29 The processor 2920 may correspond to Figure 2 The processor 210, Fig.29 The camera module 2980 may correspond to Figure 2 The camera 110, and Fig.29The memory 2930 may correspond to Figure 2 Memory 220.

[0224] refer to Fig.29 In the network environment 2900, the electronic device 2901 may communicate with the electronic device 2902 through the first network 2998 (e.g., a short-range wireless communication network), or communicate with at least one of the electronic device 2904 or the server 2908 through the second network 2999 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 2901 may communicate with the electronic device 2904 via the server 2908. According to an embodiment, the electronic device 2901 may include a processor 2920, a memory 2930, an input module 2950, ​​a sound output module 2955, a display module 2960, an audio module 2970, a sensor module 2976, an interface 2977, a connection terminal 2978, a haptic module 2979, a camera module 2980, a power management module 2988, a battery 2989, a communication module 2990, a user identification module 2996, or an antenna module 2997. In an embodiment, the electronic device 2901 may not include at least one of these components (e.g., the connection terminal 2978) or may also include one or more other components. In an embodiment, some of these components (eg, sensor module 2976, camera module 2980, or antenna module 2997) may be integrated into a single component (eg, display module 2960).

[0225] For example, the processor 2920 may execute software (e.g., program 2940) to control at least one other component (e.g., hardware or software component) connected to the processor 2920 in the electronic device 2901, and perform various data processing or calculations. According to an embodiment, as at least part of the data processing or calculation, the processor 2920 may store commands or data received from other components (e.g., sensor module 2976 or communication module 2990) in the volatile memory 2932, process the commands or data stored in the volatile memory 2932, and store the resultant data in the non-volatile memory 2934. According to an embodiment, the processor 2920 may include a main processor 2921 (e.g., CPU or AP) or an auxiliary processor 2923 (e.g., GPU, NPU, image signal processor, sensor hub processor, or communication processor), and the auxiliary processor 2923 may operate independently of the main processor 2921 or operate in cooperation with the main processor 2921. For example, when the electronic device 2901 includes a main processor 2921 and an auxiliary processor 2923, the auxiliary processor 2923 may be configured to use less power than the main processor 2921 or to be dedicated to a specified function. The auxiliary processor 2923 may be implemented separately from the main processor 2921 or may be implemented as part of the main processor 2921.

[0226] For example, the auxiliary processor 2923 may replace the main processor 2921 when the main processor 2921 is in an inactive (e.g., sleep) state, or control at least some functions or states related to at least one component of the electronic device 2901 (e.g., display module 2960, sensor module 2976, or communication module 2990) together with the main processor 2921 when the main processor 2921 is in an active state (e.g., executing an application). According to an embodiment, the auxiliary processor 2923 (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module 2980 or a communication module 2990). According to an embodiment, the auxiliary processor 2923 (e.g., an NPU) may include a hardware structure specifically for processing an AI model. The AI ​​model may be generated by machine learning. Such machine learning may be performed by the electronic device 2901 itself, or by a separate server (e.g., server 2908). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples. The AI ​​model may include multiple artificial neural network layers. The artificial neural network may be, but is not limited to, a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent DNN (BRDNN), a deep Q network (DQN), or a combination of two or more of the networks. The AI ​​model may include a software structure in addition to or in place of a hardware structure.

[0227] The memory 2930 may store various types of data used by at least one component (e.g., the processor 2920 or the sensor module 2976) of the electronic device 2901. The various types of data may include, for example, input data or output data of software (e.g., the program 2940) and commands related thereto. The memory 2930 may include a volatile memory 2932 or a non-volatile memory 2934.

[0228] The program 2940 may be stored as software in the memory 2930 , and include, for example, an operating system (OS) 2942 , middleware 2944 , or an application 2946 .

[0229] The input module 2950 may receive a command or data to be used by a component (e.g., the processor 2920) of the electronic device 2901 from outside (e.g., a user) of the electronic device 2901. The input module 2950 may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus).

[0230] The sound output module 2955 can output sound signals to the outside of the electronic device 2901. The sound output module 2955 can include, for example, a speaker or an earpiece. The speaker can be used for general purposes, such as playing multimedia or playing recordings. The earpiece can be used to answer calls. According to an embodiment, the earpiece can be implemented separately from the speaker or as part of the speaker.

[0231] The display module 2960 can visually provide information to the outside of the electronic device 2901 (e.g., a user). The display module 2960 may include, for example, a display, a holographic device or a projector, and a control circuit for controlling the corresponding device. According to an embodiment, the display module 2960 may include a touch sensor configured to detect a touch or a pressure sensor configured to measure the strength of the force generated by the touch.

[0232] The audio module 2970 may convert sound into an electrical signal, and vice versa. According to an embodiment, the audio module 2970 may obtain sound via the input module 2950, ​​or output sound via the sound output module 2955 or an external electronic device (e.g., electronic device 2902) (e.g., a speaker or earphone) directly or wirelessly connected to the electronic device 2901.

[0233] The sensor module 2976 can detect the operating state (e.g., power or temperature) or the external environment state (e.g., user state) of the electronic device 2901, and generate an electrical signal or data value corresponding to the detected state. According to an embodiment, the sensor module 2976 may include, for example, a gesture sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0234] The interface 2977 may support one or more specified protocols, and the electronic device 2901 may use these protocols to directly or wirelessly connect with an external electronic device (e.g., the electronic device 2902). According to an embodiment, the interface 2977 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0235] The connection terminal 2978 may include a connector, and the electronic device 2901 may be physically connected to an external electronic device (e.g., the electronic device 2902) via the connector. According to an embodiment, the connection terminal 2978 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0236] The haptic module 2979 may convert the electrical signal into mechanical stimulation (eg, vibration or motion) or electrical stimulation that the user can sense through tactile or kinesthetic sense. According to an embodiment, the haptic module 2979 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0237] The camera module 2980 may capture still images or videos. According to embodiments, the camera module 2980 may include one or more lenses, image sensors, image signal processors, or flashes.

[0238] The power management module 2988 may manage power supplied to the electronic device 2901. According to an embodiment, the power management module 2988 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0239] The battery 2989 may supply power to at least one component of the electronic device 2901. According to an embodiment, the battery 2989 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0240] The communication module 2990 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 2901 and an external electronic device (e.g., electronic device 2902, electronic device 2904, or server 2908), and perform communication via the established communication channel. The communication module 2990 may include: one or more communication processors that operate independently of the processor 2920 (e.g., AP) and support direct (or wired) communication or wireless communication. According to an embodiment, the communication module 2990 may include a wireless communication module 2992 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 2994 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules may communicate with an external electronic device 2904 via a first network 2998 (e.g., a short-range communication network such as Bluetooth, Wireless Fidelity (Wi-Fi) Direct (WFD), or Infrared Data Association (IrDA)) or a second network 2999 (e.g., a long-range communication network such as a conventional cellular network, a fifth generation (5G) network, a next generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip), or may be implemented as a plurality of separate components (e.g., a plurality of chips). The wireless communication module 2992 may identify or authenticate the electronic device 2901 in a communication network (e.g., the first network 2998 or the second network 2999) using user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 2996.

[0241] The wireless communication module 2992 can support 5G networks, and the next generation communication technology after the fourth generation (4G) network, such as the new radio (NR) access technology. The NR access technology can support high-speed transmission of large-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power consumption, and large-scale access of a large number of terminals (massive machine type communication (mMTC)) or high reliability and low latency (ultra-reliable and low latency communication (URLLC)). For example, the wireless communication module 2992 can support high frequency bands (e.g., millimeter wave (mmWave) bands) to achieve high data rates. The wireless communication module 2992 can support various technologies for ensuring high frequency band performance, such as beamforming, massive multiple input multiple output (MIMO), full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. The wireless communication module 2992 can support various requirements specified by the electronic device 2901, an external electronic device (e.g., an electronic device 2904), or a network system (e.g., a second network 2999). According to an embodiment, the wireless communication module 2992 can support peak data rates (e.g., 20 gigabits per second (Gbps) or higher) to implement eMBB, loss coverage (e.g., up to 164 decibels (dB)) to implement mMTC, or user plane (U-plane) delay to implement URLLC (e.g., downlink (DL) and uplink (UL) delays less than or equal to 0.5 milliseconds or round-trip delay less than or equal to 1 millisecond).

[0242] The antenna module 2997 may send a signal or power to the outside (e.g., an external electronic device) or receive a signal or power from the outside. According to an embodiment, the antenna module 2997 may include an antenna including a radiation element composed of a conductive material or a conductive pattern formed on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, the antenna module 2997 may include a plurality of antennas (e.g., an array antenna). In this case, for example, at least one antenna suitable for a communication method used in a communication network (e.g., a first network 2998 or a second network 2999) may be selected from a plurality of antennas by the communication module 2990. A signal or power may be sent or received between the communication module 2990 and the external electronic device via the selected at least one antenna. According to an embodiment, a component other than the radiation element (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module 2997. According to various embodiments, the antenna module 2997 may form a mmWave antenna module. According to an embodiment, the mmWave antenna module may include a PCB, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the PCB and capable of supporting a specified high frequency band (e.g., mmWave frequency band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the PCB and capable of sending or receiving signals in the specified high frequency band.

[0243] At least some components may be connected to each other and exchange signals (eg, commands or data) with each other by using a communication scheme between peripheral devices (eg, via a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)).

[0244] According to an embodiment, commands or data may be sent or received between the electronic device 2901 and the external electronic device 2904 via a server 2908 connected to the second network 2999. Each of the external electronic devices 2902 and 2404 may be a device of the same type or a different type as the electronic device 2901. According to an embodiment, all or some operations performed by the electronic device 2901 may be performed by one or more of the external electronic devices 2902, 2904, and 2908. For example, when the electronic device 2901 needs to perform a certain function or service automatically or in response to a request from a user or other device, as a substitute or in addition to performing the function or service by itself, the electronic device 2901 may request one or more external electronic devices to perform at least a part of the function or service. The one or more external electronic devices that receive the request may perform at least a part of the requested function or service or an additional function or service related to the request, and send the result of the execution to the electronic device 2901. The electronic device 2901 may provide the result as at least a partial response to the request with or without further processing the result. To this end, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 2901 may provide ultra-low latency services by using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 2904 may include an Internet of Things (IoT) device. The server 2908 may be an intelligent server using machine learning and / or neural networks. Depending on the embodiment, the external electronic device 2904 or the server 2908 may be included in the second network 2999. The electronic device 2901 may be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or healthcare) based on 5G communication technology and IoT-related technologies.

[0245] Fig.30 is a block diagram illustrating a camera module according to an embodiment of the present disclosure.

[0246] refer to Fig.30 , the camera module 3080 may include a lens assembly 3010 , a flash 3020 , an image sensor 3030 , an image stabilizer 3040 , a memory 3050 (eg, a buffer memory), or an image signal processor 3060 .

[0247] Fig.30 The image sensor 3030 may correspond to Figure 2 The image sensor 230 is provided.

[0248] The lens assembly 3010 may collect light emitted from an object whose image is to be captured. The lens assembly 3010 may include one or more lenses. According to an embodiment, the camera module 2980 may include a plurality of lens assemblies 3010. In this case, the camera module 2980 may form, for example, a dual lens, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies 3010 may have the same lens characteristics (e.g., viewing angle, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens characteristics different from those of the other lens assemblies. The lens assembly 3010 may include, for example, a wide-angle lens or a telephoto lens.

[0249] The flash 3020 may emit light for enhancing light emitted or reflected from an object. According to an embodiment, the flash 3020 may include one or more LEDs (e.g., a red, green, and blue (RGB) LED, a white light LED, an infrared LED, or an ultraviolet LED) or a xenon lamp. The image sensor 3030 may obtain an image corresponding to the object by converting light emitted or reflected from the object and transmitted via the lens assembly 3010 into an electrical signal. According to an embodiment, the image sensor 3030 may include one image sensor selected from image sensors with different properties (e.g., an RGB sensor, a black and white (BW) sensor, an infrared sensor, or an ultraviolet sensor), a plurality of image sensors with the same properties, or a plurality of image sensors with different properties. Each image sensor included in the image sensor 3030 may be implemented using, for example, a CCD sensor or a complementary metal oxide semiconductor (CMOS) sensor.

[0250] The image stabilizer 3040 may move at least one lens or the image sensor 3030 included in the lens assembly 3010 in a specific direction, or control the operating characteristics of the image sensor 3030 (e.g., adjusting the readout timing, etc.), in response to the movement of the camera module 2980 or the electronic device 2901 including the camera module. This compensates for at least some of the negative effects of the above-mentioned movement on the captured image. According to an embodiment, the image stabilizer 3040 may detect such movement of the camera module 2980 or the electronic device 2901 by using a gyroscope sensor (not shown) or an acceleration sensor (not shown) located inside or outside the camera module 2980. According to an embodiment, the image stabilizer 3040 may be implemented as, for example, an optical image stabilizer. The memory 3050 may at least temporarily store at least a portion of the image obtained by the image sensor 3030 for the next image processing task. For example, when image acquisition is delayed due to a shutter, or when multiple images are obtained at high speed, the obtained original image (e.g., a Bayer pattern image or a high-resolution image) is stored in the memory 3050, and the corresponding copy image (e.g., a low-resolution image) can be previewed via the display module 2960 as a display device. Thereafter, when a specified condition (e.g., a user input or a system command) is met, at least a portion of the original image stored in the memory 3050 can be obtained and processed by, for example, the image signal processor 3060. According to an embodiment, the memory 3050 may be configured as at least a portion of the memory 2930, or as a separate memory that operates independently of the memory 2930.

[0251] The image signal processor 3060 may perform one or more image processing operations on an image obtained via the image sensor 3030 or an image stored in the memory 3050. The one or more image processing operations may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Additionally or alternatively, the image signal processor 3060 may perform control (e.g., exposure time control or readout timing control, etc.) on at least one component (e.g., image sensor 3030) included in the camera module 2980. The image processed by the image signal processor 3060 may be stored back in the memory 3050 for further processing, or may be provided to a component (e.g., memory 2930, display module 2960, electronic device 2902, electronic device 2904, or server 2908) outside the camera module 2980. In an embodiment, the image signal processor 3060 may be configured as at least a part of the processor 2920, or may be configured as a separate processor that operates independently of the processor 2920. When the image signal processor 3060 is configured as a processor separate from the processor 2920 , at least one image processed by the image signal processor 3060 may be displayed via the display module 2960 as is or after further image processing by the processor 2920 .

[0252] According to an embodiment, the electronic device 2901 may include a plurality of camera modules 2980, each having different characteristics or functions. In this case, for example, at least one of the plurality of camera modules 2980 may be a wide-angle camera, and at least another one may be a telephoto camera. Similarly, at least one of the plurality of camera modules 2980 may be a front camera, and at least another one may be a rear camera.

[0253] As used in various embodiments of this document, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms such as logic, logic block, component, or circuit. A module may be an integrally formed component, or a minimum unit or part of the component configured to perform one or more functions. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0254] The various embodiments described herein may be implemented as software (e.g., a program) that includes one or more instructions stored in a storage medium readable by a machine (e.g., electronic device 100). For example, a processor of a machine (e.g., electronic device 100) may call at least one of the one or more instructions stored from a storage medium and execute the at least one instruction called. This enables the machine to perform at least one function according to at least one instruction called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. In this regard, the term "non-transitory" only means that the storage medium is a tangible device and does not include a signal (e.g., an electromagnetic wave), and the term does not distinguish between a case where data is semi-permanently stored in a storage medium and a case where data is temporarily stored in a storage medium.

[0255] According to an embodiment, the method according to various embodiments described herein may be included in a computer program product when provided. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc ROM (CD-ROM)), or distributed online (e.g., downloaded or uploaded) via an application store or distributed directly between two user devices (e.g., smart phones). For online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily generated in a machine-readable storage medium (e.g., a memory of a manufacturer's server, an application store's server, or a relay server).

[0256] According to one aspect of the present disclosure, a method for controlling an electronic device including a camera is provided. The method for controlling an electronic device includes obtaining an image frame from a camera. In addition, the method for controlling an electronic device includes identifying whether a current frame rate is lower than a reference frame rate. In addition, the method for controlling an electronic device includes: based on determining that the current frame rate is lower than the reference frame rate, identifying whether a brightness value of an image frame obtained from an image sensor is lower than a reference brightness value. In addition, the method for controlling an electronic device includes: based on determining that the brightness value of the image frame is lower than the reference brightness value, determining whether the camera corresponds to a first state in which the camera is blocked by an obstacle. In addition, the method for controlling an electronic device includes: based on determining that the camera corresponds to the first state, changing the exposure setting of the camera to the first exposure setting.

[0257] In addition, according to an embodiment of the present disclosure, the method for controlling an electronic device further includes: setting the frame rate to a second frame rate based on determining that the brightness value of the obtained image frame is higher than or equal to the reference brightness value; obtaining the image frame at the second frame rate; and recalculating the exposure setting of the camera based on the image frame obtained at the second frame rate. In addition, according to an embodiment of the present disclosure, the method for controlling an electronic device further includes: setting the frame rate to a second frame rate based on determining that the camera does not correspond to the first state; obtaining the image frame at the second frame rate; and recalculating the exposure setting of the camera based on the image frame obtained at the second frame rate.

[0258] In addition, according to an embodiment of the present disclosure, identifying whether the brightness value of an image frame is lower than a reference brightness value includes: obtaining a first image frame from an image sensor based on determining that the camera operates at a frame rate lower than the reference frame rate; and identifying whether the brightness value of the obtained first image frame is lower than the reference brightness value.

[0259] In addition, according to an embodiment of the present disclosure, the brightness value of the image frame corresponds to the average value of the brightness values ​​of all pixels in the image frame.

[0260] In addition, according to an embodiment of the present disclosure, the method of controlling an electronic device further includes: storing the exposure setting of the camera and the brightness value of the image frame in a buffer of the camera each time an image frame is obtained from the image sensor. In addition, changing the exposure setting of the camera to a first exposure setting includes: determining whether the brightness value of the image frame stored in the buffer is a valid value based on determining that the camera corresponds to the first state; changing the exposure setting of the camera to the first exposure setting based on determining that the brightness value of the image frame stored in the buffer is valid; and increasing the frame rate of the camera to a second frame rate based on determining that the brightness value of the image frame stored in the buffer is invalid.

[0261] In addition, according to an embodiment of the present disclosure, determining whether the brightness value of an image frame is a valid value includes: determining whether the brightness values ​​of a plurality of image frames stored in a buffer are stable based on the distribution of the brightness values ​​of the plurality of image frames; determining whether the brightness values ​​of the plurality of image frames stored in the buffer exceed a minimum brightness value; and determining that the brightness values ​​and exposure settings of the image frames are valid based on determining that the brightness values ​​of the plurality of image frames are stable and exceed the minimum brightness value.

[0262] In addition, according to an embodiment of the present disclosure, the method for controlling an electronic device further includes: storing the exposure setting of the camera and the brightness value of the image frame in a buffer of the camera each time an image frame is obtained from the image sensor. In addition, changing the exposure setting of the camera to a first exposure setting includes: determining the exposure setting for the image frame corresponding to the highest brightness value among the exposure settings for the plurality of image frames stored in the buffer as the first exposure setting.

[0263] In addition, according to an embodiment of the present disclosure, the method of controlling an electronic device further includes: storing the exposure setting of the camera and the brightness value of the image frame in a buffer of the camera each time an image frame is obtained from the image sensor.

[0264] In addition, according to an embodiment of the present disclosure, determining the first exposure setting includes: calculating an average value of brightness values ​​of multiple frames stored in the buffer, and determining the exposure setting for the following image frame whose brightness value is closest to the average brightness value as the first exposure setting.

[0265] Furthermore, according to an embodiment of the present disclosure, determining whether the camera corresponds to the first state includes determining whether the camera corresponds to the first state based on a detection value of a proximity sensor located around a lens of the camera.

[0266] In addition, according to an embodiment of the present disclosure, determining whether the camera corresponds to the first state includes: obtaining a depth map from a ToF camera located around the lens of the camera; and based on determining that the minimum depth value in the depth map is less than a reference depth value, determining that the camera corresponds to the first state in which the camera is blocked by an obstacle.

[0267] In addition, according to an embodiment of the present disclosure, determining whether the camera corresponds to the first state includes: obtaining state information of a protective shell coupled to the electronic device based on the detection value of the Hall sensor, the protective shell having a folded state that blocks the front side of the electronic device and an open state that opens the front side of the electronic device; and determining that the camera corresponds to the first state based on determining that the state information of the protective shell corresponds to the folded state.

[0268] Furthermore, according to an embodiment of the present disclosure, determining whether the camera corresponds to the first state includes determining that the camera corresponds to the first state based on determining that the focus state of the camera is out of focus.

[0269] Furthermore, according to an embodiment of the present disclosure, determining whether the camera corresponds to the first state includes determining whether the camera corresponds to the first state based on scene recognition of the obtained image frame.

[0270] In addition, according to an embodiment of the present disclosure, the method of controlling an electronic device further includes: increasing the frame rate of the camera to a preset first frame rate based on determining that the camera corresponds to the first state.

[0271] In addition, according to an embodiment of the present disclosure, determining whether the camera corresponds to the first state includes: obtaining a detection value of an acceleration sensor of the electronic device based on determining that the brightness value of the obtained image frame is lower than a reference brightness value; determining whether the electronic device is located in a fixed position, where a first side of the electronic device on which a camera is provided is in contact with other objects; and when the first side of the electronic device on which a camera is provided is in contact with other objects, determining that the camera corresponds to the first state.

[0272] In addition, according to an embodiment of the present disclosure, the electronic device is a foldable device. In addition, the camera is arranged on an inner surface of the foldable device, and when in a folded state, the inner surface is located inside the foldable device. In addition, determining whether the camera corresponds to the first state includes: when the folding angle of the foldable device is less than a reference angle value, determining that the camera corresponds to the first state.

[0273] In addition, according to an embodiment of the present disclosure, the method for controlling an electronic device also includes: determining a first ambient brightness value based on a detection value of an illuminance sensor of the electronic device; obtaining an exposure setting corresponding to the first ambient brightness value from a first database storing exposure settings corresponding to ambient brightness values; and determining the exposure setting obtained from the first database as the first exposure setting.

[0274] Furthermore, according to an embodiment of the present disclosure, the exposure setting includes a shutter speed, an aperture value, and an ISO value.

[0275] According to various embodiments, each of the above-mentioned components (e.g., a module or a program) may include a single entity or multiple entities, some of which may be separately provided in other components. According to various embodiments, one or more or one or more operations of the above-mentioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as performed by a corresponding one of the multiple components before integration. According to various embodiments, the operations performed by a module, a program or another component may be performed sequentially, in parallel, repeatedly or heuristically, or one or more of the operations may be performed in a different order or omitted, or one or more other operations may be added.

Claims

1. A method for controlling an electronic device including a camera, the method comprising: obtaining an image frame from the camera; Identify whether the current frame rate is lower than the reference frame rate; Based on determining that the current frame rate is lower than the reference frame rate, identifying whether a brightness value of an image frame obtained from an image sensor is lower than a reference brightness value; Based on determining that the brightness value of the image frame is lower than the reference brightness value, determining whether the camera corresponds to a first state in which the camera is blocked by an obstacle; as well as Based on determining that the camera corresponds to the first state, an exposure setting of the camera is changed to a first exposure setting.

2. The method according to claim 1, further comprising: Based on determining that the brightness value of the obtained image frame is greater than or equal to the reference brightness value: setting the frame rate to a second frame rate, acquiring image frames at the second frame rate, and recalculating exposure settings of the camera based on image frames acquired at the second frame rate; as well as Based on determining that the camera does not correspond to the first state: setting the frame rate to the second frame rate, acquiring image frames at the second frame rate, and Exposure settings of the camera are recalculated based on image frames acquired at the second frame rate.

3. The method according to claim 1 or 2, wherein: Identifying whether the brightness value of the image frame is lower than the reference brightness value includes: based on determining that the camera is operating at a frame rate lower than the reference frame rate, obtaining a first image frame from the image sensor; and It is identified whether the brightness value of the obtained first image frame is lower than the reference brightness value.

4. The method according to any one of claims 1 to 3, wherein: The brightness value of the image frame corresponds to an average value of the brightness values ​​of all pixels in the image frame.

5. The method according to any one of claims 1 to 4, further comprising: Each time an image frame is obtained from the image sensor, the exposure setting of the camera and the brightness value of the image frame are stored in a buffer of the camera, Wherein, changing the exposure setting of the camera to the first exposure setting comprises: determining whether a brightness value of an image frame stored in the buffer is a valid value based on determining that the camera corresponds to the first state; based on determining that the brightness value of the image frame stored in the buffer is valid, changing the exposure setting of the camera to the first exposure setting; and Based on determining that the brightness values ​​of the image frames stored in the buffer are invalid, increasing the frame rate of the camera to a second frame rate.

6. The method according to any one of claims 1 to 5, further comprising: Each time an image frame is obtained from the image sensor, the exposure setting of the camera and the brightness value of the image frame are stored in a buffer of the camera, Wherein, changing the exposure setting of the camera to the first exposure setting comprises: An exposure setting for an image frame corresponding to a highest brightness value among the exposure settings for a plurality of image frames stored in the buffer is determined as the first exposure setting.

7. The method according to any one of claims 1 to 6, further comprising: Each time an image frame is obtained from the image sensor, the exposure setting of the camera and the brightness value of the image frame are stored in a buffer of the camera, Wherein, determining the first exposure setting includes: calculating an average of the brightness values ​​of the plurality of frames stored in the buffer; and An exposure setting for an image frame having a luminance value closest to the average value of the luminance values ​​is determined as the first exposure setting.

8. The method according to any one of claims 1 to 7, wherein: Determining whether the camera corresponds to the first state includes determining whether the camera corresponds to the first state based on a detection value of a proximity sensor located around a lens of the camera.

9. The method according to any one of claims 1 to 8, wherein: Determining whether the camera corresponds to the first state includes: Obtaining a depth map from a time-of-flight (ToF) camera located around the lens of the camera; and Based on determining that the minimum depth value in the depth map is less than the reference depth value, determining that the camera corresponds to the first state in which the camera is blocked by the obstacle.

10. The method according to any one of claims 1 to 9, wherein: Determining whether the camera corresponds to the first state includes: Based on the detection value of the Hall sensor, obtaining state information of a protective case coupled to the electronic device, the protective case having a folded state in which the front side of the electronic device is blocked and an open state in which the front side of the electronic device is opened; and Based on determining that the state information of the protective case corresponds to the folded state, determining that the camera corresponds to the first state.

11. The method according to any one of claims 1 to 10, further comprising: Based on determining that the camera corresponds to the first state, a frame rate of the camera is increased to a preset first frame rate.

12. The method according to any one of claims 1 to 11, wherein: Determining whether the camera corresponds to the first state includes: Based on determining that the brightness value of the obtained image frame is lower than the reference brightness value, obtaining a detection value of the acceleration sensor of the electronic device; determining whether the electronic device is in a fixed position, wherein a first side of the electronic device on which the camera is disposed is in contact with another object; and When the first side of the electronic device on which the camera is disposed contacts the other object, it is determined that the camera corresponds to the first state.

13. The method according to any one of claims 1 to 12, wherein: The electronic device is a foldable device, The camera is disposed on an inner surface of the foldable device, wherein the inner surface is located inside the foldable device when in a folded state, and Determining whether the camera corresponds to the first state includes: When the folding angle of the foldable device is less than a reference angle value, it is determined that the camera corresponds to the first state.

14. An electronic device (100), comprising: A camera (110), including an image sensor (230); A memory (220) storing at least one instruction; as well as at least one processor (210), connected to the memory (220), The at least one processor (210) is configured to execute the at least one instruction to: obtaining an image frame from the camera (110), Identify whether the current frame rate is lower than the reference frame rate, Based on determining that the current frame rate is lower than the reference frame rate, identifying whether a brightness value of an image frame obtained from the image sensor (230) is lower than a reference brightness value, Based on determining that the brightness value of the image frame is lower than the reference brightness value, determining whether the camera (110) corresponds to a first state in which the camera (110) is blocked by an obstacle, and Based on determining that the camera (110) corresponds to the first state, an exposure setting of the camera (110) is changed to a first exposure setting.

15. A computer-readable recording medium having recorded thereon a program for executing the method according to any one of claims 1 to 13 on a computer.

Images