Image sensing apparatus, electronic apparatus including the same, and method of operating the electronic apparatus

Abstract

A method of operating an electronic device including an application processor and an image sensing device is provided. The method includes generating, by the image sensing device, first image data in a first shutter mode, providing, by the application processor, a first mode change signal indicating a second shutter mode different from the first shutter mode to the image sensing device based on shooting environment information associated with the first image data, and generating, by the image sensing device, second image data in the second shutter mode based on the first mode change signal.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to Korean Patent Application No. 10-2025-0003398, filed on January 9, 2025, with the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety. Background Technology

[0003] This disclosure relates to an electronic device, and more specifically, to an image sensing device that switches shutter modes depending on the shooting environment, an electronic device including the image sensing device, and a method of operating the electronic device.

[0004] Image sensors convert light received through photodiodes into electrical signals. Complementary metal-oxide-semiconductor (CMOS) image sensors offer convenient driving methods, low power consumption, and signal processing circuitry integrated on a single chip.

[0005] With the rapid increase in the use of CMOS image sensors, the demand for image sensors that can quickly switch to modes suitable for the shooting environment is growing. Summary of the Invention

[0006] One or more embodiments provide an image sensing device that switches shutter modes depending on the shooting environment, an electronic device including the image sensing device, and a method of operating the electronic device.

[0007] According to one aspect of the embodiments, a method of operating an electronic device including an application processor and an image sensing device includes: generating first image data by the image sensing device in a first shutter mode; providing the image sensing device with a first mode change signal indicating a second shutter mode different from the first shutter mode based on shooting environment information associated with the first image data; and generating second image data by the image sensing device in the second shutter mode based on the first mode change signal.

[0008] According to another aspect of the embodiments, a method for operating an on-chip system including a sensor controller and an image sensor includes: generating first image data by the image sensor in a first shutter mode; changing settings of the image sensor by the sensor controller based on shooting environment information associated with the first image data to control the image sensor to operate in a second shutter mode different from the first shutter mode; and generating second image data by the image sensor in the second shutter mode.

[0009] According to another aspect of the embodiments, an electronic device includes: an image sensing device configured to generate first image data in a first shutter mode and a second shutter mode; and an application processor configured to provide the image sensing device with a first mode change signal indicating a shutter mode different from the current shutter mode based on shooting environment information associated with the first image data. Attached Figure Description

[0010] The above and other aspects will become more apparent from the following description of embodiments in conjunction with the accompanying drawings.

[0011] Figure 1 This is a block diagram of an electronic device according to some embodiments.

[0012] Figure 2 This is a block diagram of an electronic device according to some embodiments.

[0013] Figure 3 This is a diagram illustrating an image sensing device according to some embodiments.

[0014] Figure 4 This is a diagram schematically illustrating a rolling shutter mode among shutter modes according to some embodiments.

[0015] Figure 5 This is a diagram schematically illustrating a global shutter mode among shutter modes according to some embodiments.

[0016] Figure 6 This is a circuit diagram of a pixel circuit according to some embodiments.

[0017] Figure 7 It is a timing diagram describing changes in shutter mode according to some embodiments.

[0018] Figure 8 This is a diagram describing setup data according to some embodiments.

[0019] Figure 9 This describes some embodiments. Figure 1 A diagram illustrating the operation method of an electronic device.

[0020] Figure 10 This is a flowchart describing the operation method of an application processor over time according to some embodiments.

[0021] Figure 11 This is a diagram illustrating a method of operating an electronic device according to some embodiments.

[0022] Figure 12 This is a block diagram of a system-on-a-chip according to some embodiments.

[0023] Figure 13This is a diagram illustrating the operation method of a system-on-a-chip according to some embodiments.

[0024] Figure 14 This is a block diagram of an electronic device including a multi-camera module according to some embodiments.

[0025] Figure 15 This is a block diagram illustrating a camera module according to some embodiments. Detailed Implementation

[0026] The embodiments described below are described with reference to the accompanying drawings. The embodiments described herein are provided as examples, and therefore, this disclosure is not limited thereto, and may be implemented in various other forms. Each example embodiment provided in the following description does not exclude association with one or more features of another example or another example embodiment also provided herein or not provided herein but consistent with this disclosure.

[0027] Figure 1 This is a block diagram of an electronic device 1000 according to some embodiments. (Reference) Figure 1 The electronic device 1000 may include an application processor 1100 and an image sensing device 1200.

[0028] Electronic device 1000 can be implemented using laptop computers, mobile phones, smartphones, tablet PCs, personal digital assistants (PDAs), enterprise digital assistants (EDAs), digital still cameras, digital video cameras, portable multimedia players (PMPs), mobile internet devices (MIDs), wearable computers, Internet of Things (IoT) devices, or Internet of Everything (IoE) devices.

[0029] Application processor 1100 may include main processor 1110, random access memory (RAM) 1120, image signal processor 1130, non-volatile memory interface 1140, camera interface 1150 and memory interface 1160.

[0030] The main processor 1110 can control all operations of the application processor 1100. For example, the main processor 1110 can be implemented using a central processing unit (CPU) or a microprocessor. Depending on the embodiment, the main processor 1110 can be implemented using a computing component that includes two or more independent processors (or cores), i.e., a multi-core processor. The main processor 1110 can execute programs stored in RAM 1120 (or read-only memory (ROM)) or can process data stored therein.

[0031] RAM 1120 can temporarily store programs, data, or instructions. For example, RAM 1120 can be implemented using dynamic RAM (DRAM) or static RAM (SRAM). RAM 1120 can temporarily store images input / output through interfaces 1140, 1150, and 1160, or images generated by image signal processor 1130 or main processor 1110.

[0032] In some embodiments, the application processor 1100 may further include a read-only memory (ROM). The ROM may store programs and / or data that are used continuously. The ROM may be implemented using an erasable programmable ROM (EPROM) or an electrically erasable programmable ROM (EEPROM).

[0033] The image signal processor 1130 can generate a converted image by performing image processing on image data received from the image sensing device 1200, and can store the converted image in the memory 20. The image signal processor 1130 can scale the converted image and provide the scaled image to the display device.

[0034] The non-volatile memory interface 1140 can interface data input to or output to the non-volatile memory device 10. The non-volatile memory device 10 can be implemented, for example, using a memory card (e.g., a multimedia card (MMC), an embedded MMC (eMMC), a secure digital card (SD), or a micro SD).

[0035] The camera interface 1150 can interface with image data input from an image sensing device 1200 located external to the application processor 1100. The image sensing device 1200 can generate image data of images captured using multiple photodetectors. The image data received through the camera interface 1150 can be provided to the image signal processor 1130, or stored in the memory 20 via the memory interface 1160.

[0036] For example, the camera interface 1150 can be implemented using one of the following interfaces: a serial interface, a Mobile Display Digital Interface (MDDI), an Internal Integrated Circuit (I2C) interface, a Serial Peripheral Interface (SPI), a Microcontroller Unit (MCU) interface, a Mobile Industry Processor Interface (MIPI), an Embedded Display Port (eDP) interface, a D-sub interface, an optical interface, or an High Definition Multimedia Interface (HDMI). Furthermore, the camera interface 1150 can be implemented using various serial or parallel interface methods.

[0037] However, the embodiments are not limited thereto. For example, with Figure 1Unlike the examples shown, application processor 1100 may also include additional components, or may not include some of the components described above. For example, image signal processor 1130 may be included in image sensing device 1200.

[0038] Electronic device 1000 can change the shutter mode depending on the shooting environment. Electronic device 1000 can generate data of an image captured in the changed shutter mode, or can display the captured image. Image sensing device 1200 can generate data of an image captured in one of multiple shutter modes. For example, the multiple shutter modes may include a rolling shutter mode and a global shutter mode. This will refer to... Figure 4 and Figure 5 Detailed description.

[0039] Specifically, for example, the image sensing device 1200 can generate first image data in a first shutter mode. The image sensing device 1200 can provide the first image data to the application processor 1100 via the camera interface 1150. The application processor 1100 can provide the image sensing device 1200 with a mode change signal for changing the first shutter mode to a second shutter mode different from the first shutter mode based on shooting environment information about the first image data. The shooting environment information may include data associated with factors that can cause noise in the image data generated by the image sensing device 1200, or data associated with the movement of the subject being photographed.

[0040] For example, the shooting environment information may include at least one of illumination information, gain amplification information, and motion information.

[0041] Illumination information can indicate an illuminance value corresponding to the illuminance of a captured frame. For example, application processor 1100 can calculate the illuminance value from the first image data. For example, the illuminance value can be the average illuminance of the first image data, or it can be the maximum or minimum illuminance of the first image data.

[0042] Gain amplification information indicates the multiplier used to amplify the signal output from the pixel circuitry of the image sensing device 1200. By amplifying the signal output from the pixel circuitry, the image sensing device 1200 can obtain bright image data even in dark environments. For example, the image sensing device 1200 can adjust the gain in units of x4 or x1.

[0043] Motion information may include motion vector data indicating the direction and speed of movement of an object in the frame, and motion intensity data corresponding to the magnitude of the change in pixel values. For example, the motion vector data may be represented by a two-dimensional vector. For example, application processor 1100 may calculate the amount of movement based on the magnitude of the motion vector data and the motion intensity data. The amount of movement may be represented by a scalar value.

[0044] In some embodiments, the application processor 1100 may obtain shooting environment information from image data received from the image sensing device 1200 or from the metadata of the image data.

[0045] In some embodiments, the application processor 1100 may obtain shooting environment information from metadata stored in the internal or external memory of the application processor 1100 and associated with corresponding image data. For example, the application processor 1100 may store and retain metadata to be provided to the image sensing device 1200 during the process in which the image sensing device 1200 generates the corresponding image data.

[0046] Application processor 1100 can determine whether to change the shutter mode based on shooting environment information. For example, different thresholds can be associated with different shutter modes. For example, application processor 1100 can compare a value included in the shooting environment information with a threshold, and based on the comparison, can determine to change the current shutter mode (e.g., a first shutter mode) to another shutter mode (e.g., a second shutter mode). In this case, application processor 1100 can provide image sensing device 1200 with a mode change signal indicating the shutter mode to which to change (e.g., the second shutter mode).

[0047] The image sensing device 1200 can operate in a second shutter mode based on a mode change signal. For example, the image sensing device 1200 can generate second image data in the second shutter mode in response to the mode change signal.

[0048] In some embodiments, the image sensing device 1200 may operate in a second shutter mode during a frame period that begins after receiving a mode change signal. This will refer to Figure 7 Detailed description.

[0049] In some embodiments, the image sensing device 1200 may be formed on a semiconductor substrate. The image sensing device 1200 may be implemented using a system-on-a-chip. In this case, the application processor 1100 may be formed on a semiconductor substrate independent of the semiconductor substrate of the image sensing device 1200.

[0050] According to some embodiments of the electronic device 1000, the application processor 1100 can determine in real time (or periodically) whether to change the shutter mode based on shooting environment information, and therefore, when it is necessary to change the shutter mode, the application processor 1100 can quickly change the shutter mode.

[0051] Figure 2 This is a block diagram of an electronic device 1000 according to some embodiments. (Reference) Figure 2The image sensing device 1200 may include a sensor controller 1210 and an image sensor 1220. Figure 2 The application processor 1100 and the image sensing device 1200 are respectively with Figure 1 The application processor 1100 and the image sensing device 1200 correspond to each other.

[0052] Sensor controller 1210 can control all operations of image sensor 1220. Sensor controller 1210 can change the settings of image sensor 1220. For example, sensor controller 1210 can change the shutter mode of image sensor 1220. Specifically, sensor controller 1210 can provide setting values ​​corresponding to different shutter modes to image sensor 1220. Sensor controller 1210 can change the shutter mode of image sensor 1220 by storing the setting values ​​of image sensor 1220 (e.g., in the registers of image sensor 1220).

[0053] The sensor controller 1210 can provide image data received from the image sensor 1220 to the application processor 1100. The sensor controller 1210 can receive mode change signals from the application processor 1100.

[0054] The sensor controller 1210 can obtain setting data corresponding to a target shutter mode (e.g., a second shutter mode) indicated by the mode change signal based on a mode change signal. For example, the sensor controller 1210 may include a first storage area. For example, the image sensor 1220 may include a second storage area. The first or second storage area may store the setting data corresponding to the target shutter mode.

[0055] In other words, the sensor controller 1210 can read setting data corresponding to the target shutter mode from the first storage area or the second storage area based on the mode change signal.

[0056] In some embodiments, each of the first storage region and the second storage region may include at least one of registers, SRAM, and ROM. However, the embodiments are not limited thereto. For example, the first storage region and the second storage region may each include a memory capable of storing different kinds of data.

[0057] In some embodiments, the setting data may be stored in the form of a lookup table (LUT) that includes multiple shutter modes and multiple setting values. This will refer to Figure 8 Detailed description.

[0058] Image sensor 1220 can convert light that is reflected from or emitted from a subject into a digital signal. (Refer to...) Figure 3 Detailed description of image sensor 1220.

[0059] Figure 3 It is shown in detail according to some embodiments Figure 2 A diagram of the image sensing device 1200. (Refer to...) Figure 3 The sensor controller 1210 and the image sensor 1220 are described in detail. Figure 3 The sensor controller 1210 and the image sensor 1220 are respectively connected to Figure 2 The sensor controller 1210 and the image sensor 1220 correspond to each other.

[0060] Image sensor 1220 may include pixel array 1221, row driver (e.g., row driver circuitry) 1222, timing generator (e.g., timing generation circuitry) 1223, analog-to-digital converter (ADC) circuitry 1224, control register block (e.g., control register circuitry) 1225, ramp signal generator (e.g., ramp signal generation circuitry) 1226, and buffer (e.g., buffer circuitry) 1227.

[0061] Pixel array 1221 may include a plurality of pixel circuits (PIX) arranged in the form of a matrix comprising multiple rows and multiple columns. Each of the plurality of pixel circuits (PIX) may be referred to as a unit pixel, since the plurality of pixel circuits (PIX) constitute a pixel array 1221.

[0062] Under the control of timing output 1223, row driver 1222 can send multiple control signals for controlling the operation of each of the multiple pixel circuits (PIX) to pixel array 1221.

[0063] Under the control of the control register block 1225, the timing generator 1223 can control the operation of the row driver 1222, the ADC circuit 1224, and the ramp signal output device 1226.

[0064] The ADC circuit 1224 may include multiple ADCs ADC1 to ADCn. The multiple ADCs ADC1 to ADCn can perform correlated double sampling on pixel signals output from multiple column lines CL1 to CLn implemented in the pixel array 1221, respectively. Each of the ADCs ADC1 to ADCn can compare its corresponding correlated double-sampled pixel signal with a ramp signal (e.g., a voltage level) output from the ramp signal output unit 1226, and can output a comparison signal depending on the comparison result. Each of the ADCs ADC1 to ADCn can convert the comparison signal into a digital signal, and can output the digital signal to a buffer 1227.

[0065] In some embodiments, the ADC circuit 1224 may be referred to as a readout circuit.

[0066] Under the control of the sensor controller 1210, the control register block 1225 can control the timing generator 1223, the ramp signal generator 1226, and the buffer 1227.

[0067] In some embodiments, the sensor controller 1210 may store multiple setting values ​​of setting data corresponding to the target shutter mode in the control register block 1225. That is, the sensor controller 1210 may change the values ​​stored in the multiple registers of the control register block 1225 to the setting values ​​of the setting data. Therefore, the sensor controller 1210 may change the shutter mode of the image sensor 1220.

[0068] In some embodiments, the settings stored in the register can indicate the start and end points of each pulse signal used by the image sensor 1220.

[0069] The buffer 1227 can send multiple digital pixel signals, each corresponding to a multiple digital signal output from the ADC circuit 1224, as image data to the sensor controller 1210.

[0070] Figure 4 This is a diagram schematically illustrating a rolling shutter mode according to an embodiment.

[0071] In rolling shutter mode, the image sensor can sequentially perform reset and readout operations on the target pixel circuitry, row by row. As shown in rolling shutter mode, different rows of the image sensor are reset, exposed, and readout at slightly different times. Rolling shutter mode may result in tilting between rows.

[0072] Figure 5 This is a diagram schematically illustrating the global shutter mode according to an embodiment.

[0073] In global shutter mode, the signals converted by photodiodes in all pixel circuits (hereinafter referred to as "target pixel circuits") included in the target area are simultaneously transmitted to the floating diffusion node, and digital signals corresponding to each sequentially selected row can be output.

[0074] In detail, in global shutter mode, after the image sensor simultaneously resets the target pixel circuit, the image sensor can simultaneously transfer the charge corresponding to the light received by the photodiode to the floating diffusion node within the same time period. Therefore, the target pixel circuit starts and stops its exposure simultaneously, eliminating time skew between rows. Subsequently, when rows are selected sequentially, the pixel signals of the target pixel circuit can be read out sequentially.

[0075] Figure 6 According to some embodiments Figure 3 The circuit diagram of the pixel circuit PIX. Figure 6 The diagram below shows a detailed circuit diagram of the pixel circuit PIX.

[0076] The pixel circuit PIX according to one or more embodiments can be referred to as operating in a hybrid global shutter mode, because the pixel circuit PIX is capable of operating in either rolling shutter mode or global shutter mode.

[0077] The pixel circuit PIX may include a photodiode PD, a first transistor TR1 to a ninth transistor TR9, a first capacitor C1 and a second capacitor C2.

[0078] A photodiode (PD) can receive light and generate a charge corresponding to the intensity or amount of the received light.

[0079] The first transistor TR1 can be connected between the photodiode PD and the floating diffusion node FD, and can include a gate for receiving the transmitted signal TS. The first transistor TR1 can be referred to as the transmission transistor TG.

[0080] The second transistor TR2 can be connected between the first power supply terminal and the floating diffusion node FD, and can include a gate for receiving a reset signal RS. In this case, the first power supply terminal can receive a first power supply voltage Vpix1. The second transistor TR2 can be referred to as the reset transistor RG.

[0081] The third transistor TR3 can be connected between the second power supply terminal and the first node N1, and can include a gate connected to the floating diffusion node FD. In this case, the second power supply terminal can receive a first power supply voltage Vpix1. The third transistor TR3 can perform the source follower function of outputting the voltage of the floating diffusion node FD. The third transistor TR3 can be referred to as the "first source follower transistor" SF1.

[0082] A fourth transistor TR4 can be connected between the first node N1 and column line CL11, and can include a gate for receiving the first selection signal SS1. During a readout operation, the fourth transistor TR4 can transfer the voltage of the first node N1 to column line CL11. Furthermore, the fourth transistor TR4 can perform an on / off function for column line CL11. For example, when 0V is applied to the gate of the fourth transistor TR4, the fourth transistor TR4 can turn off column line CL11. In this respect, TR4 can disconnect the first node N1 from column line CL11. The fourth transistor TR4 can be referred to as the first selection transistor SEL1.

[0083] The fifth transistor TR5 can be connected between the first node N1 and the storage node SN, and can include a gate that receives the first switching signal SW.

[0084] The sixth transistor TR6 may be connected between the storage node SN and the first terminal of the first capacitor C1, and may include a gate for receiving the first sampling signal SMP1.

[0085] The seventh transistor TR7 can be connected between the storage node SN and the first terminal of the second capacitor C2, and can include a gate for receiving the second sampling signal SMP2.

[0086] The eighth transistor TR8 can be connected between the second power supply terminal and the second node N2, and can include a gate connected to the storage node SN. In this case, the second power supply terminal can receive the second power supply voltage Vpix2. The eighth transistor TR8 can perform the source follower function of outputting the voltage of the storage node SN. The eighth transistor TR8 can be referred to as the "second source follower transistor" SF2.

[0087] The ninth transistor TR9 can be connected between the second node N2 and column line CL12, and can include a gate for receiving the second selection signal SS2. During a readout operation, the ninth transistor TR9 can transfer the voltage of the second node N2 to column line CL12. Furthermore, the ninth transistor TR9 can perform an on / off function for column line CL12. For example, when 0V is applied to the gate of the ninth transistor TR9, it can turn off column line CL12. In this respect, TR9 can disconnect the first node N2 from column line CL12. The ninth transistor TR9 can be referred to as the second selection transistor SEL2.

[0088] The second terminal of the first capacitor C1 can be connected to the second power supply terminal. The second terminal of the second capacitor C2 can also be connected to the second power supply terminal. In this respect, the first capacitor C1 and the second capacitor C2 can be connected in parallel between the storage node SN and the second power supply terminal when the sixth transistor TR6 and the seventh transistor TR7 are turned on.

[0089] The first capacitor C1 can store the charge corresponding to the reset level signal from the charge of the floating diffusion node FD. The second capacitor C2 can store the charge corresponding to the sense level signal from the charge of the floating diffusion node FD. In an embodiment, each of the reset level signal and the sense level signal can be used in a correlated double sampling manner.

[0090] When the image sensor is operating in rolling shutter mode, the fifth transistor TR5 can be turned off via the switch signal SW. Conversely, when the image sensor is operating in global shutter mode, the fifth transistor TR5 can be turned on via the switch signal SW.

[0091] In some embodiments, the image sensor can operate in one of two shutter modes, rolling shutter mode and global shutter mode, by using settings stored in a register.

[0092] For example, an image sensor can control at least one of the reset timing, light receiving timing, and readout timing by using settings stored in a register.

[0093] In some embodiments, column lines CL11 and CL12 can be connected to Figure 3 One of the multiple column lines CL1 to CLn.

[0094] Figure 7 This is a timing diagram illustrating changes in shutter mode according to some embodiments. (Refer to...) Figure 7 Describe the timing of how an image sensing device changes its shutter mode.

[0095] Image sensing devices can change the shutter mode after the current frame period ends and before the next frame period begins. The frame period indicates the time interval from the start time of outputting image data for one frame to the start time of outputting image data for the next frame. In this context, a frame refers to the set of all pixels that make up an image.

[0096] In detail, the image sensing device can operate in the target shutter mode during the frame period that begins after the mode change signal is received (i.e., the first full frame period after the mode change signal is received).

[0097] For example, during the first frame period f1, the image sensing device can operate in a first shutter mode (e.g., rolling shutter mode). The first frame period f1 represents the time period from a first time point t1, starting from a global reset for acquiring the first row of image data for the first frame, to a fourth time point t4, starting from a global reset for acquiring the first row of image data for the second frame.

[0098] The image sensing device can receive a first mode change signal from the application processor at a second time point t2, between a first time point t1 and a third time point t3, and terminate the readout operation of the last line in the frame at the third time point t3. The first mode change signal may indicate a second shutter mode (e.g., a global shutter mode). The image sensing device can obtain first setting data corresponding to the second shutter mode based on the first mode change signal. The image sensing device can continue operating in the first shutter mode from the second time point t2 to the third time point t3.

[0099] The image sensing device can apply the setting values ​​of the first setting data corresponding to the second shutter mode during the time period from t3 to t4, during which no image data is output. That is, the image sensing device can store the corresponding setting values ​​in the image sensor's register. Then, under the control of the sensor controller, the image sensor can operate in the second shutter mode during the second frame period f2 starting from the fourth time point t4, based on the setting values ​​stored in the register.

[0100] During the second frame period f2, the image sensing device may operate in a second shutter mode (e.g., global shutter mode). The second frame period f2 represents the time interval from the fourth time point t4, which begins with a global reset for acquiring the first row of image data for the second frame, to the seventh time point t7, which begins with a global reset for acquiring the first row of image data for the third frame.

[0101] The image sensing device can receive a second mode change signal from the application processor at a fifth time point t5, between a fourth time point t4 and a sixth time point t6, and terminate the readout operation of the last line in the second frame at the sixth time point t6. The second mode change signal can indicate a first shutter mode (e.g., rolling shutter mode). The image sensing device can obtain second setting data corresponding to the first shutter mode based on the second mode change signal. The image sensing device can continue operating in the second shutter mode from the fifth time point t5 to the sixth time point t6.

[0102] The image sensing device can apply the setting values ​​of the second setting data corresponding to the first shutter mode during the time period from t6 to t7, during which no image data is output. That is, the image sensing device can store the corresponding setting values ​​in the image sensor's register. Then, under the control of the sensor controller, the image sensor can operate in the first shutter mode starting from the seventh time point t7, based on the setting values ​​stored in the register.

[0103] For ease of description, an example is illustrated where the first setting data is applied in the first frame period f1, followed by the application of the second setting data in the second frame period f2; however, the embodiment is not limited to this. For example, the image sensing device may receive the second mode change signal in the third frame period f3 instead of the second frame period f2. In this case, the image sensing device may operate in the first shutter mode during the fourth frame period immediately following the third frame period f3.

[0104] Figure 8 This is a diagram illustrating setup data according to some embodiments. (Refer to...) Figure 8 Describes the setup data stored in the image sensing device in the form of a LUT.

[0105] The settings data can include multiple shutter modes and multiple setting values.

[0106] For example, the settings data may include a first shutter mode SM1 and a second shutter mode SM2. The first shutter mode SM1 may be a rolling shutter (RS) mode, and the second shutter mode SM2 may be a global shutter (GS) mode.

[0107] The settings data can include multiple setting values ​​RV1 and RVn corresponding to each shutter mode. In this case, "n" is any natural number.

[0108] The setting data can also include multiple setting values ​​r1~rn as multiple setting values ​​RV1, RVn corresponding to the first shutter mode SM1.

[0109] The settings data can include multiple settings g1 to gn as multiple settings RV1 and RVn corresponding to the second shutter mode SM2.

[0110] Figure 9 This describes some embodiments. Figure 1 A diagram illustrating the operation method of the electronic device 1000. (Reference) Figure 9 The electronic device 1000 may include an application processor 1100 and an image sensing device 1200.

[0111] In operation S110, the image sensing device 1200 can generate first image data in the first shutter mode SM1.

[0112] In operation S120, the image sensing device 1200 can provide the first image data to the application processor 1100.

[0113] In operation S130, application processor 1100 can generate a first mode change signal indicating a second shutter mode SM2 that is different from the first shutter mode SM1.

[0114] In detail, the application processor 1100 can determine whether to change the shutter mode based on shooting environment information associated with the first image data. This will refer to... Figure 10 Detailed description.

[0115] In operation S140, application processor 1100 may provide a first mode change signal to image sensing device 1200. For example, when it is determined that the shutter mode will be changed, application processor 1100 may provide a first mode change signal to image sensing device 1200.

[0116] In operation S150, the image sensing device 1200 can generate second image data in the second shutter mode SM2.

[0117] In some embodiments, the image sensing device 1200 can obtain first setting data corresponding to the second shutter mode SM2 based on a first mode change signal. For example, the image sensing device 1200 can store and retain the first setting data in a sensor controller or image sensor. The image sensing device 1200 can change the shutter mode of the image sensor by storing the first setting data in a register of the image sensor.

[0118] In some embodiments, the image sensing device 1200 may operate in a second shutter mode SM2 during the frame period that begins after receiving the first mode change signal.

[0119] Figure 10 This describes some embodiments. Figure 1 A flowchart illustrating the operation method of the application processor 1100 over time. (Refer to...) Figure 10 Describes how the application processor operates over time.

[0120] In operation S210, the application processor can receive first image data generated in the first shutter mode from the image sensing device.

[0121] In operation S220, the application processor can determine whether to change the shutter mode based on the shooting environment information associated with the first image data.

[0122] In some embodiments, the application processor may obtain at least a portion of the shooting environment information based on the first image data. Optionally, the application processor may obtain at least a portion of the shooting environment information based on the metadata of the first image data transmitted from the image sensing device along with the first image data. Optionally, the application processor may obtain at least a portion of the shooting environment information based on the metadata of the first image data stored in the application processor's internal or external memory.

[0123] In some embodiments, the application processor may determine that the first shutter mode is a global shutter mode and the illuminance value of the shooting environment information is less than a first threshold. The application processor may generate a first mode change signal that indicates the rolling shutter mode as a second shutter mode. In this case, the first threshold may be a preset value.

[0124] In some embodiments, the application processor may determine that the first shutter mode is a rolling shutter mode and that the illuminance value of the shooting environment information is greater than a second threshold. In this case, the second threshold may be greater than the first threshold and may be a preset value. The application processor may generate a first mode change signal that indicates the global shutter mode as the second shutter mode.

[0125] Based on the above description, the electronic device can operate in global shutter mode when it is bright and in rolling shutter mode when it is dark.

[0126] In some embodiments, the application processor may determine that the first shutter mode is a rolling shutter mode and that the speed value of the ambient information being captured is greater than a third threshold. The application processor may generate a first mode change signal that indicates the global shutter mode as a second shutter mode. The third threshold may be a preset value.

[0127] In some embodiments, the application processor may determine that the first shutter mode is a global shutter mode and the speed value of the shooting environment information is less than a fourth threshold. In this case, the fourth threshold may be less than a third threshold and may be a preset value. The application processor may generate a first mode change signal that indicates the rolling shutter mode as a second shutter mode.

[0128] Based on the above description, when there is a lot of movement, the electronic device can operate in global shutter mode, and when there is very little movement, the electronic device can operate in rolling shutter mode.

[0129] In some embodiments, the application processor may determine that the first shutter mode is a rolling shutter mode and that the gain amplification factor included in the shooting environment information is less than a fifth threshold. The application processor may generate a first mode change signal that indicates the global shutter mode as a second shutter mode. The fifth threshold may be a preset value.

[0130] In some embodiments, the application processor may determine that the first shutter mode is a global shutter mode and the gain amplification factor included in the shooting environment information is greater than a sixth threshold. In this case, the sixth threshold may be greater than a fifth threshold and may be a preset value. The application processor may generate a first mode change signal that indicates the rolling shutter mode as a second shutter mode.

[0131] Based on the above description, the electronic device can operate in rolling shutter mode in an environment where the probability of increased noise is high due to the large gain amplification value, and can operate in global shutter mode in an environment where the probability of increased noise is low due to the small gain amplification value.

[0132] When it is determined that the shutter mode needs to be changed, the application processor can execute operation S230; when it is determined that the shutter mode does not need to be changed, the application processor can execute operation S210 again.

[0133] In operation S230, the application processor can provide a first mode change signal to the image sensing device.

[0134] The image sensing device can change its settings based on the signal change of the first mode in order to operate in the second shutter mode after switching from the first shutter mode.

[0135] Figure 11 This describes some embodiments. Figure 2 A diagram illustrating the operation method of the electronic device 1000. (Refer to...) Figure 11 Describe how the application processor 1100, sensor controller 1210, and image sensor 1220 operate.

[0136] For convenience, references will be omitted. Figure 9 and Figure 10 The description provided is to avoid redundancy.

[0137] In operation S310, the image sensor 1220 can generate first image data in the first shutter mode SM1.

[0138] In operation S315, the image sensor 1220 can provide the first image data to the sensor controller 1210.

[0139] In operation S320, the sensor controller 1210 can provide the first image data to the application processor 1100. In operation S330, the application processor 1100 can generate a first mode change signal indicating a second shutter mode SM2 that is different from the first shutter mode SM1. In operation S340, the application processor 1100 can provide the first mode change signal to the sensor controller 1210.

[0140] In operation S341, the sensor controller 1210 can obtain first setting data corresponding to the second shutter mode SM2 based on the first mode change signal. Specifically, the sensor controller 1210 can obtain the first setting data from one of the first storage area of ​​the sensor controller 1210 and the second storage area of ​​the image sensor 1220.

[0141] In operation S342, between the end of the output of image data in the current frame period and the start of the next frame period, the sensor controller 1210 can provide the setting value of the first setting data to the image sensor 1220.

[0142] In operation S343, between the time point at which the setting value is provided to the image sensor 1220 and the time point at which the next frame period begins, the setting value can be applied to the image sensor 1220 under the control of the sensor controller 1210. Specifically, in this case, the sensor controller 1210 can store the setting value in a register of the image sensor 1220.

[0143] In operation S350, during the next frame period, the image sensor 1220 can generate second image data in the second shutter mode.

[0144] Figure 12 This is a block diagram of a System-on-Chip 2000 according to some embodiments. Reference Figure 12 This illustrates a system-on-a-chip including a sensor controller 2100 and an image sensor 2200. References are omitted for convenience. Figure 3 The description provided is to avoid redundancy.

[0145] The components of a system-on-a-chip can be formed on a semiconductor substrate.

[0146] The sensor controller 2100 can control all operations of the image sensor 2200. The sensor controller 2100 can change the shutter mode of the image sensor 2200.

[0147] In detail, the sensor controller 2100 can receive first image data generated in the first shutter mode from the image sensor 2200.

[0148] The sensor controller 2100 can change the settings of the image sensor 2200 based on shooting environment information associated with the first image data, so that the image sensor 2200 operates in a second shutter mode different from the first shutter mode.

[0149] In some embodiments, the sensor controller 2100 may include a microcontroller unit (MCU). At least a portion of the sensor controller 2100 may be implemented by firmware.

[0150] In some embodiments, the sensor controller 2100 can compare shooting environment information with at least one threshold and determine whether to change the shutter mode of the image sensor 2200. Detailed comparison methods are similar to those described in the reference. Figure 10 The method described.

[0151] In some embodiments, the sensor controller 2100 may obtain at least a portion of the shooting environment information from the first image data. Optionally, the sensor controller 2100 may obtain at least a portion of the shooting environment information from the metadata of the first image data provided by the image sensor 2200, together with the first image data. Optionally, the sensor controller 2100 may be provided stored on an external application processor (e.g., Figure 1 The metadata of the first image data in the application processor, and at least a portion of the shooting environment information can be obtained from the provided metadata.

[0152] In some embodiments, the sensor controller 2100 may determine to change the shutter mode based on shooting environment information associated with the first image data. In this regard, the sensor controller 2100 may determine (or select) a second shutter mode as the target shutter mode. The sensor controller 2100 may obtain first setting data corresponding to the second shutter mode. The first setting data may be stored in a first storage area of ​​the sensor controller 2100 (e.g., Figure 2 The first storage area) or the second storage area of ​​the image sensor 2200 (e.g., Figure 2 In the second storage area.

[0153] In some embodiments, the sensor controller 2100 may provide at least one setting value of the first setting data to the image sensor 2200. The image sensor 2200 may store at least one setting value in at least one register. Therefore, the shutter mode of the image sensor 2200 can be changed.

[0154] In some embodiments, the image sensor 2200 may operate in the target shutter mode during a frame period that begins after the sensor controller 2100 determines the time point at which the shutter mode of the image sensor 2200 is changed.

[0155] Figure 13 This describes some embodiments. Figure 12 A diagram illustrating the operation of the on-chip system. (Refer to...) Figure 13 The system-on-a-chip may include a sensor controller 2100 and an image sensor 2200.

[0156] In operation S410, the image sensor 2200 can generate first image data in the first shutter mode SM1.

[0157] In operation S420, the image sensor 2200 can provide first image data to the sensor controller 2100.

[0158] In operation S430, the sensor controller 2100 can determine to change the shutter mode of the image sensor 2200 to the second shutter mode SM2 based on shooting environment information associated with the first image data.

[0159] In operation S440, the sensor controller 2100 can change the settings of the image sensor 2200 to operate in the second shutter mode SM2.

[0160] In some embodiments, operation S440 may include the sensor controller determining a second shutter mode as a target shutter mode based on shooting environment information associated with the first image data, the sensor controller obtaining first setting data corresponding to the second shutter mode, and the sensor controller changing the settings of the image sensor based on the first setting data.

[0161] In some embodiments, changing the settings of an image sensor by a sensor controller based on first setting data may include: the sensor controller providing at least one setting value of the first setting data to the image sensor, and the image sensor storing at least one setting value in at least one register.

[0162] In the system-on-chip 2000 according to an embodiment, the sensor controller 2100 can analyze shooting environment information and determine whether to change the shutter mode of the image sensor 2200.

[0163] Figure 14 It is a block diagram of an electronic device that includes multiple camera modules. Figure 15 This is a detailed explanation. Figure 14 A block diagram of the camera module.

[0164] Reference Figure 14 The electronic device 3000 may include a camera module group 3100, an application processor 3200, a PMIC 3300, and external memory 3400. The camera module group 3100 may include... Figure 2 The image sensor 1220. The application processor 3200 can be used with... Figure 1 The application processor corresponds to 1100.

[0165] The camera module group 3100 may include multiple camera modules 3100a, 3100b, and 3100c. Figure 14 The illustration shows an electronic device comprising three camera modules 3100a, 3100b, and 3100c, but the embodiments are not limited thereto. In some embodiments, the camera module group 3100 may be modified to include only two camera modules. Furthermore, in some embodiments, the camera module group 3100 may be modified to include "n" camera modules (n being a natural number of 4 or greater).

[0166] Below, we will refer to Figure 15 The following description provides a more comprehensive and detailed description of the configuration of camera module 3100b, but the description can be applied equally to the remaining camera modules 3100a and 3100c.

[0167] refer to Figure 15The camera module 3100b may include a prism 3105, an optical path folding element (OPFE) 3110, an actuator 3130, an image sensing device 3140, and a storage device 3150.

[0168] The prism 3105 may include a reflective plane 3107 of light-reflecting material and may change the path of light “L” incident from the outside.

[0169] In some embodiments, prism 3105 can change the path of light "L" incident along the first direction (X) to a second direction (Y) perpendicular to the first direction (X). Furthermore, prism 3105 can change the path of light "L" incident along the first direction (X) to a second direction (Y) perpendicular to the first (X-axis) direction by rotating the reflective plane 3107 of the light-reflecting material about the central axis 3106 in direction "A" or by rotating the central axis 3106 in direction "B". In this case, OPFE 3110 can move in a third direction (Z) perpendicular to the first direction (X) and the second direction (Y).

[0170] In some embodiments, such as Figure 15 As shown, the maximum rotation angle of prism 3105 in direction "A" can be equal to or less than 15 degrees in the positive A direction and greater than 15 degrees in the negative A direction, but the embodiments are not limited to this.

[0171] In some embodiments, the prism 3105 may move within about 20 degrees, between 10 degrees and 20 degrees, or between 15 degrees and 20 degrees in the positive B direction or the negative B direction; here, the prism 3105 may move at the same angle in the positive B or negative B direction, or may move at a similar angle within about 1 degree.

[0172] In some embodiments, the prism 3105 can move the reflective plane 3107 of the light-reflecting material in a third direction (e.g., the Z direction) parallel to the direction in which the central axis 3106 extends.

[0173] OPFE 3110 may include, for example, an optical lens consisting of "m" lenses (where m is a natural number). Here, the "m" lenses can be moved in a second direction (Y) to change the optical zoom ratio of camera module 3100b. For example, when the default optical zoom ratio of camera module 3100b is "Z", the optical zoom ratio of camera module 3100b can be changed to 3Z, 5Z, or 5Z or greater by moving the "m" optical lenses included in OPFE 3110.

[0174] The actuator 3130 can move the OPFE 3110 or the optical lens (hereinafter referred to as the "optical lens") to a specific position. For example, the actuator 3130 can adjust the position of the optical lens so that the image sensor 3142 is placed at the focal length of the optical lens for accurate sensing.

[0175] Image sensing device 3140 may include image sensor 3142, control logic 3144, and memory 3146. Image sensor 3142 can sense an image of a target using light "L" provided through an optical lens. Control logic 3144 can control the overall operation of camera module 3100b. For example, control logic 3144 can control the operation of camera module 3100b based on control signals provided through control signal line CSLb.

[0176] The memory 3146 can store information required for the operation of the camera module 3100b, such as calibration data 3147. Calibration data 3147 may include information required by the camera module 3100b to generate image data using externally supplied light "L". Calibration data 3147 may include, for example, information about the degree of rotation, information about the focal length, information about the optical axis, etc. In the case where the camera module 3100b is implemented as a multi-state camera where the focal length varies depending on the position of the optical lens, calibration data 3147 may include the focal length value for each position (or state) of the optical lens and information about autofocus.

[0177] The memory 3150 can store image data sensed by the image sensor 3142. The memory 3150 can be disposed outside the image sensing device 3140 and can be implemented in the form of a stack of the memory 3150 and the sensor chip constituting the image sensing device 3140. In some embodiments, the memory 3150 can be implemented with an electrically erasable programmable read-only memory (EEPROM), but the embodiments are not limited thereto.

[0178] Let's refer to each other. Figure 14 and Figure 15 In some embodiments, each of the plurality of camera modules 3100a, 3100b, and 3100c may include an actuator 3130. Thus, depending on the operation of the actuator 3130 therein, the same calibration data 3147 or different calibration data 3147 may be included in the plurality of camera modules 3100a, 3100b, and 3100c.

[0179] In some embodiments, one of the plurality of camera modules 3100a, 3100b and 3100c (e.g. 3100b) may be a folded lens-shaped camera module, including the aforementioned prism 3105 and OPFE 3110, and the remaining camera modules (e.g. 3100a and 3100c) may be vertically shaped camera modules, which do not include the aforementioned prism 3105 and OPFE 3110; however, the embodiments are not limited thereto.

[0180] In some embodiments, one of the plurality of camera modules 3100a, 3100b, and 3100c (e.g., 3100c) may be a depth camera of a vertical shape, for example, that extracts depth information by using infrared (IR). In this case, the application processor 3200 may combine image data provided from the depth camera with image data provided from any other camera module (e.g., 3100a or 3100b) and may generate a three-dimensional (3D) depth image.

[0181] In some embodiments, at least two camera modules (e.g., 3100a and 3100b) of the plurality of camera modules 3100a, 3100b and 3100c may have different fields of view. In this case, at least two camera modules (e.g., 3100a and 3100b) of the plurality of camera modules 3100a, 3100b and 3100c may include different optical lenses, but the embodiments are not limited thereto.

[0182] Furthermore, in some embodiments, the fields of view of the multiple camera modules 3100a, 3100b, and 3100c may be different. In this case, the multiple camera modules 3100a, 3100b, and 3100c may include different optical lenses, but are not limited thereto.

[0183] In some embodiments, the plurality of camera modules 3100a, 3100b and 3100c may be configured to be physically separate from each other. That is, the plurality of camera modules 3100a, 3100b and 3100c may not use the sensing area of ​​a single image sensor 3142, but rather the plurality of camera modules 3100a, 3100b and 3100c may each include an independent image sensor 3142.

[0184] return Figure 14 The application processor 3200 may include an image processing device 3210, a memory controller 3220, and internal memory 3230. The application processor 3200 may be implemented separately from the multiple camera modules 3100a, 3100b, and 3100c. For example, the application processor 3200 and the multiple camera modules 3100a, 3100b, and 3100c may be implemented using separate semiconductor chips.

[0185] The image processing device 3210 may include a plurality of sub-image processors 3212a, 3212b and 3212c, an image generator 3214 and a camera module controller 3216.

[0186] The image processing device 3210 may include a plurality of sub-image processors 3212a, 3212b and 3212c, the number of which corresponds to the number of the plurality of camera modules 3100a, 3100b and 3100c.

[0187] Image data generated from camera modules 3100a, 3100b, and 3100c can be provided to corresponding sub-image processors 3212a, 3212b, and 3212c via separate image signal lines ISLa, ISLb, and ISLc, respectively. For example, image data generated from camera module 3100a can be provided to sub-image processor 3212a via image signal line ISLa, image data generated from camera module 3100b can be provided to sub-image processor 3212b via image signal line ISLb, and image data generated from camera module 3100c can be provided to sub-image processor 3212c via image signal line ISLc. This image data transmission can be performed, for example, using a camera serial interface (CSI) based on MIPI (Mobile Industry Processor Interface), but the embodiments are not limited thereto.

[0188] In some embodiments, a sub-image processor may be arranged to correspond to multiple camera modules. For example, sub-image processors 3212a and 3212c may be implemented as a single unit, rather than as... Figure 14 As shown, they are separated from each other; in this case, one of the multiple image data provided from camera module 3100a and camera module 3100c can be selected by a selection element (e.g., a multiplexer), and the selected image data can be provided to the integrated sub-image processor.

[0189] Image data provided to sub-image processors 3212a, 3212b, and 3212c can be provided to image generator 3214. Depending on the image generation information or mode signal, image generator 3214 can generate an output image using the image data provided from sub-image processors 3212a, 3212b, and 3212c.

[0190] Specifically, depending on the image generation information or mode signal, the image generator 3214 can generate an output image by merging at least a portion of image data generated from camera modules 3100a, 3100b, and 3100c, which have different fields of view. Furthermore, depending on the image generation information or mode signal, the image generator 3214 can generate an output image by selecting one of the image data generated from camera modules 3100a, 3100b, and 3100c, which have different fields of view.

[0191] In some embodiments, the image generation information may include (i.e., an indication) a zoom signal or zoom factor. Furthermore, in some embodiments, the mode signal may be, for example, a signal based on a mode selected by the user.

[0192] When the image generation information (Generating Information) is a zoom signal (or zoom factor) and camera modules 3100a, 3100b, and 3100c have different fields of view, the image generator 3214 can perform different operations depending on the type of zoom signal. For example, when the zoom signal is a first signal, the image generator 3214 can merge image data output from camera module 3100a and image data output from camera module 3100c, and can generate an output image by using the merged image signal and image data output from camera module 3100b that was not used in the merging operation. When the zoom signal is a second signal different from the first signal, without an image data merging operation, the image generator 3214 can select one of the image data output from camera modules 3100a, 3100b, and 3100c respectively, and can output the selected image data as the output image. However, the embodiments are not limited to this, and the way image data is processed can be modified as needed.

[0193] In some embodiments, the image generator 3214 can generate merged image data with increased dynamic range by receiving multiple image data with different exposure times from at least one of multiple sub-image processors 3212a, 3212b and 3212c and performing high dynamic range (HDR) processing on the multiple image data.

[0194] The camera module controller 3216 can provide control signals to camera modules 3100a, 3100b, and 3100c respectively. The control signals generated by the camera module controller 3216 can be provided to the corresponding camera modules 3100a, 3100b, and 3100c respectively through separate control signal lines CSLa, CSLb, and CSLc.

[0195] Depending on the Generating Information used to generate image generation information, including zoom signals or mode signals, one of the multiple camera modules 3100a, 3100b, and 3100c can be designated as the master camera (e.g., 3100b), and the remaining camera modules (e.g., 3100a and 3100c) can be designated as slave cameras. This designation information can be included in control signals, and the control signals including the designation information can be provided to the corresponding camera modules 3100a, 3100b, and 3100c via separate control signal lines CSLa, CSLb, and CSLc, respectively.

[0196] The camera module operating as a master camera or a slave camera can be changed depending on the zoom factor or operating mode signal. For example, when the field of view of camera module 3100a is wider than that of camera module 3100b and the zoom factor indicates a low zoom ratio, camera module 3100b can operate as the master camera, and camera module 3100a can operate as the slave camera. Conversely, when the zoom factor indicates a high zoom ratio, camera module 3100a can operate as the master camera, and camera module 3100b can operate as the slave camera.

[0197] In some embodiments, the control signals provided from the camera module controller 3216 to each of the camera modules 3100a, 3100b, and 3100c may include a synchronization enable signal. For example, when camera module 3100b acts as the main camera and camera modules 3100a and 3100c act as slave cameras, the camera module controller 3216 may send a synchronization enable signal to camera module 3100b. Camera module 3100b, provided with the synchronization enable signal, may generate a synchronization signal based on the provided synchronization enable signal and may provide the generated synchronization signal to camera modules 3100a and 3100c via the synchronization signal line SSL. Camera modules 3100b, 3100a, and 3100c may synchronize with the synchronization signal to send image data to the application processor 3200.

[0198] In some embodiments, control signals provided from camera module controller 3216 to each of camera modules 3100a, 3100b, and 3100c may include mode information based on mode signals. Based on the mode information, the plurality of camera modules 3100a, 3100b, and 3100c may operate in a first operating mode and a second operating mode relative to the sensing speed.

[0199] In the first operating mode, multiple camera modules 3100a, 3100b, and 3100c can generate image signals at a first speed (e.g., generate image signals at a first frame rate), encode the image signals at a second speed (e.g., encode image signals at a second frame rate higher than the first frame rate), and send the encoded image signals to the application processor 3200. In this case, the second speed can be less than 30 times the first speed.

[0200] Application processor 3200 can store the received image signal (i.e., the encoded image signal) in the provided memory 3230 or in an external memory 3400 located outside the application processor 3200. Then, application processor 3200 can read the encoded image signal from memory 3230 or external memory 3400 and decode it, and can display image data generated based on the decoded image signal. For example, one of the sub-image processors 3212a, 3212b, and 3212c of image processing device 3210 can perform decoding and can also perform image processing on the decoded image signal.

[0201] In the second operating mode, multiple camera modules 3100a, 3100b, and 3100c can generate image signals at a third speed (e.g., a third frame rate image signal lower than the first frame rate) and send the image signals to the application processor 3200. The image signals provided to the application processor 3200 can be unencoded signals. The application processor 3200 can perform image processing on the received image signals, or it can store the image signals in memory 3230 or external memory 3400.

[0202] The PMIC 3300 can supply power, for example, power supply voltage, to multiple camera modules 3100a, 3100b, and 3100c respectively. For example, under the control of the application processor 3200, the PMIC 3300 can supply a first power to camera module 3100a via power signal line PSLa, a second power to camera module 3100b via power signal line PSLb, and a third power to camera module 3100c via power signal line PSLc.

[0203] In response to a power control signal PCON from the application processor 3200, the PMIC 3300 can generate power corresponding to each of the plurality of camera modules 3100a, 3100b, and 3100c, and can adjust the power level. The power control signal PCON can include a power adjustment signal for each operating mode of the plurality of camera modules 3100a, 3100b, and 3100c. For example, the operating mode can include a low power mode. In this case, the power control signal PCON can include information about the camera module operating in low power mode and the set power level. The power levels supplied to the plurality of camera modules 3100a, 3100b, and 3100c respectively can be the same or different from each other. Furthermore, the power level can be changed dynamically.

[0204] According to an embodiment, an image sensing device that switches shutter modes according to the shooting environment, an electronic device including the image sensing device, and a method for operating the electronic device are provided.

[0205] In addition, an image sensing device and an electronic device including the image sensing device are provided, which can perform rapid switching between shutter modes when the application processor or sensor controller determines in real time whether to change the shutter mode based on shooting environment information and changes the shutter mode, that is, provide improved performance.

[0206] While various aspects of the embodiments have been specifically shown and described, it should be understood that various changes in form and detail may be made therein without departing from the spirit and scope of the appended claims.

Claims

1. A method of operating an electronic device including an application processor and an image sensing device, the method comprising: The first image data is generated by the image sensing device in the first shutter mode; The application processor provides the image sensing device with a first mode change signal that indicates a second shutter mode that is different from the first shutter mode, based on shooting environment information associated with the first image data; as well as The image sensing device generates second image data based on the first mode change signal in the second shutter mode.

2. The method according to claim 1, wherein, The first shutter mode is one of the rolling shutter mode and the global shutter mode, and the second shutter mode is the other of the rolling shutter mode and the global shutter mode.

3. The method according to claim 1, wherein, Image sensing devices are system-on-a-chip (SoCs).

4. The method according to claim 1, wherein, The second image data generated based on the first mode change signal in the second shutter mode includes: The image sensing device obtains first setting data corresponding to the second shutter mode based on the first mode change signal; The image sensing device changes the settings of its image sensor based on first setting data; and The image sensor generates second image data based on the modified settings.

5. The method according to claim 4, wherein, The initial setup data is stored in the sensor controller or image sensor of the image sensing device.

6. The method according to claim 1, wherein, The image sensing device operates in the second shutter mode during the frame period that begins after the time point at which it receives the first mode change signal from the application processor.

7. The method according to claim 1, further comprising: The application processor obtains shooting environment information from at least one of the first image data, the metadata of the first image data provided together with the first image data from the image sensing device, and the metadata of the first image data stored in the application processor.

8. The method according to claim 1, wherein, The shooting environment information includes any one or any combination of illumination information, gain amplification information, and motion information.

9. The method according to claim 8, wherein, The first shutter mode is the global shutter mode and the second shutter mode is the rolling shutter mode. The provision of a first mode change signal indicating a second shutter mode to the image sensing device includes: The application processor determines that the illuminance value indicated by the illuminance information is less than a first threshold. The application processor generates a first mode change signal indicating a second shutter mode based on an illuminance value that is less than a first threshold; and The application processor provides the first mode change signal to the image sensing device.

10. The method according to claim 8, wherein, The first shutter mode is the rolling shutter mode, and the second shutter mode is the global shutter mode. The provision of a first mode change signal indicating a second shutter mode to the image sensing device includes: The application processor determines that the illuminance value indicated by the illuminance information is greater than a second threshold; The application processor generates a first mode change signal indicating a second shutter mode based on an illuminance value greater than a second threshold; and The application processor provides the first mode change signal to the image sensing device.

11. The method according to claim 8, wherein, The first shutter mode is the rolling shutter mode, and the second shutter mode is the global shutter mode. The provision of a first mode change signal indicating a second shutter mode to the image sensing device includes: The application processor determines that the amount of movement indicated by the movement information is greater than a third threshold; The application processor generates a first mode change signal indicating a second shutter mode based on the amount of movement exceeding a third threshold; and The application processor provides the first mode change signal to the image sensing device.

12. The method according to claim 8, wherein, The first shutter mode is the global shutter mode and the second shutter mode is the rolling shutter mode. The provision of a first mode change signal indicating a second shutter mode to the image sensing device includes: The application processor determines that the amount of movement indicated by the movement information is less than the fourth threshold; The application processor generates a first mode change signal indicating a second shutter mode based on the movement amount being less than a fourth threshold; and The application processor provides the first mode change signal to the image sensing device.

13. The method according to claim 8, wherein, The first shutter mode is the rolling shutter mode, and the second shutter mode is the global shutter mode. The provision of a first mode change signal indicating a second shutter mode to the image sensing device includes: The application processor determines that the gain amplification value indicated by the gain amplification information is less than the fifth threshold; The application processor generates a first mode change signal indicating the second shutter mode based on a multiplier value being less than a fifth threshold; and The application processor provides the first mode change signal to the image sensing device.

14. The method according to claim 8, wherein, The first shutter mode is a global shutter mode, and the second shutter mode is a rolling shutter mode. The provision of a first mode change signal indicating a second shutter mode to the image sensing device includes: The application processor determines that the multiple indicated by the gain amplification information is greater than the sixth threshold; The application processor generates a first mode change signal indicating the second shutter mode based on a multiplier value greater than a sixth threshold; and The application processor provides the first mode change signal to the image sensing device.

15. A method of operating an on-chip system including a sensor controller and an image sensor, the method comprising: The first image data is generated by the image sensor in the first shutter mode; The sensor controller changes the image sensor settings based on shooting environment information associated with the first image data to control the image sensor to operate in a second shutter mode different from the first shutter mode. as well as The second image data is generated by the image sensor in the second shutter mode.

16. The method according to claim 15, wherein, Changing the image sensor settings includes: The sensor controller determines the second shutter mode as the target shutter mode based on the shooting environment information associated with the first image data; The sensor controller obtains first setting data corresponding to the second shutter mode; and The sensor controller changes the settings of the image sensor based on the first set data.

17. The method according to claim 16, wherein, Changing the image sensor settings based on the first setting data includes: The sensor controller provides at least one setting value of the first setting data to the image sensor; and At least one setting value is stored in at least one register of the image sensor.

18. The method according to claim 16, wherein, The image sensor operates in the second shutter mode during the frame period that begins after the time point from which the sensor controller determines the second shutter mode as the target shutter mode.

19. The method of claim 15, further comprising: Shooting environment information is obtained from any one or any combination of the first image data, the metadata of the first image data provided by the image sensor together with the first image data, and the metadata of the first image data stored in the application processor communicating with the system on chip.

20. An electronic device, comprising: An image sensing device is configured to generate first image data in a first shutter mode and a second shutter mode; as well as The application processor is configured to provide the image sensing device with a first mode change signal indicating a shutter mode different from the current shutter mode, based on shooting environment information associated with the first image data.

Images