Image acquisition device and electronic device including the same

Abstract

An image acquisition device and an electronic device including the image acquisition device are provided. The image acquisition device includes: a first image sensor configured to acquire a first image based on detecting light of a first wavelength band; a second image sensor configured to acquire a second image based on detecting light of a second wavelength band wider than the first wavelength band; and a processor configured to acquire a third image having a spatial resolution corresponding to the first image and a color gamut corresponding to the second image based on the first image and the second image. The image acquisition device can provide an image with high spatial resolution and wide color gamut.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on and claims the benefit of priority from Korean Patent Application No. 10-2021-0067898 filed in the Korean Intellectual Property Office on May 26, 2021, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] Apparatuses and methods consistent with example embodiments relate to an image acquisition apparatus and an electronic apparatus including the image acquisition apparatus. Background Art

[0004] The image sensor is a device that receives light incident from a subject and photoelectrically converts the received light to generate an electric signal.

[0005] The image sensor uses a color filter composed of an array of filter elements that selectively transmit red light, green light, and blue light for color representation, senses the amount of light passing through each filter element, and then forms a color image of the object through image processing.

[0006] When such an image is acquired, since the wavelength band transmitted by the filter element provided in the color filter is limited, the color gamut representing the object is limited. Summary of the invention

[0007] One or more example embodiments provide an image acquisition apparatus providing a wide color gamut image and an electronic apparatus including the same.

[0008] According to one aspect of an example embodiment, an image acquisition device may include: a first image sensor configured to obtain a first image based on detecting light of a first wavelength band; a second image sensor configured to obtain a second image based on detecting light of a second wavelength band wider than the first wavelength band; and a processor configured to obtain a third image having a spatial resolution corresponding to the first image and a color gamut corresponding to the second image based on the first image and the second image.

[0009] The processor may also be configured to correct pixel values ​​of the first image based on pixel values ​​of the second image to obtain a third image.

[0010] The processor may also be configured to register the first image and the second image on a two-dimensional plane, and correct pixel values ​​of the first image based on a correspondence between pixels of the first image and pixels of the second image.

[0011] The processor may be further configured to register the first image and the second image after performing aberration correction on the first image and the second image.

[0012] The processor may also be configured to determine the third image or the first image as an output image for display.

[0013] The processor may also be configured to determine the first image and a difference image between the first image and the third image for display.

[0014] The first image may indicate a first color gamut, and the third image may indicate a second color gamut wider than the first color gamut.

[0015] The first image sensor may include: a first pixel array including a first sensor layer in which a plurality of first sensing elements are arranged in an array; and a color filter arranged on the first sensor layer and including red, green, and blue filters arranged alternately.

[0016] The second image sensor may include: a second pixel array including a second sensor layer in which a plurality of second sensing elements are arranged in an array; and a spectral filter arranged on the second sensor layer, wherein a plurality of filter groups are repeatedly arranged in the spectral filter. Each of the plurality of filter groups may include a plurality of unit filters having different transmission bands.

[0017] Each transmission wavelength band of the plurality of unit filters may include a visible light band and have a wavelength band wider than the visible light band.

[0018] Each of the plurality of filter banks includes 16 unit filters arranged in a 4×4 array.

[0019] The first pixel array and the second pixel array may be horizontally spaced apart from each other on the same circuit board to have the same focal length and the same field of view.

[0020] The circuit board may include a first circuit element that processes a first signal from the first sensor layer and a second circuit element that processes a second signal from the second sensor layer.

[0021] The image acquisition device may include a timing controller configured to synchronize operations of the first circuit element and the second circuit element.

[0022] The image acquisition device may include a memory, wherein data related to the first image and data related to the second image are stored in the memory.

[0023] The memory may be provided in the circuit board.

[0024] The image acquisition device may include: a first imaging optical system, which forms a first image of an object on a first image sensor and includes one or more first lenses; and a second imaging optical system, which forms a second image of an object on a second image sensor and includes one or more second lenses.

[0025] The first imaging optical system and the second imaging optical system may be arranged to have the same focal length and the same field of view.

[0026] According to an aspect of another example embodiment, an electronic device may include: an RGB sensor configured to detect first light of a first band reflected from an object; a hyperspectral image sensor for detecting second light of a second band reflected from the object, wherein the second band is wider than the first band; and a processor configured to: obtain a first image based on the first light of the first band, and a second image based on the second light of the second band; align the first image with the second image based on position information of edges detected from the first image and the second image; and while the first image is aligned with the second image, convert the color of the first image based on the second image.

[0027] The electronic device may correspond to a mobile phone, a smart phone, a tablet computer, a smart tablet computer, a digital camera, a video recorder, a laptop computer, a television, a smart television, a smart refrigerator, a security camera, a robot, or a medical camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and / or other aspects will become more apparent by describing certain example embodiments with reference to the accompanying drawings, in which:

[0029] Figure 1 is a block diagram of a schematic structure of an image acquisition apparatus according to an example embodiment;

[0030] Figure 2 is a color coordinate diagram showing a color gamut indicated by an image obtained by an image acquisition apparatus according to an example embodiment;

[0031] Figure 3 is a conceptual diagram of a schematic structure of an image acquisition device according to an exemplary embodiment;

[0032] Figure 4 is a diagram of a circuit configuration of a first image sensor and a second image sensor provided in an image acquisition apparatus according to an example embodiment;

[0033] Figure 5 is a graph of a wavelength spectrum of a first image sensor provided in an image acquisition device according to an example embodiment;

[0034] Figures 6 to 8is a diagram of an exemplary pixel arrangement of a first image sensor provided in an image acquisition device according to an example embodiment;

[0035] Fig. 9 is a graph of a wavelength spectrum of a second image sensor provided in an image acquisition device according to an example embodiment;

[0036] Figures 10 to 12 is a diagram of an exemplary pixel arrangement of a second image sensor provided in an image acquisition device according to an example embodiment;

[0037] Fig.13 is a flowchart illustrating an image processing process of an image acquisition device according to an example embodiment; and

[0038] Fig.14 is a block diagram of a schematic structure of an electronic device according to an example embodiment;

[0039] Fig.15 yes Fig.14 A block diagram of a camera module included in an electronic device of FIG.

[0040] Figures 16 to 25 are diagrams illustrating various examples of electronic devices to which the image acquisition device according to the exemplary embodiment is applied. DETAILED DESCRIPTION

[0041] Example embodiments are described in more detail below with reference to the accompanying drawings.

[0042] In the following description, similar reference numerals are used for similar elements even in different drawings. The content defined in the description (e.g., detailed configuration and elements) is provided to help fully understand the exemplary embodiments. However, it should be clear that the exemplary embodiments can be practiced even in the absence of these specifically defined contents. In addition, since well-known functions or configurations would obscure the description with unnecessary details, they are not described in detail.

[0043] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. Expressions such as "at least one of..." when following a list of elements modify the entire list of elements rather than modifying the individual elements in the list.

[0044] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. The embodiments described below are merely examples, and it should be understood that the embodiments may be modified in various forms. Throughout the accompanying drawings, similar reference numerals refer to similar elements. In the accompanying drawings, the sizes of the constituent elements may be exaggerated for clarity.

[0045] For example, when an element is referred to as being “on” or “over” another element, it can be directly on the other element, or intervening elements may also be present.

[0046] It should be understood that although the terms "first", "second", etc. can be used in this article to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. These terms do not limit the material or structure of the component.

[0047] As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. In addition, it should be understood that when a unit is referred to as "comprising" another element, it does not exclude the possibility that one or more other elements may exist or may be added.

[0048] In addition, the terms "... device", "... machine" and "module" described in the specification refer to a unit for processing at least one function and / or operation, and can be implemented by hardware components or software components and a combination thereof.

[0049] Use of the terms "a", "an" and "the" and similar referential terms should be construed to cover both the singular and the plural.

[0050] The operations constituting the method may be performed in any suitable order, unless explicitly stated that they should be performed in the described order. In addition, the use of any and all examples or exemplary language (e.g., "such as (for example)") provided herein is intended only to better illustrate the inventive concept and does not impose limitations on the scope of the present disclosure, unless otherwise stated.

[0051] Figure 1 is a block diagram of a schematic structure of an image acquisition apparatus according to an example embodiment, and Figure 2 is a color coordinate diagram illustrating a color gamut indicated by an image obtained by an image acquisition apparatus according to example embodiments.

[0052] The image acquisition device 1000 includes: a first image sensor 100, which acquires a first image IM1 based on a first wavelength band; a second image sensor 200, which acquires a second image IM2 based on a second wavelength band wider than the first wavelength band; and a processor 500, which performs signal processing on the first image IM1 and the second image IM2 and forms a third image IM3. The first image sensor 100 can detect and process light of the first wavelength band to obtain the first image IM1, and the second image sensor 200 can detect and process light of the second wavelength band to obtain the second image IM2.

[0053] The first image sensor 100 is a sensor used in an RGB camera and may be a CMOS image sensor using a Bayer color filter array. The first image IM1 obtained by the first image sensor 100 may be an RGB image based on red, green, and blue and may have a color gamut range obtained by an RGB camera. For example, the first image IM1 may have a standard RGB (s-RGB) color gamut, a BT.709 color gamut, a DCI-P3 color gamut, or an Adobe RGB color gamut. Figure 2 The BT.709 color gamut and the DCI-P3 color gamut are exemplarily shown.

[0054] The second image sensor 200 is a sensor that senses light of more wavelengths than the first image sensor 100. The second image sensor 200 may use, for example, 16 channels or 31 channels or other number of channels. The width of each channel is set to be smaller than the R, G and B bands, and the total bandwidth including the bandwidths of all channels includes the RGB bandwidth (i.e., the visible light bandwidth), and may be wider than the RGB bandwidth. For example, the total bandwidth may have a bandwidth of about 350nm to about 1000nm. The second image IM2 obtained by the second image sensor 200 may be a hyperspectral image, may include a band larger than the RGB band (e.g., the visible light band), and may be a wavelength-based image in which the ultraviolet to infrared band wider than the visible light band is divided into 16 or more channels. The second image IM2 may be an image obtained by using all available channels of the second image sensor 200, or may be an image obtained by selecting a specific channel. The spatial resolution of the second image IM2 may be lower than the spatial resolution of the first image IM1, but is not limited thereto.

[0055] The first image sensor 100 and the second image sensor 200 may be configured as separate chips or a single chip in which the first image sensor 100 and the second image sensor 200 are stacked up and down via a three-dimensional (3D) stacking technology.

[0056] The third image IM3 is an image having a spatial resolution corresponding to the first image IM1 of the first image sensor 100 and a color gamut corresponding to the second image IM2 of the second image sensor 200. The third image IM3 has the same spatial resolution as that of the first image IM1 and a larger color gamut, for example, a color gamut greater than or equal to . Figure 2 The quality of the formed third image IM3 can be evaluated by calculating the color difference with the first image IM1.

[0057] The third image IM3 can be obtained by correcting the pixel values ​​of the first image IM1 with reference to or based on the pixel values ​​of the second image IM2. To this end, the first image IM1 and the second image IM2 can be image-registered on a two-dimensional plane, and the correspondence and / or spatial relationship between the pixels of the first image IM1 and the pixels of the second image IM2 can be identified. Next, it is determined whether each corresponding pixel needs color conversion, and the pixel value of the first image IM1 can be corrected accordingly.

[0058] Prior to image registration between the first image IM1 and the second image IM2, phase difference correction may be performed on the first image IM1 and the second image IM2. Phase difference (e.g., distortion, geometric phase difference, and chromatic aberration) may occur due to imaging lenses that form optical images on the first image sensor 100 and the second image sensor 200, and image registration may be performed after correcting the phase difference.

[0059] The image acquisition device 1000 may further include an image output unit 700. The image output unit 700 may output a third image IM3, or selectively output one of the third image IM3 or the first image IM1. In addition, additional information as to whether the output image is a normal RGB image (e.g., the first image IM1) or an image with an expanded color gamut (e.g., the third image IM3) may also be output. The image output unit 700 may also output a difference image of the difference between the third image IM3 and the first image IM1, or may output the first image IM1 and the difference image together. At this time, it is determined whether color conversion has been performed for each region, and only the difference image of the region where color conversion has been performed may be output. Such selective output may be determined by user input.

[0060] The image processing process of processor 500 will be described in more detail below.

[0061] The image registration unit 520 can perform image registration of the first image IM1 and the second image IM2 by using the relative position information between the first image sensor 100 and the second image sensor 200. The image registration unit 520 identifies the positional relationship between each pixel of the image based on the spatial resolution of the image obtained by the first image sensor 100 and the second image sensor 200, and the field of view, focal length, etc. of the optical system used to obtain the image. At this time, based on the image of one sensor, the image of another sensor can be overlaid on it. For example, pixels in the second image IM2 corresponding to the pixels of the first image IM1 can be found based on the first image IM1 obtained by the first image sensor 100. To this end, scaling, translation, rotation, affine transformation, perspective transformation, etc. can be performed on the pixels of the second image IM2. One or more pixels corresponding to the pixels of the first image IM1 can be stored in the second image IM2, and the pixel values ​​corresponding to the pixels of the first image in the second image IM2 can be generated by mixing the pixels of the second image IM2 at a specific ratio according to the position. Image registration can also be performed using the image of each channel of the second image IM2. In order to improve the accuracy of registration, registration can be performed in units of sub-pixels. In sub-pixel registration, pixel positions can be represented by real numbers instead of integers.

[0062] The image registration unit 520 can also improve the registration efficiency by making the first image sensor 100 and the second image sensor 200 focus the object at the same position through focus control. In addition, by making the two sensors have the same field of view, image registration can be performed quickly and accurately. For example, when the imaging optical systems used to form optical images on the first image sensor 100 and the second image sensor 200 have the same focal length and the same field of view, there is only a translation between the first image IM1 and the second image IM2, and the relative position between the first image sensor 100 and the second image sensor 200 and the focal length of each optical system can be used to calculate the relevant parameters.

[0063] When the spatial resolution of the second image IM2 is higher than that of the first image IM1, image registration can be performed by downsampling the second image IM2. In this case, image registration can be performed more accurately by downsampling using filtering that considers edge information (such as bilateral filtering and guided filtering).

[0064] When the spatial resolution of the second image IM2 is lower than that of the first image IMI, a second image sample corresponding to each pixel position of the first image can be generated for each channel by interpolation. Similarly, interpolation can be performed by considering edge information (eg, bilateral filtering and guided filtering).

[0065] Alternatively, image registration may be performed after making the spatial resolution of the second image IM2 equal to the spatial resolution of the first image IM1. De-mosaicing may be performed to make the two images have the same resolution. In this case, when the two optical systems that form the optical images on the two image sensors, respectively, have the same focal length and field of view, image registration may be performed considering only the translation without additional interpolation.

[0066] Before performing registration, phase difference in the first image IM1 and the second image IM2 may be corrected. That is, registration may be performed after correcting the influence of distortion, geometric phase difference, chromatic aberration, etc. caused by lenses included in the imaging optical system for obtaining the first image IM1 and the second image IM2.

[0067] The image registration unit 520 can detect edges in the image where the brightness or intensity changes significantly. The image registration unit 520 can extract edge feature information in the image and match features (e.g., edges) in the two images. Because color distortion may occur when the image registration is mismatched in the boundary area of ​​the object, distortion in the boundary area can be prevented by extracting edge information and performing registration to align the edges in the two images. In addition to edges, other image features (e.g., corners) can be used to perform image registration.

[0068] For example, the image registration unit 520 may detect the boundary of an object where the image brightness in the image changes sharply as an edge. When there are points in the image with discontinuity in depth, discontinuity in surface orientation, changes in material properties, and / or changes in scene illumination, such points may be identified as edges corresponding to the boundary of the object. The image registration unit 520 may obtain position information (e.g., x-coordinates and y-coordinates) of the edges detected from the first image IM1 and the second image IM2, and may register the first image IM1 with the second image IM2 based on the position information of the edges.

[0069] The color conversion unit 540 may use the pixel values ​​included in the second image IM2 to perform linear transformation using the first image IM1. When the number of channels of the second image sensor 200 is n, the linear transformation may be performed using a 3×n matrix. In other words,

[0070] v'=Lv,

[0071] Wherein v is an n-dimensional vector consisting of pixel values ​​of the second image, L is a 3×n matrix, and v′ is a 3-dimensional vector indicating R, G, B values.

[0072] Alternatively, the color mapping may be performed by converting v to an XYZ coordinate space and then converting it back to an RGB color space. Alternatively, instead of the aforementioned linear transformation, a more complex transformation may be used that, for example, takes into account cross-channels or quadratic terms. Alternatively, a nonlinear transformation method may be used.

[0073] During color mapping, it is determined whether each area requires color mapping, and color mapping with an expanded color gamut may be performed only on the required areas. For example, after the image area is divided into a plurality of rectangular blocks, when the color gamut indicated by each block of the second image is within the color gamut indicated by the first image, color mapping may not be performed on the area. When the second image IM2 is outside the color gamut indicated by the first image IM1 in the corresponding area, color mapping is performed on the corresponding area. In order to display a wide color gamut (WCG) image (third image) with an expanded color gamut, the image may be stored using a bit depth greater than that of a general RGB image.

[0074] In an example embodiment, the first image sensor 100 and the second image sensor 200 may be implemented by an RGB sensor and a hyperspectral image (HSI) sensor, respectively. The HSI sensor may be configured to process light of a wider bandwidth than that to be processed by the RGB sensor, and may have a sensing accuracy higher than that of the RGB sensor. The image acquisition device 1000 may include: an RGB sensor and an HSI sensor; and a processor configured to combine the first image IM1 and the second image IM2 to obtain an image with a wide color gamut. The image acquisition device 1000 may be referred to as a wide color gamut (WCG) sensor.

[0075] Figure 3 is a conceptual diagram of a schematic structure of an image acquisition device according to an exemplary embodiment, and Figure 4 is a diagram of a circuit configuration of a first image sensor and a second image sensor provided in an image acquisition apparatus according to example embodiments.

[0076] The image acquisition device 1000 includes: a first image sensor 100, which acquires a first image IM1 based on a first band; a second image sensor 200, which acquires a second image IM2 based on a second band wider than the first band; and a processor 500, which performs signal processing on the first image IM1 and the second image IM2 and forms a third image IM3. The image acquisition device 1000 may also include: a memory 300, which stores data related to the first image IM1 and the second image IM2; and an image output unit 700 that outputs an image.

[0077] The image acquisition device 1000 may further include: a first imaging optical system 190 that forms an optical image of the object OBJ on the first image sensor 100; and a second imaging optical system 290 that forms an optical image of the object OBJ on the second image sensor 200. Although each of the first imaging optical system 190 and the second imaging optical system 290 is shown as including one lens, this is illustrative and not limited to this. The first imaging optical system 190 and the second imaging optical system 290 may be configured to have the same focal length and the same field of view. In this case, the process of aligning the first image IM1 and the second image IM2 to form the third image IM3 may be simpler. However, the present disclosure is not limited to this.

[0078] The first image sensor 100 includes a first pixel array PA1, and the first pixel array PA1 includes: a first sensor layer 110, in which a plurality of first sensing elements form an array; and a color filter 120 arranged on the first sensor layer 110. The color filter 120 may include a red filter, a green filter, and a blue filter arranged alternately. A first micro lens array 130 may be on the first pixel array PA1. Figures 5 to 8 Various examples of pixel arrangement applied to the first pixel array PA1 are described.

[0079] The second image sensor 200 includes a second pixel array PA2, and the second pixel array PA2 includes: a second sensor layer 210, in which a plurality of second sensing elements form an array; and a spectral filter 220 arranged on the second sensor layer 210. The spectral filter 220 includes a plurality of filter groups, and each of the plurality of filter groups may include a plurality of unit filters having different transmission bands. The spectral filter 220 may be configured to filter light having a wider band than the band of the color filter 120 in a more subdivided manner than the color filter 120, wherein the band of the light is, for example, a band in the ultraviolet wavelength or infrared wavelength range. The first microlens array 230 may be on the second pixel array PA2. Referring to FIG. Figures 10 to 12 An example of a pixel arrangement applied to the second pixel array PA2 is described.

[0080] The first sensor layer 110 and the second sensor layer 210 may include, but are not limited to, a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor.

[0081] The first pixel array PA1 and the second pixel array PA2 may be horizontally arranged on the same circuit board SU, for example, spaced apart from each other in the X direction.

[0082] The circuit board SU may include a first circuit element processing a signal from the first sensor layer 110 and a second circuit element processing a signal from the second sensor layer 210. However, the present disclosure is not limited thereto, and the first circuit element and the second circuit element may be respectively disposed on separate substrates.

[0083] Although the memory 300 storing the data of the first image IM1 and the second image IM2 is shown as being separated from the circuit board SU, this is only an example, and the memory 300 may be arranged on the same layer as the circuit elements in the circuit board SU or arranged as a separate layer. The memory 300 may be a line memory storing images line by line or a frame buffer storing the entire image. The memory 300 may be a static random access memory (SRAM) or a dynamic random access memory (DRAM). The line memory may be implemented by a line buffer included in a DRAM chip.

[0084] Various circuit elements required by the image acquisition device 1000 may be integrated and arranged on the circuit board SU. For example, a logic layer including various analog circuits and digital circuits may be provided, and a memory layer for storing data may be provided. The logic layer and the memory layer may be configured as different layers or the same layer.

[0085] refer to Figure 4 , a row decoder 102, an output circuit 103, and a timing controller (TC) 101 are connected to the first pixel array PA1. The row decoder 102 selects and activates one of the rows of the first pixel array PA1 in response to a row address signal output from the timing controller 101. The row decoder 102 can select and activate multiple rows row by row, one row at a time. The output circuit 103 outputs photoelectric sensing signals from multiple pixels arranged along the selected row in columns. To this end, the output circuit 103 may include a column decoder and an analog-to-digital converter (ADC). For example, the output circuit 103 may include: a plurality of ADCs arranged for each column between the column decoder and the first pixel array PA1, or an ADC arranged at the output end of the column decoder. The timing controller 101, the row decoder 102, and the output circuit 103 may be implemented as one chip or separate chips. At least some of the circuit elements shown may be arranged in Figure 3 The processor for processing the first image IM1 output through the output circuit 103 may be implemented as a single chip together with the timing controller 101, the row decoder 102, and the output circuit 103.

[0086] The row decoder 202, the output circuit 203 and the timing controller (TC) 201 are also connected to the second pixel array PA2, and similarly to the above, the signal from the second pixel array PA2 can be processed. In addition, the processor for processing the second image IM2 output from the output circuit 203 can be implemented as a single chip together with the TC 201, the row decoder 202 and the output circuit 203.

[0087] Although the first pixel array PA1 and the second pixel array PA2 are illustrated as having the same pixel size and number in the drawings, this is an example and is not limited thereto.

[0088] When operating two different types of sensors, timing control may be required depending on the different resolutions and output speeds, as well as the size of the area required for image registration. For example, when reading an image column based on the first image sensor 100, the image column corresponding to the area in the second image sensor 200 may have been stored in the buffer or may need to be read a new one. These timings need to be calculated correctly and read out. Alternatively, the operation of the first image sensor 100 and the second image sensor 200 can be synchronized using the same synchronization signal. For example, the timing controller (TC) 400 can also be configured to send a synchronization signal sync to the first image sensor 100 and the second image sensor 200.

[0089] Figure 5 is a graph of a wavelength spectrum of a first image sensor provided in an image acquisition device according to example embodiments, and Figures 6 to 8 is a diagram of an exemplary pixel arrangement of a first image sensor provided in an image acquisition device according to example embodiments.

[0090] refer to Figure 6 In the color filter 120 provided in the first pixel array PA1, filters for filtering red (R), green (G), and blue (B) bands are arranged in a Bayer pattern. That is, one unit pixel includes sub-pixels arranged in a 2×2 array, and a plurality of unit pixels are repeatedly arranged two-dimensionally. A red color filter and a green color filter are arranged in one row of the unit pixel, and a green color filter and a blue color filter are arranged in a second row. The pixel arrangement may be implemented in other ways than the Bayer pattern.

[0091] For example, refer to Figure 7 , a CYGM arrangement can also be implemented, in which a magenta pixel (M), a cyan pixel (C), a yellow pixel (Y), and a green pixel (G) constitute a unit pixel. Figure 8, an RGBW arrangement may also be implemented, in which a green pixel (G), a red pixel (R), a blue pixel (B), and a white pixel (W) constitute a unit pixel. In addition, although not shown, the unit pixel may have a 3×2 array form. In addition, the pixels of the first pixel array PA1 may be arranged in various ways according to the color characteristics of the first image sensor 100.

[0092] Fig. 9 is a graph of a wavelength spectrum of a second image sensor provided in an image acquisition device according to example embodiments, and Figures 10 to 12 is a diagram of an exemplary pixel arrangement of a second image sensor provided in an image acquisition device according to example embodiments.

[0093] refer to Fig.10 , the spectral filter 220 provided in the second pixel array PA2 may include a plurality of filter groups 221 arranged in a two-dimensional form. Each filter group 221 may include 16 unit filters F1 to F16 arranged in a 4×4 array.

[0094] The first unit filter F1 and the second unit filter F2 may have center wavelengths UV1 and UV2 in the ultraviolet band, and the third to fifth unit filters F3 to F5 may have center wavelengths B1 to B3 in the blue light band. The sixth to eleventh unit filters F6 to F11 may have center wavelengths G1 to G6 in the green light band, and the twelfth to fourteenth unit filters F12 to F14 may have center wavelengths R1 to R3 in the red light band. In addition, the fifteenth and sixteenth unit filters F15 and F16 may have center wavelengths NIR1 and NIR2 in the near infrared band.

[0095] Fig.11 FIG. 2 is a plan view of another example of a filter group 222 provided in the spectral filter 220. Fig.11 , the filter group 222 may include nine unit filters F1 to F9 arranged in a 3×3 array. The first unit filter F1 and the second unit filter F2 may have center wavelengths UV1 and UV2 of the ultraviolet band, and the fourth unit filter F4, the fifth unit filter F5, and the seventh unit filter F7 may have center wavelengths B1 to B3 of the blue light band. The third unit filter F3 and the sixth unit filter F6 may have center wavelengths G1 and G2 of the green light band, and the eighth unit filter F8 and the ninth unit filter F9 may have center wavelengths R1 and R2 of the red light band.

[0096] Fig.12 FIG. 2 is a plan view of another example of a filter group 223 provided in the spectral filter 220. Fig.12, the filter group 223 may include 25 unit filters F1 to F25 arranged in a 5×5 array. The first unit filter F1 to the third unit filter F3 may have central wavelengths UV1 to UV3 in the ultraviolet band, and the sixth unit filter F6, the seventh unit filter F7, the eighth unit filter F8, the eleventh unit filter F11, and the twelfth unit filter F12 may have central wavelengths B1 to B5 in the blue light band. The fourth unit filter F4, the fifth unit filter F5, and the ninth unit filter F9 may have central wavelengths G1 to G3 in the green light band, and the tenth unit filter F10, the thirteenth unit filter F13, the fourteenth unit filter F14, the fifteenth unit filter F15, the eighteenth unit filter F18, and the nineteenth unit filter F19 may have central wavelengths R1 to R6 in the red light band. In addition, the twentieth unit filter F20, the twenty-third unit filter F23, the twenty-fourth unit filter F24, and the twenty-fifth unit filter F25 may have central wavelengths NIR1 to NIR4 in the near infrared band.

[0097] The above-mentioned unit filter provided in the spectral filter 220 may have a resonant structure including two reflectors, and the transmitted waveband may be determined according to the characteristics of the resonant structure. The transmission waveband may be adjusted according to the material of the reflector, the material of the dielectric material in the cavity, and the thickness of the cavity. In addition, a structure using a grating, a structure using a distributed Bragg emitter (DBR), etc. may be applied to the unit filter.

[0098] In addition, the pixels of the second pixel array PA2 may be arranged in various ways according to color characteristics of the second image sensor 200 .

[0099] Fig.13 is a flowchart illustrating an image processing procedure of an image acquisition apparatus according to an example embodiment.

[0100] In operation S810 , a first image is obtained from the first image sensor 100 , and in operation S820 , a second image is obtained from the second image sensor 200 .

[0101] In operation S830, if necessary, phase difference correction may be performed on the first image and / or the second image. For example, the registration may be performed after correcting the influence of distortion, geometric phase difference, chromatic aberration, etc. caused by a lens included in an imaging optical system for obtaining the first image IM1 and the second image IM2.

[0102] In operation S840, the first image and the second image are registered. In operation S842, image registration of the first image and the second image is performed based on the resolutions of the first image and the second image. For example, when the spatial resolution of the second image IM2 is higher than the spatial resolution of the first image IM1, the image registration can be performed by downsampling the second image IM2. When the spatial resolution of the second image IM2 is lower than the spatial resolution of the first image IM1, a second image sample corresponding to each pixel position of the first image can be generated for each channel by interpolation. Edge information such as bilateral filtering and guided filtering can be considered during image registration.

[0103] Alternatively, in operation S846, image registration may be performed after adjusting the first image and the second image to the same resolution. De-mosaicing may be performed to make the resolutions of the first image and the second image the same. In this case, when two optical systems that form optical images on the two image sensors, respectively, have the same focal length and field of view, image registration may be performed considering only translation without additional interpolation.

[0104] In the color conversion operation S860, the pixel values ​​of the first image are corrected with reference to or based on the pixel values ​​of the second image, and a third image is formed. In this case, it can be determined whether each area needs color conversion, and color conversion to expand the color gamut can be performed only on the required area. For example, after the image area is divided into a plurality of blocks, it is determined for each block whether the color gamut indicated by the second image is within the color gamut indicated by the first image. Only when the second image IM2 is outside the color gamut indicated by the first image IM1 in the corresponding area, color conversion can be performed on the corresponding area.

[0105] In the image output operation S870, any one or more of the first image, the third image, and the difference image between the first image and the third image may be selectively output. Additional information related to the image may also be output.

[0106] The image acquisition device 1000 can be used in various high-performance optical devices or high-performance electronic devices. The electronic device can be, for example, a smart phone, a mobile phone, a cellular phone, a personal digital assistant (PDA), a laptop computer, a personal computer (PC), various portable devices, home appliances, security cameras, medical cameras, automobiles, Internet of Things (IoT) devices, or other mobile or non-mobile computing devices, but is not limited thereto.

[0107] In addition to the image acquisition device 1000, the electronic device may further include a processor (e.g., an application processor (AP)) for controlling an image sensor (which is provided in the processor), and a plurality of hardware or software components may be controlled by driving an operating system or an application program through the processor, and various data processing and operations may be performed. The processor may further include a graphics processing unit (GPU) and / or an image signal processor. When the processor includes an image signal processor, the image (or video) obtained by the image sensor may be stored and / or output using the processor.

[0108] Fig.14 is a block diagram of a schematic structure of an electronic device according to an example embodiment. Fig.14 In the network environment ED00, the electronic device ED01 can communicate with another electronic device ED02 through a first network ED98 (near field wireless communication network, etc.), or can communicate with another electronic device ED04 and / or a server ED08 through a second network ED99 (far field wireless communication network, etc.). The electronic device ED01 can communicate with the electronic device ED04 through the server ED08. The electronic device ED01 may include a processor ED20, a memory ED30, an input device ED50, a sound output device ED55, a display device ED60, an audio module ED70, a sensor module ED76, an interface ED77, a tactile module ED79, a camera module ED80, a power management module ED88, a battery ED89, a communication module ED90, a user identification module ED96 and / or an antenna module ED97. In the electronic device ED01, some of the components (display device ED60, etc.) may be omitted or other components may be added. Some of these components may be implemented in one integrated circuit. For example, the sensor module ED76 (fingerprint sensor, iris sensor, illumination sensor, etc.) may be implemented by being embedded in the display device ED60 (display, etc.).

[0109] The processor ED20 can execute software (program ED40, etc.) to control one or more other components (hardware or software components, etc.) connected to the processor ED20 in the electronic device ED01, and can perform various data processing or operations. As part of the data processing or operation, the processor ED20 can load instructions and / or data received from other components (sensor module ED76, communication module ED90, etc.) into the volatile memory ED32, can process the instructions and / or data stored in the volatile memory ED32, and can store the resulting data in the non-volatile memory ED34. The processor ED20 may include a main processor ED21 (central processing unit, application processor, etc.) and an auxiliary processor ED23 (GPU, image signal processor, sensor hub processor, communication processor, etc.) that can operate independently or together. The auxiliary processor ED23 uses less power than the main processor ED21 and can perform specialized functions.

[0110] The auxiliary processor ED23 can control functions and / or states related to some components of the electronic device ED01 (e.g., display device ED60, sensor module ED76, communication module ED90, etc.) on behalf of the main processor ED21 when the main processor ED21 is in an inactive (e.g., sleep) state, or together with the main processor ED21 when the main processor ED21 is in an active (e.g., application execution) state. The auxiliary processor ED23 (image signal processor, communication processor, etc.) can be implemented as part of other functionally related components (camera module ED80, communication module ED90, etc.).

[0111] The memory ED30 can store various data required by the components of the electronic device ED01 (processor ED20, sensor module ED76, etc.). The data may include, for example, input data and / or output data of software (such as program ED40, etc.) and commands related thereto. The memory ED30 may include a volatile memory ED32 and / or a non-volatile memory ED34. The non-volatile memory ED32 may include an internal memory ED36 fixed in the electronic device ED01 and a removable external memory ED38.

[0112] The program ED40 may be stored as software in the memory ED30 , and may include an operating system ED42 , middleware ED44 , and / or applications ED46 .

[0113] The input device ED50 may receive commands and / or data from outside the electronic device ED01 (user, etc.) to be used for components (processor ED20, etc.) of the electronic device ED01. The input device ED50 may include a microphone, a mouse, a keyboard, and / or a digital pen (stylus, etc.).

[0114] The audio output device ED55 can output audio signals to the outside of the electronic device ED01. The audio output device ED55 can include a speaker and / or an earpiece. The speaker can be used for general purposes such as multimedia playback or recording playback, and the earpiece can be used to receive incoming calls. The earpiece can be combined as part of the speaker or can be implemented as a separate device.

[0115] The display device ED60 can visually provide information to the user of the electronic device ED01. The display device ED60 may include a display, a holographic device or a projector, and a control circuit for controlling the device. The display device ED60 may include a touch circuit configured to sense a touch and / or a sensor circuit (pressure sensor, etc.) configured to measure the strength of the force generated by the touch.

[0116] The audio module ED70 can convert sound into an electrical signal and vice versa. The audio module ED70 can obtain sound through the input device ED50, or can output sound through the sound output device ED55 and / or a speaker and / or earphone of another electronic device (electronic device ED02, etc.) directly or wirelessly connected to the electronic device ED01.

[0117] The sensor module ED76 can detect the operating state (power, temperature, etc.) or external environment state (user state, etc.) of the electronic device ED01, and can generate an electrical signal and / or data value corresponding to the detected state. The sensor module ED76 can include a gesture sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, and / or an illumination sensor.

[0118] Interface ED77 can support one or more specified protocols that can be used to connect electronic device ED01 directly or wirelessly with other electronic devices (electronic device ED02). Interface ED77 can include a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface and / or an audio interface.

[0119] The connection terminal ED78 may include a connector through which the electronic device ED01 can be physically connected to other electronic devices (electronic device ED02, etc.) The connection terminal ED78 may include an HDMI connector, a USB connector, an SD card connector, and / or an audio connector (headphone connector, etc.).

[0120] The haptic module ED79 may convert the electrical signal into mechanical stimulation (vibration, motion, etc.) or electrical stimulation that the user may sense through tactile or kinesthetic sense. The haptic module ED79 may include a motor, a piezoelectric element, and / or an electrical stimulation device.

[0121] The camera module ED80 can capture both static and moving images. The camera module ED80 may include the above-mentioned image acquisition device 1000, and may include an additional lens assembly, an image signal processor, and / or a flash. The lens assembly included in the camera module ED80 may collect light emitted from an object to be imaged.

[0122] The power management module ED88 may manage power supplied to the electronic device ED01. The power management module ED88 may be implemented as a part of a power management integrated circuit PMIC.

[0123] The battery ED89 may supply power to the components of the electronic device ED01. The battery ED89 may include a non-rechargeable main battery, a rechargeable auxiliary battery, and / or a fuel cell.

[0124] The communication module ED90 can support the establishment of a direct (wired) communication channel and / or a wireless communication channel between the electronic device ED01 and other electronic devices (electronic device ED02, electronic device ED04 or server ED08, etc.), and communicate through the established communication channel. The communication module ED90 can operate independently of the processor ED20 (application processor, etc.), and can include one or more communication processors that support direct communication and / or wireless communication. The communication module ED90 may include a wireless communication module ED92 (cellular communication module, short-range wireless communication module, global navigation satellite system (GNSS)), etc.) and / or a wired communication module ED94 (local area network (LAN) communication module, power line communication module, etc.). The corresponding communication module among these communication modules can communicate with other electronic devices through a first network ED98 (a local area network such as Bluetooth, WiFi direct connection or infrared data association (IrDA)) or a second network ED99 (a communication network such as a cellular network, the Internet or a computer network (LAN, WAN, etc.)). These various types of communication modules can be integrated into a single component (a single chip, etc.), or can be implemented as multiple separate components (multiple chips). The wireless communication module ED92 can identify and authenticate the electronic device ED01 in a communication network (eg, the first network ED98 and / or the second network ED99 ) using user information (International Mobile Subscriber Identifier (IMSI) etc.) stored in the user identification module ED96 .

[0125] The antenna module ED97 can send signals and / or power to the outside (other electronic devices, etc.) or receive signals and / or power from it. The antenna may include a radiator implemented by a conductive pattern formed on a substrate (PCB, etc.). The antenna module ED97 may include one or more antennas. When multiple antennas are included, the communication module ED90 can select an antenna suitable for a communication method used in a communication network such as a first network ED98 and / or a second network ED99 from the multiple antennas. Signals and / or power can be transmitted or received between the communication module ED90 and other electronic devices through the selected antenna. Other components (RFIC, etc.) besides the antenna may be included as part of the antenna module ED97.

[0126] Some of the components may be connected to each other and exchange signals (commands, data, etc.) through a communication method between peripheral devices (bus, general purpose input and output (GPIO), serial peripheral bus (SPI), mobile industry processor interface (MIPI), etc.).

[0127] Commands or data can be sent or received between the electronic device ED01 and the external electronic device ED04 through the server ED08 connected to the second network ED99. The other electronic devices ED02 and ED04 may be the same as or different from the electronic device ED01. All or some operations performed by the electronic device ED01 may be performed by one or more of the other electronic devices ED02, ED04 and ED08. For example, when the electronic device ED01 needs to perform a specific function or service, the electronic device ED01 may request one or more other electronic devices to perform some or all of the functions or services instead of directly performing the functions or services. One or more other electronic devices that have received the request may perform additional functions or services related to the request, and may transmit the results of the execution to the electronic device ED01. To this end, cloud computing, distributed computing and / or client-server computing technologies may be used.

[0128] Fig.15 yes Fig.14 The camera module ED80 is a block diagram of a camera module ED80 included in an electronic device. The camera module ED80 may include the above-mentioned image acquisition device 1000, or may have a structure modified therefrom. Fig.15 The camera module ED80 may include a lens assembly CM10, a flash CM20, an image sensor CM30, an image stabilizer CM40, a memory CM50 (buffer memory, etc.), and / or an image signal processor CM60.

[0129] The image sensor CM30 may include a first image sensor 100 and a second image sensor 200 provided in the above-mentioned image acquisition device 1000. The first image sensor 100 and the second image sensor 200 may obtain an image corresponding to an object by converting light emitted or reflected from the object and transmitted through the lens assembly CM10 into an electrical signal. The first image sensor 100 may obtain an RGB image, and the second image sensor 200 may obtain a hyperspectral image in a wavelength range from ultraviolet to infrared.

[0130] In addition to the above-mentioned first image sensor 100 and second image sensor 200, the image sensor CM30 may also include one or more sensors selected from image sensors having different properties, such as other RGB sensors, black and white (BW) sensors, IR sensors, or UV sensors. Each of the sensors included in the image sensor CM30 may be implemented as a charge coupled device (CCD) sensor and / or a complementary metal oxide semiconductor (CMOS) sensor.

[0131] The lens assembly CM10 can collect light emitted from the object to be imaged. The camera module ED80 may include a plurality of lens assemblies CM10, and in this case, may be a dual camera, a 360-degree camera, or a spherical camera. Some of the plurality of lens assemblies CM10 may have the same lens properties (field of view, focal length, autofocus, F number, optical zoom, etc.) or different lens properties. The lens assembly CM10 may include a wide-angle lens or a telephoto lens.

[0132] The lens assembly CM10 may be configured to allow two image sensors included in the image sensor CM30 to form an optical image of a subject at the same position and / or may control a focus.

[0133] The flash CM20 may emit light for enhancing light emitted or reflected from an object. The flash CM20 may include one or more light emitting diodes (red-green-blue (RGB) LEDs, white LEDs, infrared LEDs, ultraviolet LEDs, etc.) and / or xenon lamps.

[0134] The image stabilizer CM40 may move one or more lenses included in the lens assembly CM10 or the image sensor CM30 in a specific direction in response to the movement of the camera module ED80 or the electronic device CM01 including the camera module ED80, or may control the operating characteristics of the image sensor CM30 (adjust the readout timing, etc.) to compensate for the negative effects caused by the movement. The image stabilizer CM40 may detect the movement of the camera module ED80 or the electronic device ED01 using a gyro sensor (not shown) or an acceleration sensor (not shown) arranged inside or outside the camera module ED80. The image stabilizer CM40 may be implemented in an optical form.

[0135] In the memory CM50, some or all of the data obtained by the image sensor 1000 may be stored for the next image processing operation. For example, when a plurality of images are obtained at high speed, the obtained raw data (Bayer pattern data, high-resolution data, etc.) may be stored in the memory CM50, and only a low-resolution image may be displayed, and then the raw data of the selected (user selected, etc.) image may be transmitted to the image signal processor CM60. The memory CM50 may be integrated into the memory ED30 of the electronic device ED01 or may be configured as a separate memory that operates independently.

[0136] The image signal processor CM60 may perform one or more image processing operations on the image obtained by the image sensor CM30 or the image data stored in the memory CM50. Figures 1 to 13 As described above, the first image (eg, RGB image) and the second image (eg, hyperspectral image) obtained by the two image sensors included in the image sensor CM30 are processed to form a third image with an expanded color gamut. The configuration of the processor 500 for this purpose may be included in the image signal processor CM60.

[0137] In addition, one or more image processing may include depth map generation, three-dimensional modeling, panorama generation, feature point extraction, image synthesis and / or image compensation (noise reduction, resolution adjustment, brightness adjustment, blur, sharpening, softening, etc.). The image signal processor CM60 may control (exposure time control or readout timing control, etc.) the components (image sensor CM30, etc.) included in the camera module ED80. The image processed by the image signal processor CM60 may be stored again in the memory CM50 for further processing, or may be provided to external components (memory ED30, display device ED60, electronic device ED02, electronic device ED04, server ED08, etc.) of the camera module ED80. The image signal processor CM60 may be integrated into the processor ED20 or may be configured as a separate processor that operates independently of the processor ED20. When the image signal processor CM60 is configured as a processor separate from the processor ED20, the image processed by the image signal processor CM60 may be displayed by the display device ED60 after being further processed by the processor ED20.

[0138] The electronic device ED01 may include a plurality of camera modules ED80 having corresponding properties or functions. In this case, one of the plurality of camera modules ED80 may be a wide-angle camera, and the other camera modules may be telephoto cameras. Similarly, one of the plurality of camera modules ED80 may be a front camera, and the other camera modules may be a rear camera.

[0139] Figures 16 to 25 are diagrams illustrating various examples of electronic devices to which the image acquisition device according to the exemplary embodiment is applied.

[0140] The image acquisition device according to the embodiment can be applied to Fig.16 The mobile phone or smart phone 5100m shown, Fig.17 The tablet computer or smart tablet computer 5200 shown, Fig.18 The digital camera or video recorder 5300 shown, Fig.19 Laptop computer 5400 or Fig. 20 The television or smart television 5500 shown. For example, the smart phone 5100m or the smart tablet computer 5200 may include a plurality of high-resolution cameras each having a high-resolution image sensor installed. The high-resolution camera may be used to extract depth information of an object in an image, adjust the defocus of an image, or automatically recognize an object in an image.

[0141] In addition, the image acquisition device 1000 can be applied to Fig.21 The smart refrigerator 5600 shown, Fig. 22 The security camera 5700 shown, Fig.23 The robot 5800 shown, Fig.24 Medical camera 5900 shown, etc. For example, smart refrigerator 5600 can automatically identify food in the refrigerator using image acquisition device 1000, and can notify the user through a smart phone of the presence of specific food, the type of food put in or transported, etc. Security camera 5700 can provide ultra-high resolution images by using high sensitivity and can identify objects or people in images even in dark environments. High-resolution images can be provided by providing robot 5800 at disaster or industrial sites that cannot be directly accessed by humans. Medical camera 5900 can provide high-resolution images for diagnosis or surgery, and can dynamically adjust the field of view.

[0142] In addition, the image acquisition device 1000 can be applied to Fig.25 The vehicle 6000 shown. The vehicle 6000 may include a plurality of vehicle cameras 6010, 6020, 6030, and 6040 arranged in various positions. Each of the vehicle cameras 6010, 6020, 6030, and 6040 may include an image acquisition device according to an example embodiment. The vehicle 6000 may provide the driver with various information related to the interior or environment of the vehicle 6000 using the plurality of vehicle cameras 6010, 6020, 6030, and 6040, and may provide information required for autonomous driving by automatically recognizing an object or person in an image.

[0143] The above-mentioned image acquisition device can provide an image having a high resolution and representing a wide color gamut by using two different types of image sensors.

[0144] The above-mentioned image acquisition device can be used in various electronic devices.

[0145] The above exemplary embodiments are merely examples and should not be construed as limiting. The present teaching can be easily applied to other types of devices. In addition, the description of the exemplary embodiments is intended to be illustrative rather than to limit the scope of the claims, and a variety of substitutions, modifications, and variations will be apparent to those skilled in the art.

Claims

1. An image acquisition device, comprising: A first image sensor configured to obtain a first image based on detecting light in a first wavelength band; a second image sensor configured to obtain a second image based on detecting light in a second wavelength band wider than the first wavelength band; as well as A processor is configured to obtain a third image having a spatial resolution corresponding to the first image and a color gamut corresponding to the second image based on the first image and the second image by the following operations: determining whether color conversion is required for each of multiple areas of the first image based on the second image, and performing color conversion only on the following areas of the first image: the color gamut indicated by the second image for the area is outside the range of the color gamut indicated by the first image.

2. The image acquisition device according to claim 1, in, The processor is further configured to correct pixel values ​​of the first image based on pixel values ​​of the second image to obtain the third image.

3. The image acquisition device according to claim 2, in, The processor is also configured to register the first image and the second image on a two-dimensional plane, and correct pixel values ​​of the first image based on a correspondence between pixels of the first image and pixels of the second image.

4. The image acquisition device according to claim 3, in, The processor is further configured to register the first image and the second image after performing aberration correction on the first image and the second image.

5. The image acquisition device according to claim 1, wherein: The processor is further configured to determine the third image or the first image as an output image for display.

6. The image acquisition device according to claim 1, in, The processor is also configured to determine the first image and a difference image between the first image and the third image for display.

7. The image acquisition device according to claim 1, in, The first image indicates a first color gamut, and the third image indicates a second color gamut wider than the first color gamut.

8. The image acquisition device according to claim 1, in, The first image sensor comprises: a first pixel array including a first sensor layer in which a plurality of first sensing elements are arranged in an array; and The color filters are arranged on the first sensor layer and include red color filters, green color filters, and blue color filters that are alternately arranged.

9. The image acquisition device according to claim 8, in, The second image sensor comprises: a second pixel array including a second sensor layer in which a plurality of second sensing elements are arranged in an array; and a spectral filter arranged on the second sensor layer, wherein a plurality of filter groups are repeatedly arranged in the spectral filter, and Each of the plurality of filter groups includes a plurality of unit filters having different transmission bands.

10. The image acquisition device according to claim 9, in, Each transmission wavelength band of the plurality of unit filters includes a visible light wavelength band and has a wavelength band wider than the visible light wavelength band.

11. The image acquisition device according to claim 10, in, Each of the plurality of filter banks includes 16 unit filters arranged in a 4×4 array.

12. The image acquisition device according to claim 9, in, The first pixel array and the second pixel array are horizontally spaced apart from each other on the same circuit board to have the same focal length and the same field of view.

13. The image acquisition device according to claim 12, in, The circuit board includes a first circuit element that processes a first signal from the first sensor layer and a second circuit element that processes a second signal from the second sensor layer.

14. The image acquisition device according to claim 13, further comprising: A timing controller is configured to synchronize operations of the first circuit element and the second circuit element. 15 . The image acquisition device according to claim 12 , further comprising a memory, wherein the memory stores data related to the first image and data related to the second image.

16. The image acquisition device according to claim 15, in, The memory is disposed in the circuit board.

17. The image acquisition device according to claim 1, further comprising: a first imaging optical system that forms the first image of the object on the first image sensor and includes one or more first lenses; as well as A second imaging optical system forms the second image of the object on the second image sensor and includes one or more second lenses.

18. The image acquisition device according to claim 17, in, The first imaging optical system and the second imaging optical system are arranged to have the same focal length and the same field of view.

19. An electronic device comprising: an RGB sensor configured to detect a first light of a first wavelength band reflected from an object; a hyperspectral image sensor for detecting second light of a second wavelength band reflected from the object, wherein the second wavelength band is wider than the first wavelength band; as well as The processor is configured as: Obtaining a first image based on a first light of the first wavelength band, and a second image based on a second light of the second wavelength band; aligning the first image with the second image based on position information of edges detected from the first image and the second image; determining whether color conversion needs to be performed on each of a plurality of regions of the first image based on the second image; as well as While the first image is aligned with the second image, the color of the first image is converted by performing color conversion only on a region of the first image for which a color gamut indicated by the second image is outside a range of a color gamut indicated by the first image.

20. The electronic device according to claim 19, wherein: The electronic device corresponds to: Mobile phone, tablet computer, digital camera, laptop computer, television, smart refrigerator or robot.

Images