Image sensor module and image compression method

Abstract

To provide an image sensor module and an image compressing method for compressing pixel data in an isolated region efficiently.SOLUTION: An image compressing method for compressing image data generated by an image sensor according to the present invention includes the steps of receiving pixel values of a plurality of target pixels in a target pixel group to be compressed in the image data, and reference values of a plurality of reference pixels used to compress the target pixel group, generating a virtual reference map by applying an offset value for each of the reference values, compressing the pixel values of the target pixel group on the basis of the virtual reference map, and generating a bit stream on the basis of the compression information and the compression result based on the virtual reference map.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image sensor, and more particularly to an image sensor module and an image compression method for compressing pixel data of an isolated area. [Background technology]

[0002] In recent years, as the demand for high quality photographs, videos, etc. increases, the number of sensing pixels in the pixel array of an image sensor increases, thereby increasing the size of image data generated by the image sensor. The image data is transmitted to an image processing device, and the image data is compressed to improve transmission efficiency, and the compressed image data is transmitted to the image processing device. The image data includes a variety of two-dimensional or multi-dimensional image patterns.

[0003] When compressing pixel data contained in a specific region of an image pattern, it is important to develop a compression method that improves compression efficiency and reduces compression loss. Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention has been made in consideration of the above-mentioned problems with conventional image sensors, and an object of the present invention is to provide an image sensor module and an image compression method that efficiently compress pixel data of isolated areas. [Means for solving the problem]

[0005] In order to achieve the above object, an image compression method according to the present invention is an image compression method for compressing image data generated by an image sensor, the image compression method comprising the steps of: receiving pixel values ​​of a plurality of target pixels of a target pixel group to be compressed in the image data and reference values ​​of a plurality of reference pixels used in compressing the target pixel group; applying an offset value to each of the reference values ​​to generate a virtual reference map; compressing the pixel values ​​of the target pixel group based on the virtual reference map; and generating a bitstream based on compression results and compression information based on the virtual reference map. The reference value corresponds to a restored pixel value of the reference pixel that was compressed prior to the target pixel group, and a difference between the pixel value of the pixel included in the target pixel group and the reference value is equal to or greater than a threshold value. It is characterized by:

[0006] In order to achieve the above object, an image sensor module according to the present invention includes an image sensor that generates image data including a plurality of pixels; an encoder that sequentially compresses the image data generated by the image sensor in units of pixel groups to generate compressed data including a plurality of bit streams, and compresses the target pixel group to be compressed using at least one encoding method among a plurality of encoding methods; and an interface that outputs the compressed data to an external image processing device, wherein the encoder applies offset values ​​to reference values ​​of reference pixels arranged adjacent to the target pixel group using a first encoding method among the plurality of encoding methods to generate a virtual reference map, and compresses the target pixel group based on the virtual reference map. The reference value corresponds to a restored pixel value of the reference pixel that was compressed prior to the target pixel group, and a difference between the pixel value of the pixel included in the target pixel group and the reference value is equal to or greater than a threshold value. It is characterized by:

[0007] An image processing system according to an embodiment of the present invention includes an image sensor that senses received optical signals and generates image data; an encoder that sequentially compresses a plurality of pixel groups of the image data to generate a plurality of bit streams; and a decoder that decompresses the plurality of bit streams to restore the image data. The encoder applies offset values ​​to reference values ​​of reference pixels disposed adjacent to a target pixel group to be compressed to generate a virtual reference map, and compresses the target pixel group based on the virtual reference map. [Effects of the Invention]

[0008] According to the image sensor module and image compression method of the present invention, when there is a large difference between the pixel values ​​of a pixel group to be compressed among a plurality of pixel groups of image data and a reference value used for compression, an offset value is applied to the reference value to generate a virtual reference map, and the target pixel group is compressed based on the virtual reference map, or the compressed data corresponding to the target pixel group is decompressed, thereby improving compression efficiency and reducing compression loss. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a schematic configuration of an image processing system according to an embodiment of the present invention; [Figure 2] 2 is a diagram illustrating an example of a pixel array and image data applied to an image sensor module according to an embodiment of the present invention; [Figure 3A] 1 is a block diagram showing a schematic configuration of an encoder according to an embodiment of the present invention; [Figure 3B] FIG. 10 is a block diagram showing a schematic configuration of an encoder according to another embodiment of the present invention. [Figure 4A] 1 is an exemplary diagram illustrating a reference map and a virtual reference map according to an embodiment of the present invention; [Figure 4B]1 is an exemplary diagram illustrating a reference map and a virtual reference map according to an embodiment of the present invention; [Figure 5A] 1 is a conceptual diagram illustrating a compression method according to an embodiment of the present invention; [Figure 5B] 1 is a conceptual diagram illustrating a compression method according to an embodiment of the present invention; [Figure 6A] FIG. 2 illustrates a bitstream according to one embodiment of the present invention. [Figure 6B] FIG. 2 illustrates a bitstream according to one embodiment of the present invention. [Figure 7] 1 is a conceptual diagram illustrating a compression method according to an embodiment of the present invention; [Figure 8] 1 is an exemplary diagram illustrating image data and a virtual reference map according to one embodiment of the present invention. [Figure 9] 1 is a conceptual diagram illustrating a compression method according to an embodiment of the present invention; [Figure 10] 2 is a flowchart illustrating a method for compressing image data according to an embodiment of the present invention. [Figure 11A] 1 is a block diagram showing a schematic configuration of an image sensor module according to an embodiment of the present invention; [Figure 11B] 1 is a block diagram showing a schematic configuration of an image sensor module according to an embodiment of the present invention; [Figure 12] 1 is a block diagram showing a schematic configuration of a decoder according to an embodiment of the present invention; [Figure 13] 1 is a block diagram showing a schematic configuration of an image processing system according to an embodiment of the present invention; [Figure 14] 1 is a table illustrating compression information according to one embodiment of the present invention. [Figure 15A] 1 is a block diagram showing a schematic configuration of an electronic device including a multi-camera module according to an embodiment of the present invention. [Figure 15B] FIG. 15B is a block diagram showing a detailed configuration of the camera module of FIG. 15A. [Figure 16]1 is a block diagram showing a schematic configuration of an electronic device according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0010] Next, specific examples of embodiments for carrying out the image sensor module and image compression method according to the present invention will be described with reference to the drawings.

[0011] FIG. 1 is a block diagram showing a schematic configuration of an image processing system according to an embodiment of the present invention, and FIG. 2 is a diagram showing an example of a pixel array and image data applied to an image sensor module according to an embodiment of the present invention. The image processing system 10 senses an image of a subject, processes the sensed image or stores the sensed image in memory, and stores the processed image in memory. According to one embodiment of the present invention, the image processing system 10 is embodied in a digital camera, a digital camcorder, a mobile phone, a tablet computer, or any other portable electronic device.

[0012] Portable electronic devices include laptop computers, mobile phones, smartphones, tablet PCs, personal digital assistants (PDAs), enterprise digital assistants (EDAs), digital still cameras, digital video cameras, audio devices, portable multimedia players (PMPs), personal navigation devices (PNDs), MP3 players, handheld game consoles, e-books, wearable devices, and the like. In addition, the image processing system 10 may be installed as a component in electronic devices such as drones and advanced driver assistance systems (ADAS), or in cars, furniture, manufacturing equipment, doors, various measuring instruments, and the like.

[0013] Referring to FIG. 1, an image processing system 10 includes an image sensor module 100 and an image processing device 200. In one embodiment, the image sensor module 100 includes an image sensor 110, an encoder 120, and an interface 130. In one embodiment, the image sensor module 100 is also embodied by multiple semiconductor chips.

[0014] For example, the pixel array (PXA in FIG. 2) of the image sensor 110 is integrated on one semiconductor chip, and the logic circuit of the image sensor 110, the encoder 120, and the interface 130 are integrated on another semiconductor chip, and multiple semiconductor chips are electrically connected to each other through connecting members, or multiple semiconductor chips are stacked and electrically connected to each other through through vias. However, the present invention is not limited to this, and the image sensor module 100 may be implemented as a single semiconductor chip.

[0015] In one embodiment, the image processing device 200 comprises an interface 210 , a memory 220 , a decoder 230 , and an image signal processor 240 . The image sensor module 100 captures an external subject (or object) and generates image data IDT. The image sensor module 100 includes an image sensor 110 that can convert an optical signal of an object incident through a lens LS into an electrical signal. The image sensor 110 has a pixel array (PXA in FIG. 2) in which a plurality of sensing pixels (SPX in FIG. 2) are arranged two-dimensionally, and outputs image data IDT including a plurality of pixel values ​​corresponding to each of the plurality of sensing pixels SPX in the pixel array PXA.

[0016] The pixel array PXA includes a plurality of row lines, a plurality of column lines, and a plurality of sensing pixels SPX each connected to the row lines and column lines and arranged in a matrix. Each of the plurality of sensing pixels SPX of the pixel array PXA senses a light signal of at least one color among a plurality of reference colors. For example, the plurality of reference colors may include red, green, and blue, or red, green, blue, and white, but may also include other colors. For example, the multiple reference colors may include cyan, yellow, green, and magenta. The pixel array PXA generates pixel signals containing information about the reference color of each of the plurality of sensing pixels SPX.

[0017] For example, as shown in FIG. 2, the pixel array PXA includes a red sensing pixel (SPX_R), a blue sensing pixel (SPX_B), and two green sensing pixels (SPX_Gr) and (SPX_Gb). The green sensing pixel located in the same row as the red sensing pixel (SPX_R) is called the first green sensing pixel (PX_Gr), and the green sensing pixel located in the same row as the blue sensing pixel (PX_B) is called the second green sensing pixel (PX_Gb). A red sensing pixel (SPX_R), a blue sensing pixel (SPX_B), a first green sensing pixel (SPX_Gr), and a second green sensing pixel (SPX_Gb) are arranged in a matrix, which is referred to as a pixel pattern PT. Within the pixel array PXA, a plurality of pixel patterns PT are repeatedly arranged.

[0018] For example, as shown in FIG. 2, the pixel pattern PT may include red sensing pixels (SPX_R) arranged in a 2×2 matrix, blue sensing pixels (SPX_B) arranged in a 2×2 matrix, first green sensing pixels (SPX_Gr) arranged in a 2×2 matrix, and second green sensing pixels (SPX_Gb) arranged in a 2×2 matrix. Such a pixel pattern PT is called a tetra pattern.

[0019] However, the technical idea of ​​the present invention is not limited to this, and the pixel pattern PT comprises a red sensing pixel (SPX_R), a blue sensing pixel (SPX_B), a first green sensing pixel (SPX_Gr), and a second green sensing pixel (SPX_Gb) arranged in a 2x2 matrix, and such a pixel pattern PT is referred to as a Bayer pattern.

[0020] Alternatively, the pixel pattern PT may comprise red sensing pixels (SPX_R) arranged in an n×n matrix (n is an integer greater than or equal to 3), blue sensing pixels (SPX_B) arranged in an n×n matrix (n is an integer greater than or equal to 3), a first green sensing pixel (SPX_Gr) arranged in an n×n matrix, and a second green sensing pixel (SPX_Gb) arranged in an n×n matrix.

[0021] Image data IDT is generated based on pixel signals output from the pixel array PXA. The image data IDT has a color pattern corresponding to the pixel pattern PT of the pixel array PXA. As an example, if the pixel array PXA has a Bayer pattern, the image data IDT also has a Bayer pattern. As another example, if the pixel array PXA has a tetra pattern, the image data IDT has a tetra pattern or a Bayer pattern.

[0022] For example, if the pixel array PXA has a tetra pattern, one pixel signal is output from four sensing pixels SPX of the same color included in the pixel pattern SPX, or four pixel signals are output by outputting a pixel signal from each of the four sensing pixels SPX. When one pixel signal is output, the image data IDT has a Bayer pattern, and when four pixel signals are output, the image data IDT has a tetrahedral pattern as shown in FIG.

[0023] The image data IDT includes a red pixel (PX_R), a blue pixel (PX_B), a first green pixel (PX_Gr), and a second green pixel (PX_Gb) that are repeatedly arranged. The pixel PX of the image data IDT means data corresponding to the sensing pixel SPX of the pixel array PXA, in other words, pixel data. The red pixel (PX_R), blue pixel (PX_B), first green pixel (PX_Gr), and second green pixel (PX_Gb) correspond to the red sensing pixel (SPX_R), blue sensing pixel (SPX_B), first green sensing pixel (SPX_Gr), and second green sensing pixel (SPX_Gb) of the pixel array PXA, respectively.

[0024] The image data IDT includes a plurality of pixel groups PG, where the pixel groups PG are set to have a predetermined number of pixels PX arranged in a row and column sequentially or arranged in one direction according to the color pattern of the image data IDT, or are also set to have pixels PX corresponding to the same reference color and adjacent to each other. For example, as shown in FIG. 2, if the image data IDT has a tetra pattern, the pixel group PG is set to include four pixels PX that correspond to the same reference color (e.g., red, blue, green, etc.) and are adjacent to each other. As another example, if the image data IDT has a Bayer pattern, the pixel group PG is also set to include a predetermined number (for example, four) of pixels PX arranged in rows and columns.

[0025] Referring now to FIG. 1, each of the plurality of sensing pixels SPX includes at least one light-sensing element (or photoelectric conversion element). The light-sensing element senses light and converts the sensed light into an electrical signal. For example, the light-sensing element may be a photodiode, a phototransistor, a photogate, a pinned photodiode (PPD), or a combination thereof. Each of the plurality of sensing pixels SPX includes at least one light-sensing element and a pixel circuit for outputting a pixel signal corresponding to an electrical signal generated by the light-sensing element. For example, the pixel circuit may have a four-transistor structure with a transfer transistor, a reset transistor, an amplifier transistor, and a select transistor. However, without being limited thereto, the pixel circuit may have a 1-transistor structure, a 3-transistor structure, a 4-transistor structure, or a 5-transistor structure, or a structure in which multiple pixels share some transistors. In one embodiment, each pixel circuit may include a memory or an analog-to-digital converter.

[0026] In one embodiment, a plurality of color filters that transmit optical signals of a specific wavelength band (in other words, optical signals of a specific hue) are arranged on a plurality of pixels so as to correspond to each of the pixels of the pixel array PXA, and at least one photo-sensing element provided in the pixel converts the optical signal transmitted through the corresponding color filter into an electrical signal. Thereby, each of the plurality of sensing pixels SPX of the pixel array PXA outputs at least one pixel signal for at least one assigned reference color. However, the present invention is not limited thereto, and at least one photo-sensing element included in the sensing pixel SPX may selectively convert an optical signal of a specific wavelength band among incident light into an electrical signal.

[0027] In one embodiment, the image data IDT includes raw image data including a plurality of pixel values ​​obtained by digital-to-analog conversion of a plurality of pixel signals output from the pixel array PXA, or image data that has been pre-processed on the raw image data. The image sensor module 100 compresses the image data IDT using the encoder 120 to improve data transmission speed, reduce power consumption due to data transmission, and use efficient data storage space, and transmits the compressed data CDT to the image processing device 200.

[0028] The encoder 120 receives the image data IDT from the image sensor 110 and compresses the image data IDT to generate compressed data CDT. The compressed data CDT is embodied in the form of an encoded bitstream. Hereinafter, the coded bitstream will be simply referred to as the bitstream. The bitstream includes the compression result and compression information (for example, mode information indicating the compression method). The encoder 120 encodes the image data IDT in units of pixel groups PG to generate compressed data CDT. The encoder 120 encodes one pixel group PG to generate one bitstream, and generates compressed data CDT based on the bitstreams of all pixel groups PG in the image data IDT. By encoding the pixel group PG, the pixel group PG is compressed, and hereinafter, in the present invention, the term "encoding" is also used to mean the same as "compression."

[0029] The encoder 120 performs the compression using a reference map generated based on pixel values ​​corresponding to the pixel group PG being compressed, ie, pixels compressed prior to the pixel group being compressed. Specifically, the encoder 120 compresses the pixel value of the target pixel based on a reference value of at least one reference pixel that is adjacent to the at least one target pixel in the target pixel group in the reference map. The reference value is generated based on the pixel value of the reference pixel, and for example, the reference value is a value generated by compressing and then decompressing the pixel value of the reference pixel.

[0030] There is a high possibility that the pixel value of the target pixel and the pixel value of the adjacent reference pixel will be similar to each other. Furthermore, the pixel values ​​of the target pixels in the target pixel group are likely to be similar to each other. Therefore, the encoder 120 compresses the target pixel group using a DPCM (Differential Pulse Code Modulation) method, which encodes the target pixel of the target pixel group based on the difference between the target pixel and surrounding pixels, for example, the difference between the pixel value of the target pixel and the reference value of an adjacent reference pixel, or the difference between the pixel value of the target pixel and the pixel value of another target pixel in the target pixel group. This increases the compression efficiency (or compression ratio) and reduces data loss due to compression.

[0031] However, the pixel values ​​of the pixels of the target pixel group included in the isolated region in the image data IDT differ greatly from the pixel values ​​of the reference pixels. In other words, the pixel value of the target pixel and the reference value of the reference pixel have a low correlation. Here, the isolated region refers to a region that corresponds to an edge region in at least two directions (for example, two directions perpendicular to each other) of the edge region of a two-dimensional or multi-dimensional image pattern occurring in the image data IDT.

[0032] According to an embodiment of the present invention, when there is a large difference between the pixel value of a target pixel and the reference value of a reference pixel, the encoder 120 applies an offset value to each reference value included in the reference map to generate a virtual reference map VRM, and compresses the target pixel group using a DPCM method or other encoding method based on the virtual reference map VRM. The virtual reference map VRM includes compensated reference values ​​in which the difference between the pixel value of the target pixel and the reference value of the reference pixel is reduced by the offset value. Thus, the encoding scheme based on the virtual reference map according to the embodiment of the present invention is referred to as an offset encoding scheme. The offset encoding method will be described in detail later with reference to FIGS. 4A to 10.

[0033] If the encoder 120 compresses the pixel group of the isolated region using the aforementioned DPCM method or compresses the pixel group of the isolated region using a method that encodes the pixel group based on some of the most significant data bits, including the MSB (Most Significant Bit), among the multiple data bits that indicate the pixel value of each target pixel, a large amount of data loss will occur. This causes degradation of image quality in the restored image data generated by decompressing the compressed data CDT, resulting in artifacts in the image data. However, as described above, when the encoder 120 compresses pixel groups in isolated regions using the offset encoding method according to an embodiment of the present invention, the compression efficiency increases and data loss decreases.

[0034] The encoder 120 provides the compressed data CDT to the image processing device 200 through an interface 130 . For example, the interface 130 may be implemented by a Camera Serial Interface (CSI) based on the Mobile Industry Processor Interface (MIPI). Meanwhile, the type of interface 130 is not limited thereto, and may be implemented according to various protocol standards.

[0035] The image processing device 200 converts the compressed data CDT received from the image sensor module 100 to generate an image to be displayed on a display (not shown). Specifically, the image processing device 200 receives compressed data CDT from the image sensor module 100, decompresses the compressed data CDT to generate decompressed data DDT, e.g., restored image data, and performs image processing on the decompressed data DDT. In one embodiment, the image processing device 200 receives the compressed data CDT from the image sensor module 100 through the interface 210 . The interface 210 may be implemented as MIPI, like the interface 130 included in the image sensor module 100, but is not limited thereto. The image processing device 200 stores the received compressed data CDT in the memory 220 .

[0036] Memory 220 is a repository for storing data. The compressed data CDT is stored in the memory 220. In addition, the memory 220 stores other data, such as an operating system (OS), various programs, and various data (eg, compressed data CDT). The memory 220 includes volatile memory such as DRAM (Dynamic Random Access Memory) or SRAM (Static RAM), or non-volatile memory such as PRAM (Phase Change RAM), ReRAM (Resistive RAM), MRAM (Magnetic RAM), or flash memory. Although the memory 220 is shown in FIG. 1 as being provided within the image processing device 200, the present invention is not limited thereto, and the memory 220 may be provided separately outside the image processing device 200.

[0037] The decoder 230 reads the compressed data CDT from the memory 220 and decompresses the compressed data CDT to generate decompressed data DDT. The decoder 230 provides the decompressed data DDT to an image signal processor 240 . The decoder 230 decompresses the compressed data CDT in units of pixel groups PG according to a decompression method (or decoding method) performed by the encoder 120 of the image sensor module 100 . At this time, the decoder 230 determines the compression method applied to the pixel group PG based on the compression information included in the bitstream of the compressed data CDT. The decoder 230 decompresses the target pixel of the target pixel group based on a reference map that contains reference values ​​corresponding to pixels, or reference pixels, that were decompressed prior to the target pixel group being decompressed. In one embodiment, the decoder 230 applies an offset value to each reference value of the reference pixel to generate a virtual reference map VRM, and decompresses the target pixel group based on the virtual reference map VRM. At this time, the offset value is the same as the offset value used in the encoder 120 .

[0038] The image signal processor 240 performs various image processing operations on the received decompressed data DDT. By way of non-limiting example, the image signal processor 240 performs at least one of the following image processing on the decompressed data DDT: bad pixel correction, offset correction, lens distortion correction, color gain correction, shading correction, gamma correction, noise removal, and sharpening. In one embodiment, depending on the capabilities of the image sensor module 100, some of the image processing described above may be omitted. For example, if the image sensor module 100 includes a high-quality image sensor 110, defective pixel correction (especially static defective pixel correction) or offset correction in the image processing may be omitted.

[0039] Meanwhile, each of the encoder 120 and the decoder 230 may be implemented in software or hardware, or may be implemented in a combination of software and hardware, such as firmware. When the encoder 120 and the decoder 230 are implemented in software, the respective functions described above are implemented as programmed source code, which is loaded onto recording media provided in the image sensor module 100 and the image processing device 200, respectively, and the functions of the encoder 120 and the decoder 230 are implemented by the processors (e.g., image processing processors) provided in the image sensor module 100 and the image processing device 200, respectively, executing the software. When the encoder 120 and the decoder 230 are implemented in hardware, the encoder 120 and the decoder 230 include logic circuits and registers, and perform the above-described respective functions based on the settings of the registers.

[0040] Although the image processing system 10 is shown in FIG. 1 as including the image sensor module 100 and the image processing device 200, the present invention is not limited thereto. For example, the image processing system 10 may be embodied to include only some of the image sensor modules 100 and image processing devices 200, or to include multiple image sensor modules 100. Also, although the decoder 230 and the image signal processor 240 are shown as separate components in FIG. 1, the present invention is not limited to this. For example, the image signal processor 240 may also be embodied to include a decoder 230 .

[0041] 3A and 3B are block diagrams showing a schematic configuration of an encoder according to an embodiment of the present invention. 3A and 3B show the encoder 120 of FIG. 1 and 3A, the encoder 120 includes a reference pixel detector 121, a compression circuit 122, a mode selector 123, a reconstructed image generator 124, and a reference buffer 125.

[0042] The reference pixel detector 121 receives image data IDT from the image sensor (110 in FIG. 1) and receives from the reference buffer 125 a reference map containing reference values ​​of reference pixels used in compressing the target pixel group. The reference pixel detector 121 detects reference values ​​of reference pixels that are positionally adjacent to the target pixel group from the restored image data stored in the reference buffer 125, in other words, the restored pixel values ​​of the reference pixels, and receives the reference values ​​from the reference buffer 125 as a reference map. The reference pixel detector 121 provides the target pixel group of the image data IDT and the reference map to the compression circuit 122 .

[0043] The compression circuit 122 compresses the target pixel group based on the reference map. The compression circuit 122 includes an offset mode compressor OMC and a normal mode compressor NMC, and the offset mode compressor OMC and the normal mode compressor NMC compress the target pixel group using different compression methods (encoding methods) from each other.

[0044] The offset mode compressor OMC encodes the target pixel group in an offset encoding manner based on the virtual reference map VRM, as described above with reference to FIG. For example, the offset mode compressor OMC applies an offset value to a reference value of a reference map to generate a virtual reference map VRM, and encodes a target pixel group in a DPCM manner based on the virtual reference map VRM. After the target pixel group is compressed, the virtual reference map VRM is removed.

[0045] The normal mode compressor NMC encodes the target pixel group in the DPCM manner based on the reference map. The normal mode compressor NMC outputs first encoded data EDT1, and the offset mode compressor OMC outputs second encoded data EDT2. In FIG. 3A, the compression circuit 122 is shown as including an offset mode compressor OMC and a normal mode compressor NMC, but is not limited thereto, and the compression circuit 122 may further include a compressor that encodes the target pixel group using an encoding scheme different from that of the offset mode compressor OMC and the normal mode compressor NMC.

[0046] The mode selector 123 selects one of the compressed data received from the compression circuit 122, for example, the first encoded data EDT1 and the second encoded data EDT2, and outputs the selected encoded data as compressed data CDT. The mode selector 123 decodes the first encoding data EDT1 and the second encoding data EDT2 using a decoding method according to the encoding method, and selects one of the first encoding data EDT1 and the second encoding data EDT2 based on an error rate resulting from the decoding. The error rate refers to the difference between the decoded data, in other words, the decoded pixel value and the pixel value before decoding, and the smaller the difference, the lower the error rate. The lower the error rate, the less the degradation of the image quality of the decompressed data DDT, in other words, the restored image data, generated by the image processing device 200. Therefore, the mode selector 123 selects the encoded data with the lower error rate from the first encoded data EDT1 and the second encoded data EDT2 as the compressed data CDT, and outputs the compressed data CDT.

[0047] In one embodiment, the normal mode compressor NMC (or other separate compressor) encodes a target pixel group to generate encoded data, e.g., first encoding data EDT1, and when the error rate of the first encoding data EDT1 exceeds a reference error rate, the offset mode compressor OMC operates. The offset mode compressor OMC encodes the target pixel group to generate second encoded data EDT2. When the error rate of the first encoded data EDT1 is equal to the reference error rate, the mode selector 123 outputs the first encoded data EDT1 as the compressed data CDT. When the error rate of the first encoded data EDT1 exceeds the reference error rate, the mode selector 123 outputs the second encoded data EDT2 received from the offset mode compressor OMC as compressed data CDT.

[0048] The restored image generator 124 decodes the compressed data CDT to generate restored image data. The restored image generator 124 decodes each of the plurality of bitstreams included in the compressed data CDT using a decoding method corresponding to the encoding method, thereby restoring pixel values ​​of the target pixel group. The pixel corresponding to the restored pixel value is used as a reference pixel for other pixel groups to be compressed.

[0049] The reference buffer 125 stores the reconstructed image data and provides the reference pixel detector 121 with reference values ​​for the reference pixels used in compressing the target pixel group. In an exemplary embodiment, the reference buffer 125 is made up of a line memory and stores reference pixels located around a target pixel of a target pixel group. In an exemplary embodiment, the reference buffer 125 is also implemented by a volatile memory such as a DRAM or SRAM. However, the reference buffer 125 is not limited to this, and may also be implemented by a non-volatile memory such as ReRAM or PRAM.

[0050] Referring to FIG. 3B, the encoder 120 a includes a reference pixel detector 121 , a pre-detector 126 , a compression circuit 122 , a mode selector 123 , a reconstructed image generator 124 , and a reference buffer 125 . Compared to the encoder 120 of FIG. 3A, the encoder 120 a further comprises a pre-detector 126 . The operations of the reference pixel detector 121, the compression circuit 122, the mode selector 123, the restored image generator 124, and the reference buffer 125 have been described with reference to FIG. 3A, so a duplicated description will be omitted.

[0051] The pre-detector 126 activates or deactivates the offset mode compressor OMC. In one embodiment, the pre-detector 126 activates or deactivates the offset mode compressor OMC based on the reference value of the reference pixel. For example, if the difference between the reference value and the pixel value of the target pixel is greater than or equal to a threshold value (or a specific code value), the offset mode compressor OMC is activated, and if the difference is less than the threshold value, the offset mode compressor OMC is deactivated. In one embodiment, the pre-detector 126 comprises a register and activates or deactivates the offset mode compressor OMC based on a control signal stored in the register. For example, the control signal is received from the image processing device 200 .

[0052] If the offset mode compressor OMC is deactivated, another compressor included in the compression circuit 122, for example, the normal mode compressor NMC, encodes the target pixel group, and the encoded data, for example, the first encoding data EDT1, is output as compressed data CDT. If the offset mode compressor OMC is activated, at least some of the compressors included in the compression circuit 122 and the offset mode compressor OMC each encode the target pixel group, and the mode selector 123 outputs the encoded data with the lowest error rate as the compressed data CDT.

[0053] In one embodiment, even if the offset mode compressor OMC is activated, the offset mode compressor OMC may have a lower priority among the compressors included in the compression circuit 122 . For example, the normal mode compressor NMC first encodes a target pixel group to generate first encoded data EDT1, and when the error rate of the first encoded data EDT1 exceeds the reference error rate, the offset mode compressor OMC operates. The offset mode compressor OMC encodes the target pixel group to generate second encoded data EDT2.

[0054] When the error rate of the first encoded data EDT1 is equal to the reference error rate, the mode selector 123 outputs the first encoded data EDT1 as the compressed data CDT. When the error rate of the first encoded data EDT1 exceeds the reference error rate, the mode selector 123 outputs the second encoded data EDT2 received from the offset mode compressor OMC as compressed data CDT. In one embodiment, when the offset mode compressor OMC is activated, other compressors provided in the compression circuit 122, such as the normal mode compressor NMC, are deactivated, and the second encoding data EDT2 generated by the offset mode compressor OMC is output as compressed data CDT. This reduces the power consumption of the encoder 120.

[0055] 4A and 4B are exemplary diagrams illustrating a reference map and a virtual reference map according to an embodiment of the present invention. In Figures 4A and 4B, the numbers in parentheses indicate the value of the pixel, in other words the pixel value, the reference value or the compensated reference value.

[0056] Referring to FIG. 4A, the image data IDT includes a plurality of pixel groups PG, which are compressed sequentially along a set direction. In one embodiment, the image data IDT is compressed in pixel groups PG sequentially from left to right and top to bottom. However, the present invention is not limited to this, and the image data IDT may be compressed sequentially from right to left or from bottom to top. In the target pixel group TG, specifically, compression is performed on the target pixels (T0, T1, T2, T3) of the target pixel group TG. The target pixel group TG is compressed based on pixels contained in the pixel group PG that correspond to the same color among the surrounding pixel groups PG where compression has been performed. The surrounding pixels used in compressing the target pixel group TG are called reference pixels.

[0057] The reference values ​​of the reference pixels adjacent to the target pixel group TG in the reconstructed image data RIDT stored in the reference buffer 125 are generated as a reference map RM. Here, the reference value means a value generated by compressing and then decompressing a pixel value. For example, the reference map RM includes reference values ​​for reference pixels (e.g., R11, R12, R13, R14, R21, R22, R23, R24, R31, R32, R33, R34). The target pixel group TG is compressed based on the reference map RM. For example, the target pixel group TG or the target pixel T0 of the target pixel group TG is encoded based on the most relevant reference value among the reference values ​​of the adjacent reference pixels (R13, R14, R23, R24, R32, R34) or the reference value of the reference pixel adjacent in a predetermined direction.

[0058] On the other hand, the difference between the pixel values ​​of the target pixels (T0, T1, T2, T3) of the target pixel group TG and the reference values ​​of the reference pixels also becomes large. For example, the pixel value of the target pixel T0 is 283, and the reference values ​​of the adjacent reference pixels (R14, R23, R32) are 137, 148, and 127, respectively. If the threshold is set to 125, the difference between the pixel value and the reference value is greater than or equal to 125, so the target pixel group TG is an isolated area. The target pixel group TG is compressed based on a virtual reference map (VRM in FIG. 4B). For example, the offset mode compressor OMC in FIG. 3 generates a virtual reference map VRM based on the reference map RM, and compresses the target pixel group TG based on the virtual reference map VRM.

[0059] Referring to FIG. 4B, an offset value is applied to each of the reference values ​​of the reference pixels (e.g., R11, R12, R13, R14, R21, R22, R23, R24, R31, R32, R33, R34) to generate a virtual reference map VRM. The offset value is assumed to be 128. The offset value may be a positive or negative value. An offset value is added to each of the reference values ​​of the reference pixels (e.g., R11, R12, R13, R14, R21, R22, R23, R24, R31, R32, R33, R34) to generate a compensated reference value, and the virtual reference map VRM includes the compensated reference values. The pixel value of the target pixel and the compensation reference value are similar, and the correlation between the pixel value and the compensation reference value is high. In this way, when the target pixel group TG is in an isolated area and has a low correlation with the reference map RM, the offset mode compressor OMC applies an offset value to the reference map RM to generate a highly correlated virtual reference map VRM, and compresses the target pixel group TG based on the virtual reference map VRM.

[0060] 5A and 5B are conceptual diagrams for explaining a compression method according to an embodiment of the present invention. The compression method described in this invention is performed in the offset mode compressor OMC of FIGS. 3A and 3B and is based on the DPCM method. 5A and 5B, the offset mode compressor OMC compresses target pixels (T0, T1, T2, T3) of a target pixel group TG based on a compensation reference value of at least one reference pixel included in a virtual reference map VRM. For example, the reference pixel R32 of the virtual reference map VRM is used to compress the target pixel group TG.

[0061] The offset mode compressor OMC calculates a difference value d0 between the compensated reference value of the reference pixel R32 and the pixel value of the target pixel T0. For example, the compensation reference value 255 is subtracted from the pixel value 283, and the difference value d0 is 28. At this time, if the pixel value is smaller than the compensation reference value, the difference value d0 has a negative value. Alternatively, the pixel value is subtracted from the compensation reference value to generate a difference value d0. The offset mode compressor OMC calculates difference values ​​(d1, d2, d3) between target pixels (T0, T1, T2, T3). The offset mode compressor OMC generates a bitstream BS for the target pixel group TG based on the difference value d0 between the compensated reference value of the reference pixel R32 and the pixel value of the target pixel T0, and the difference values ​​(d1, d2, d3) between the target pixels (T0, T1, T2, T3).

[0062] The bitstream BS comprises a header HD and data blocks DB. The header HD includes compression information, such as mode information including the encoding method used for compression (e.g., DPCM method, offset mode method, etc.), compression rate, loss information, etc., and the data block DB includes information based on the pixel values ​​of the target pixels (T0, T1, T2, T3), such as multiple difference values ​​(d0, d1, d2, d3). When the bitstream BS is transmitted to the image processing device (200 in FIG. 1) as compressed data CDT, the decoder (230 in FIG. 1) determines the compression method (i.e., encoding method) of the bitstream BS based on the mode information contained in the header HD, and decompresses the compressed data CDT by decoding the bitstream BS based on the decompression method (i.e., decoding method) corresponding to the compression method and the information contained in the data block DB.

[0063] 6A and 6B are diagrams illustrating a bitstream according to one embodiment of the present invention. 6A and 6B show examples of implementations of the bitstream BS of FIGS. 5A and 5B. Assume that the image data IDT, in other words, the pixel values ​​of the target pixels (e.g., T0, T1, T2, T3) before compression is represented by data containing 10 bits, and that the target pixel group TG is compressed at a compression rate of 50%, resulting in the bit stream BS being data containing 20 bits.

[0064] 6A and 6B, the data blocks DB of the bit streams (BSa, BSb) are assigned to the 1st to 16th bits (B0 to B15), and the header HD is assigned to the 17th to 20th bits (B16 to B19). As mentioned above, the header HD contains mode information. For example, 4 bits are assigned to the header HD, so that the header HD includes one of 2^4 (=16) pieces of mode information. The data block DB includes a plurality of remaining areas, for example, first to fourth remaining areas (RD0, RD1, RD2, RD3).

[0065] For example, the difference values ​​(d0, d1, d2, d3) calculated in FIGS. 5A and 5B are included in the first to fourth remaining regions (RD0, RD1, RD2, RD3), respectively. In the present invention, values ​​(eg, difference values) included in the bitstream BSa are expressed as negative and positive numbers through the most significant sign bit representation. For example, if the difference value is represented by 4-bit data, the most significant bit indicates the sign, and the remaining 3 bits indicate the absolute value. The difference value is expressed as a 4-bit binary code representing an integer between -8 and 7. However, the present invention is not limited to this, and the one's complement method, absolute value method, etc. may also be used.

[0066] In one embodiment, the first to fourth remainder regions (RD0, RD1, RD2, RD3) are allocated the same number of bits. For example, the fourth remaining area RD3 is assigned to the first to fourth bits (B0 to B3), the third remaining area RD2 is assigned to the fifth to eighth bits (B4 to B7), the second remaining area RD1 is assigned to the ninth to twelfth bits (B8 to B11), and the first remaining area RD0 is assigned to the thirteenth to sixteenth bits (B12 to B15).

[0067] Referring to FIG. 6B, the data block DB includes a plurality of remaining areas, for example, first to fourth remaining areas (RD0, RD1, RD2, RD3) and an additional area SUB. In one embodiment, the first to fourth remainder areas (RD0, RD1, RD2, RD3) are allocated different numbers of bits. For example, the fourth remaining area RD3 is assigned to the first to third bits (B0 to B2), the third remaining area RD2 is assigned to the fourth to sixth bits (B3 to B5), the second remaining area RD1 is assigned to the seventh to tenth bits (B6 to B9), and the first remaining area RD0 is assigned to the eleventh to fifteenth bits (B10 to B14).

[0068] In FIG. 5B, the difference value d0 between the compensation reference value of the reference pixel R32 and the pixel value of the target pixel T0 is also greater than the difference values ​​(d1, d2, d3) between the relatively more adjacent target pixels (T0, T1, T2, T3). Therefore, the most bits are allocated to the first remainder region RD0. An additional area SUB is allocated to the 16th bit B15. The additional region SUB includes a code indicating the amount of shift (e.g., right or left shift amount) required when the difference values ​​(d0, d1, d2, d3) included in the first to fourth remainder regions (RD0, RD1, RD2, RD3) are decoded. For example, if the code of the additional area SUB is "0", it indicates no shift, and if it is "1", it indicates that the difference values ​​(d0, d1, d2, d3) are set to be shifted left twice. If the code (e.g., binary code) of the additional area SUB is "1" and the difference value d0 is "110100", the code value included in the second remainder area RD1 is "1101". At this time, the most significant bit is not shifted because it indicates the sign.

[0069] When restoring the compressed data CDT, the decoder (230 in FIG. 1) of the image processing device (200 in FIG. 1) calculates the difference values ​​(d0, d1, d2, d3) by performing two left-shift operations on the values ​​read from the first to fourth remaining areas (RD0, RD1, RD2, RD3) in consideration of the code of the additional area SUB. For example, the decoder 230 performs two left shift operations on the code value "101" excluding the most significant bit of the code value "1101" included in the second remainder area RD1 to generate the code value "110100". The decoder 230 determines that the difference value d is −20, which is indicated by the code value “110100”.

[0070] The bitstream BS of FIG. 5 has been exemplarily described with reference to FIGS. 6A and 6B. However, the present invention is not limited to this, and the number of bits and structure of the bitstream BS may vary depending on the compression method, compression rate, and the like. In addition, the number of bits allocated to each region of the bitstream BS, for example, the header HD and the data block DB, or the number of bits allocated to each region of the data block BD, for example, the remaining region (RD0, RD1, RD2, RD3) and the additional region SUB, may also vary.

[0071] FIG. 7 is a conceptual diagram for explaining a compression method according to an embodiment of the present invention. The compression method of FIG. 7 is performed in the offset mode compressor OMC of FIG. Referring to FIG. 7, the offset mode compressor OMC compresses target pixels (T0, T1, T2, T3) of a target pixel group TG based on a compensation reference value of at least one reference pixel included in the virtual reference map VRM. For example, the reference pixel R32 of the virtual reference map VRM is used to compress the target pixel group TG.

[0072] The offset mode compressor OMC calculates the average value of the pixel values ​​of the target pixels (T0, T1, T2, T3). The offset mode compressor OMC then calculates the difference value d0 between the compensation reference value of the reference pixel R32 and the average value. For example, the average value of the target pixels (T0, T1, T2, T3) is calculated as 342, and the compensation reference value of 255 is subtracted from 342 to obtain a difference value d0 of 87. If the pixel value is less than the compensation reference value, the difference value d0 has a negative value.

[0073] The offset mode compressor OMC calculates the difference values ​​(d1, d2, d3, d4) between the average values ​​and the pixel values ​​of the target pixels (T0, T1, T2, T3). The offset mode compressor OMC generates a bitstream BSc for the target pixel group TG based on the difference value d0 and the difference values ​​(d1, d2, d3, d4). The data block DB includes a plurality of remaining areas, for example, first to fourth remaining areas (RD0, RD1, RD2, RD3) and an average area AVE. The average area AVE includes the difference value d0, and the first to fourth remainder areas (RD0, RD1, RD2, RD3) include the difference values ​​(d1, d2, d3, d4), respectively.

[0074] FIG. 8 is an exemplary diagram showing image data and a virtual reference map in accordance with an exemplary embodiment of the present invention. In FIG. 8, the image data IDTa includes a Bayer pattern. In one embodiment, a pixel group PG is set up in units of four consecutively arranged pixels. In one embodiment, pixel group PG comprises two red pixels and two green pixels, or two blue pixels and two green pixels.

[0075] Compression is performed in units of pixel groups PG, with the target pixel group TG being compressed first and then compressed based on the reference pixels of the pixel groups PG that correspond to the same hue within the adjacent pixel groups PG (e.g., R11, R12, R13, R14, R21, R22, R23, R24, R31, R32, R33, R34). There is a significant difference between the pixel values ​​of the target pixel group TG and the reference pixels (R11, R12, R13, R14, R21, R22, R23, R24, R31, R32, R33, R34), and it is determined that the target pixel group TG corresponds to an isolated region. As a result, the target pixel group TG is compressed using the offset encoding method.

[0076] It is assumed that when the offset value is 128 and the pixel value of the reference pixel is compressed and then decompressed, the pixel value before compression and the pixel value after decompression, in other words, the reference value, are equal. The offset mode compressor (OMC in FIG. 3A) adds an offset value 128 to each reference value of the reference map to generate a virtual reference map VRG. The reference values ​​of the reference pixels (R11, R12, R13, R14, R21, R22, R23, R24, R31, R32, R33, R34) are similar to the pixel values ​​of the target pixels (T0, T1, T2, T3). The offset mode compressor (OMC in FIG. 3A) compresses the target pixel group TG based on the virtual reference map VRM.

[0077] FIG. 9 is a conceptual diagram for explaining a compression method according to an embodiment of the present invention. The compression method of FIG. 9 is performed on the image data IDTa of FIG. 8 using the virtual reference map VRM. Referring to FIG. 9, the offset mode compressor (OMC in FIG. 3A) compresses target pixels (T0, T1, T2, T3) of a target pixel group TG based on a compensation reference value of at least one reference pixel included in a virtual reference map VRM. For example, the reference pixels (R33, R34) of the virtual reference map VRM are used to compress the target pixel group TG. When compression is performed, a reference pixel of the same hue as the target pixel is used.

[0078] The offset mode compressor OMC calculates a difference value d0 between the compensated reference value of the reference pixel R33 and the target pixel T0, and also calculates a difference value d1 between the pixel values ​​of the target pixel T0 and the target pixel T2. The offset mode compressor OMC also calculates a difference value d2 between the compensated reference value of the reference pixel R34 and the target pixel T1, and calculates a difference value d3 between the pixel values ​​of the target pixel T2 and the target pixel T3. The offset mode compressor OMC generates a bitstream BS based on the difference values ​​(d0, d1, d2, d3). The difference values ​​(d0, d1, d2, d3) are contained in the data block DB of the bitstream BS.

[0079] FIG. 10 is a flowchart illustrating a method for compressing image data according to an embodiment of the present invention. The compression method of FIG. 10 includes the offset encoding scheme described above and is performed in the offset mode compressor OMC of FIGS. 3A and 3B. The compression method based on the offset encoding scheme described above is applied to this embodiment.

[0080] 3A and 10, the offset mode compressor OMC receives pixel values ​​of a target pixel group and reference pixel values ​​of reference pixels (step S110). In other words, the offset mode compressor OMC receives image data IDT containing a group of pixels of interest and a reference map containing reference values ​​for reference pixels. The target pixel group is a pixel group for which compression is to be performed, and comprises target pixels for which compression is to be performed, and for which pixel values ​​corresponding to the target pixels are received. Reference pixels are pixels surrounding the target pixel that are used to compress the target pixel group. The pixel values ​​of the surrounding pixels are compressed and then restored to generate the reference value.

[0081] The offset mode compressor OMC applies an offset value to each of the reference values ​​of the reference pixels to generate a virtual reference map (step S120). A positive or negative offset value is added to each reference value to generate a compensated reference value. The difference between the compensation reference value and the pixel value of the target pixel is less than the difference between the reference value and the pixel value of the target pixel. The virtual reference map includes a plurality of compensated reference values.

[0082] In one embodiment, the offset values ​​are preset, and the offset mode compressor OMC applies the preset offset value to each of the reference values. As a non-limiting example, if the pixel value or reference value is represented by data (e.g., binary code data) including 8 bits representing 256 values ​​(e.g., 0 code to 255 code), the offset value is set to a value that is half the maximum pixel value, e.g., 255, e.g., 127. The offset value includes one bit of data (for example, the MSB) indicating the sign and seven bits of data indicating 127. However, the offset value is not limited to this, and may be preset in various values ​​and manners.

[0083] In one embodiment, the offset mode compressor OMC comprises decision logic for determining the offset value, the decision logic determining the offset value based on a reference value. For example, the offset value is determined depending on the gradation range in which the reference value is included. If the gradation area containing the reference value corresponds to a low gradation area, the positive candidate offset value corresponding to the gradation area among the positive candidate offset values ​​is determined as the offset value, and if the gradation area containing the reference value corresponds to a high gradation area, the negative candidate offset value corresponding to the gradation area among the negative candidate offset values ​​is determined as the offset value. As another example, the candidate offset value among multiple candidate offset values ​​set based on the difference value between the reference value and the pixel value of the target pixel that corresponds to the actual difference value (in other words, the difference value between the pixel value of the current target pixel and the reference value of the current reference pixel) is determined to be the offset value. Besides this, the offset value can be determined by various methods.

[0084] The offset mode compressor OMC compresses pixel values ​​of a group of target pixels, in other words, target pixels, based on a virtual reference map. For example, the offset mode compressor OMC compresses the target pixel group according to the compression method, specifically, the encoding scheme, described with reference to FIGS. However, the present invention is not limited to this, and various encoding methods can be applied.

[0085] The offset mode compressor OMC generates a bitstream including the compression result and the compression information (step S140). As mentioned above, the bitstream comprises a header and data blocks, where the header contains compression information and the data blocks contain the compression results, such as difference values ​​between the reference value of a reference pixel and the pixel value of a target pixel, or difference values ​​between the pixel values ​​of the target pixel. Meanwhile, as described with reference to FIG. 3A, the compression method may further include a step of, after the bitstream is generated, decompressing the bitstream to generate restored pixel values, and a step of generating restored image data including reference pixels to be used in compressing the next target pixel group, which is compressed after the target pixel group, based on the restored pixel values.

[0086] 11A and 11B are block diagrams showing a schematic configuration of an image sensor module according to one embodiment of the present invention. Referring to FIG. 11A, the image sensor module 100 includes an image sensor 110, processing logic 150, an encoder 120, and an interface 130.

[0087] The image sensor 110 comprises a pixel array PXA and a drive and readout circuit DRC. As described above, the pixel array PXA includes a plurality of pixels PX arranged in rows and columns. The drive and read circuit DRC controls the pixel array PXA and converts pixel signals received from the pixel array PXA into pixel values. The drive and read circuit DRC generates raw image data RDT containing pixel values ​​corresponding to each of the received pixel signals. The processing logic 150 performs pre-processing on the raw image data RDT. For example, pre-processing includes image processing such as bad pixel correction, crosstalk correction, noise removal, binning, resizing, and color space conversion.

[0088] The encoder 120 compresses the image data IDT (or original image data RDT) received from the processing logic 150 to generate compressed data CDT. The encoder 120 compresses the image data IDT in units of pixel groups, and compresses the target pixel group using the neighboring pixel groups that have been compressed. As described above, if the target pixel group falls within an isolated region, the encoder 120 generates a virtual reference map and compresses the target pixel group based on the virtual reference map. The compressed data CDT is provided to an interface 130, which transmits the compressed data CDT to the image processing device (200 in FIG. 1).

[0089] Referring to FIG. 11B, the image sensor module 100a further includes a memory 160. The memory 160 may include volatile memory such as DRAM or SRAM, or non-volatile memory such as PRAM, ReRAM, MRAM, or flash memory. The compressed data CDT generated by the encoder 120 is stored in the memory 160 . The compressed data CDT is read from the memory 160 and output through the interface 130 .

[0090] FIG. 12 is a block diagram showing a schematic configuration of a decoder according to an embodiment of the present invention. The decoder 230 decompresses the compressed data CDT to generate decompressed data DDT, e.g., restored image data, by performing the series of steps that the encoder (120 in FIG. 1) goes through to encode the image data IDT in reverse order. The decoder 230 decompresses the compressed data CDT using a decoding method that corresponds to the encoding method used by the encoder 120 . The decoder 230 decodes the compressed data CDT in units of bitstreams.

[0091] The decoder 230 comprises a reference pixel detector 231 , a mode determiner 232 , a decompressor 233 and a reference buffer 234 . The reference pixel detector 231 receives the compressed data CDT and receives from a reference buffer 234 a reference map used for decompressing the bitstream contained in the image data CDT that is to be decompressed. The reference map contains reference values ​​for surrounding pixels, in other words, reference pixels, of a pixel group associated with a reference bitstream.

[0092] The reference pixel detector 231 detects reference values ​​of reference pixels that are positionally adjacent to the target pixel group, in other words, the restored pixel values ​​of the reference pixels, from the restored image data stored in the reference buffer 235, and receives the reference values ​​from the reference buffer 235 as a reference map. The reference pixel detector 231 provides the target bitstream and the reference map of the compressed data CDT to a mode decider 232 or a decompressor 233 . The mode determiner 232 decodes the header of the bitstream and determines mode information, compression rate, loss information, etc. as the decoding results. According to one embodiment of the present invention, the mode determiner 232 determines from the result of decoding the header whether the compression was performed using the offset encoding method or another encoding method (for example, the DPCM method).

[0093] The decompressor 233 restores the target pixel from the bitstream based on the determined compression mode, compression rate, loss information, and the like. According to one embodiment of the present invention, the decompressor 233 applies an offset to the reference map to generate a virtual reference map and decodes the bitstream based on the virtual reference map. The decompressor 233 applies the offset value used by the encoder 120 to each reference value contained in the reference map to calculate a compensated reference value. The virtual reference map includes the compensation reference values. The decompressor 233 decodes the bitstream using at least one compensation reference value of the virtual reference map or a compensation reference value for each pixel group. The pixel groups generated by decoding the bitstream are output as decompressed data DDT. The reference buffer 234 stores the decompressed data DDT, i.e., the restored image data. In one embodiment, the reference buffer 234 stores a group of pixels from the restored image data that corresponds to the next bitstream to be decompressed. In one embodiment, a memory or buffer (e.g., memory 220) included in the image processing device (200 in FIG. 1) is used as the reference buffer 234.

[0094] FIG. 13 is a block diagram showing a schematic configuration of an image processing system according to an embodiment of the present invention. FIG. 13 shows an alternative embodiment of the image processing system 10 of FIG. Referring to FIG. 13, an image processing system 10b includes an image sensor module 100b and an image processing device 200b.

[0095] The image sensor module 100b includes an image sensor 110 and an interface 130. The image sensor module 100b further includes a memory. The image processing device 200 b includes an interface 210 , an encoder 250 , a decoder 230 , an image signal processor 240 , and a memory 220 . The encoder 250 in FIG. 13 corresponds to the encoder 120 in FIG. Comparing the image processing system 10b of FIG. 13 with the image processing system 10 of FIG. 1, the image processing device 200b, which is not the image sensor module 100b, includes an encoder 250, and the image processing device 200b compresses the image data IDT. The remaining configurations are substantially identical. A description of the configuration of the image processing system 10b that overlaps with the configuration of the image processing system 10 in FIG. 1 will be omitted.

[0096] Referring to FIG. 13, the image sensor 110b generates image data IDT (original image data or pre-processed image data). The image data IDT is transmitted through the interface 130 to the image processing device 200b. The image processing device 200b receives the image data IDT from the image sensor module 100b, compresses the image data IDT, and stores the compressed data CDT in the memory 220. The decoder 230 then reads the compressed data CDT stored in the memory 220 and decompresses the compressed data CDT. The decoder 230 provides decompressed data CDT, eg, reconstructed image data, to an image signal processor 240 . As described above, compression and decompression are performed in units of pixel groups, and pixel groups that correspond to isolated regions of the image data CDT are compressed or decompressed based on the virtual reference map.

[0097] FIG. 14 is a table illustrating compression information according to one embodiment of the present invention. FIG. 14 exemplarily illustrates compression modes (compression methods) according to the standard proposed by the Mobile Industry Processor Interface (IPI) Alliance. Referring to FIG. 14, the image data of the tetra pattern (IDT in FIG. 2) is compressed by various compression modes. However, the present invention is not limited to this, and image data in which a red pixel group, a blue pixel group, a first green pixel group, and a second pixel group, each including pixels arranged in an n×n matrix, are repeatedly arranged, can also be compressed in various compression modes.

[0098] The compression modes used are AD (Average-based Directional Differential) mode, eMPD (extended Multi-Pixel-based Differential) mode, eHVD (extended Horizontal or Vertical Direction-based Differential) mode, eHVA (extended Horizontal or Vertical Average-based Differential) mode, OD (Oblique Direction-based Differential) mode, eOUT (extended OUTlier compensation) mode, OUT mode, and FNR (Fixed quantization and No-Reference) mode. Meanwhile, the names of the compression modes described above are merely examples, and the present invention is not limited to the above examples.

[0099] In AD mode, the target pixel group is encoded using the DPCM method. For example, a bitstream (eg, BSc in FIG. 7) is generated based on the difference between the average pixel value of the target pixel group and the reference value of the reference pixel, and the difference between each pixel value and the average pixel value. The AD modes are divided into MODE0, MODE1, MODE2, and MODE3 according to the detailed implementation algorithm. Four bits are allocated to the header indicating the compression method, so each of the 16 compression modes expresses header information using different bits. For example, MODE0 is represented by bits "0000", MODE1 by bits "0001", MODE2 by bits "0010", and MODE3 by bits "0011".

[0100] The OD mode compresses the image data IDT having a diagonal structure. The OD mode is divided into MODE4 (bits '0100') and MODE5 (bits '0101') according to the detailed implementation algorithm. Similarly, eMPD mode includes MODE8 (bits "1000"), MODE9 (bits "1001"), MODE10 (bits "1010"), and MODE11 (bits "1011"), eHVD mode includes MODE12 (bits "1100") and MODE13 (bits "1101"), eHVA mode includes MODE14 (bits "1110"), eOUT mode includes MODE15 ("1111"), and OUT mode includes MODE7 ("0111"). The FNR mode includes MODE6 (bits "0110"). In one embodiment, MODE7 ("0111") is included in the eOUT mode, depending on the value stored in the register.

[0101] The offset encoding scheme according to one embodiment of the present invention corresponds to the eOUT mode, and the offset mode compressor (OMC in FIG. 3A) generates a bitstream with a header indicating the eOUT mode. In one embodiment, the mode selector (reference numeral 123 in Figures 3A and 3B) sequentially evaluates the AD mode, eMPD mode, eHVD mode, eHVA mode, OD mode, eOUT mode, and FNR mode, and selects the optimal mode based on compression evaluation indicators such as compression ratio and loss information. However, the technical concept of the present invention is not limited to the presented mode evaluation procedure.

[0102] FIG. 15A is a block diagram showing a schematic configuration of an electronic device including a multi-camera module according to an embodiment of the present invention, and FIG. 15B is a block diagram showing a detailed configuration of the camera module of FIG. 15A. Referring to FIG. 15A, the electronic device 1000 includes a camera module group 1100, an application processor 1200, a PMIC 1300, and an external memory 1400.

[0103] The camera module group 1100 includes a plurality of camera modules (1100a, 1100b, 1100c). For example, although the figure shows an embodiment in which three camera modules (1100a, 1100b, 1100c) are arranged, the present invention is not limited to this. In one embodiment, the camera module group 1100 may be implemented with only two camera modules, or may be modified to include k camera modules (k is a natural number equal to or greater than 4). The detailed configuration of camera module 1100b will be described in more detail below with reference to FIG. 15B, but the following description also applies to other camera modules 1100a and 1100b depending on the embodiment.

[0104] Referring to FIG. 15B, a camera module 1100b includes a prism 1105, an optical path folding element (hereinafter, "OPFE") 1110, an actuator 1130, an image sensing device 1140, and a storage unit 1150. The prism 1105 has a reflecting surface 1107 made of a light-reflecting material, and changes the path of light L incident from the outside. In one embodiment, the prism 1105 changes the path of light L incident in a first direction (X) to a second direction (Y) perpendicular to the first direction (X). In addition, the prism 1105 rotates the reflecting surface 1107 of the light-reflecting material in direction A around the central axis 1106, or rotates the central axis 1106 in direction B, thereby changing the path of the light L incident in the first direction (X) to a second direction (Y) perpendicular to the first direction (X). At this time, the OPFE 1110 also moves in a third direction (Z) perpendicular to the first direction (X) and the second direction (Y).

[0105] In one embodiment, as shown in the figure, the maximum rotation angle of prism 1105 in the A direction is less than or equal to 15° in the plus (+) A direction and greater than 15° in the minus (-) A direction, but this embodiment is not limited thereto. In one embodiment, prism 1105 moves in the plus (+) or minus (-) B direction by 20° in or out, or between 10° and 20°, or between 15° and 20°, where the angle of movement is the same angle in the plus (+) or minus (-) B direction, or to a similar angle in the range of around 1°. In one embodiment, the prism 1105 moves the reflective surface 1106 of the light-reflecting material in a third direction (eg, Z direction) parallel to the extension direction of the central axis 1106.

[0106] In one embodiment, the camera module 1100b is composed of two or more prisms, through which the path of light L incident in a first direction (X) is varied to a second direction (Y) perpendicular to the first direction (X), back to the first direction (X) or a third direction (Z), and then back to the second direction (Y), etc. The OPFE 1110 includes, for example, m groups of optical lenses (where m is a natural number). The m lenses move in a second direction (Y) to change the optical zoom magnification of the camera module 1100b. For example, if the basic optical zoom magnification of camera module 1100b is Z, when m optical lenses provided in OPFE 1110 are moved, the optical zoom magnification of camera module 1100b is changed to an optical zoom magnification of 3Z, 5Z, or more than 5Z.

[0107] The actuator 1130 moves the OPFE 1110 or the optical lens (hereinafter referred to as the optical lens) to a specific position. For example, the actuator 1130 adjusts the position of the optical lens so that the image sensor 1142 is located at the focal length of the optical lens for accurate sensing. The image sensing device 1140 includes an image sensor 1142 , control logic 1144 , and memory 1146 . The image sensor module 100 in FIG. 1 or the image sensor module 100b in FIG.

[0108] The image sensor 1142 senses an image of a sensing target using light L provided through an optical lens. Control logic 1144 controls the overall operation of camera module 1100b and processes sensed images. For example, the control logic 1144 controls the operation of the camera module 1100b based on a control signal provided through the control signal line CSLb, extracting image data corresponding to a specific image (e.g., a person's face, arm, leg, etc. in the image) from the sensed image, or performing image processing such as noise removal. In one embodiment, the control logic 1144 includes an encoder (120 in FIG. 1) to compress the sensed or processed image. As described above, the encoder 120 compresses the image in units of pixel groups, and compresses pixel groups in isolated regions using an offset encoding method.

[0109] Memory 1146 stores information necessary for the operation of camera module 1100b, such as calibration data 1147. The calibration data 1147 is information required when the camera module 1100b generates image data using externally provided light L, and includes, for example, information regarding the degree of rotation, the focal length, and the optical axis. If the camera module 1100b is implemented as a multi-state camera in which the focal length changes depending on the position of the optical lens, the calibration data 1147 includes focal length values ​​for each position (or state) of the optical lens and information regarding autofocusing. In one embodiment, the compressed data is stored in memory 1146 . The memory 1146 is also used as a reference buffer 125 for the encoder 120 .

[0110] The storage unit 1150 stores the image data sensed by the image sensor 1142 . The storage unit 1150 is disposed outside the image sensing device 1140 and is stacked on a sensor chip constituting the image sensing device 1140 . In one embodiment, the image sensor 1142 is configured on a first chip, and the control logic 1144, storage unit 1150, and memory 1146 are configured on a second chip, and the two chips may be stacked. In one embodiment, the storage unit 1150 is implemented as an EEPROM (Electrically Erasable Programmable Read-Only Memory), but the present embodiment is not limited thereto. In one embodiment, the image sensor 1142 is configured by a pixel array, and the control logic 1144 includes an analog to digital converter and an image signal processor for processing the sensed image.

[0111] 15A and 15B, in one embodiment, each of the multiple camera modules (1100a, 1100b, 1100c) includes an actuator 1130. As a result, each of the multiple camera modules (1100a, 1100b, 1100c) is provided with the same or different calibration data 1147 depending on the operation of the actuator 1130 provided therein. In one embodiment, one of the multiple camera modules (1100a, 1100b, 1100c) (e.g., 1100b) may be a folded lens type camera module having the above-mentioned prism 1105 and OPFE 1110, and the remaining camera modules (e.g., 1100a and 1100c) may be vertical type camera modules not having the prism 1105 and OPFE 1110, but are not limited to this.

[0112] In one embodiment, one of the camera modules (1100a, 1100b, 1100c) (for example, 1100c) is a vertical depth camera that extracts depth information using, for example, infrared rays (IR). In this case, the application processor 1200 merges image data provided by such a depth camera with image data provided by other camera modules (e.g., 1100a or 1100b) to generate a three-dimensional depth image. In one embodiment, at least two camera modules (eg, 1100a and 1100b) of the plurality of camera modules (1100a, 1100b, 1100c) have different fields of view (field of view angles). In this case, for example, the optical lenses of at least two camera modules (for example, 1100a and 1100b) among the plurality of camera modules (1100a, 1100b, 1100c) are different from each other, but this is not limitative.

[0113] In one embodiment, the camera modules 1100a, 1100b, and 1100c each have a different viewing angle. For example, but not limited to, camera module 1100a is an ultra-wide camera, camera module 1100b is a wide camera, and camera module 1100c is a telephoto camera. In this case, the optical lenses provided in each of the camera modules (1100a, 1100b, 1100c) are different from each other, but the present invention is not limited to this. In one embodiment, each of the multiple camera modules (1100a, 1100b, 1100c) is positioned physically separated from one another. In other words, the sensing area of ​​one image sensor 1142 is not divided and used by multiple camera modules (1100a, 1100b, 1100c), but an independent image sensor 1142 is arranged inside each of the multiple camera modules (1100a, 1100b, 1100c).

[0114] Referring again to FIG. 15A, the application processor 1200 includes an image processing unit 1210, a memory controller 1220, and an internal memory 1230. The application processor 1200 and the camera modules 1100a, 1100b, and 1100c may be implemented as separate semiconductor chips. The image processing device 1210 includes a plurality of sub-image processors (1212a, 1212b, 1212c), an image generator 1214, and a camera module controller 1216. The image processing device 1210 includes a plurality of sub-image processors (1212a, 1212b, 1212c) corresponding to the number of the plurality of camera modules (1100a, 1100b, 1100c).

[0115] Image data generated from camera module 1100a is provided to sub-image processor 1212a via image signal line ISLa, image data generated from camera module 1100b is provided to sub-image processor 1212b via image signal line ISLb, and image data generated from camera module 1100c is provided to sub-image processor 1212c via image signal line ISLc. Such image data transmission is performed using, for example, a Camera Serial Interface (CSI) based on MIPI (Mobile Industry Processor Interface), but is not limited thereto.

[0116] In one embodiment, at least one of the sub-image processors (1212a, 1212b, 1212c) includes a decoder (230 in FIG. 1). The multiple sub-image processors (1212a, 1212b, 1212c) each include a decoder 230 to decompress compressed image data when the corresponding camera module (1100a, 1100b, 1100c) includes an encoder (120 in Figure 1). In one embodiment, the image processing device 200b of Figure 13 is implemented with at least one of a plurality of sub-image processors (1212a, 1212b, 1212c), and at least one of the plurality of sub-image processors (1212a, 1212b, 1212c) includes an encoder (250 of Figure 13) and a decoder (230 of Figure 13).

[0117] On the other hand, in one embodiment, one sub-image processor may be arranged to correspond to multiple camera modules. For example, sub-image processor 1212a and sub-image processor 1212c are not implemented separately from each other as shown in the figure, but are integrated into a single sub-image processor, and image data provided from camera module 1100a and camera module 1100c is selected through a selection element (e.g., a multiplexer) and then provided to the integrated sub-image processor. At this time, the sub-image processor 1212b is not integrated and is provided with image data from the camera module 1100b.

[0118] Also, in one embodiment, image data generated from camera module 1100a is provided to sub-image processor 1212a via image signal line ISLa, image data generated from camera module 1100b is provided to sub-image processor 1212b via image signal line ISLb, and image data generated from camera module 1100c is provided to sub-image processor 1212c via image signal line ISLc. The image data processed by sub-image processor 1212b is immediately provided to image generator 1214, but either the image data processed by sub-image processor 1212a or the image data processed by sub-image processor 1212c is selected through a selection element (e.g., a multiflexor) and then provided to image generator 1214.

[0119] Each sub-image processor (1212a, 1212b, 1212c) performs image processing such as defective pixel correction, 3A adjustment (auto-focus correction, auto-white balance, auto-exposure), noise reduction, sharpening, gamma adjustment, and remosaic on the image data provided by the camera modules (1100a, 1100b, 1100c). In one embodiment, re-mosaic signal processing is performed in each camera module (1100a, 1100b, 1100c) before being provided to the sub-image processors (1212a, 1212b, 1212c). The image data processed by each of the sub-image processors (1212a, 1212b, 1212c) is provided to an image generator 1214. The image generator 1214 generates an output image using image data provided by each of the sub-image processors (1212a, 1212b, 1212c) according to image generation information or mode signals.

[0120] Specifically, the image generator 1214 generates an output image by combining at least a portion of the image data generated from the sub-image processors (1212a, 1212b, 1212c) according to image generation information or mode signals. In addition, the image generator 1214 selects one of the image data generated by the sub-image processors (1212a, 1212b, 1212c) according to image generation information or a mode signal, and generates an output image. In one embodiment, the image generation information includes a zoom signal or zoom factor. Also, in one embodiment, the mode signal is a signal based on a mode selected by, for example, a user.

[0121] When the image generation information is a zoom signal (zoom factor) and each camera module 1100a, 1100b, and 1100c has a different observation field of view (viewing angle), the image generator 1214 operates differently depending on the type of zoom signal. For example, when the zoom signal is the first signal, the output image is generated using the image data output from sub-image processor 1212a, the image data output from sub-image processor 1212a among the image data output from sub-image processor 1212c, and the image data output from sub-image processor 1212b. If the zoom signal is a second signal different from the first signal, the image generator 1214 generates an output image using the image data output from the sub-image processor 1212a, the image data output from the sub-image processor 1212c among the image data output from the sub-image processor 1212c, and the image data output from the sub-image processor 1212b. If the zoom signal is a third signal different from the first and second signals, the image generator 1214 does not perform such image data merging, but instead selects one of the image data output from each sub-image processor (1212a, 1212b, 1212c) to generate an output image. However, this embodiment is not limited to this, and the method of processing image data can be modified as needed.

[0122] In one embodiment, the image processing device 1210 further comprises a selection unit (not shown) that selects the outputs of the sub-image processors (1212a, 1212b, 1212c) and transmits them to the image generator 1214. In this case, the selectors perform different operations depending on the zoom signal or zoom factor. For example, when the zoom signal is the fourth signal (for example, when the zoom magnification is the first magnification), the selection unit selects one of the outputs of the sub-image processors (1212a, 1212b, 1212c) and transmits it to the image generator 1214. In addition, when the zoom signal is a fifth signal different from the fourth signal (for example, the zoom magnification is second magnification), the selection unit sequentially transmits p outputs (p is a natural number greater than or equal to 2) of the outputs of the sub-image processors (1212a, 1212b, 1212c) to the image generator 1214.

[0123] For example, the selection unit transmits the outputs of the sub-image processor 1212b and the sub-image processor 1212c to the image generator 1214 in sequence. The selection unit also transmits the outputs of the sub-image processor 1212a and the sub-image processor 1212b to the image generator 1214 in sequence. The image generator 1214 merges the p outputs provided in sequence to generate a single output image. Here, image processing such as demosaic, downscaling to video / preview resolution size, gamma correction, and HDR (High Dynamic Range) processing is performed in advance in the sub-image processors (1212a, 1212b, 1212c), and the processed image data is then transmitted to the image generator 1214. Therefore, even if the processed image data is provided to the image generator 1214 through a selector on one signal line, the image merging operation of the image generator 1214 can be performed at high speed.

[0124] In one embodiment, the image generator 1214 receives multiple image data with different exposure times from at least one of the multiple sub-image processors (1212a, 1212b, 1212c) and performs HDR (high dynamic range) processing on the multiple image data to generate merged image data with an increased dynamic range. The camera module controller 1216 provides control signals to each of the camera modules (1100a, 1100b, 1100c). The control signals generated by the camera module controller 1216 are provided to the corresponding camera modules (1100a, 1100b, 1100c) through mutually separated control signal lines (CSLa, CSLb, CSLc).

[0125] One of the multiple camera modules (1100a, 1100b, 1100c) is designated as a master camera (e.g., 1100b) by image generation information or a mode signal including a zoom signal, and the remaining camera modules (e.g., 1100a and 1100c) are designated as slave cameras. Such information is included in a control signal and provided to the corresponding camera modules (1100a, 1100b, 1100c) through mutually separated control signal lines (CSLa, CSLb, CSLc). The zoom factor or operation mode signal changes which camera module acts as the master and which as the slave. For example, if the viewing angle of camera module 1100a is wider than that of camera module 1100b and the zoom factor indicates a lower zoom magnification, camera module 1100a acts as the master and camera module 1100b acts as the slave. Conversely, when the zoom factor indicates a high zoom magnification, camera module 1100b acts as the master and camera module 1100a acts as the slave.

[0126] In one embodiment, the control signals provided by the camera module controller 1216 to each camera module (1100a, 1100b, 1100c) include a sync enable signal. For example, if camera module 1100b is the master camera and camera module 1100a and camera module 1100c are slave cameras, camera module controller 1216 transmits a synchronization enable signal to camera module 1100b. Upon receiving such a synchronization enable signal, camera module 1100b generates a synchronization signal based on the synchronization enable signal and provides the generated synchronization signal to camera module 1100a and camera module 1100c via synchronization signal line SSL. The camera module 1100b, the camera module 1100a, and the camera module 1100c are synchronized by such a synchronization signal and transmit image data to the application processor 1200.

[0127] In one embodiment, the control signals provided by the camera module controller 1216 to the multiple camera modules (1100a, 1100b, 1100c) include mode information via a mode signal. Based on such mode information, the multiple camera modules (1100a, 1100b, 1100c) operate in a first operation mode and a second operation mode in relation to the sensing speed. In a first operating mode, the multiple camera modules (1100a, 1100b, 1100c) generate image signals at a first rate (e.g., generate image signals at a first frame rate), encode the image signals at a second rate higher than the first rate (e.g., encode image signals at a second frame rate higher than the first frame rate), and transmit the encoded image signals to the application processor 1200. At this time, the second speed is 30 times or less than the first speed.

[0128] The application processor 1200 stores the received image signal, in other words, the encoded image signal, in its internal memory 1230 or in the external memory 1400 of the application processor 1200, and then reads and decodes the encoded image signal from the memory 1230 or the external memory 1400, and displays image data generated based on the decoded image signal. For example, a corresponding sub-image processor among the plurality of sub-image processors (1212a, 1212b, 1212c) of the image processing device 1210 performs decoding and image processing on the decoded image signal.

[0129] In the second operating mode, the multiple camera modules (1100a, 1100b, 1100c) generate image signals at a third rate lower than the first rate (e.g., generate image signals at a third frame rate lower than the first frame rate) and transmit the image signals to the application processor 1200. The image signal provided to the application processor 1200 is an unencoded signal. The application processor 1200 performs image processing on the received image signal or stores the image signal in the memory 1230 or the external memory 1400 .

[0130] The PMIC 1300 supplies power, for example, a power supply voltage, to each of the multiple camera modules (1100a, 1100b, 1100c). For example, under the control of application processor 1200, PMIC 1300 supplies a first power to camera module 1100a through power signal line PSLa, a second power to camera module 1100b through power signal line PSLb, and a third power to camera module 1100c through power signal line PSLc. In response to a power control signal PCON from the application processor 1200, the PMIC 1300 generates power corresponding to each of the multiple camera modules (1100a, 1100b, 1100c) and adjusts the power level.

[0131] The power control signal PCON includes a power adjustment signal for each operation mode of the plurality of camera modules (1100a, 1100b, 1100c). For example, the operation mode includes a low power mode, and the power control signal PCON then includes information about the camera module operating in the low power mode and the power level to be set. The levels of power provided to each of the multiple camera modules (1100a, 1100b, 1100c) may be equal or different. Also, the power level changes dynamically.

[0132] FIG. 16 is a block diagram showing a schematic configuration of an electronic device according to one embodiment of the present invention. The electronic device 2000 in FIG. 16 is a portable terminal. Referring to FIG. 16, the electronic device 2000 includes an application processor 2100, a camera module 2200, a working memory 2300, a storage unit 2400, a display unit 2600, a user interface 2700, and a wireless transceiver unit 2500. The application processor 2100 is implemented as a system-on-chip SoC that controls the overall operation of the image processing system 2000 and runs application programs, an operating system, and the like. The application processor 2100 provides image data provided from the camera module 2200 to the display device 2600 or stores it in the storage unit 2400 .

[0133] The image sensor module 100 described with reference to FIGS. 1 to 11B is applied to a camera module 2200. The camera module 2200 includes an encoder 2210 that compresses image data to generate compressed data and transmits the compressed data to the application processor 2100 . As previously mentioned, the encoder 2210 compresses at least a portion of the image data, for example, at least one pixel group, using an offset encoding scheme if the pixel group falls within an isolated region.

[0134] The application processor 2100 includes a decoder 2110 that decompresses the compressed data using a decoding method that corresponds to the compression method of the encoder 2210, for example, the encoding method. The decoder 2110 decompresses the compressed data received from the camera module 2200 to generate restored image data, and the application processor 2100 performs image processing on the restored image data. The application processor 2100 displays the restored image data or the processed image data on the display device 2600 or stores it in the storage unit 2400 . The working memory 2300 may be implemented as a volatile memory such as a DRAM or SRAM, or a non-volatile resistive memory such as an FeRAM, a RRAM, or a PRAM. The working memory 200 stores programs and / or data that are processed or executed by the application processor 2100 .

[0135] The storage unit 2400 may be implemented as a non-volatile memory device such as a NAND flash or a resistive memory, and may also be provided as a memory card (MMC, eMMC, SD, microSD, etc.). The storage unit 2400 stores image data received from the camera module 2200 or data processed or generated by the application 2100 . The user interface 2700 may be implemented with various devices capable of receiving user input, such as a keyboard, a curtain key panel, a touch panel, a fingerprint sensor, or a microphone. User interface 2700 receives user input and provides signals corresponding to the received user input to application processor 2100. The wireless transceiver unit 2500 includes a transceiver 2510, a modem 2520, and an antenna 2530.

[0136] The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the technical scope of the present invention. [Explanation of symbols]

[0137] 10, 10b Image Processing System 100, 100a, 100b Image Sensor Module 110 Image Sensor 120, 120a encoder 121 Reference Pixel Detector 122 Compression Circuit 123 Mode Selector 124 Restored Image Generator 125 Reference Buffer 126 Advance Detector 130, 210 interface 150 Processing Logic 160, 220 memory 200, 200b Image processing device 230 decoder 240 Image Signal Processor

Claims

1. 1. An image compression method for compressing image data generated by an image sensor, comprising: receiving pixel values ​​of a plurality of target pixels in a target pixel group to be compressed within the image data and reference values ​​of a plurality of reference pixels to be used in compressing the target pixel group; applying an offset value to each of the reference values ​​to generate a virtual reference map; compressing the pixel values ​​of the target pixel group based on the virtual reference map; generating a bitstream based on the compression result and compression information based on the virtual reference map; The image compression method, wherein the reference value corresponds to a restored pixel value of the reference pixel that was compressed prior to the target pixel group, and a difference between the pixel value of the pixel included in the target pixel group and the reference value is equal to or greater than a threshold value.

2. 2. The image compression method of claim 1, wherein generating the virtual reference map comprises setting the offset value based on the reference value.

3. 2. The image compression method of claim 1, wherein the compressing the pixel values ​​comprises calculating a first difference value between the compensated reference value included in the virtual reference map and a pixel value of a first pixel among a plurality of pixels included in the target pixel group, and a plurality of second difference values ​​between the pixel values ​​of the plurality of target pixels.

4. 4. The image compression method of claim 3, wherein the bitstream includes data indicating the first difference value and the second difference value, and mode information indicating a compression method.

5. 5. The image compression method of claim 4, wherein the number of bits allocated to the area including the first difference value is greater than the number of bits allocated to the area including each of the second difference values.

6. The image compression method of claim 4, wherein the bitstream further includes additional information indicating the amount of bit shift of the data or the offset value.

7. 2. The image compression method of claim 1, wherein the compressing the pixel values ​​comprises calculating a first difference value between an average value of the pixel values ​​and at least one compensated reference value included in the virtual reference map, and a second difference value between the average value and each of the pixel values.

8. after generating the bitstream, decompressing the bitstream to generate recovered pixel values; 2. The image compression method of claim 1, further comprising: generating a restored image including reference pixels that are compressed after the target pixel group and are used to compress a next target pixel group based on the restored pixel values.

9. an image sensor for generating image data including a plurality of pixels; an encoder that sequentially compresses the image data generated by the image sensor in units of pixel groups to generate compressed data including a plurality of bitstreams, and compresses the pixel groups to be compressed using at least one of a plurality of encoding methods; an interface for outputting the compressed data to an external image processing device; the encoder applies an offset value to each of reference values ​​of reference pixels disposed adjacent to the target pixel group according to a first encoding method among the plurality of encoding methods to generate a virtual reference map; compressing the group of pixels of interest based on the virtual reference map; The reference value corresponds to a restored pixel value of the reference pixel that was compressed prior to the target pixel group, and a difference between the pixel value of the pixel included in the target pixel group and the reference value is greater than or equal to a threshold.

Images