Camera module, image processing system, and image compression method

Abstract

To provide a camera module, an image processing system, and an image compressing method.SOLUTION: An image compressing method for compressing each of a plurality of pixel groups constituting image data according to the present invention includes the steps of detecting a defective pixel from among a plurality of pixels in the pixel groups, generating a flag expressing positional information of the defective pixel, calculating a difference value between a reference pixel value and pixel values of the pixels other than the defective pixel in the pixel groups, and generating a bit stream including the flag and the difference value.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a camera module, an image processing system, and an image compression method, and more particularly to a camera module, an image processing system, and an image compression method that compress image data taking into account defective pixels. [Background technology]

[0002] Recently, as demand for high-quality photographs and videos has increased, the size of image sensor pixels has become smaller and more numerous. As a result, a process problem has arisen in which image sensor pixels contain defective pixels of various shapes at various positions. Since a large number of defective pixels of various shapes are not used to generate photographs and videos, there has been a problem of degrading the performance of the image sensor. Therefore, there is a need for a technology to correct image data output from defective pixels. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-100898 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention has been made in consideration of the above-mentioned conventional problems, and an object of the present invention is to provide a camera module, an image processing device, and an image compression method that compress image data taking into account defective pixels. [Means for solving the problem]

[0005] In order to achieve the above object, according to one aspect of the present invention, an image compression method for compressing each of a plurality of pixel groups constituting image data includes the steps of: detecting defective pixels from among a plurality of pixels in a pixel group; generating a flag indicating position information of the defective pixels; calculating a difference between a pixel value of the remaining pixels in the pixel group excluding the defective pixels and a reference pixel value; and generating a bitstream including the flag and the difference value.

[0006] In order to achieve the above object, one aspect of the present invention provides a camera module comprising: an image sensor that generates image data including a plurality of pixels; an encoder that divides the plurality of pixels into a plurality of pixel groups and compresses the plurality of pixel groups to generate compressed data including a plurality of bit streams; and a memory that stores reference information including pixel values ​​of the pixels compressed by the encoder, wherein the encoder detects defective pixels included in a first pixel group, compresses pixel values ​​of first pixels included in the first pixel group based on the defective pixel detection result and the reference information to generate a first bit stream corresponding to the first pixel group, and updates the reference information based on corrected pixel values ​​obtained by correcting the pixel values ​​of the defective pixels.

[0007] In order to achieve the above object, one aspect of the present invention provides an image processing system comprising an image sensor that generates image data including a plurality of pixels, an encoder that sequentially compresses a plurality of pixel groups that make up the image data to generate a plurality of bit streams, and a decoder that decompresses the plurality of bit streams to restore the image data, wherein the encoder detects defective pixels in a second pixel group for each of the plurality of pixel groups, compresses the pixel values ​​of the second pixel group according to reference information generated based on the pixel values ​​of a first pixel group that was compressed before the second pixel group, and updates the reference information based on the results of the detection of the defective pixels. [Effects of the Invention]

[0008] According to the image compression method, camera module, and image processing system of the present invention, reference information used for compression or decompression is generated taking into account defective pixels, and image data is compressed or compressed data is decompressed based on the generated reference information, thereby significantly reducing the number of defective pixel detection operations, thereby reducing the amount of calculation and power consumption. [Brief explanation of the drawings]

[0009] [Figure 1] 1 illustrates an example of an image processing system according to an embodiment of the present invention. [Figure 2] FIG. 2 illustrates an example of an encoder according to an embodiment of the present invention. [Figure 3] 1 is a conceptual diagram illustrating an example of an image compression method according to an embodiment of the present invention. [Figure 4] 10 is a conceptual diagram illustrating another example of an image compression method according to an embodiment of the present invention. [Figure 5A] FIG. 2 illustrates an example of a bitstream according to one embodiment of the present invention. [Figure 5B] FIG. 10 is a diagram illustrating another example of a bitstream according to an embodiment of the present invention. [Figure 6] FIG. 10 is a diagram illustrating yet another example of a bitstream according to an embodiment of the present invention. [Figure 7] 2 is a flowchart illustrating an image compression method according to an embodiment of the present invention. [Figure 8] FIG. 2 illustrates a decoder according to one embodiment of the present invention. [Figure 9A] 1 is a conceptual diagram illustrating an example of an image decompression method according to an embodiment of the present invention. [Figure 9B] 10 is a conceptual diagram illustrating another example of an image decompression method according to an embodiment of the present invention. [Figure 10] FIG. 2 illustrates another example of an image processing system according to an embodiment of the present invention. [Figure 11] FIG. 10 illustrates another example of an encoder according to an embodiment of the present invention. [Figure 12A] 10 is a table illustrating an example of compression information according to an embodiment of the present invention. [Figure 12B] 10 is a table illustrating another example of compressed information according to an embodiment of the present invention. [Figure 13] FIG. 10 illustrates yet another example of an image processing system according to an embodiment of the present invention. [Figure 14] 1 illustrates an electronic device according to one embodiment of the present invention. [Figure 15] 1 illustrates a portion of an electronic device according to one embodiment of the present invention. [Figure 16] 1 is a diagram showing a specific configuration of a camera module according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, specific examples of embodiments of the present invention will be described in detail with reference to the drawings.

[0011] 1 is a diagram illustrating an example of an image processing system according to an embodiment of the present invention. Referring to FIG. 1, the image processing system 10 includes a camera module 100 and an image processing device 200. In this embodiment, the camera module 100 includes an image sensor 110, an encoder 120, a memory 130, and an interface (I / F) 140. In this embodiment, the image processing device 200 includes an interface 210, a memory 220, a decoder 230, and an image signal processor (ISP) 240.

[0012] For example, the image processing system 10 may be implemented as a personal computer (PC), an Internet of Things (IoT) device, or a portable electronic device. Examples of portable electronic devices include laptop computers, mobile phones, smartphones, tablet PCs, personal digital assistants (PDAs), enterprise digital assistants (EDAs), digital still cameras, digital video cameras, audio devices, portable multimedia players (PMPs), personal navigation devices (PNDs), MP3 players, handheld game consoles, e-books, and wearable devices. The image processing system 10 may also be installed in electronic devices such as drones and advanced driver assistance systems (ADASs), or electronic devices installed as components in vehicles, furniture, manufacturing equipment, doors, various measuring instruments, and the like.

[0013] The camera module 100 captures an external subject (or object) and generates image data IDT. In this embodiment, the camera module 100 includes an image sensor 110 that converts an optical signal of the subject into an electrical signal. For example, the image sensor 110 includes a pixel array 111 in which a plurality of pixels are two-dimensionally arranged. Each of the plurality of pixels is assigned one of a plurality of reference colors. For example, the plurality of reference colors may include RGB (red, green, blue) or RGBW (red, green, blue, white), but may also include other colors. For example, the plurality of reference colors may include cyan, yellow, green, and magenta. The pixel array 111 generates a pixel signal including information about the reference color of each of the plurality of pixels.

[0014] The pixel array 111 includes a plurality of row lines, a plurality of column lines, a plurality of pixels connected to the row lines and the column lines and arranged in a matrix, and a plurality of color filters arranged to correspond to the plurality of pixels. For example, referring to FIG. 1, the color filter has a configuration in which 2×2 cells each including a red pixel R, a blue pixel B, and two green pixels (Gr, Gb) are repeatedly arranged. Such a pattern is called a Bayer pattern.

[0015] In another example, the color filter has a repeating configuration of pixel groups corresponding to each reference color. For example, the color filter has a repeating configuration of a red pixel group including red pixels R arranged in a 2×2 array, a first green pixel group including first green pixels Gr arranged in a 2×2 array, a blue pixel group including blue pixels B arranged in a 2×2 array, and a second green pixel group including green pixels Gb arranged in a 2×2 array. Such a pattern is called a tetra pattern.

[0016] In yet another example, the color filter has a repeating configuration of a red pixel group including red pixels R arranged in a 3 × 3 array, a first green pixel group including first green pixels Gr arranged in a 3 × 3 array, a blue pixel group including blue pixels B arranged in a 3 × 3 array, and a second green pixel group including green pixels Gb arranged in a 3 × 3 array. Such a pattern is called a nona pattern.

[0017] However, the present invention is not limited thereto, and the color filter is repeatedly arranged with a red pixel group, a blue pixel group, a first green pixel group, and a second pixel group, each of which includes pixels arranged in a 2n×2n or 3n×3n (n is a positive integer) array.

[0018] As a non-limiting example, the image sensor 110 may be implemented using a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (COMS), or may be implemented using various other types of photoelectric conversion elements. In this embodiment, the image sensor 110 outputs image data IDT that has been pre-processed for pixel signals generated by the pixel array 111.

[0019] Meanwhile, in the above example, the image data IDT is described as including information about a reference color (e.g., RGB information), but the present invention is not limited thereto. Specifically, the image sensor 110 converts the RGB information of each pixel into YUV information including information about luminance and chrominance through color space conversion, so that the image data IDT includes YUV information corresponding to each pixel. Even when the image data IDT includes YUV information, the technical concept of the present invention is substantially the same.

[0020] The camera module 100 compresses the image data IDT using the encoder 120 to reduce power consumption during data transmission and to efficiently use data storage space. Specifically, the encoder 120 receives the image data IDT from the image sensor 110 and compresses the image data IDT to generate compressed data CDT. The compressed data CDT is implemented in the form of an encoded bit stream. Hereinafter, the encoded bit stream will be simply referred to as a bit stream.

[0021] In this embodiment, the encoder 120 compresses the image data IDT in units of pixel groups. Here, a pixel group is set to include a predetermined number of pixels sequentially arranged according to the pattern of the image data IDT, or to include adjacent pixels corresponding to the same reference color. For example, if the image data IDT is a Bayer pattern, a pixel group is set to include a predetermined number of pixels (e.g., four) sequentially arranged horizontally or vertically. As another example, if the image data IDT is a tetrahedral pattern (or a nona pattern), a pixel group is set to include four (or nine) adjacent pixels corresponding to the same reference color (e.g., red, blue, green, etc.). The encoder 120 compresses one pixel group to generate one bitstream, and generates compressed data CDT based on the bitstreams of all pixel groups in the image data IDT.

[0022] When compressing a specific pixel group, the encoder 120 performs compression using a first reference map RM1 generated based on pixel values ​​corresponding to pixels compressed prior to the pixel group. Specifically, the encoder 120 determines a reference value in the first reference map RM1 based on the pixel value of at least one pixel adjacent to the target pixel to be compressed within the pixel group, and compresses the pixel value of the target pixel based on the reference value and the pixel value of the target pixel.

[0023] When the encoder 120 completes the compression of a pixel group, it adds the pixel values ​​of the compressed pixel group to the existing first reference map RM1 to generate (or update) a new first reference map RM1, and then uses the new first reference map RM1 to compress the next sequential pixel group.

[0024] The pixel value of the target pixel and the pixel values ​​of the adjacent pixels are likely to be similar to each other. Therefore, when the image data IDT is compressed to generate compressed data CDT using the above method, compression efficiency is improved and data loss is reduced. For convenience of explanation, information including pixel values ​​corresponding to previously compressed pixels will be referred to as a first reference map RM1 below. However, the present invention is not limited to this and may be referred to by various names, such as reference information.

[0025] Meanwhile, the image data IDT includes pixel values ​​due to defective pixels. Here, defective pixels include static defective pixels that are continuously turned on or off, and dynamic defective pixels that are randomly turned on or off. Since static defective pixels are fixed in position, their pixel values ​​can be corrected through a pre-processing operation of the image sensor 110, which allows for simple calculations. On the other hand, since dynamic defective pixels are not fixed in position and require more calculations for correction, correction through a pre-processing operation of the image sensor 110 is somewhat more difficult.

[0026] Since the encoder 120 is configured for compression, not correction, of the image data IDT, it compresses the image data IDT even if it contains defective pixels. Therefore, when the encoder 120 performs compression using the above method, the pixel values ​​of the defective pixels are included in the first reference map RM1. Because the pixel values ​​of the defective pixels are significantly different from the pixel values ​​of the surrounding pixels, the compression efficiency when using the first reference map RM1 is reduced, which can lead to increased data loss.

[0027] To improve compression efficiency and prevent data loss, the encoder 120 detects defective pixels among the pixels included in the first reference map RM1. If the first reference map RM1 includes defective pixels, the encoder 120 determines reference values ​​based on the pixel values ​​of the remaining pixels (i.e., normal pixels) excluding the defective pixels. However, the above-described defective pixel detection operation requires a large amount of calculations and increases power consumption because it is performed for each compression operation for all pixels.

[0028] Therefore, the encoder 120 according to the present invention goes beyond the compression method described above to correct pixel values ​​of defective pixels (especially dynamic defective pixels) and include the corrected pixel values ​​in the first reference map RM1, so that the first reference map RM1 does not include pixel values ​​of defective pixels, and therefore the defective pixel detection operation for the first reference map RM1 is omitted.

[0029] The memory 130 is a storage location for storing data, such as the image data IDT or the first reference map RM1, and may be a volatile memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM), or a non-volatile memory such as a phase change random access memory (PRAM), a resistive random access memory (ReRAM), or a flash memory.

[0030] The encoder 120 provides the generated compressed data CDT to the image processing device 200 via the interface 140. For example, the interface 140 is implemented by a camera serial interface (CSI) based on the MIPI (Mobile Industry Processor Interface). However, the type of the interface 140 is not limited thereto and may be implemented by various protocol standards.

[0031] The image processing device 200 converts image data received from the camera module 100 to generate an image to be displayed on a display (not shown). Specifically, the image processing device 200 receives compressed data CDT from the camera module 100, decompresses the compressed data CDT to generate decompressed data DDT, and performs image processing operations based on the decompressed data DDT to generate a final image.

[0032] In this embodiment, the image processing device 200 receives the compressed data CDT from the camera module 100 via an interface 210. The interface 210 is implemented using MIPI, like the interface 140, but is not limited to this. The image processing device 200 stores the received compressed data CDT in a memory 220.

[0033] The memory 220 is a storage location for storing data, and stores, for example, an operating system (OS), various programs, and various data (for example, compressed data CDT). The memory 220 is a volatile memory such as a DRAM or an SRAM, or a non-volatile memory such as a PRAM, ReRAM, or a flash memory.

[0034] The decoder 230 reads the compressed data CDT from the memory 220, decompresses the compressed data CDT to generate decompressed data DDT, and outputs the generated decompressed data DDT to the image signal processor 240.

[0035] In this embodiment, the decoder 230 decompresses the compressed data CDT in units of pixel groups using a second reference map RM2 generated based on pixel values ​​corresponding to pixels decompressed prior to the pixel group.

[0036] According to one embodiment of the present invention, the second reference map RM2 includes corrected pixel values ​​of defective pixels (especially dynamic defective pixels), as in the first reference map RM1 described above. In other words, if a defective pixel is present in a decompressed pixel group, the decoder 230 corrects the pixel value of the defective pixel and generates (or updates) a new second reference map RM2 including the corrected pixel value. That is, the decoder 230 updates the second reference map RM2 with the compensated value of the defective pixel. The decoder 230 then decompresses the next pixel group based on the new second reference map RM2. The second reference map RM2 is stored in the memory 220. Hereinafter, for convenience of explanation, information including pixel values ​​corresponding to pixels decompressed before the pixel group to be decompressed will be referred to as the second reference map RM2. However, the present invention is not limited to this and various names may be used.

[0037] Meanwhile, each of the encoder 120 and the decoder 230 may be implemented by software or hardware, or a combination of software and hardware such as firmware. When the encoder 120 and the decoder 230 are implemented by software, the respective functions described above are embodied in programmed source code, which is loaded onto recording media provided in the camera module 100 and the image processing device 200, respectively, and the functions of the encoder 120 and the decoder 230 are implemented by processors (e.g., image processing processors) provided in the camera module 100 and the image processing device 200 executing the software. When the encoder 120 and the decoder 230 are implemented by hardware, the encoder 120 and the decoder 230 include logic circuits and registers, and perform the respective functions described above based on register settings.

[0038] The image signal processor 240 performs various image processing on the received decompressed data DDT. In a non-limiting example, the image signal processor 240 performs at least one of the following image processing on the decompressed data DDT: bad pixel correction, offset correction, lens distortion correction, color gain correction, shading correction, gamma correction, denoising, and sharpening. In some embodiments, some of the above-described image processing may be omitted depending on the capabilities of the camera module 100. For example, if the camera module 100 includes a high-quality image sensor 110, bad pixel correction (especially static bad pixel correction) or offset correction may be omitted from the image processing.

[0039] 1 illustrates the image processing system 10 as including a camera module 100 and an image processing device 200, but the present invention is not limited thereto. For example, the image processing system 10 may be embodied to include only some of the camera module 100 and the image processing device 200, or to include a plurality of camera modules 100. Also, while FIG. 1 illustrates the decoder 230 and the image signal processor 240 as separate components, the present invention is not limited thereto. For example, the image signal processor 240 may be embodied to include the decoder 230.

[0040] 1, the memory 130 and the memory 220 are illustrated and described as being included in the camera module 100 and the image processing device 200, respectively, but the present invention is not limited thereto. For example, the memory 130 and the memory 220 may be embodied so as to be located outside the camera module 100 or the image processing device 200, respectively.

[0041] The image processing system according to the technical concept of the present invention generates a reference map used for compression or decompression in consideration of defective pixels, and compresses image data or decompresses compressed data based on the generated reference map, thereby significantly reducing the number of defective pixel detection operations, thereby reducing power consumption of the image processing system.

[0042] Figure 2 is a diagram illustrating an example of an encoder according to an embodiment of the present invention. Specifically, Figure 2 illustrates the encoder 120 of Figure 1. Referring to Figures 1 and 2, the encoder 120 includes a bad pixel detector 121, a compressor 123, and a first reference map generator 125.

[0043] The bad pixel detector 121 receives image data IDT from the image sensor 110. The bad pixel detector 121 detects bad pixels for a pixel group to be compressed based on the received image data IDT. A detailed description of this will be given below with reference to Figures 3 and 4. The bad pixel detector 121 transmits bad pixel information BP including the detection result to the compressor 123 and / or the first reference map generator 125.

[0044] The compressor 123 generates compressed data CDT by compressing the image data IDT using the bad pixel information BP and the first reference map RM1. Specifically, the compressor 123 identifies bad pixels in a pixel group to be compressed in the image data IDT based on the bad pixel information BP received from the bad pixel detector 121. Then, the compressor 123 compresses pixel values ​​of the remaining pixels (i.e., normal pixels) in the pixel group excluding the bad pixels using the first reference map RM1.

[0045] For example, the compressor 123 determines a reference value based on the pixel value of at least one pixel adjacent to the pixel group or a normal pixel in the first reference map RM1. The compressor 123 then generates a bitstream BS corresponding to the pixel group by compressing the reference value and the pixel values ​​of normal pixels that are not defective pixels. This will be described in detail later with reference to FIGS. 3 and 4.

[0046] The compressor 123 repeats the above-described operation for each pixel group of the image data IDT to generate a bitstream BS corresponding to the pixel group, and then generates compressed data CDT based on the bitstream BS.

[0047] The compressor 123 transmits the generated compressed data CDT to the interface 140. The interface 140 transmits the compressed data CDT to the image processing device 200. In addition, the compressor 123 transmits each bitstream BS to the first reference map generator 125 every time it generates the bitstream BS.

[0048] The first reference map generator 125 generates a new first reference map RM1(NEW) based on the received bad pixel information BP and bitstream BS. Specifically, the first reference map generator 125 first decodes the bitstream BS to restore pixel values ​​of the pixel group. The first reference map generator 125 then identifies the bad pixels in the pixel group based on the bad pixel information BP. The first reference map generator 125 then corrects the pixel values ​​of the bad pixels among the restored pixel values ​​to values ​​similar to those of the surrounding pixels. The first reference map generator 125 then adds the restored pixel values ​​of the non-defective pixels in the pixel group and the corrected pixel values ​​of the bad pixels to the existing first reference map RM1 to generate the new first reference map RM1(NEW). This will be described in detail below with reference to FIGS. 3 and 4.

[0049] In this way, when one pixel group is compressed, the first reference map generator 125 adds pixel values ​​corresponding to the compressed pixel group (i.e., restored pixel values ​​of the pixel group and / or corrected pixel values ​​of defective pixels) to generate a new first reference map RM1(NEW) for compressing the next pixel group. The first reference map generator 125 stores the generated first reference map RM1(NEW) in the memory 130. The compressor 123 reads the new first reference map RM1(NEW) stored in the memory 130 and performs compression on the next pixel group based on the read first reference map RM1.

[0050] Meanwhile, each of the bad pixel detector 121, the compressor 123, and the first reference map generator 125 may be implemented by software, hardware, or a combination of software and hardware, such as firmware. When the bad pixel detector 121, the compressor 123, and the first reference map generator 125 are implemented by software, the respective functions described above are implemented by programmed source code, which is loaded onto a recording medium included in the camera module 100. A processor (e.g., a microprocessor) included in each of the camera modules 100 executes the software to implement the functions of the bad pixel detector 121, the compressor 123, and the first reference map generator 125. When the bad pixel detector 121, the compressor 123, and the first reference map generator 125 are implemented by hardware, the bad pixel detector 121, the compressor 123, and the first reference map generator 125 include logic circuits and registers and perform the respective functions described above based on the register settings.

[0051] 3 is a conceptual diagram illustrating an example of an image compression method according to an embodiment of the present invention. Specifically, FIG. 3 illustrates a compression method for Bayer pattern image data IDT by the encoder 120. The following description will be given on the assumption that one pixel group is set to include four pixels arranged sequentially. Also, an embodiment will be described in which the encoder 120 compresses a pixel group PG including a defective pixel R7 and normal pixels Gr7, R8, and Gr8. For ease of explanation, a pixel to be processed among the pixels in the pixel group PG will be referred to as a target pixel.

[0052] The bad pixel detector 121 detects bad pixels among the pixels in the pixel group PG. Specifically, the bad pixel detector 121 detects whether a target pixel is a bad pixel based on the pixel value of at least one first pixel adjacent to the target pixel in the image data IDT. In some embodiments, the bad pixel detector 121 detects whether a target pixel is a bad pixel based on the pixel value of at least one first pixel adjacent to the target pixel that corresponds to the same reference color as the target pixel. In this case, various criteria can be set to determine whether a pixel is adjacent to the target pixel. For example, the pixel may be determined to be adjacent to a pixel that is directly adjacent to the target pixel or a pixel located within a certain distance from the target pixel.

[0053] Specifically, if there are multiple first pixels, the bad pixel detector 121 calculates an average pixel value of the first pixels and then calculates a difference between the average pixel value and the pixel value of the target pixel. If the calculated difference exceeds a threshold value, the bad pixel detector 121 detects the target pixel as a bad pixel. Here, the threshold value is preset by a user or a manufacturer and has a different value for each piece of image data IDT. The bad pixel detector 121 generates bad pixel information by repeating the above operation for each pixel in the pixel group PG. At this time, the bad pixel information includes position information of the bad pixel.

[0054] 3, the bad pixel detector 121 calculates an average pixel value of the first pixels (R3, R6, R8, and R11) adjacent to the target pixel R7 in the image data IDT, corresponding to the same red hue as the target pixel R7. The bad pixel detector 121 then calculates the difference between the calculated average pixel value and the pixel value of the pixel R7. If the calculated difference exceeds a threshold value, the bad pixel detector 121 detects the pixel R7 as a bad pixel.

[0055] Meanwhile, the defective pixel detection operation of the defective pixel detector 121 is not limited to the above example, and various methods can be applied. For example, the defective pixel detector 121 detects whether a target pixel is a defective pixel based on the pixel value of at least one first pixel adjacent to the target pixel, regardless of the reference color.

[0056] 3, the bad pixel detector 121 detects whether pixel R7 is a bad pixel based on the pixel value of the first pixel (R6, Gr6, Gr7, R8) adjacent to pixel R7 in the horizontal direction in the image data IDT. As another example, the bad pixel detector 121 detects whether pixel R7 is a bad pixel based on the pixel value of the first pixel (R3, Gb3, Gb7, R11) adjacent to pixel R7 in the vertical direction. As yet another example, the bad pixel detector 121 detects whether pixel R7 is a bad pixel based on the pixel values ​​of the remaining pixels (Gr7, R8, Gr8) of the pixel group to which pixel R7 belongs.

[0057] The compressor 123 identifies the defective pixels in the pixel group based on the defective pixel information received from the defective pixel detector 121, and compresses the pixel values ​​of the remaining pixels excluding the defective pixels.

[0058] Specifically, the compressor 123 identifies at least one second pixel in the first reference map RM1 that corresponds to the same reference color as the target pixel that is not a defective pixel and that is adjacent to the target pixel (or pixel group PG). The compressor 123 then determines a reference value based on the pixel value of the identified second pixel. The compressor 123 then compresses the pixel value of the target pixel based on the reference value and the pixel value of the target pixel. At this time, various criteria can be set for determining the adjacent second pixel.

[0059] In some embodiments, when there are multiple second pixels, the compressor 123 determines a pixel at a specific position among the second pixels (e.g., to the left, above, or diagonally opposite the target pixel) as the reference value. Alternatively, the compressor 123 calculates an average pixel value of the identified second pixels and determines the calculated average pixel value as the reference value. Then, the compressor 123 calculates a difference value RES between the reference value and the pixel value of the target pixel.

[0060] The compressor 123 repeats the above operation for all pixels in the pixel group PG to calculate a plurality of difference values ​​RES and generate a bitstream BS including the plurality of difference values ​​RES. A compression method performed based on the difference values ​​RES in this manner is called differential pulse code modulation (DPCM).

[0061] 3, the compressor 123 identifies, in the first reference map RM1, second pixels (Gb3, Gb4, Gb7, Gb8, Gr3, Gr6, Gr8) adjacent to pixel Gr7, where the target pixel corresponds to the same reference color as pixel Gr7. The compressor 123 then determines, for example, the pixel value of pixel Gr6 located to the left of pixel Gr7 in the identified second pixels (Gb3, Gb4, Gb7, Gb8, Gr3, Gr6, Gr8), as a reference value. The compressor 123 then calculates a difference value RES between the reference value and the pixel value of pixel Gr7, and generates a bitstream BS including the calculated difference value RES.

[0062] When the compression of pixel group PG is completed, first reference map generator 125 adds pixel values ​​corresponding to the compressed pixel group PG to the existing first reference map RM1 to generate a new first reference map RM1(NEW). For example, first reference map generator 125 decodes bitstream BS of pixel group PG received from compressor 123 to restore pixel values ​​of pixel group PG, and adds the restored pixel values ​​to the existing first reference map RM1 to generate a new first reference map RM1(NEW). Meanwhile, the method by which first reference map generator 125 restores pixel values ​​based on bitstream BS is substantially the same as the decompression method of decoder 230, which will be described later. In another example, first reference map generator 125 may add pixel values ​​corresponding to pixel group PG in image data IDT to the existing first reference map RM1.

[0063] Meanwhile, according to the technical idea of ​​the present invention, when a defective pixel exists in the pixel group PG, the first reference map generator 125 first corrects the pixel value of the defective pixel and then adds it to the existing first reference map RM1.

[0064] In some embodiments, the first reference map generator 125 corrects the pixel value of the defective pixel in various ways based on the pixel value of a third pixel adjacent to the defective pixel. Specifically, the first reference map generator 125 identifies a third pixel adjacent to the defective pixel that corresponds to the same reference color as the defective pixel. Then, the first reference map generator 125 corrects the pixel value of the defective pixel based on the pixel value of the identified third pixel.

[0065] 3, the first reference map generator 125 may correct the pixel value of pixel R7, which corresponds to the same red hue as pixel R7 (the defective pixel), to the pixel value of pixel R3 located above pixel R7. As another example, the first reference map generator 125 may correct the pixel value of pixel R7 to the pixel value of pixel R6 located to the left of pixel R7. As yet another example, the first reference map generator 125 may correct the pixel value of pixel R7 to the pixel value of pixel R8 located to the right of pixel R7. As yet another example, the first reference map generator 125 may correct the pixel value of pixel R7 to the average pixel value of the third pixels (R2 and R4) located diagonally opposite pixel R7.

[0066] Then, the first reference map generator 125 adds the corrected pixel value R7b of pixel R7 according to any one of the above methods to the existing first reference map RM1. The first reference map generator 125 also adds the pixel values ​​of normal pixels (Gr7, R8, Gr8) in the pixel group PG to the existing first reference map RM1 to generate a new first reference map RM1(NEW).

[0067] FIG. 4 is a conceptual diagram illustrating another example of an image compression method according to an embodiment of the present invention. Specifically, FIG. 4 illustrates a compression method for tetra-pattern image data IDT by the encoder 120. The following description will be given on the assumption that one pixel group is set to include adjacent pixels corresponding to the same reference color. Also, an embodiment will be described in which the encoder 120 compresses a pixel group PG including a defective pixel Gb5 and normal pixels Gb6, Gb7, and Gb8. Meanwhile, in the description of FIG. 4, details that overlap with the description of FIG. 3 will be omitted.

[0068] The bad pixel detector 121 detects whether a target pixel is a bad pixel based on the pixel value of at least one first pixel adjacent to the target pixel in the image data IDT. In some embodiments, the bad pixel detector 121 detects whether a target pixel is a bad pixel based on the pixel value of at least one first pixel adjacent to the target pixel, corresponding to the same reference color as the target pixel.

[0069] For example, referring to FIG. 4, the bad pixel detector 121 calculates an average pixel value of the first pixels (Gb6, Gb7, Gb8) adjacent to the target pixel Gr5 in the image data IDT, corresponding to the same green hue as the target pixel Gr5. The bad pixel detector 121 then calculates a difference between the calculated average pixel value and the pixel value of the target pixel. If the calculated difference exceeds a threshold value, the bad pixel detector 121 detects the target pixel as a bad pixel. However, the bad pixel detection operation of the bad pixel detector 121 is not limited to the above example, and various methods can be used.

[0070] The compressor 123 identifies the defective pixels in the pixel group based on the defective pixel information received from the defective pixel detector 121, and compresses the pixel values ​​of the remaining pixels excluding the defective pixels.

[0071] Specifically, the compressor 123 identifies at least one second pixel adjacent to the target pixel (or pixel group PG) in the first reference map RM1, corresponding to the same reference color as the target pixel that is not a defective pixel. The compressor 123 then determines a reference value based on the pixel value of the identified second pixel. The compressor 123 then compresses the pixel value of the target pixel based on the reference value and the pixel value of the target pixel.

[0072] In some embodiments, when there are multiple second pixels, the compressor 123 determines the pixel value of a pixel at a specific position among the second pixels (e.g., to the left, above, or diagonally opposite the target pixel) as the reference value. Alternatively, the compressor 123 calculates an average pixel value of pixel values ​​of pixel groups compressed before the pixel group PG to be currently compressed, and determines the calculated average pixel value as the reference value. For example, if a first pixel group PG1 is compressed before a second pixel group PG2, the compressor 123 uses the average pixel value calculated for the first pixel group PG1 as the reference value when compressing the second pixel group PG2. The compressor 123 then calculates a difference value RES between the reference value and the pixel value of the target pixel, and generates a bitstream BS including the calculated difference value RES.

[0073] The compressor 123 calculates a plurality of difference values ​​RES by repeating the above-described operations for all pixels in the pixel group PG, and generates a bitstream BS including the plurality of difference values ​​RES.

[0074] 4, the compressor 123 identifies, in the first reference map RM1, second pixels (Gb1, Gb2, Gb3, Gb4) adjacent to the pixel Gb6, which correspond to the same reference color as the non-defective pixel Gb6. The compressor 123 then determines, for example, the average pixel value of the identified second pixels (Gb2, Gb4, Gb7) as the reference value. The compressor 123 then calculates a difference value RES between the reference value and the pixel value of the pixel Gb6, and generates a bitstream BS including the calculated difference value RES.

[0075] When the compression of the pixel group PG is completed, the first reference map generator 125 adds pixel values ​​corresponding to the compressed pixel group PG to the existing first reference map RM1 to generate a new first reference map RM1(NEW). For example, the first reference map generator 125 decodes the bitstream BS received from the compressor 123 to restore pixel values ​​of the pixel group PG, and adds the restored pixel values ​​to the existing first reference map RM1 to generate a new first reference map RM1(NEW).

[0076] That is, the first reference map generator 125 first corrects the pixel values ​​of the defective pixels in the pixel group PG, and then adds them to the existing first reference map RM1, thereby generating a new first reference map RM1.

[0077] In some embodiments, the first reference map generator 125 compensates for the pixel value of the defective pixel in various ways based on the pixel value of a third pixel adjacent to the defective pixel. Specifically, the first reference map generator 125 identifies at least one third pixel adjacent to the defective pixel that corresponds to the same reference color as the defective pixel. Then, the first reference map generator 125 compensates for the pixel value of the defective pixel based on the pixel value of the identified third pixel.

[0078] For example, referring to FIG. 4, the first reference map generator 125 may correct the pixel value of pixel Gb5 to the pixel value of the third pixel Gb2 located to the left of pixel Gb5, corresponding to the same green hue as pixel Gb5, which is a defective pixel. As another example, the first reference map generator 125 may correct the pixel value of pixel Gb5 to the pixel value of the third pixel located above pixel Gb5. As yet another example, the first reference map generator 125 may correct the pixel value of pixel Gb5 to the pixel value of the third pixel Gb6 located to the right of pixel Gb5. As yet another example, the first reference map generator 125 may correct the pixel value of pixel Gb5 to the average pixel value of pixels Gr4 and Gr7 located diagonally opposite pixel Gb5. As yet another example, the first reference map generator 125 may correct the pixel value of pixel Gb5 to the average pixel value of the remaining non-defective pixels (Gb6, Gb7, and Gb8) in pixel group PG.

[0079] Then, the first reference map generator 125 adds the corrected pixel value Gb5b of the target pixel according to any one of the above examples to the existing first reference map RM1. The first reference map generator 125 also adds the pixel values ​​of the normal pixels (Gb6, Gb7, Gb8) in the pixel group PG to the existing first reference map RM1 to generate a new first reference map RM1(NEW).

[0080] 4, the image data IDT is assumed to be a tetra pattern, but the present invention is not limited to this. For example, even when the image data IDT is a nona pattern, substantially the same method as described above is applied.

[0081] 5A and 5B are diagrams illustrating an encoded bitstream according to an embodiment of the present invention. Specifically, FIG. 5A and FIG. 5B are diagrams illustrating a bitstream BS generated by compressing pixel values ​​of pixels included in one pixel group PG. For ease of understanding, the following describes an embodiment in which one pixel group PG includes four pixels (P1 to P4), and each pixel is composed of 10 bits.

[0082] 5A and 5B, the bitstream BS is composed of 20 bits, including 4 bits indicating a header HEADER, 4 bits indicating a bad pixel flag BP FLAG, and 12 bits indicating a difference value RES. The header HEADER contains information related to the compression of the image data IDT. For example, the header HEADER contains information such as a compression method, compression mode, compression rate, and loss information. The decoder 230 checks the header HEADER and decompresses the bitstream BS according to the scheme corresponding to the header HEADER.

[0083] The bad pixel flag BP FLAG contains information indicating which pixels in the pixel group PG are bad pixels. Specifically, each of the four bits constituting the bad pixel flag BP FLAG corresponds to a pixel (P1 to P4) in the pixel group PG. For example, the first bit of the bad pixel flag BP FLAG corresponds to the first pixel P1, the second bit corresponds to the second pixel P2, the third bit corresponds to the third pixel P3, and the fourth bit corresponds to the fourth pixel P4. When each of the four bits of the bad pixel flag BP FLAG has a specific value (e.g., 1 or 0), it means that the pixel corresponding to that bit is a bad pixel.

[0084] The difference value RES includes the difference value of the normal pixel in the pixel group PG. Here, the difference value of the normal pixel refers to the difference value between the reference value determined based on the first reference map RM1 described in FIGS. 3 and 4 and the pixel value of the target pixel (e.g., a normal pixel or a defective pixel). If the difference value between the reference value and the pixel value of the target pixel is not within the range that can be represented by the bits allocated in the bitstream BS, the least significant bit (LSB) of the difference value is removed. If the difference value is within the representable range, the difference value is included in the bitstream BS.

[0085] According to an embodiment of the present invention, as the number of defective pixels in a pixel group PG increases, the number of bits allocated to the difference values ​​of normal pixels also increases. For example, referring to FIG. 5A, since there is one defective pixel, the fourth pixel P4, in the pixel group PG, the difference values ​​(RES1, RES2, RES3) of the remaining three normal pixels (P1, P2, P3) are configured with 12 bits. For example, 4 bits are allocated to each of the difference values ​​(RES1, RES2, RES3). On the other hand, referring to FIG. 5B, since there are two defective pixels, the third pixel P3 and the fourth pixel P4, in the pixel group PG, the difference values ​​(RES1, RES2) of the remaining two normal pixels (P1, P2) are configured with 12 bits. For example, 6 bits are allocated to each of the difference values ​​(RES1, RES2). As the number of allocated bits increases, the range of the difference values ​​increases, thereby reducing data loss. Meanwhile, the number of bits allocated to each of the difference values ​​of the top pixels may be the same or different.

[0086] The decoder 230 restores pixel values ​​of the pixel group PG based on the bitstream BS. Specifically, the decoder 230 first restores pixel values ​​of normal pixels based on the difference value RES. Then, the decoder 230 restores pixel values ​​of defective pixels based on the restored pixel values ​​of normal pixels. The restoration operation of the decoder 230 will be described in detail below with reference to FIG. 9.

[0087] On the other hand, the number of bits constituting each of the header HEADER, the bad pixel flag BP FLAG, and the difference value RES is not limited to the above example, and it goes without saying that the number of bits can be set to differ depending on the embodiment.

[0088] Figure 6 is a diagram illustrating yet another example of an encoded bitstream according to an embodiment of the present invention. Specifically, Figure 6 is a diagram illustrating a modified embodiment of Figure 5. In the description of Figure 6, the content overlapping with the description of Figure 5 will be omitted.

[0089] Referring to FIG. 6, the bitstream BS is made up of 20 bits, including 4 bits indicating a header HEADER, 2 bits indicating a bad pixel flag BP FLAG, and 14 bits indicating a difference value RES.

[0090] 6 is applied when a pixel group PG contains one bad pixel, and the two bits constituting the bad pixel flag BP FLAG have values ​​indicating which of the four pixels (P1 to P4) in the pixel group PG is a bad pixel. For example, if the first pixel P1 is a bad pixel, the bad pixel flag BP FLAG has a value of "00." If the second pixel P2 is a bad pixel, the bad pixel flag BP FLAG has a value of "01." If the third pixel P3 is a bad pixel, the bad pixel flag BP FLAG has a value of "10." If the fourth pixel P4 is a bad pixel, the bad pixel flag BP FLAG has a value of "11."

[0091] In this embodiment, only two bits are allocated to the bad pixel flag BP FLAG, and therefore more bits are allocated to the difference value RES than in the embodiment of Figure 5A. Therefore, data loss of the difference value RES is reduced. Meanwhile, in Figure 6, five bits are allocated to the difference value RES1 of the first pixel P1, five bits are allocated to the difference value RES2 of the second pixel P2, and four bits are allocated to the difference value RES3 of the third pixel P3. However, the present invention is not limited to this, and it goes without saying that different numbers of bits may be set depending on the embodiment.

[0092] 5A to 6, the 40 bits constituting the pixel group PG are compressed into 20 bits of the bit stream BS, so the image data IDT can be compressed at a compression rate of 50%. However, the present invention is not limited to this, and the number of bits of the bit stream BS can be set to be more or less than 20 bits, thereby increasing or decreasing the compression rate.

[0093] Figure 7 is a flowchart illustrating an image compression method according to an embodiment of the present invention. Specifically, Figure 7 is a flowchart illustrating an image compression method for the image processing system 10 of Figure 1. At least one of the steps of Figure 7 is performed by the encoder 120.

[0094] 1 and 7, the encoder 120 detects defective pixels within each of a plurality of pixel groups constituting the image data IDT (step S110). Specifically, the encoder 120 detects defective pixels in the image data IDT based on at least one of pixel values ​​of pixels within the pixel group and pixels adjacent to the pixel group. For example, the encoder 120 calculates an average pixel value of pixels adjacent to a target pixel to be detected as a defective pixel among pixels included in the pixel group. The encoder 120 then detects whether the target pixel is a defective pixel based on a difference between the calculated average pixel value and the pixel value of the target pixel. For example, the encoder 120 detects the target pixel as a defective pixel if the difference between the average pixel value and the pixel value of the target pixel exceeds a threshold value.

[0095] The encoder 120 then generates a flag indicating position information of the defective pixel (step S120). Here, the flag is embodied to have a value corresponding to the position of at least one defective pixel included in the pixel group. The encoder 120 then calculates a difference between the value of the remaining pixels in the pixel group, excluding the defective pixel, and the reference pixel value (step S130). Specifically, the encoder 120 determines the reference pixel value based on reference information (e.g., first reference map RM1) including pixel values ​​corresponding to pixels compressed before the pixel group. For example, the encoder 120 determines the reference pixel value based on the pixel value of at least one pixel adjacent to the pixel group in the reference information. A detailed description of a method for determining the reference pixel value has been provided with reference to FIGS. 3 and 4, and therefore will not be repeated here.

[0096] The encoder 120 then generates a bitstream including the flags and the difference values ​​(step S140). At this time, the encoder 120 generates compression information indicating the compression method applied to the pixel group. The encoder 120 then includes the generated compression information in the bitstream. For example, the encoder 120 includes the compression information in the header of the bitstream.

[0097] After generating the bitstream, the encoder 120 updates the reference information based on pixel values ​​corresponding to the pixels in the pixel group. Specifically, the encoder 120 corrects pixel values ​​of defective pixels in the pixel group. For example, the encoder 120 corrects pixel values ​​of defective pixels based on pixel values ​​of pixels adjacent to the defective pixels. Detailed descriptions of the method for correcting pixel values ​​of defective pixels have been provided with reference to FIGS. 3 and 4, and therefore will not be repeated. The encoder 120 updates the reference information by adding pixel values ​​of the remaining pixels in the pixel group excluding the defective pixel and the corrected pixel value of the defective pixel to the reference information.

[0098] Figure 8 is a diagram illustrating a decoder according to an embodiment of the present invention. Specifically, Figure 8 illustrates the decoder 230 of Figure 1. Referring to Figures 1 and 8, the decoder 230 includes a decompressor 231 and a second reference map generator 233.

[0099] The decompressor 231 receives the compressed data CDT and the second reference map RM2 from the memory 220 and generates decompressed data DDT by decompressing the compressed data CDT using the second reference map RM2. Specifically, the decompressor 231 first sequentially decompresses a plurality of bitstreams BS included in the compressed data CDT using the second reference map RM2 generated based on the pixel values ​​of the decompressed pixels. A detailed description of this will be given later with reference to Figures 9A and 9B.

[0100] When the decompressor 231 completes decompression of one bitstream BS, it provides restored pixel information DP including pixel values ​​of the decompressed pixels to the second reference map generator 233. In addition, the decompressor 231 identifies bad pixels through bad pixel flags BP FLAG of the bitstream BS, and provides bad pixel information BP regarding the identified bad pixels to the second reference map generator 233.

[0101] The second reference map generator 233 generates a new second reference map RM2(NEW) based on the received restored pixel information DP and defective pixel information BP. Specifically, the second reference map generator 233 identifies defective pixels in the restored pixel group based on the defective pixel information BP. The second reference map generator 233 then corrects the pixel values ​​of the defective pixels among the restored pixel values ​​to values ​​similar to the pixel values ​​of the surrounding pixels. The second reference map generator 233 then adds or replaces the restored pixel values ​​of the normal pixels in the pixel group and the corrected pixel values ​​of the defective pixels to the existing second reference map RM2 to generate the new second reference map RM2(NEW). This will be described in detail below with reference to FIGS. 9A and 9B.

[0102] The second reference map generator 233 stores the generated second reference map RM2(NEW) in the memory 220. The decompressor 231 reads the new second reference map RM2(NEW) stored in the memory 220 and performs decompression on the next pixel group based on the read second reference map RM2(NEW).

[0103] Meanwhile, each of the decompressor 231 and the second reference map generator 233 may be implemented by software or hardware, or a combination of software and hardware such as firmware. When the decompressor 231 and the second reference map generator 233 are implemented by software, the respective functions described above are implemented by programmed source code, which is loaded into a recording medium included in the image processing device 200, and the functions of the decompressor 231 and the second reference map generator 233 are implemented by a processor (e.g., a microprocessor) included in each of the image processing devices 200 executing the software. When the decompressor 231 and the second reference map generator 233 are implemented by hardware, the decompressor 231 and the second reference map generator 233 include logic circuits and registers and perform the respective functions described above based on register settings.

[0104] 9A and 9B are conceptual diagrams illustrating an image decompression method according to an embodiment of the present invention. Specifically, FIGS. 9A and 9B illustrate a decompression method for Bayer pattern image data IDT by the decoder 230. The following description will be given on the assumption that one pixel group is configured to include four sequentially arranged pixels. Also, an embodiment will be described in which the decoder 230 decompresses a sixth bitstream BS6 corresponding to a pixel group PG including pixels R7, Gr7, R8, and Gr8, and pixel R7 is a defective pixel.

[0105] 9A, the decompressor 231 decompresses the sixth bitstream BS6 using the second reference map RM2, which includes pixel values ​​of the first to fifth bitstreams BS1 to BS5 that were decompressed before the sixth bitstream BS6.

[0106] Specifically, the decompressor 231 checks for defective pixels based on the defective pixel flag BP FLAG of the sixth bitstream BS6, and then decompresses the normal pixels that are not defective pixels.

[0107] In some embodiments, the decompressor 231 selects at least one pixel adjacent to the target pixel, which is a pixel group PG or a normal pixel of the pixel group PG, in the second reference map RM2. The decompressor 231 then determines a reference value based on the pixel value of the selected pixel. The method of determining the reference value corresponds to the method by which the compressor 123 determines the reference value in the process of generating the sixth bitstream BS6. For example, if the compressor 123 determines a pixel at a specific position (e.g., to the left of the target pixel) among pixels adjacent to the target pixel in the first reference map RM1 as the reference value, the decompressor 231 determines the reference value by referring to a pixel at a specific position (e.g., to the left of the target pixel) among pixels adjacent to the target pixel in the second reference map RM2. According to some embodiments, information regarding the method by which the compressor 123 determines the reference value may be included in the header HEADER of the bitstream BS.

[0108] The decompressor 231 then restores the pixel value of the target pixel based on the difference value RES between the determined reference value and the target pixel included in the sixth bitstream BS6. Specifically, the decompressor 231 restores the pixel value of the target pixel by adding the difference value RES of the target pixel to the determined reference value. For example, if the target pixel is pixel Gr7, the decompressor 231 restores the pixel value of the target pixel Gr7 by adding the difference value RES of the target pixel Gr7 to the reference value.

[0109] After completing the restoration of the normal pixels, the decompressor 231 restores the pixel values ​​of the defective pixels. Meanwhile, the sixth bitstream BS6 does not include the difference values ​​RES of the defective pixels. Therefore, the decoder 230 calculates the average pixel value of the restored normal pixels and restores the pixel values ​​of the defective pixels by adding a critical value to the calculated average pixel value. Here, the critical value is the same as the critical value used in the defective pixel detection operation described in FIG. 3, but the present invention is not limited thereto, and various values ​​can be set as the critical value.

[0110] For example, if the defective pixel is pixel R7, the decoder 230 calculates an average pixel value of the pixel values ​​of the restored normal pixels (Gr7, R8, Gr8) and restores the pixel value of the defective pixel R7 by adding a critical value to the calculated average pixel value. Meanwhile, the method by which the decoder 230 restores a defective pixel is not limited to the above example, and various other methods may be used to restore a defective pixel.

[0111] When decompression for pixel group PG is completed, second reference map generator 233 adds the pixel values ​​of the decompressed pixel group PG to the existing second reference map RM2 to generate a new second reference map RM2(NEW).

[0112] Specifically, if there are no bad pixels in the pixel group, the second reference map generator 233 adds the decompressed pixel values ​​to the existing second reference map RM2 to generate a new second reference map RM2(NEW).

[0113] On the other hand, if a pixel group includes a defective pixel, the second reference map generator 233 corrects the pixel value of the defective pixel among the restored pixel values ​​based on the pixel values ​​of pixels adjacent to the defective pixel. The method of correcting the pixel value of the defective pixel is performed similarly to the method of correcting the pixel value of the defective pixel described with reference to FIG.

[0114] 9B , the second reference map generator 233 corrects the pixel value of pixel R7, which corresponds to the same red hue as pixel R7 (the target pixel that is a defective pixel), to the pixel value of pixel R3 located above pixel R7. As another example, the second reference map generator 233 corrects the pixel value of pixel R7 to the pixel value of pixel R6 located to the left of pixel R7. As yet another example, the second reference map generator 233 corrects the pixel value of pixel R7 to the pixel value of pixel R8 located to the right of pixel R7. As yet another example, the second reference map generator 233 corrects the pixel value of pixel R7 to the average pixel value of pixels (R2 and R4) located diagonally opposite pixel R7.

[0115] Then, the second reference map generator 233 adds the restored pixel values ​​of the normal pixels and the corrected pixel values ​​of the defective pixels to the existing second reference map RM2 to generate a new second reference map RM2(NEW). The corrected value of the target pixel R7 is pixel value R7b in the new second reference map RM2(NEW).

[0116] In this way, when one pixel group PG is decompressed, the second reference map generator 233 adds pixel values ​​corresponding to the pixel group PG (i.e., the pixel values ​​of the restored normal pixels and the corrected pixel values ​​of the defective pixels of the pixel group PG) to generate a new second reference map RM2(NEW) for decompressing the next pixel group in the sequence.

[0117] 10 is a diagram illustrating another example of an image processing system according to an embodiment of the present invention, specifically, a modified embodiment of the image processing system 10 of FIG.

[0118] 10, an image processing system 10a includes a camera module 100a and an image processing device 200a. In this embodiment, the camera module 100a includes an image sensor 110a, an encoder 120a, a memory 130a, and an interface 140a. In this embodiment, the image processing device 200a includes an interface 210a, a memory 220a, a decoder 230a, and an image signal processor 240a.

[0119] Compared to the image system 10 of Figure 1, the image system 10a of Figure 10 has a mode selector 127a added to the encoder 120a, and the remaining configuration is substantially the same. Of the configuration of the image processing system 10a, a description of the configuration that overlaps with the configuration of the image processing system 10 of Figure 1 will be omitted.

[0120] According to an embodiment of the present invention, the encoder 120a compresses the image data IDT in a plurality of compression modes to generate a plurality of compressed data CDT. Here, each of the plurality of compression modes is set based on a compression target, a compression rate, an error rate, and / or loss information. The encoder 120a then includes mode information in a bitstream header during compression. A detailed description of the plurality of compression modes will be provided below with reference to FIGS. 12A and 12B.

[0121] For example, the encoder 120 generates a first reference map RM1 including pixel values ​​of original defective pixels that have not been corrected, and compresses the image data IDT in a first mode that performs a defective pixel detection operation on the first reference map RM1 during a compression operation using the first reference map RM1, with information indicating the first mode being included in the header of the bitstream.

[0122] 1 to 9, the encoder 120a generates a first reference map RM1 including corrected pixel values ​​of defective pixels, and compresses the image data IDT in a second mode in which the compression operation is performed without detecting defective pixels in the first reference map RM1, and information indicating the second mode is included in the header of the bitstream.

[0123] However, the present invention is not limited to this, and the encoder 120a can compress the image data IDT in three or more modes. It goes without saying that the compression method for each of the multiple modes can be set differently from the example described above.

[0124] The mode selector 127a of the encoder 120a selects one of a plurality of modes and transmits the compressed data CDT corresponding to the selected mode to the image processing device 200a via the interface 140a. The mode selector 127a is a switch implemented in software and / or hardware. The mode selector 127a selects one of the plurality of modes based on the compression target, compression rate, error rate, and / or loss information. The mode information is included in the bitstream header. For example, the compression rate may be set based on user input or manufacturing specifications, and the mode selector 127a selects one of the plurality of modes based on the predefined compression rate. In some embodiments, the mode selector 127a selects one of the plurality of modes based on error data of the plurality of modes. Here, the error data refers to data indicating the difference between the image data IDT and data obtained by restoring the compressed data CDT of a particular mode. The mode selector 127a selects the mode with the smallest error based on the error data of the plurality of modes and transmits the compressed data CDT corresponding to the selected mode to the image processing device 200a.

[0125] According to an embodiment of the present invention, the decoder 230a decompresses the compressed data CDT in multiple modes to generate multiple decompressed data DDT. Specifically, the decoder 230a checks mode information based on the header of the bitstream included in the compressed data CDT. Then, the decoder 230a decompresses the bitstream using a decompression method corresponding to the checked mode information.

[0126] For example, if the header of the bitstream contains mode information indicating the above-mentioned first mode, the decoder 230a decompresses the bitstream using a decompression method corresponding to the first mode. Also, if the header of the bitstream contains mode information indicating the above-mentioned second mode, the decoder 230a decompresses the bitstream using a decompression method corresponding to the second mode.

[0127] FIG. 11 is a diagram illustrating another example of an encoder according to an embodiment of the present invention.

[0128] Specifically, Figure 11 illustrates the encoder 120a of Figure 10. Referring to Figures 10 and 11, the encoder 120a includes a bad pixel detector 121a, a compressor 123a, a first reference map generator 125a, and a mode selector 127a.

[0129] 11, the bad pixel detector 121a, the compressor 123a, and the first reference map generator 125a compress the image data IDT using a compression scheme corresponding to each of a plurality of modes to generate a plurality of compressed data CDT. The compressor 123a then generates error data ED based on the plurality of compressed data CDT. In some embodiments, the bad pixel detector 121a, the compressor 123a, and / or the first reference map generator 125a compress the image data IDT according to a plurality of modes under the control of the mode selector 127a.

[0130] Specifically, the compressor 123a decompresses each of the plurality of compressed data CDT to generate decompressed data, calculates the error between the decompressed data and the image data IDT, generates error data ED including errors of a plurality of modes, and transmits the error data ED to the mode selector 127a.

[0131] The mode selector 127a identifies the mode with the smallest error based on the error data ED, and transmits the compressed data CDT corresponding to the identified mode to the image processing device 200a via the interface 140a.

[0132] Meanwhile, in a modified embodiment, the compressor 123a generates error data ED for each pixel group. Specifically, the compressor 123a calculates the error between each of n pixel groups (n is a positive integer) constituting the image data IDT and each of n pixel groups constituting the decompressed data. The compressor 123a then generates error data ED including multiple error modes for each pixel group. The mode selector 127a identifies the mode with the smallest error for each pixel group based on the error data ED and transmits a bitstream corresponding to the identified mode to the image processing device 200a. For example, the mode selector 127a determines the mode for decompressing the compressed data CDT so that the decompressed data DDT has the lowest error rate. The term "error" refers to errors generated during the restoration of the compressed image data CDT by the decoder 230a. However, the term "error" is not limited to this and includes errors due to damage that may occur to the original image.

[0133] Meanwhile, in the above example, bad pixel detector 121a, compressor 123a, and first reference map generator 125a are illustrated and described as performing compression using compression schemes corresponding to a plurality of modes to generate a plurality of compressed data CDTs, but the present invention is not limited thereto. For example, encoder 120a may be embodied to include bad pixel detector 121a, compressor 123a, and / or first reference map generator 125a corresponding to a plurality of modes. For example, encoder 120a may be embodied to include bad pixel detector 121a, compressor 123a, and / or first reference map generator 125a corresponding to a first mode, and bad pixel detector 121a, compressor 123a, and / or first reference map generator 125a corresponding to a second mode.

[0134] 10 and 11, the mode selector 127a is illustrated and described as being included in the encoder 120a, but the present invention is not limited to this. For example, the mode selector 127a may be implemented in the camera module 100a as a separate configuration from the encoder 120a.

[0135] 12A and 12B are tables illustrating compression information according to an embodiment of the present invention. In particular, FIG. 12A and FIG. 12B illustrate compression modes (compression methods) according to the standard proposed by the MIPI (Mobile Industry Processor Interface) Alliance. Please refer to both FIG. 10 and FIG. 11.

[0136] 12A, the Bayer pattern image data IDT (FIG. 3) is compressed using various compression modes, such as pixel-based directional differential (PD) mode, diagonal direction-based differential (DGD) mode, extended slanted horizontal or vertical direction-based differential (eSHV) mode, outlier compensation (OUT) mode, and fixed quantization and no-reference (FNR) mode. The names of the compression modes mentioned above are merely examples, and the present invention is not limited to these examples.

[0137] The PD mode performs DPCM (differential pulse code modulation) on Bayer pattern image data IDT. PD modes are divided into MODE0, MODE1, MODE2, MODE3, MODE12, and MODE13 depending on the detailed implementation algorithm. 4 bits are allocated to the header indicating the compression method, and the 16 compression modes each express header information with different bits. For example, MODE0 is expressed by bits '0000', MODE1 by bits '0001', MODE2 by bits '0010', MODE3 by bits '0011', MODE12 by bits '1100', and MODE13 by bits '1101'.

[0138] The DGD mode performs DPCM on diagonal image data IDT. The DGD modes are classified into MODE4 (bits '010'), MODE5 (bits '0101'), MODE8 (bits '1000'), MODE9 (bits '1001'), MODE10 (bits '1010'), and MODE11 (bits '1011') according to the detailed implementation algorithm.

[0139] Similarly, the eSHV mode includes MODE14 (bits "1110") and MODE15 ("1111"), the OUT mode includes MODE7 (bits "0111"), and the FNR mode includes MODE6 (bits "01110"). According to one embodiment of the present invention, MODE7 refers to the OUT mode or the saturation mode, which includes the BP mode for processing defective pixels, and either one of the two modes (saturation mode or OUT mode) is selected depending on the operating environment.

[0140] In one embodiment, the mode selector 127a sequentially evaluates the PD mode, DGD mode, eSHV mode, OUT mode, and FNR mode, and selects the optimal mode based on compression evaluation criteria such as compression ratio and loss information. However, the technical concept of the present invention is not limited to the evaluation order of the presented modes.

[0141] 12B, the image data IDT (FIG. 4) of the tetra pattern is compressed using various compression modes. However, the present invention is not limited thereto, and image data in which a red pixel group, a blue pixel group, a first green pixel group, and a second pixel group, each including pixels arranged in a 2n×2n or 3n×3n (n is a positive integer) array, are repeatedly arranged can also be compressed using various compression modes.

[0142] The compression modes used are AD (average-based directional differential) mode, eHVD (extended horizontal or vertical direction-based differential) mode, OD (oblique direction-based differential) mode, eMPD (extended multi-pixel-based differential) mode, eHVA (extended horizontal or vertical average-based differential) mode, eOUT (extended outlier compensation) mode, and FNR mode.

[0143] In AD mode, DPCM is performed on image data IDT, where one pixel group PG constituting a Bayer pattern contains multiple pixels. AD mode is classified into MODE0, MODE1, MODE2, and MODE3 according to the detailed implementation algorithm. Since 4 bits are allocated to the header indicating the compression method, the 16 compression modes can express header information with different bits. For example, MODE0 is expressed by bits '0000', MODE1 by bits '0001', MODE2 by bits '0010', and MODE3 by bits '0011'.

[0144] The OD mode compresses diagonal image data IDT. The OD mode is divided into MODE4 (bits '0100') and MODE5 (bits '0101') according to the detailed implementation algorithm.

[0145] Similarly, the eMPD mode includes MODE8 (bits “1000”), MODE9 (bits “1001”), MODE10 (bits “1010”), and MODE11 (bits “1011”), the eHVD mode includes MODE12 (bits “1100”) and MODE13 (bits “1101”), the eHVA mode includes MODE14 (bits “1110”), the (e)OUT mode (eOUT mode or OUT mode) includes MODE15 (bits “1111”) and MODE7 (bits “0111”), and the FNR mode includes MODE6 (bits “0110”). According to an embodiment of the present invention, MODE7 refers to the (e)OUT mode or saturation mode, which includes the BP mode for processing defective pixels, and either one of the two modes (saturation mode or (e)OUT mode) is selected depending on the operating environment.

[0146] In one embodiment, the mode selector 230 sequentially evaluates the AD mode, eHVD mode, OD mode, eMPD mode, eHVA mode, eOUT mode, and FNR mode, and selects the optimal mode based on compression evaluation metrics such as compression ratio and loss information. However, the technical concept of the present invention is not limited to the evaluation order of the presented modes.

[0147] 13 is a diagram illustrating yet another example of an image processing system according to an embodiment of the present invention, specifically, a modified embodiment of the image processing system 10 of FIG.

[0148] 13, an image processing system 10b includes a camera module 100b and an image processing device 200b. In this embodiment, the camera module 100b includes an image sensor 110b and an interface 140b.

[0149] In this embodiment, the image processing device 200b includes an interface 210b, a memory 220b, a decoder 230b, an image signal processor 240b, and an encoder 250b. The encoder 250b in Figure 13 corresponds to the encoder 120 in Figure 1 or the encoder 120a in Figure 10.

[0150] Comparing the image processing system 10b of Figure 13 with the image processing system 10 of Figure 1, the difference is that the encoder 250b is included in the image processing device 200b rather than the camera module 100b, but the remaining configuration is substantially the same. Description of the configuration of the image processing system 10b that overlaps with the configuration of the image processing system 10 of Figure 1 will be omitted.

[0151] 13, the image sensor 110b generates image data IDT. The image data IDT is transmitted to the image processing device 200b via the interface 140b. Meanwhile, according to an embodiment, the camera module 100b further includes an image signal processor (not shown), which processes the image data IDT and transmits the processed image data IDT to the image processing device 200b.

[0152] The image processing device 200b receives image data IDT via an interface 210b. The encoder 250b compresses the image data IDT using a first reference map RM1 stored in the memory 220b to generate compressed data CDT. The method by which the encoder 250b generates the compressed data CDT is substantially the same as that described with reference to FIGS. 1 to 11, and therefore a redundant description will be omitted. The compressed data CDT is stored in the memory 220b and read from the memory 220b by the decoder 230b. The decoder 230b decompresses the image data IDT using a second reference map RM2 stored in the memory 220b to generate decompressed data DDT. The method by which the decoder 230b generates the decompressed data DDT is substantially the same as that described with reference to FIGS. 1 to 11, and therefore a redundant description will be omitted.

[0153] FIG. 14 is a diagram illustrating an electronic device according to one embodiment of the present invention.

[0154] 14, an electronic device 1000 includes a camera module 1100, an application processor 1200, a display 1300, a memory 1400, a storage 1500, a user interface 1600, and a wireless transceiver 1700. The camera module 1100 in Fig. 14 corresponds to the camera module 100 in Fig. 1, the camera module 100a in Fig. 10, or the camera module 100b in Fig. 13, and the application processor 1200 in Fig. 14 includes the image processing device 200 in Fig. 1, the image processing device 200a in Fig. 10, or the image processing device 200b in Fig. 13. Descriptions that overlap with those described with reference to Figs. 1, 10, and 13 will be omitted.

[0155] The application processor 1200 is provided as a system-on-chip (SoC) that controls the overall operation of the electronic device 1000 and runs application programs, an operating system, and the like.

[0156] The memory 1400 stores programs and / or data to be processed or executed by the application processor 1200. The storage 1500 is implemented by a non-volatile memory device such as a NAND flash or a resistive memory, and for example, the storage 1500 is provided by a memory card (MMC, eMMC, SD, microSD), etc. The storage 1500 stores data and / or programs related to execution algorithms that control the image processing operations of the application processor 1200, and the data and / or programs are loaded into the memory 1400 when the image processing operations are performed.

[0157] The user interface 1600 may be implemented by a variety of devices for receiving user input, such as a keyboard, a curtain key panel, a touch panel, a fingerprint sensor, or a microphone. The user interface 1600 receives the user input and provides a signal corresponding to the received user input to the application processor 1200. The wireless transceiver 1700 includes a modem 1710, a transceiver 1720, and an antenna 1730.

[0158] Fig. 15 is a diagram illustrating a portion of an electronic device according to an embodiment of the present invention. Fig. 16 is a diagram illustrating a specific configuration of a camera module according to an embodiment of the present invention. Specifically, Fig. 15 is a diagram illustrating an electronic device 2000 that is a part of the electronic device 1000 of Fig. 14, and Fig. 16 is a diagram illustrating a specific configuration of the camera module 2100b of Fig. 15.

[0159] 15, an electronic device 2000 includes a multi-camera module 2100, an application processor 2200, and a memory 2300. The memory 2300 performs the same function as the memory 1400 shown in FIG. 14, and therefore, a redundant description will be omitted.

[0160] The electronic device 2000 utilizes a CMOS image sensor to capture and / or store images of a subject and may be embodied by a mobile phone, tablet computer, or other portable electronic device, including laptop computers, mobile phones, smartphones, tablet PCs, wearable devices, and the like.

[0161] The multi-camera module 2100 includes a first camera module 2100a, a second camera module 2100b, and a third camera module 2100c. The multi-camera module 2100 performs the same functions as the camera module 100 of Fig. 1, the camera module 100a of Fig. 10, or the camera module 100b of Fig. 13. While Fig. 15 illustrates the multi-camera module 2100 as including three camera modules (2100a to 2100c), the present invention is not limited thereto, and the multi-camera module 2100 may include any number of camera modules.

[0162] The detailed configuration of camera module 2100b will be described in more detail below with reference to FIG. 16, but the following description also applies equally to other camera modules (2100a, 2100c) depending on the embodiment.

[0163] Referring to FIG. 16, the second camera module 2100b includes a prism 2105, an optical path folding element (OPFE) 2110, an actuator 2130, an image sensing device 2140, and a memory 2150.

[0164] Prism 2105 includes a reflective surface 2107 made of a light-reflecting material, and changes the path of light L incident from the outside.

[0165] According to one embodiment, the prism 2105 changes the path of light L incident in a first direction X to a second direction Y perpendicular to the first direction X. The prism 2105 also changes the path of light L incident in the first direction X to the perpendicular second direction Y by rotating a reflective surface 2107 of a light-reflecting material around a central axis 2106 in a direction A or by rotating the central axis 2106 in a direction B. At this time, the OPFE 2110 also moves in a third direction Z perpendicular to the first direction X and the second direction Y.

[0166] The OPFE 2110 includes, for example, m groups of optical lenses (where m is a natural number). The m lenses move in the second direction Y to change the optical zoom ratio of the camera module 2100b.

[0167] The actuator 2130 moves the OPFE 2110 or the optical lens (hereinafter referred to as the optical lens) to a specific position. For example, the actuator 2130 adjusts the position of the optical lens so that the image sensor 2142 is located at the focal length of the optical lens for accurate sensing.

[0168] The image sensing device 2140 includes an image sensor 2142, control logic 2144, an encoder 2145, and a memory 2146. The image sensor 2142 senses an image of a sensing target using light L provided through an optical lens. The image sensor 2142 of FIG. 16 is functionally similar to the image sensor 110 of FIG. 1, the image sensor 110a of FIG. 10, or the image sensor 110b of FIG. 13, and therefore a duplicated description will be omitted. The control logic 2144 controls the overall operation of the second camera module 2100b.

[0169] The encoder 2145 encodes the sensed image data. The encoder 2145 of Fig. 16 performs a function similar to that of the encoder 120 of Fig. 1, the encoder 120a of Fig. 10, or the encoder 250b of Fig. 13, and therefore a duplicated description will be omitted. Meanwhile, for convenience of explanation, the encoder 2145 is illustrated as a separate functional unit different from the other functional units, but is not limited thereto, and may be included in the control logic 2144 to compress and encode the image data.

[0170] The memory 2146 stores information necessary for the operation of the second camera module 2100b, such as correction data 2147. The correction data 2147 includes information necessary for the second camera module 2100b to generate image data using light L provided from the outside. The correction data 2147 includes, for example, information regarding the degree of rotation, information regarding the focal length, and information regarding the optical axis. If the second camera module 2100b is implemented in the form of a multi-state camera in which the focal length changes depending on the position of the optical lens, the correction data 2147 includes a focal length value for each position (or state) of the optical lens and information regarding autofocusing.

[0171] The memory 2150 stores image data sensed via the image sensor 2142. In some embodiments, the memory 2150 stores compressed data generated by the encoder 2145. The memory 2150 is disposed outside the image sensing device 2140 and is embodied in a stacked form on a sensor chip constituting the image sensing device 2140. In one embodiment, the memory 2150 is embodied as an EEPROM (electrically erasable programmable read only memory), but the present embodiment is not limited thereto.

[0172] 15 and 16, in this embodiment, each of the multiple camera modules (2100a, 2100b, 2100c) includes an actuator 2130. As a result, each of the multiple camera modules (2100a, 2100b, 2100c) includes correction data 2147 that is the same as or different from one another depending on the operation of the actuator 2130 included therein.

[0173] In this embodiment, one of the multiple camera modules (2100a, 2100b, 2100c) (e.g., the second camera module 2100b) is a folded lens type camera module including the above-mentioned prism 2105 and OPFE 2110, and the remaining camera modules (2100a, 2100c) are vertical type camera modules that do not include the prism 2105 and OPFE 2110, but this embodiment is not limited thereto.

[0174] In one embodiment, one of the camera modules (2100a, 2100b, 2100c) (e.g., the third camera module 2100c) is a vertical depth camera that extracts depth information using, for example, infrared ray (IR). In this case, the application processor 2200 can merge image data provided from the vertical depth camera with image data provided from a different camera module (e.g., the first camera module 2100a or the second camera module 2100b) to generate a 3D depth image.

[0175] In one embodiment, at least two camera modules (e.g., the first camera module 2100a or the second camera module 2100b) of the plurality of camera modules (2100a, 2100b, 2100c) have different fields of view (FOVs). In this case, for example, but not limited to, at least two camera modules (e.g., the first camera module 2100a or the second camera module 2100b) of the plurality of camera modules (2100a, 2100b, 2100c) have different optical lenses. For example, the first camera module 2100a of the plurality of camera modules (2100a, 2100b, 2100c) has a narrower field of view (FOV) than the second camera module 2100b and the third camera module 2100c. However, the multi-camera module 2100 is not limited to this, and may further include a camera module having a wider field of view (FOV) than the camera modules (2100a, 2100b, 2100c) that are originally used.

[0176] In some embodiments, the camera modules (2100a, 2100b, 2100c) each have a different viewing angle, and in this case, the optical lenses included in the camera modules (2100a, 2100b, 2100c) are also different from each other, but are not limited to this.

[0177] In some embodiments, the multiple camera modules (2100a, 2100b, 2100c) are arranged to be physically separated from one another. That is, instead of the multiple camera modules (2100a, 2100b, 2100c) sharing the sensing area of ​​one image sensor 2142, an independent image sensor 2142 is arranged inside each of the multiple camera modules (2100a, 2100b, 2100c).

[0178] The application processor 2200 includes a plurality of sub-processors (2210a, 2210b, 2210c), an image generator 2220, a camera module controller 2230, a memory controller 2400, and an internal memory 2250. The application processor 2200 is implemented separately from the plurality of camera modules (2100a, 2100b, 2100c). For example, the application processor 2200 and the plurality of camera modules (2100a, 2100b, 2100c) are implemented separately from each other on separate semiconductor chips.

[0179] Image data or compressed data generated from each camera module (2100a, 2100b, 2100c) is provided to the corresponding sub-processor (2210a, 2210b, 2210c) via mutually separated image signal lines (ISLa, ISLb, ISLc). Such image data transmission is performed using, for example, a camera serial interface (CSI) based on MIPI, but the present embodiment is not limited thereto.

[0180] In one embodiment, one sub-processor is arranged to correspond to multiple camera modules. For example, the first sub-processor 2210a and the third sub-processor 2210c are not implemented separately from each other as shown in the figure, but are integrated into one sub-processor, and the image data or compressed data provided from the camera module 2100a and the camera module 2100c is selected via a selection element (e.g., a multiplexer) and then provided to the integrated sub-image processor.

[0181] Each of the sub-processors (2210a, 2210b, 2210c) includes decoder 230 of Figure 1, decoder 230a of Figure 10, or decoder 230b of Figure 13. The sub-processors (2210a, 2210b, 2210c) decompress the received compressed data to generate decompressed data and output the generated decompressed data to image generator 2220. Image generator 2220 corresponds to image signal processor 240 of Figure 1, image signal processor 240a of Figure 10, or image signal processor 240b of Figure 13.

[0182] The camera module controller 2230 provides control signals to each of the camera modules (2100a, 2100b, 2100c). The control signals generated by the camera module controller 2230 are provided to the corresponding camera modules (2100a, 2100b, 2100c) via separate control signal lines (CSLa, CSLb, CSLc).

[0183] Although the embodiments of the present invention have been described in detail above with reference to the drawings, the present invention is not limited to the above-described embodiments and can be modified in various ways without departing from the technical concept of the present invention. [Explanation of symbols]

[0184] 10, 10a, 10b Image Processing System 100, 100a, 100b camera modules 110, 110a, 110b Image sensor 111, 111a, 111b pixel array 120, 120a, 250b, 2145 encoders 121, 121a Bad pixel detector 123, 123a compressor 125, 125a First reference map generator 127a Mode Selector 130, 130a, 220, 220a, 220b, 1400, 2146, 2300 memory 140, 140a, 140b, 210, 210a, 210b Interface (I / F) 200, 200a, 200b Image processing device 230, 230a, 230b decoders 231 Decompressor 233 Secondary Reference Map Generator 240, 240a, 240b Image Signal Processor (ISP) 1000, 2000 electronic equipment 1100 Camera Module 1200, 2200 Application Processor (AP) 1300 display 1500, 2150 storage 1600 User Interface 1700 Radio Transceiver 1710 modem 1720 Transceiver 2100, 2100a, 2100b, 2100c Multi-Camera Module 2105 Prism 2106 Center axis 2107 Reflective surface 2110 Optical Path Folding Element (OPFE) 2130 Actuator 2140 Image Sensing Device 2142 Image Sensor 2144 Control Logic 2147 Correction Data 2210a, 2210b, 2210c Sub-Processors 2220 Image Generator 2230 Camera Module Controller 2240 memory controller 2250 internal memory BP Bad Pixel Information BP FLAG Bad pixel flag BS Bitstream CDT Compressed Data DP restored pixel information HEADER Header IDT Image Data PG Pixel Group RES, RES1, RES2, RES3 difference value RM1, RM2 1st and 2nd reference maps

Claims

1. 1. An image compression method for compressing each of a plurality of pixel groups constituting image data, comprising: detecting a bad pixel from among the plurality of pixels in the pixel group; generating a flag indicating location information of the defective pixel; calculating a difference between a pixel value of each pixel remaining in the pixel group excluding the defective pixel and a reference pixel value; generating a bitstream including the flag and the difference value.

2. 2. The image compression method of claim 1, wherein the detecting a defective pixel includes determining whether the target pixel is a defective pixel based on at least one of pixel values ​​of a plurality of pixels in the pixel group and at least one pixel value of at least one pixel adjacent to the target pixel in the pixel group.

3. The step of determining whether the target pixel is a defective pixel includes: calculating an average pixel value of pixels adjacent to a target pixel in the pixel group; 3. The method of claim 2, further comprising the step of: calculating a difference between the average pixel value and the pixel value of the target pixel.

4. 2. The image compression method of claim 1, further comprising determining the reference pixel value based on reference information including pixel values ​​of pixels compressed prior to the pixel group.

5. 5. The image compression method of claim 4, wherein the determining the reference pixel value comprises determining the reference pixel value based on a pixel value of at least one pixel adjacent to the pixel group in the reference information.

6. 5. The image compression method of claim 4, further comprising updating the reference information based on pixel values ​​of the plurality of pixels in the pixel group after generating the bitstream.

7. The step of updating the reference information comprises: correcting pixel values ​​of the defective pixels in the pixel group; and updating pixel values ​​of the remaining pixels in the pixel group excluding the defective pixel and correction pixel values ​​of the defective pixel with the reference information.

8. 8. The image compression method of claim 7, wherein the step of correcting the pixel value of the defective pixel in the pixel group further comprises correcting the pixel value of the defective pixel based on a pixel value of at least one pixel adjacent to the defective pixel.

9. generating compression information indicative of a compression method to be applied to the pixel group; 2. The image compression method of claim 1, wherein generating the bitstream comprises generating the bitstream including the compression information.

10. A camera module, an image sensor for generating image data including a plurality of pixels; an encoder that divides the plurality of pixels into a plurality of pixel groups and compresses the plurality of pixel groups to generate compressed data including a plurality of bitstreams; a memory for storing reference information including pixel values ​​of pixels compressed by the encoder; The encoder comprises: Detecting defective pixels in the first pixel group; compressing pixel values ​​of first pixels included in the first pixel group based on the defective pixel detection result and the reference information to generate a first bitstream corresponding to the first pixel group; updating the reference information based on a corrected pixel value obtained by correcting the pixel value of the defective pixel; The encoder determines a reference pixel value based on at least one pixel adjacent to the first pixel in the reference information; Calculating a difference between the pixel value of the remaining pixels excluding the defective pixel and the reference pixel value; generating the first bitstream including the difference values; The camera module is characterized in that the encoder generates a flag indicating position information of the defective pixel, and generates the first bitstream including the flag.

11. The encoder comprises: correcting the pixel value of the defective pixel based on the pixel value of at least one pixel adjacent to the defective pixel; 11. The camera module of claim 10, wherein the reference information is updated by adding pixel values ​​of remaining pixels excluding the defective pixel among the first pixels and a correction pixel value of the defective pixel to the reference information.

12. The camera module of claim 11 , wherein the encoder detects the defective pixel based on at least one of pixel values ​​of the first pixel and pixels adjacent to the first pixel in the image data.

13. 11. The camera module of claim 10, wherein the encoder compresses the image data a plurality of times according to a plurality of modes corresponding to a plurality of compression methods to generate a plurality of compressed data, and further comprises a mode selector that selects a mode having a smallest error with the image data based on the plurality of compressed data from the plurality of modes.

14. The camera module of claim 13 , wherein the encoder generates compressed information including information about the selected mode, and generates the first bitstream including the generated compressed information.

15. 1. An image processing system comprising: an image sensor for generating image data including a plurality of pixels; an encoder that sequentially compresses a plurality of pixel groups that constitute the image data to generate a plurality of bitstreams; a decoder for decompressing the plurality of bitstreams to recover the image data; The encoder comprises: Detecting defective pixels in a second pixel group in each of the plurality of pixel groups; compressing the pixel values ​​of the second pixel group according to reference information generated based on pixel values ​​of the first pixel group compressed before the second pixel group; updating the reference information based on the result of the detection of the defective pixel; When compressing the pixel values, generating a flag indicating position information of the defective pixel; calculating a difference between pixel values ​​of the remaining pixels in the second pixel group excluding the defective pixel and a reference pixel value determined based on the reference information; an image processing system for generating a bitstream including the flag and the difference value;

16. 16. The image processing system of claim 15, wherein the defective pixel is detected based on at least one pixel value of a pixel included in the second pixel group and a pixel adjacent to the second pixel group.

17. When updating the reference information, correcting the pixel value of the defective pixel based on at least one of pixel values ​​of pixels adjacent to the defective pixel; 16. The image processing system of claim 15, wherein the correction pixel value of the defective pixel and pixel values ​​of the remaining pixels excluding the defective pixel are updated with the reference information.

Images