Frame buffer compressor, and image processing device having the same

Abstract

An image processing device according to one embodiment of the present invention comprises a multimedia IP (Intellectual Property) that processes image data to generate source data, and a frame buffer compressor that compresses the source data to generate compressed data and decompresses the compressed data to generate output data. The frame buffer compressor can generate quantized data by performing quantization on the source data using a plurality of pixel position offsets corresponding to the coordinates of each of the plurality of image pixels included in the source data and a predetermined quantization coefficient, and can generate the compressed data by processing the quantized data.

Description

Technology Field

[0001] The present invention relates to a frame buffer compressor and an image processing device including the same. Background Technology

[0003] With the emergence of the need for high-definition and high-resolution images, the amount of memory accessed by various multimedia IPs (Intellectual Properties) of image processing devices—that is, the bandwidth—can increase significantly. As bandwidth increases, the processing capacity of the image processing device may reach its limit, potentially leading to a decrease in the speed of processing high-definition and high-resolution images. Accordingly, methods to compress data size when multimedia IPs access memory are being considered. Data can be compressed before being written to memory, and the compressed data can be decompressed before being read from memory.

[0004] Recently, there has been a trend toward increasing Dynamic Range Compression (DRC) gain to improve image quality in dark areas. However, as DRC gain increases, small errors can be boosted, potentially leading to color distortion or warping. In other words, errors occurring during the data compression and decompression processes can degrade the color and quality of an image. The problem to be solved

[0006] One of the objectives of the technical concept of the present invention is to provide a frame buffer compressor and an image processing device including the same, wherein the error occurring during the data compression and decompression process is not biased in a specific direction by calculating a pixel position offset corresponding to the coordinates of each of the plurality of image pixels included in the source data and applying the pixel position offset to the process of quantizing the source data. means of solving the problem

[0008] An image processing device according to one embodiment of the present invention comprises a multimedia IP (Intellectual Property) that processes image data to generate source data, and a frame buffer compressor that compresses the source data to generate compressed data and decompresses the compressed data to generate output data. The frame buffer compressor generates quantized data by performing quantization on the source data using a plurality of pixel position offsets corresponding to the coordinates of each of the plurality of image pixels included in the source data and a predetermined quantization coefficient, and generates the compressed data by processing the quantized data.

[0010] A frame buffer compressor according to one embodiment of the present invention comprises an encoder that compresses source data to generate compressed data, and a decoder that decompresses the compressed data to generate output data. The encoder generates quantized data by performing quantization on the source data using pixel position offsets calculated by modulo operation on the coordinates of each of a plurality of image pixels included in the source data, and a predetermined quantization coefficient, and generates the compressed data by processing the quantized data.

[0012] A data processing method according to one embodiment of the present invention includes the steps of receiving source data from a multimedia IP, a frame buffer compressor calculating a first offset determined by the coordinates of each of a plurality of image pixels included in the source data, a frame buffer compressor determining a predetermined coefficient and a second offset determined by half of the predetermined coefficient, and a frame buffer compressor performing an operation of subtracting the first offset from the source data and adding the second offset to the source data and dividing the result by the predetermined coefficient. Effects of the invention

[0014] According to one embodiment of the present invention, a frame buffer compressor calculates a pixel position offset corresponding to the coordinates of each of a plurality of image pixels included in source data and can apply the pixel position offset to the process of quantizing source data. Accordingly, errors occurring during the data compression and decompression processes can be prevented from being biased in a specific direction.

[0015] The various and beneficial advantages and effects of the present invention are not limited to those described above and will be more easily understood in the process of explaining specific embodiments of the present invention. Brief explanation of the drawing

[0017] FIG. 1 is a simplified diagram of an image processing device according to one embodiment of the present invention. FIG. 2 is a block diagram simply showing a frame buffer compressor according to one embodiment of the present invention. FIG. 3 is a simple block diagram showing an encoder according to one embodiment of the present invention. FIG. 4 is a circuit diagram simply illustrating a pixel array included in a sensor according to one embodiment of the present invention. FIG. 5 is a diagram simply illustrating image data generated by a sensor according to one embodiment of the present invention. FIG. 6 is a diagram simply illustrating image data generated by an image sensor according to one embodiment of the present invention. FIG. 7 is a drawing provided to explain source data according to one embodiment of the present invention. FIG. 8 is a drawing provided to explain quantization data according to one embodiment of the present invention. FIG. 9 is a drawing provided to explain output data according to an embodiment of the present invention. FIG. 10 is a drawing provided to explain error data according to one embodiment of the present invention. FIG. 11 is a simplified diagram of an image processing device according to one embodiment of the present invention. FIG. 12 is a simplified diagram of an image processing device according to one embodiment of the present invention. FIG. 13 is a simplified diagram of an image processing device according to one embodiment of the present invention. Specific details for implementing the invention

[0018] Hereinafter, preferred embodiments of the present invention are described as follows with reference to the attached drawings.

[0020] FIG. 1 is a simplified diagram of an image processing device according to one embodiment of the present invention.

[0021] Referring to FIG. 1, the image processing device (1) may include a multimedia IP (Intellectual Property) (10), a frame buffer compressor (20), a memory (30), and a system bus (40).

[0022] The multimedia IP (10) may be a part that directly performs processing of images, etc. of the image processing device (1). The multimedia IP (10) may include a plurality of modules related to the processing of images and videos. For example, the multimedia IP (10) may process image processing, image capture, and / or image output. As another example, the multimedia IP (10) may process video recording and playback, such as video camcoding and playback. In other words, the multimedia IP (10) may include processing modules that need to access memory (30) to process videos or images.

[0023] A multimedia IP (10) can receive image data from a sensor (not shown). The sensor may be an image sensor that receives light and generates image data. The image data may be raw data of a video or image. The image sensor may include a pixel array, and the pixel array may include a plurality of pixels arranged in a Bayer pattern. In this case, the color arrangement of the plurality of image pixels included in the image data may be matched with the color arrangement of the plurality of pixels included in the pixel array. Accordingly, the color arrangement of the plurality of image pixels included in the image data may be determined by the color arrangement of the plurality of pixels included in the pixel array.

[0024] The multimedia IP (10) can process image data and convert it into source data. The source data is data generated by the multimedia IP (10) and may also include data being processed by the multimedia IP (10). That is, the multimedia IP (10) can repeat the process of storing the source data in memory (30) and updating it again through several steps. In other words, the source data may include all data to be output from the multimedia IP (10) and stored in memory (30).

[0025] Source data can be stored in memory (30) in the form of compressed data. A frame buffer compressor (20) can compress source data to generate compressed data. Memory (30) can store the compressed data. A frame buffer compressor (20) can process the compressed data to generate recon data, and decompress the recon data to generate output data. The frame buffer compressor (20) can output the output data to a multimedia IP (10). For example, the source data and the output data may be the same or different depending on the compression method.

[0026] The multimedia IP (10) may include an image signal processor (ISP) (11), a shake correction module (G2D) (12), a multi-format codec (MFC) (13), a processing module (PD) (14), and a display (15).

[0027] An image signal processor (11) can preprocess image data and convert it into source data. The image data may be raw data following a Bayer pattern. In one embodiment, the image signal processor (11) can convert the image data into RGB data and convert the RGB data into YUV source data.

[0028] The RGB method can be described as a data format that expresses color based on the three primary colors of light. In other words, the RGB method can be described as a method of expressing images using the colors red, green, and blue.

[0029] The YUV method corresponds to a data format that separates and represents luminance (luma) signals and chroma signals. Y represents the luminance signal, and U(Cb) and V(Cr) can represent the chroma signals, respectively. U represents the difference between the luminance signal and the blue signal component, and V represents the difference between the luminance signal and the red signal component. Here, the items Y, U(Cb), and V(Cr) can be defined as planes. For example, data regarding the luminance signal can be referred to as data in the Y plane, and data regarding the chroma signals can be referred to as data in the U(Cb) plane or data in the V(Cr) plane.

[0030] For example, YUV data can be converted from RGB data using conversion formulas such as Y=0.3R+0.59G+0.11B, U=(BY)x0.493, and V=(RY)x0.877. Since the human eye is more sensitive to luminance signals than to color signals, YUV data may be easier to compress compared to RGB data. Therefore, the image signal processor (11) can convert the image data into YUV source data. The image signal processor (11) can store the converted source data in memory (30).

[0031] The shake correction module (12) can perform shake correction on image or video data. Shake correction may include detecting camera shake in the image or video data and removing it. The shake correction module (12) can perform shake correction by reading the image data or output data decompressed from the frame buffer compressor (20). The shake correction module (12) can correct the shake of the image data, source data, and / or output data to generate or update new source data and store it in memory (30).

[0032] The multi-format codec (13) may be a codec that compresses video data. The multi-format codec (13) may compress video data by utilizing the relationship between multiple frames. The multi-format codec (13) may compress image data, source data, and / or output data from memory (30). The multi-format codec (13) may compress the image data, source data, and / or output data to generate or update new source data and store it in memory (30).

[0033] The processing module (14) may be a system unit comprising a plurality of units that perform various processing tasks, such as images and videos. The processing module (14) may include an image processing unit, a video processing unit, a display processing unit, a graphic processing unit, and / or a neural processing unit. The processing module (14) may generate or update source data from image data.

[0034] The image processing unit can perform the role of improving image quality by executing various image processing tasks, such as enhancing image resolution, color correction, filtering, and boundary detection. The video processing unit handles video-related tasks, such as decoding, encoding, compression, and decoding of video signals, and can optimize video quality and process video data in real time. The display processing unit processes data to be ultimately displayed on the screen, and can control and optimize the resolution, color, brightness, etc., of the video output to the screen.

[0035] A graphics processing unit is a unit that specializes in processing graphics-related calculations and can perform 3D rendering, texture mapping, game graphics processing, UI graphics processing, etc. A graphics processing unit can maximize graphics performance by utilizing high-speed parallel processing capabilities. A neural network processing unit can efficiently process artificial intelligence (AI) and machine learning algorithms. A neural network processing unit can perform AI-based tasks such as image recognition, speech recognition, or natural language processing.

[0036] The display (15) can display output data from the memory (30) on the screen. Additionally, the display (15) can display source data processed by the image signal processor (110), shake correction module (12), multi-format codec (13), and processing module (14) on the screen. However, it may not be limited to this.

[0037] The image signal processor (11), shake correction module (12), multi-format codec (13), processing module (14), and display (15) of the multimedia IP (10) can each operate individually. In other words, each of the image signal processor (11), shake correction module (12), multi-format codec (13), processing module (14), and display (15) can individually access memory (30) to write or read data.

[0038] The frame buffer compressor (20) can generate compressed data by compressing source data before the multimedia IP (10) individually accesses the memory (30). The frame buffer compressor (20) can generate output data by decompressing the compressed data. The output data can be transmitted to the multimedia IP (10). In other words, the compressed data compressed by the frame buffer compressor (20) can be stored in the memory (30). The output data decompressed by the frame buffer compressor (20) can be loaded by the multimedia IP (10).

[0039] Whenever the image signal processor (11), shake correction module (12), multi-format codec (13), processing module (14), and display (15) of the multimedia IP (10) individually access the memory (30), the frame buffer compressor (20) can compress the source data into compressed data and transmit it to the memory (30). Additionally, whenever there is a data request to the image signal processor (11), shake correction module (12), multi-format codec (13), processing module (14), and display (15) of the multimedia IP (10), the frame buffer compressor (20) can decompress the compressed data from the memory (30) into output data and transmit it to each of the image signal processor (11), shake correction module (12), multi-format codec (13), processing module (14), and display (15) of the multimedia IP (10).

[0040] The memory (30) can store compressed data generated by the frame buffer compressor (20). Additionally, the memory (30) can provide the stored compressed data to the frame buffer compressor (20) so that the frame buffer compressor (20) can decompress it.

[0041] According to one embodiment illustrated in FIG. 1, the system bus (40) can be connected to the frame buffer compressor (20) and the memory (30). Specifically, the system bus (40) can be connected to the frame buffer compressor (20) and the memory (30), respectively. The multimedia IP (10) can be connected to the frame buffer compressor (20). That is, the multimedia IP (10) can be connected to the memory (30) through the frame buffer compressor (20) and the system bus (40).

[0042] The frame buffer compressor (20) can perform the task of converting source data into compressed data or compressed data into output data when each of the image signal processor (11), shake correction module (12), multi-format codec (13), processing module (14), and display (15) of the multimedia IP (10) accesses memory.

[0043] In the compression process of a general frame buffer compressor, quantization can be performed by applying a half-rounding technique to the source data. In this process, source data that yields a decimal value of 0.5 for a predetermined quantization factor can be rounded to generate quantized data. When decompressing the compressed data, output data can be generated by reflecting the aforementioned predetermined quantization factor in the quantized data. At this time, the difference between the output data and the source data can be calculated as error data.

[0044] By applying the half-rounding technique, the Mean Squared Error (MSE) of multiple error data can be minimized and the Peak Signal-to-Noise Ratio (PSNR) can be improved. However, since source data with a decimal value of 0.5 is generated as quantized data by rounding for the aforementioned predetermined quantization coefficient, a problem may occur in which the error data for multiple source data are biased in a specific direction.

[0045] A frame buffer compressor (20) according to one embodiment of the present invention can generate quantized data by performing quantization on source data using a plurality of pixel position offsets corresponding to the coordinates of each of a plurality of image pixels included in source data, and a predetermined quantization coefficient. The frame buffer compressor (20) can generate compressed data by processing the quantized data.

[0046] According to one embodiment of the present invention, the plurality of pixel position offsets may be calculated as 0 or 1, and the predetermined quantization coefficient may be a positive even number. The frame buffer compressor (20) may generate quantized data by adding half of the quantization coefficient to the source data and subtracting the pixel position offset, then dividing the result by the quantization coefficient. Accordingly, the error data may not be biased in a specific direction and may exhibit a random tendency. Additionally, the implementation of calculating the pixel position offset may be simplified.

[0048] FIG. 2 is a block diagram simply showing a frame buffer compressor according to one embodiment of the present invention.

[0049] Referring to FIG. 2, the frame buffer compressor (100) may include an encoder (110) and a decoder (120).

[0050] The encoder (110) can compress source data (SD) to generate compressed data (CD). Referring to FIGS. 1 and FIGS. 2 together, the encoder (110) can compress source data (SD) from a multimedia IP (10) to generate compressed data (CD). The compressed data (CD) can be transmitted to memory (30) via a system bus (40).

[0051] The decoder (120) can decompress compressed data (CD) stored in memory (30) to generate output data (OD). For example, the compressed data (CD) can be transferred from memory (30) to a frame buffer compressor (20). The compressed data (CD) transferred to the frame buffer compressor (20) can be decompressed by the decoder (120).

[0052] The output data (OD) can be transmitted to the multimedia IP (10). At this time, the output data (OD) can be transmitted to each of the image signal processor (11), shake correction module (12), multi-format codec (13), processing module (14), and display (15) of the multimedia IP (10).

[0054] FIG. 3 is a simple block diagram showing an encoder according to one embodiment of the present invention.

[0055] The image processing device may include a multimedia IP, a frame buffer compressor, memory, and a system bus. The frame buffer compressor may include an encoder and a decoder. Specific embodiments of the image processing device may be similar to those described above in FIGS. 1 and FIG. 2.

[0056] Referring to FIG. 3, the encoder (200) includes a first mode selector (260), a quantization module (210), a prediction module (220), an entropy encoding module (230), a padding module (240), and a compression management module (250).

[0057] The first mode selector (260) can determine whether the encoder (200) operates in a lossless mode or a lossy mode. The first mode selector (260) can receive a signal from the multimedia IP to determine whether to perform lossless compression or lossy compression. In lossless compression, data can be compressed without loss, and the compression ratio may vary depending on the data. In lossy compression, some of the data may be lost during compression, and the compression ratio may be higher than that of lossless compression.

[0058] When the encoder (200) operates in a lossless mode, the source data can be compressed along a lossless path of an embodiment illustrated in FIG. 3. The first mode selector (260) can guide the data flow along the lossless path to a prediction module (220), an entropy encoding module (230), and a padding module (240). In this case, the first mode selector (260) can transmit the source data (SD) input to the encoder (200) to the prediction module (220).

[0059] When the encoder (200) operates in a loss mode, the source data can be compressed along a loss path of an embodiment illustrated in FIG. 3. The first mode selector (260) can guide the data flow along the loss path to a quantization module (210), a prediction module (220), an entropy encoding module (230), and a padding module (240). In this case, the first mode selector (260) can transmit the source data (SD) input to the encoder (200) to the quantization module (210). The quantization module (210) can perform quantization on the source data (SD) to generate quantized data (QD) and transmit the quantized data (QD) to the prediction module (220).

[0060] The prediction module (220) can generate prediction data by performing intra-prediction on source data (SD) or quantized data (QD). The prediction module (220) can perform predictions on a pixel-by-pixel basis by utilizing the spatial correlation of the data. For example, the prediction module (220) can generate prediction data based on the statistical characteristics of surrounding data (e.g., pixel values ​​up, down, left, and right). The prediction module (220) can generate residual data based on the difference between the source data and the prediction data, and can increase the efficiency of subsequent entropy encoding by reducing data redundancy.

[0061] The entropy encoding module (230) can generate entropy data by performing entropy encoding on the prediction data. The entropy encoding module (230) can apply entropy encoding algorithms such as Huffman Coding, Arithmetic Coding, or CABAC (Context-Adaptive Binary Arithmetic Coding). The entropy encoding module (230) can represent data with a minimum number of bits by assigning short codes to data with high occurrence frequency and long codes to data with low frequency. Accordingly, the storage space for the data is reduced, and the transmission efficiency of the data can be improved.

[0062] The padding module (240) can generate padding data by performing padding on the entropy data. The padding module (240) can generate padding data having a predefined size by adding meaningless data (e.g., zero data) to the entropy data. In one embodiment, when the encoder (200) operates in a lossy mode, the size of the padding data can be defined based on the size of the source data and a fixed compression ratio. If the size of the source data is 100 and the fixed compression ratio is 50%, the size of the padding data can be defined as 50. In another embodiment, when operating in a lossless mode, the size of the padding data can be defined based on the size of the source data. If the size of the source data is 100, the size of the padding data can be defined as 100.

[0063] A quantization module (210) according to one embodiment of the present invention can generate quantized data (QD) by performing quantization on source data (SD) using a plurality of pixel position offsets corresponding to the coordinates of each of a plurality of image pixels included in source data (SD) and a predetermined quantization coefficient.

[0064] According to one embodiment of the present invention, the plurality of pixel position offsets may be calculated as 0 or 1, and the predetermined quantization coefficient may be a positive even number. The quantization module (210) may generate quantized data (QD) by adding half of the quantization coefficient to the source data (SD) and subtracting the pixel position offset, and then dividing the result by the quantization coefficient.

[0066] FIG. 4 is a circuit diagram simply illustrating a pixel array included in a sensor according to one embodiment of the present invention.

[0067] The image processing device may include a multimedia IP, a frame buffer compressor, memory, and a system bus. The frame buffer compressor may include an encoder and a decoder. Specific embodiments of the image processing device may be similar to those described above in FIGS. 1 to 3.

[0068] Referring to FIG. 4, the decoder (300) includes a second mode selector (360), an unpadding module (340), a decompression management module (350), an entropy decoding module (330), a prediction compensation module (320), and an inverse quantization module (310).

[0069] The second mode selector (360) can determine whether the compressed data (CD) stored in memory is lossless compressed or lossy compressed. For example, in the case of a lossless mode, the second mode selector (360) can guide the compressed data (CD) along a lossless path. The compressed data (CD) can be guided to an unpadding module (340), an entropy decoding module (330), and a prediction compensation module (320). As another example, in the case of a lossy mode, the second mode selector (360) can guide the compressed data (CD) along a lossy path. The compressed data (CD) can be guided to an unpadding module (340), an entropy decoding module (330), a prediction compensation module (320), and an inverse quantization module (310).

[0070] The unpadding module (340) can remove meaningless data (e.g., zero data) added by the padding module (240 in FIG. 3) of the encoder. For example, the unpadding module (340) can generate entropy data by removing zero data from compressed data (CD). The entropy data generated by the unpadding module (340) can be transmitted to the entropy decoding module (330).

[0071] The entropy decoding module (330) can generate prediction data from the entropy data transmitted from the unpadding module (340). The compressed data (CD) input to the decoder (300) includes a k value. The entropy decoding module (330) can perform entropy decoding using an entropy table identified from the k value. For example, the entropy decoding module (330) can generate residual data corresponding to the entropy data using the entropy table. The entropy decoding module (330) can generate prediction data using the residual pixel data. The prediction data generated from the entropy decoding module (330) can be transmitted to the prediction compensation module (320). The entropy decoding module (330) can perform decompression through Huffman coding, exponential Gollum coding, or Gollum rice coding.

[0072] The prediction compensation module (320) can generate recon data (RD) or output data (OD) by performing intra-prediction compensation on the prediction data. The prediction compensation module (320) can decompress the prediction data by performing intra-prediction by the prediction module (220 in FIG. 4) in reverse order. If the compressed data (CD) is compressed in a lossy mode, the prediction compensation module (320) can generate recon data (RD) and transmit it to the inverse quantization module (310). If the compressed data (CD) is compressed in a lossless mode, the prediction compensation module (320) can generate output data (OD). In this case, the decoder (300) can transmit the output data (OD) to the multimedia IP.

[0073] The inverse quantization module (310) can generate output data (OD) from quantization coefficients from compressed data (CD) and recon data (RD) transmitted from the prediction compensation module (320). The inverse quantization module (310) can perform inverse quantization on the recon data using quantization coefficients and generate output data (OD) as a result. In one embodiment, output data can be generated by multiplying the recon data by quantization coefficients. In another embodiment, output data can be generated by performing a bit shift operation on the recon data.

[0074] Referring to FIGS. 3 and FIGS. 4 together, the inverse quantization module (310) can generate output data (OD) from recon data (RD). At this time, the output data (OD) generated by the inverse quantization module (310) may be different from the source data (SD) input to the encoder (200). This is because when quantization is performed on the source data (SD) in the quantization module (210) of the encoder (200), some of the source data (SD) may be lost and cannot be restored. Therefore, the inverse quantization module (310) can be utilized only in loss mode.

[0076] FIG. 5 is a diagram simply illustrating image data generated by a sensor according to one embodiment of the present invention.

[0077] Referring to FIG. 5, a pixel array (400) of an image sensor according to one embodiment of the present invention may include a plurality of pixel regions (410, 420, 430) arranged along a first direction (X-axis direction) and a second direction (Y-axis direction). In one embodiment illustrated in FIG. 5, color filters may be arranged along the plurality of pixel regions (410, 420, 430). For example, the pixel array (400) may include red pixel regions (410), green pixel regions (420), and blue pixel regions (430). Each of the red pixel regions (410) may include a red color filter, each of the green pixel regions (420) may include a green color filter, and each of the blue pixel regions (430) may include a blue color filter.

[0079] FIG. 6 is a diagram simply illustrating image data generated by an image sensor according to one embodiment of the present invention.

[0080] FIG. 6 may be an example of image data (450) generated by an image sensor, and the image data (450) may include a plurality of image pixels (460, 470, 480). The plurality of image pixels (460, 470, 480) may be arranged along a first direction and a second direction, similar to pixel regions (410, 420, 430) arranged in a pixel array (400 in FIG. 5).

[0081] In one embodiment, a plurality of image pixels (460, 470, 480) may correspond to pixel regions (410, 420, 430). Accordingly, each of the plurality of image pixels (460, 470, 480) may represent one of red, green, or blue colors, and the image data (450) may be generated in a Bayer data format. If the pixel array (400) includes color filters in a pattern other than a Bayer pattern, the plurality of image pixels (460, 470, 480) included in the image data (450) may also have a color arrangement according to the pattern of the color filters included in the pixel array (400).

[0082] An image processing device of one embodiment of the present invention can convert image data (450) into source data in the YUV format. Each of the plurality of source pixel data included in the source data may correspond to a plurality of image pixels (460, 470, 480) of one embodiment illustrated in FIG. 6. The plurality of image pixels (460, 470, 480) may correspond to coordinates according to a first direction and a second direction, and said coordinates may include a first coordinate value according to the first direction and a second coordinate value according to the second direction.

[0084] FIG. 7 is a drawing provided to explain source data according to an embodiment of the present invention. FIG. 8 is a drawing provided to explain quantized data according to an embodiment of the present invention.

[0085] First, referring to FIG. 7, the source data may include a plurality of source pixel data. In one embodiment illustrated in FIG. 7, the source data may include a plurality of source pixel data (s(0,0) to s(3,3)) having four rows and four columns. The coordinates of the plurality of source pixel data (s(0,0) to s(3,3)) may correspond to the coordinates of each of the plurality of image pixels.

[0086] Each of the plurality of source pixel data (s(0,0) to s(3,3)) may have a value, and said value may be expressed in a binary format. In one embodiment, the plurality of source pixel data (s(0,0) to s(3,3)) may have a value between 0 and 255, and said value may be expressed in a 9-bit binary format. If the source data is in the YUV format, said source pixel data may represent at least one of luminance and chrominance. However, it may not be limited thereto.

[0087] A frame buffer compressor of one embodiment of the present invention can generate quantized data by performing quantization on the source data using a plurality of pixel position offsets corresponding to the coordinates of each of a plurality of image pixels and a predetermined quantization coefficient. The quantized data may include a plurality of quantized pixel data corresponding to each of the plurality of image pixels. In other words, quantization can be performed in coordinate units corresponding to the plurality of image pixels.

[0088] A frame buffer compressor of one embodiment of the present invention can generate quantized pixel data by calculating source pixel data, a quantization offset determined as half of the quantization coefficient, a pixel position offset, and a quantization coefficient. The quantization coefficient may be a reference interval that divides the size of the source pixel data into equal intervals. For example, the quantization coefficient may be a positive even number, and the quantization offset may be half of the quantization coefficient.

[0089] Quantized pixel data can satisfy Equation 1. Specifically, quantized pixel data (QPD) can be calculated by adding a quantization offset (QOFF) to source pixel data (SPD) and subtracting a pixel position offset (POFF), then dividing the result by a quantization coefficient (QS).

[0090]

[0091] A pixel position offset (PDFF) can be calculated using the coordinates of an image pixel. Referring to FIG. 6, the coordinates of the image pixel may include a first coordinate value according to a first direction and a second coordinate value according to a second direction. A frame buffer compressor can calculate the corresponding pixel position offset by performing a modulo operation on the first coordinate value and the second coordinate value.

[0092] In one embodiment of the present invention, a frame buffer compressor can calculate a pixel position offset by performing a modulo-2 operation on the sum of a first coordinate value and a second coordinate value. By performing a modulo-2 operation, the pixel position offset may be 0 or 1.

[0093] Referring to FIG. 8, quantized data generated by applying a quantization factor 4 to the source data of one embodiment illustrated in FIG. 7 can be illustrated. The quantized data may include a plurality of quantized pixel data (p(0,0) to p(3,3)) having 4 rows and 4 columns. Each of the plurality of source pixel data (p(0,0) to p(3,3)) may correspond to the coordinates of each of the plurality of image pixels and the plurality of source pixel data (s(0,0) to s(3,3)).

[0094] Referring to (0,0) in FIGS. 7 and 8, the pixel position offset calculated by performing a modulo-2 operation on the sum of the first coordinate value (0) and the second coordinate value (0) is 0. By adding the quantization offset (4 / 2=2) to the source pixel data (s(0,0)=88) and subtracting the pixel position offset (0), and dividing the result by the quantization coefficient (4), the quantized pixel data (q(0,0)) can be calculated as 22.

[0095] Referring to (0,1) in FIGS. 7 and 8, the pixel position offset calculated by performing a modulo-2 operation on the sum of the first coordinate value (0) and the second coordinate value (1) is 1. By adding the quantization offset (4 / 2=2) to the source pixel data (s(0,1)=89) and subtracting the pixel position offset (1), and dividing the result by the quantization coefficient (4), the quantized pixel data (q(0,1)) can be calculated as 22.

[0096] Referring to (0,2) in FIGS. 7 and 8, the pixel position offset calculated by performing a modulo-2 operation on the sum of the first coordinate value (0) and the second coordinate value (2) is 0. By adding the quantization offset (4 / 2=2) to the source pixel data (s(0,2)=93) and subtracting the pixel position offset (0), and dividing the result by the quantization coefficient (4), the quantized pixel data (q(0,2)) can be calculated as 23.

[0097] Referring to (0,3) in FIGS. 7 and 8, the pixel position offset calculated by performing a modulo-2 operation on the sum of the first coordinate value (0) and the second coordinate value (3) is 1. By adding the quantization offset (4 / 2=2) to the source pixel data (s(0,3)=96) and subtracting the pixel position offset (1), and dividing the result by the quantization coefficient (4), the quantized pixel data (q(0,3)) can be calculated as 24.

[0098] The remaining quantized pixel data (q(1,0) to q(3,3)) can also be calculated and produced in the same way as above.

[0099] In one embodiment of the present invention, a method for calculating a pixel position offset may be configured such that the pixel position offset is calculated as 0 or 1. The pixel position offset may be calculated by performing a modulo-2 operation on the sum of a first coordinate value, a second coordinate value, and 1. The pixel position offset may be calculated by performing a modulo-2 operation on the first coordinate value or by performing a modulo-2 operation on the second coordinate value. The pixel position offset may be calculated by performing a modulo-2 operation on the product of the first coordinate value and the second coordinate value.

[0100] Alternatively, the pixel position offset may be calculated by performing an exclusive OR (XOR) operation on the result of a modulo-4 operation on the first coordinate value and the result of a modulo-4 operation on the second coordinate value, and then performing a modulo-2 operation on the result of the exclusive OR operation. However, the method for calculating the pixel position offset may not be limited to this.

[0101] In another embodiment of the present invention, the processes of calculating a pixel position offset using the coordinates of an image pixel and reflecting the pixel position offset in the quantized data can be similarly applied to the filtering process of the source data. For example, when performing spatial domain filtering, the methods for calculating the pixel position offset described above can be applied to calculate a first offset.

[0102] A predetermined coefficient and a second offset determined as half of the predetermined coefficient can be determined. According to the above mathematical formula 1, source data can be filtered using the first offset, the second offset, and the predetermined coefficient.

[0104] FIG. 9 is a drawing provided to explain output data according to an embodiment of the present invention.

[0105] A frame buffer compressor can generate output data from recon data. The recon data may be data generated after the compressed data passes through a mode selector, an unpadding module, an entropy encoding module, and a prediction compensation module. The recon data may include multiple recon pixel data, and the coordinates of the multiple recon pixel data may correspond to the coordinates of each of the multiple image pixels. For example, the recon data may be identical to the quantized data. In other words, the recon pixel data and the quantized pixel data corresponding to the same coordinates may be identical.

[0106] A frame buffer compressor can generate output data by decompressing recon data. A frame buffer compressor can generate output data by applying a predetermined quantization coefficient to the quantized data. For example, the output data may be the product of the recon data and the quantization coefficient. The output data may include multiple output pixel data (O(0,0) to O(3,3)).

[0107] The output pixel data can satisfy Equation 2. Specifically, the output pixel data (OPD) can be calculated by multiplying the recon pixel data (RPD) by a quantization factor (QS). Below, with reference to FIG. 8, the output pixel data when the quantization factor is 4 will be explained.

[0108]

[0109] Referring to (0,0) in FIGS. 7 and 8, the output pixel data (O(0,0)) can be calculated as 88 as a result of multiplying the recon pixel data (22) by the quantization coefficient (4). Referring to (0,1) in FIGS. 7 and 8, the output pixel data (O(0,1)) can be calculated as 88 as a result of multiplying the recon pixel data (22) by the quantization coefficient (4).

[0110] Referring to (0,2) in FIGS. 7 and 8, the output pixel data (O(0,3)) can be calculated as 92 when the recon pixel data (23) is multiplied by the quantization coefficient (4). Referring to (0,3) in FIGS. 7 and 8, the output pixel data (O(0,3)) can be calculated as 96 when the recon pixel data (24) is multiplied by the quantization coefficient (4).

[0111] The remaining output pixel data (O(1,0) to O(3,3)) can also be calculated and produced in the same way as above.

[0113] FIG. 10 is a drawing provided to explain error data according to one embodiment of the present invention.

[0114] Referring to FIGS. 7 to 9 together, the method of producing quantized data by performing quantization on source data may not correspond to the method of producing output data by performing inverse quantization on recon data. In other words, Equation 1 used for quantization and Equation 2 used for inverse quantization may not be symmetrical.

[0115] In the quantization process, lossy compression can be performed by applying quantization coefficients and pixel position offsets to the source data. In the dequantization process, lost data cannot be recovered, so a difference may occur between the output data and the source data.

[0116] Referring to FIG. 10, the error data may include a plurality of error pixel data (e(0,0) to e(3,3)). Each of the plurality of error pixel data (e(0,0) to e(3,3)) may correspond to the coordinates of each of the plurality of image pixels and a plurality of source pixel data (s(0,0) to s(3,3)). The error pixel data may be the difference between the output pixel data and the source pixel data.

[0117] Referring to (0,0) in FIGS. 7 to 10, the error pixel data (e(0,0)) can be calculated as 0 as a result of subtracting the source pixel data (88) from the output pixel data (88). Referring to (0,1) in FIGS. 7 to 10, the error pixel data (e(0,0)) can be calculated as -1 as a result of subtracting the source pixel data (89) from the output pixel data (88).

[0118] Referring to (0,2) in FIGS. 7 to 10, the error pixel data (e(0,2)) can be calculated as -1 as a result of subtracting the source pixel data (93) from the output pixel data (92). Referring to (0,3) in FIGS. 7 to 10, the error pixel data (e(0,3)) can be calculated as 0 as a result of subtracting the source pixel data (96) from the output pixel data (96).

[0119] The remaining output pixel data (O(1,0) to O(3,3)) can also be calculated and produced in the same way as above.

[0120] Error pixel data according to one embodiment of the present invention may exhibit a random tendency without being biased in a specific direction. In one embodiment illustrated in FIG. 10, the error pixel data may have values ​​between -1 and 2. Since the error pixel data consists only of 0 and positive values, or not only of 0 and negative values, the sum of the multiple error pixel data may converge to 0.

[0122] FIG. 11 is a simplified diagram of an image processing device according to one embodiment of the present invention.

[0123] FIG. 11 is a block diagram illustrating an image processing device (500) according to an embodiment of the present invention. Specific embodiments of the image processing device may be similar to those previously described in FIG. 1 to FIG. 10. Hereinafter, the differences will be described in detail.

[0124] Referring to FIG. 11, the multimedia IP (510), frame buffer compressor (520), and memory (530) can each be directly connected to the system bus (544). The frame buffer compressor (520) is not directly connected to the multimedia IP (510), but can be connected to each other through the system bus (540).

[0125] The multimedia IP (510) can transmit data to and from the frame buffer compressor (520) via the system bus (540). During the compression process, the multimedia IP (510) can transmit source data to the frame buffer compressor (520) via the system bus (540). The frame buffer compressor (520) generates compressed data from the source data and can transmit the compressed data back to memory (530) via the system bus (540).

[0126] Additionally, during the decompression process, the frame buffer compressor (520) can receive the compressed data stored in the memory (530) through the system bus (540). The frame buffer compressor (520) can decompress the received compressed data into output data. The frame buffer compressor (520) can transmit the output data to the multimedia IP (510) through the system bus (540).

[0127] In one embodiment illustrated in FIG. 11, the frame buffer compressor (520) may be connected via the system bus (540) without being individually connected to the image signal processor (511), shake correction module (512), multi-format codec (513), processing module (514), and display (515) of the multimedia IP (510). Thus, the hardware configuration can be simplified and the operating speed can be improved.

[0129] FIG. 12 is a simplified diagram of an image processing device according to one embodiment of the present invention.

[0130] FIG. 12 is a block diagram illustrating an image processing device (600) according to an embodiment of the present invention. Specific embodiments of the image processing device may be similar to those described above in FIG. 1 to FIG. 10. Hereinafter, the differences will be described in detail.

[0131] Referring to FIG. 12, the multimedia IP (610) and the frame buffer compressor (620) can be directly connected to the system bus (640). The memory (630) can be connected to the system bus (640) through the frame buffer compressor (620). In other words, the memory (630) is not directly connected to the system bus (640), but can be connected to the system bus (440) only through the frame buffer compressor (620).

[0132] Additionally, the image signal processor (611), shake correction module (612), multi-format codec (613), processing module (614), and display (615) of the multimedia IP (610) can be directly connected to the system bus (640). The image signal processor (611), shake correction module (612), multi-format codec (613), processing module (614), and display (615) of the multimedia IP (610) can access the memory (630) only after passing through the frame buffer compressor (620).

[0133] In one embodiment illustrated in FIG. 12, a frame buffer compressor (620) may be involved in all access to the memory (630). By directly connecting the frame buffer compressor (620) to the system bus (640) and allowing the memory (630) to be connected to the system bus (640) through the frame buffer compressor (620), errors in data transmission can be reduced and speed improved.

[0135] FIG. 13 is a simplified diagram of an image processing device according to one embodiment of the present invention.

[0136] FIG. 13 is a block diagram illustrating an image processing device (700) according to an embodiment of the present invention. Specific embodiments of the image processing device may be similar to those previously described in FIG. 1 to FIG. 10. Hereinafter, the differences will be described in detail.

[0137] Referring to FIG. 13, the system bus (740) can be directly connected to the multimedia IP (710) and memory (730). The frame buffer compressor (720) can be connected to the multimedia IP (710). The frame buffer compressor (720) can receive source data from the multimedia IP (710). The frame buffer compressor (720) can compress the source data to generate compressed data and transmit it back to the multimedia IP (710). The multimedia IP (710) can store the compressed data in memory (730) via the system bus (740).

[0138] During the decompression process, the multimedia IP (710) can receive compressed data from the memory (730) via the system bus (740). The multimedia IP (710) can transfer the compressed data to the frame buffer compressor (720). The frame buffer compressor (720) can decompress the compressed data to generate output data and transfer it back to the multimedia IP (710).

[0140] The present invention is not limited by the embodiments described above and the attached drawings, but is intended to be limited by the appended claims. Accordingly, various substitutions, modifications, and changes may be made by those skilled in the art within the scope of the technical concept of the present invention as described in the claims, and such are also to be considered to fall within the scope of the present invention. Explanation of the symbols

[0142] 1, 500, 600, 700: Image processing unit 10, 510, 610, 710: Multimedia IP 20, 100, 520, 620, 720: Frame buffer compressor 200: Encoder 210: Quantization Module 300: Decoder 310: Inverse Quantization Module

Claims

Claim 1 An image processing device comprising: a multimedia IP (Intellectual Property) that processes image data to generate source data; and a frame buffer compressor that compresses the source data to generate compressed data and decompresses the compressed data to generate output data; wherein the frame buffer compressor performs quantization on the source data to generate quantized data using a plurality of pixel position offsets corresponding to the coordinates of each of the plurality of image pixels included in the source data, a predetermined quantization coefficient, and a quantization offset determined as half of the quantization coefficient, and processes the quantized data to generate the compressed data. Claim 2 An image processing device according to claim 1, wherein the quantization coefficient is a positive even number. Claim 3 An image processing device according to claim 1, wherein the plurality of pixel position offsets are 0 or 1. Claim 4 An image processing device according to paragraph 3, wherein each of the coordinates of the plurality of image pixels comprises a first coordinate value according to a first direction and a second coordinate value according to a second direction intersecting the first direction. Claim 5 In paragraph 4, the frame buffer compressor calculates the pixel position offset by performing a modulo operation on the first coordinate value and the second coordinate value for each of the plurality of image pixels, an image processing device. Claim 6 In claim 5, the frame buffer compressor calculates the pixel position offset by performing a modulo-2 operation on the sum of the first coordinate value and the second coordinate value for each of the plurality of image pixels. Claim 7 In claim 5, the frame buffer compressor calculates the pixel position offset by performing a modulo-2 operation on the sum of the first coordinate value, the second coordinate value, and 1 for each of the plurality of image pixels. Claim 8 In claim 5, the frame buffer compressor calculates the pixel position offset by performing a modulo-2 operation on the first coordinate value for each of the plurality of image pixels, an image processing device. Claim 9 In claim 5, the frame buffer compressor calculates the pixel position offset by performing a modulo-2 operation on the second coordinate value for each of the plurality of image pixels, an image processing device. Claim 10 In claim 5, the frame buffer compressor calculates the pixel position offset by performing a modulo-2 operation on the product of the first coordinate value and the second coordinate value for each of the plurality of image pixels. Claim 11 An image processing device according to claim 5, wherein the frame buffer compressor calculates the pixel position offset by performing an exclusive OR (XOR) operation on the result of a modulo-4 operation on the first coordinate value and the result of a modulo-4 operation on the second coordinate value for each of the plurality of image pixels, and performing a modulo-2 operation on the result of the exclusive OR operation. Claim 12 An image processing device according to claim 1, wherein the source data includes a plurality of source pixel data corresponding to each of the plurality of image pixels, and the quantization data includes a plurality of quantization pixel data corresponding to each of the plurality of image pixels, and the frame buffer compressor generates the quantization pixel data by calculating the source pixel data, the quantization offset, the pixel position offset, and the quantization coefficient for each of the plurality of image pixels. Claim 13 In claim 12, the frame buffer compressor generates the quantized pixel data by adding the quantization offset to the source pixel data and subtracting the pixel position offset, and dividing the result by the quantization coefficient. Claim 14 An image processing device according to claim 1, further comprising: a sensor that generates the image data; and a memory that stores and outputs the compressed data. Claim 15 A frame buffer compressor comprising: an encoder that compresses source data to generate compressed data; and a decoder that decompresses the compressed data to generate output data; wherein the encoder performs quantization on the source data to generate quantized data using pixel position offsets calculated by modulo operation on the coordinates of each of a plurality of image pixels included in the source data, and a predetermined quantization coefficient, and processes the quantized data to generate the compressed data. Claim 16 In paragraph 15, a frame buffer compressor in which the pixel position offsets are 0 or 1. Claim 17 In paragraph 15, the decoder is a frame buffer compressor that generates output data using the compressed data and the quantization coefficients. Claim 18 In paragraph 17, the above output data is a frame buffer compressor in which the above output data is the product of the recon data generated by processing the above compressed data and the above quantization coefficient. Claim 19 A data processing method comprising: receiving source data from a multimedia IP; a frame buffer compressor calculating a first offset determined by the coordinates of each of a plurality of image pixels included in the source data; a frame buffer compressor determining a predetermined coefficient and a second offset determined by half of the predetermined coefficient; and a frame buffer compressor performing an operation of subtracting the first offset from the source data and adding the second offset, and dividing the result by the predetermined coefficient. Claim 20 A data processing method according to claim 19, wherein the step of calculating the first offset comprises, for each of the plurality of image pixels, calculating the first offset by performing a modulo-2 operation on the sum of the first coordinate value and the second coordinate value.

Images