Image processing device, image processing method, image processing program, and image processing system

Abstract

To enable the prediction accuracy of luminance values of images to be heightened.SOLUTION: An encoder 10 is constituted to comprise: a prediction adjustment amount table 21 that stores feature information regarding blocks in a prescribed image and prediction adjustment amount identification information that makes it possible to identify a prediction adjustment amount for adjusting a prediction residual of AC components of luminance for blocks corresponding to the feature information, where the feature information and the prediction adjustment amount identification information are associated with each other; a feature information identification unit 15 that identifies feature information regarding a relevant block in the image of interest; and a feature amount identification unit 16 that identifies, from the prediction adjustment amount identification information of the prediction adjustment amount table 21, a prediction adjustment amount that corresponds to the feature information identified by the feature information identification unit 15. The feature information is a pattern that indicates a relationship of DC components of luminance between a block and some of a plurality of adjacent blocks.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device or the like that processes images such as still images and moving images. [Background technology]

[0002] Conventionally, image compression processing has been performed to reduce the volume of image data. For example, AC component prediction is known as a fundamental technology of image compression processing.

[0003] AC analysis prediction is a process for predicting AC components in a target block by referencing DC components of the target block and its surrounding blocks among blocks set on an image plane. The AC components predicted by AC analysis prediction can be used to generate child blocks that exist on the target block. This is described, for example, in Non-Patent Document 1.

[0004] In video compression using AC component prediction, the DC component of the target block and the difference component obtained by AC component prediction are coded.

[0005] Furthermore, as a technique related to AC component prediction, Patent Document 1 discloses a technique for improving prediction accuracy by predicting AC components orthogonal to directions other than the horizontal and vertical directions.

[0006] Furthermore, Patent Document 2 discloses a technique for further improving prediction accuracy by correcting prediction errors from global information, which can reduce the effects of overshoot and undershoot on the contours of an image. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-239191 [Patent Document 2] Patent Publication No. 2021-72510 [Non-patent literature]

[0008] [Non-Patent Document 1] Ryuji Tokunaga, Fractals and Image Processing - Fundamentals and Applications of Differential Dynamics, Corona Publishing, 2002 Summary of the Invention [Problem to be solved by the invention]

[0009] In image compression processing, there is a demand for improving the compression rate of images, and it is important to improve the accuracy of prediction of image luminance values.

[0010] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique for improving the prediction accuracy of the luminance values ​​of an image. [Means for solving the problem]

[0011] In order to achieve the above object, an image processing device according to a first aspect includes an adjustment amount storage unit that stores feature information about a block in a specified image and prediction adjustment amount identification information that can identify a prediction adjustment amount for adjusting a prediction residual error of an AC component of luminance for the block corresponding to the feature information, in association with each other; a feature information identification unit that identifies feature information about a target block in a target image; and an adjustment amount identification unit that identifies, from the prediction adjustment amount identification information in the adjustment amount storage unit, a prediction adjustment amount that corresponds to the feature information identified by the feature information identification unit, wherein the feature information is a pattern that indicates a relationship in terms of a DC component of luminance between the block and a partial block that is a part of a plurality of adjacent blocks. [Effects of the Invention]

[0012] According to the present invention, it is possible to improve the prediction accuracy of the luminance value of an image. [Brief explanation of the drawings]

[0013] [Figure 1]FIG. 1 is a diagram showing the overall configuration of an image processing system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a block and a child block according to an embodiment. [Figure 3] FIG. 3 is a diagram illustrating the notation of blocks and DC components of the blocks according to one embodiment. [Figure 4] FIG. 4 is a diagram illustrating feature information of a target block according to an embodiment. [Figure 5] FIG. 5 is a diagram illustrating the configuration of a predicted adjustment amount table according to one embodiment. [Figure 6] FIG. 6 is a diagram illustrating calculation of feature information of a target block according to an embodiment. [Figure 7] FIG. 7 is a flowchart of an encoding process according to an embodiment. [Figure 8] FIG. 8 is a flowchart of a decoding process according to an embodiment. [Figure 9] FIG. 9 is a diagram illustrating the configuration of a computer device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] The following description of the embodiments will be given with reference to the drawings. Note that the embodiments described below do not limit the scope of the invention as claimed, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solution of the invention.

[0015] First, an image processing system according to an embodiment will be described.

[0016] FIG. 1 is a diagram showing the overall configuration of an image processing system according to an embodiment.

[0017] The image processing system 1 includes an encoder 10 as an example of an image processing device, and a decoder 30 as an example of an image processing device. The encoder 10 and the decoder 30 are connected via a network 50. The network 50 is, for example, a local area network (LAN) or a wide area network (WAN).

[0018] The encoder 10 includes a target image input unit 11, a DC conversion unit 12, a Hadamard transform unit 13, an AC component prediction unit 14, a feature information identification unit 15, an adjustment amount identification unit 16, a prediction residual calculation unit 17, an encoding unit 18, an adjustment amount update unit 19, a compressed data output unit 20, and a prediction adjustment amount table 21.

[0019] The target image input unit 11 inputs an image to be compressed (target image). The target image may be, for example, a still image or a single frame image of a moving image. The target image may be acquired from an auxiliary storage device 106 (see FIG. 9) in the encoder 10, or may be acquired from an imaging device (not shown).

[0020] The DC converter 12 generates DC components for a block (target block) to be processed in the target image based on the luminance value of each pixel in the target image. For example, the block in the highest layer (first layer) of the target block is 16 pixels by 16 pixels. For blocks in layers below the first layer, the block in the second layer is 8 pixels by 8 pixels, the block in the third layer is 4 pixels by 4 pixels, and the block in the fourth layer is 2 pixels by 2 pixels. The DC component in the target block is, for example, the average luminance value of the pixels included in the target block. In the following processing, the order in which blocks in the target image are selected as target blocks is assumed to be a predetermined order. In this embodiment, blocks in the horizontal direction starting from the top left of the image are selected as target blocks. After all horizontal blocks have been selected as target blocks, the blocks one layer below are selected as target blocks in the same manner. The order of the target blocks is not limited to this, and any order that can be commonly recognized by the decoder 30 may be used. Furthermore, the DC conversion unit 12 calculates the DC component of each of the child blocks that make up the target block based on the luminance value of each pixel for the child blocks.

[0021] Here, the relationship between each block and its child blocks, and the notation of a block and its DC component in this specification will be explained.

[0022] FIG. 2 is a diagram illustrating a block and a child block according to an embodiment.

[0023] Here, the block 60 has, for example, four equal child blocks 61. Here, the DC component (average brightness of pixels) of the upper right child block 61 in the block 60 is δ ++ The DC component of the lower right child block 61 is set as δ +- The DC component of the upper left child block 61 is set as δ -+ The DC component of the lower left child block is δ -- Let's say.

[0024] FIG. 3 is a diagram illustrating the notation of blocks and DC components of the blocks according to one embodiment.

[0025] In this specification, the target block is represented as B[0,0], and the blocks surrounding the target block are represented as B[a,b] as shown in Figure 3. The DC component of the block B[a,b], i.e., the average luminance of the pixels included in B[a,b], is represented as D[a,b]. For example, the DC component of the target block (B[0,0]) is represented as D[0,0], the block immediately to the right of the target block (target right block) is represented as B[1,0] and its DC component is D[1,0], the block immediately to the left of the target block (target left block) is represented as B[-1,0] and its DC component is D[-1,0], the block immediately above the target block (target upper block) is represented as B[0,1] and its DC component is D[0,1], and the block immediately below the target block (target lower block) is represented as B[0,-1] and its DC component is D[0,-1].

[0026] 1, the Hadamard transform unit 13 performs a Hadamard transform to calculate a horizontal AC component α, a vertical AC component β, a diagonal AC component γ, and a DC component δ of the target block. Note that the DC component δ is the same as the DC component of the target block, so it does not need to be calculated.

[0027] Here, the AC components α, β, γ, and DC components δ of the target block can be expressed as shown in the following equations (1) to (4) using the DC components of the child blocks of the target block.

[0028] AC component α=(δ ++ +δ +- -δ -+ -δ -- ) / 4 ···(1) AC component β=(δ ++ -δ +- +δ -+ -δ -- ) / 4 ···(2) AC component γ=(δ ++ -δ +- -δ -+ +δ -- ) / 4 ···(3) DC component δ=(δ +++δ +- +δ -+ +δ -- ) / 4 (4) The DC component δ is the DC component D[0,0] of the target block B[0,0].

[0029] The AC component prediction unit 14 calculates a predicted value of the AC component of the target block according to the AC component prediction. For example, for the AC component α, the AC component prediction unit 14 calculates a predicted value of the AC component of the target block from the DC components of the block (target right block) B[1,0] on the right side of the target block B[0,0] and the block (target left block) B[-1,0] on the left side of the target block B[0,0].

[0030] For example, the predicted value α′ for the AC component α can be expressed as shown in the following equation (5).

[0031] Predicted value α' = (D[1,0] - D[-1,0]) / 8 (5)

[0032] Note that the AC component prediction unit 14 may use other methods for predicting the predicted value of the AC component. If the target right block or target left block does not exist, D[0,0] of the target block B[0,0] or the DC component when the target block is composed of all black pixels may be used instead of the DC component of the target right block or the DC component of the target left block. Furthermore, the process of calculating the predicted value β' for the AC component β may be performed by rotating the coordinate system used in the process for the AC component α by 90 degrees.

[0033] Furthermore, for example, with respect to the AC component γ, the AC component prediction unit 14 calculates a predicted value of the AC component γ from the DC components of the four blocks (B[1,1], B[-1,1], B[1,-1], B[-1,-1]) located diagonally adjacent to the target block B[0,0].

[0034] For example, the predicted value γ′ for the AC component γ can be expressed as shown in the following equation (6).

[0035] Predicted value γ'=(δ --,B[0,1] -δ +-,B[0,1] +δ ++,B[-1,0] -δ +-,B[-1,0] -2×δ -+ +δ ++ +δ -- ) / 4 ···(6) where δ --,B[a,b] , δ +-,B[a,b] , δ ++,B[a,b] , δ +-,B[a,b] indicate the DC components of the child blocks at the corresponding positions in block B[a, b].

[0036] The AC component predictor 14 may use another method for predicting the predicted value of the AC component γ.

[0037] The feature information identifying unit 15 is an example of a first feature information identifying unit, and identifies feature information about the target block. In this embodiment, the feature information about the AC component α of the target block is identified based on the difference between the DC component of the target block and the DC components of some of the child blocks (partial blocks) included in multiple blocks adjacent to the target block B[0,0].

[0038] FIG. 4 is a diagram illustrating feature information of a target block according to an embodiment.

[0039] In this embodiment, the feature information identification unit 15 identifies feature information about the AC component α of the target block based on the difference between the DC component D[0,0] of the target block B[0,0] and the DC components (d[-1,0], d[1,0], d[0,-1]) of the partial blocks (for example, the two child blocks adjacent to B[0,0] in each of B[-1,0], B[1,0], B[0,-1]) of the three blocks (B[-1,0], B[1,0], B[0,-1]) adjacent to the target block B[0,0].

[0040] Specifically, the feature information (x, y, z) for the AC component α can be expressed by equation (7). (x,y,z)=(ε(d[-1,0]),ε(d[1,0]),ε(d[0,-1])) ···(7) Here, ε(x)=Q(xD[0,0]), where x in e(x) is a variable and Q is a function that performs quantization.

[0041] For example, in an image in which brightness is expressed in 256 levels of 8 bits, the function Q may be the function shown in the following equation (8). Q(x) = -3 (when x / 32≦-2.5) 3 (when x 32≧2.5) Round(x / 32) (otherwise) (8) Here, x is a variable, and Round is a function that rounds off decimal places.

[0042] Although the feature information is quantized using the function Q to reduce the amount of data, quantization may not be performed.

[0043] The DC component of a partial block is calculated by averaging the DC components of the child blocks included in the partial block. Specifically, d[-1,0] can be calculated based on the following equation (9). d[-1,0]=(δ ++,B[-1,0] +δ +-,B[-1,0] ) / 2 ···(9)

[0044] The DC component of the partial block may be estimated based on the DC components of the target block, the block containing the partial block (adjacent block), and the adjacent block on the opposite side of the target block. Specifically, d[1,0] can be calculated based on the following equation (10). d[1,0]=(D[0,0]-D[2,0]) / 8+D[1,0] ···(10)

[0045] This feature information (x, y, z) is used as an index for managing information corresponding to the target block in the prediction adjustment amount table 21. This feature information is a pattern indicating the relationship between the DC component of luminance between the target block and a partial block that is a part of multiple adjacent blocks, and if this feature information is the same, it means that the target blocks are identical or similar to each other. By managing and using the prediction adjustment amount corresponding to such an index, it is possible to reduce the prediction residual, which will be described later. Furthermore, because the feature information is a pattern based on the luminance between a block and partial blocks of surrounding blocks, it is possible to classify and manage blocks in more detail by the feature information.

[0046] For the feature information for the AC component β, the same processing can be performed with the coordinate system for the processing for the AC component α rotated by 90 degrees.

[0047] In addition, the feature information identification unit 15 identifies feature information about the AC component γ of the target block based on the difference between the DC component D[0,0] of the target block B[0,0] and the DC components (d[-1,0], d[1,0], d[0,-1], d[0,1]) of the partial blocks (the two child blocks adjacent to B[0,0] in each of B[-1,0], B[1,0], B[0,-1], B[0,1]) of the four blocks (B[-1,0], B[1,0], B[0,-1], B[0,1]) adjacent to the target block B[0,0] vertically and horizontally.

[0048] Specifically, the feature information (x, y, z, w) for the AC component γ can be expressed by equation (11). Note that the AC component γ in an image is a component of a two-dimensional hyperbolic paraboloid, so the feature information is four-dimensional. (x,y,z,w)=(ε(d[-1,0]),ε(d[1,0]),ε(d[0,-1]),ε(d[0,1])) ···(11)

[0049] The prediction adjustment amount table 21 manages information (prediction adjustment amount available information) that can specify the prediction adjustment amount for each piece of feature information of the target block.

[0050] FIG. 5 is a diagram illustrating the configuration of a predicted adjustment amount table according to one embodiment.

[0051] In this embodiment, the prediction adjustment amount table 21 includes a prediction adjustment amount table 22 for the AC components α and β of the target block, and a prediction adjustment amount table 23 for the AC component γ of the target block.

[0052] The prediction adjustment amount table 22 manages entries of prediction adjustment amount specifying information for each piece of feature information for the AC components α and β of the current block. Each entry in the prediction adjustment amount table 22 includes fields for feature information 22a, a pattern appearance count 22b, and a prediction residual total 22c. The feature information 22a stores feature information (x, y, z) for the AC components α and β. The pattern appearance count 22b stores the number of times (n) a pattern corresponding to the feature information of the entry appears. The prediction residual total 22c stores the sum (ψ) of prediction residuals corresponding to the pattern corresponding to the entry. In this embodiment, the prediction adjustment amount is determined by dividing the sum of prediction residuals by the number of appearances. One prediction adjustment amount table 22 is generated for each still image or each image corresponding to a keyframe in a video. The prediction adjustment amount table 22 may be shared by multiple adjacent images in a video, or may be shared between similar images.

[0053] The prediction adjustment amount table 23 manages entries of prediction adjustment amount specifying information for each feature information for the AC component γ of the current block. Each entry in the prediction adjustment amount table 23 includes fields for feature information 23a, a pattern appearance count 23b, and a prediction residual total 23c. The feature information 23a stores feature information (x, y, z, w) for the AC component γ. The pattern appearance count 23b stores the number of times (m) a pattern corresponding to the feature information of the entry appears. The prediction residual total 23c stores the sum (φ) of prediction residuals corresponding to the pattern corresponding to the entry. In this embodiment, the prediction adjustment amount is determined by dividing the sum of prediction residuals by the number of appearances. One prediction adjustment amount table 23 is generated for each still image or each image corresponding to a keyframe in a video. The prediction adjustment amount table 23 may be shared by multiple adjacent images in a video, or may be shared between similar images.

[0054] Returning to the description of FIG. 1 , the adjustment amount specifying unit 16 is an example of a first adjustment amount specifying unit, and specifies a prediction adjustment amount corresponding to the feature information specified by the feature information specifying unit 15. Specifically, the adjustment amount specifying unit 16 specifies an entry in the prediction adjustment amount table 22 corresponding to the feature information (x, y, z) for the AC components α and β, and specifies the prediction adjustment amount by dividing the total prediction residual ψ of this entry by the number of pattern occurrences n. This prediction adjustment amount is the average value of the prediction residuals for patterns with the same feature information. Since the prediction adjustment amount is specified based on the feature information in this way, it is possible to appropriately specify a prediction adjustment amount for target blocks with common feature amounts. In this embodiment, the same prediction adjustment amount table 22 is used when specifying prediction adjustment amounts corresponding to the feature information for the AC components α and β. The reason for using the same prediction adjustment amount table 22 in this way is that there may be partial images in an image that are obtained by rotating a certain partial image by 90 degrees, and these partial images have the same properties. This is expected to reduce the memory capacity required to store the prediction adjustment amount table 22, and also to enable the prediction residual information to be shared with other devices, thereby enabling the prediction adjustment amount to be set with higher accuracy.

[0055] Furthermore, the adjustment amount specifying unit 16 specifies an entry in the prediction adjustment amount table 23 corresponding to the feature information (x, y, z, w) for the AC component γ, and specifies the prediction adjustment amount by dividing the total prediction residuals φ of this entry by the number of pattern occurrences m. This prediction adjustment amount is the average value of the prediction residuals for patterns with the same feature information. Since the prediction adjustment amount is specified using feature information in this way, it is possible to appropriately specify the prediction adjustment amount for target blocks with common feature amounts.

[0056] Furthermore, in this embodiment, the same prediction adjustment amount table 21 (22, 23) is used even when specifying prediction adjustment amounts corresponding to feature information for target blocks of different hierarchical levels. The reason for using the same prediction adjustment amount table 21 in this way is that images are sometimes fractal (have self-similarity), and target blocks of different sizes may have the same properties. This is expected to reduce the amount of memory required to store the prediction adjustment amount table 21, and to enable the prediction residual information to be shared, thereby enabling the prediction adjustment amount to be set with higher accuracy.

[0057] The prediction residual calculation unit 17 calculates the prediction residual e for each AC component for the current block. In this embodiment, the prediction residuals e for the AC components α, β, and γ are calculated as α , e β , e γ is calculated using the following equations (12), (13), and (14). Note that the prediction residual e β The process of calculating the prediction residual e for the AC component α α The same processing can be performed with the coordinate system used in the processing for calculating the coordinates rotated by 90 degrees. Prediction residual e α =α―α'-ψ(x,y,z) / n(x,y,z) ···(12) Prediction residual e β =β−β'-ψ(x,y,z) / n(x,y,z) ···(13) Prediction residual eγ =γ−γ'-φ(x,y,z,w) / m(x,y,z,w) ···(14) Here, ψ(x,y,z) denotes the total prediction residual ψ for the target block whose feature information is (x,y,z), n(x,y,z) denotes the number of pattern appearances n for the target block whose feature information is (x,y,z), φ(x,y,z,w) denotes the total prediction residual φ for the target block whose feature information is (x,y,z,w), and m(x,y,z,w) denotes the number of pattern appearances m for the target block whose feature information is (x,y,z,w).

[0058] The encoding unit 18 encodes the prediction residual e calculated by the prediction residual calculation unit 17. α , e β , e γ The encoding unit 18 performs a lossy transformation such as quantization on the DC component D of each block, and then performs entropy coding such as Huffman coding or arithmetic coding to generate compressed data for the target block. The encoding unit 18 also generates compressed data for the DC component D of each most significant block using a known compression technique.

[0059] The adjustment amount update unit 19 updates the prediction residual e calculated by the prediction residual calculation unit 17. α , e β Based on this, the adjustment amount updating unit 19 updates the prediction adjustment amount specifying information in the prediction adjustment amount table 22. Specifically, the adjustment amount updating unit 19 updates the prediction residual total ψ and the pattern appearance count n in the prediction adjustment amount table 22 as shown in the following equations (15) and (16). ψ n+1 (x,y,z)=ψ n (x,y,z)+e (15) n n+1 (x,y,z)=n n (x,y,z)+1 (16) where: n+1 indicates the updated version, n indicates the time before the update, and e indicates the time before the update. α or e β Shows.

[0060] Furthermore, in this embodiment, the adjustment amount update unit 19 also updates the prediction adjustment amount specifying information of other patterns that have spatial symmetry relationships with the pattern of the target block in the prediction adjustment amount table 22, in accordance with the spatial symmetry relationships in the image. There may be bilaterally symmetrical partial images in an image, and particularly in images such as computer graphics (CG), there is a high possibility that there will be many bilaterally symmetrical parts that use the same components. In such cases, it becomes possible to set the prediction adjustment amount corresponding to each part with higher accuracy.

[0061] For a left-right inverted pattern (y, x, z) that has a left-right inverted relationship with the pattern of feature information (x, y, z), the total prediction residual ψ and the number of pattern occurrences n in the prediction adjustment amount table 22 are updated as shown in equations (17) and (18).

[0062] ψ n+1 (y,x,z)=ψ n (y,x,z)-e (17) n n+1 (y,x,z)=n n (y,x,z)+1 (18)

[0063] Furthermore, for a positive / negative inversion pattern (-x, -y, -z) having a positive / negative inversion relationship with the pattern of feature information (x, y, z), the prediction residual sum ψ and the pattern occurrence number n in the prediction adjustment amount table 22 are updated as shown in equations (19) and (20).

[0064] ψ n+1 (-x,-y,-z)=-ψ n (-x,-y,-z)-e (19) n n+1 (-x,-y,-z)=n n (-x,-y,-z)+1 (20)

[0065] Furthermore, for a left-right positive-negative inverted pattern (-y, -x, -z) that has a relationship in which the left and right and positive and negative are inverted with respect to the pattern of feature information (x, y, z), the total prediction residual ψ and the number of pattern occurrences n in the prediction adjustment amount table 22 are updated as shown in equations (21) and (22).

[0066] ψ n+1 (-y,-x,-z)=-ψ n (-y,-x,-z)+e (21) n n+1 (-y,-x,-z)=n n (-y,-x,-z)+1 (22)

[0067] It should be noted that, for the plurality of pieces of feature information having the spatial symmetry relationship in the image described above, one piece of feature information may be registered as a representative in the prediction adjustment amount table 22, and the prediction residual sum ψ for the remaining pieces of feature information may be determined based on the sign relationship (same or opposite) with the prediction residual sum ψ of the representative piece of feature information. In this way, the memory capacity required for the prediction adjustment amount table 22 can be reduced.

[0068] The adjustment amount update unit 19 also updates the prediction residual e calculated by the prediction residual calculation unit 17. γ Based on this, the adjustment amount updating unit 19 updates the prediction adjustment amount specifying information in the prediction adjustment amount table 23. Specifically, the adjustment amount updating unit 19 updates the prediction residual total φ and the pattern appearance count m in the prediction adjustment amount table 23 as shown in the following equations (23) and (24). φ n+1 (x,y,z,w)=φ n (x,y,z,w)+e (23) m n+1 (x,y,z,w)=m n (x,y,z,w)+1 (24) where: n+1 indicates the updated version, n indicates the time before the update, and e indicates the time before the update. γ Shows.

[0069] Furthermore, in this embodiment, the adjustment amount update unit 19 also updates the prediction adjustment amount specifying information of other patterns that have a spatial relationship with the pattern of the target block in the prediction adjustment amount table 23, in accordance with the spatial relationship in the image. In an image, particularly in an image such as Computer Graphics (CG), there is a high possibility that there will be many parts that have been converted using the same components, and in such cases, it becomes possible to set the prediction adjustment amount corresponding to each part with greater precision.

[0070] For a left-right inverted pattern (y, x, z, w) that has a left-right inverted relationship with the pattern of feature information (x, y, z, w), the total prediction residual φ and the number of pattern occurrences m in the prediction adjustment amount table 23 are updated as shown in equations (25) and (26).

[0071] φ n+1 (y,x,z,w)=φ n (y,x,z,w)-e (25) m n+1 (y,x,z,w)=m n (y,x,z,w)+1 (26)

[0072] Similarly, for a positive / negative inverted pattern (-x,-y,-z,-w) having a positive / negative inverted relationship with the pattern of feature information (x,y,z,w), as shown in equations (27) and (28), for a positive / negative inverted and left / right inverted pattern (-y,-x,-z,-w) having a positive / negative inverted and left / right inverted relationship with the pattern of feature information (x,y,z,w), as shown in equations (29) and (30), for a 90-degree rotated pattern (z,w,x,y) having a 90-degree rotated relationship with the pattern of feature information (x,y,z,w), as shown in equations (31) and (32), for a left / right inverted and up / down inverted pattern (y,x,w,z) having a left / right inverted and up / down inverted relationship with the pattern of feature information (x,y,z,w), as shown in equations (32), As shown in equations (33) and (34), for a 90-degree rotated, horizontally flipped, and vertically flipped pattern (w,z,y,x) which has a 90-degree rotated, horizontally flipped, and vertically flipped relationship with the pattern of feature information (x,y,z,w), the total prediction residual φ and the number of pattern appearances m in the prediction adjustment amount table 23 are updated as shown in equations (35) and (36). Also, for a 90-degree rotated and vertically flipped pattern (w,z,x,y) which has a 90-degree rotated and vertically flipped relationship with the pattern of feature information (x,y,z,w), the total prediction residual φ and the number of pattern appearances m are updated as shown in equations (37) and (38). Also, for a vertically flipped pattern (x,y,w,z) which has a vertically flipped relationship with the pattern of feature information (x,y,z,w), the total prediction residual φ and the number of pattern appearances m are updated as shown in equations (39) and (40).

[0073] Furthermore, as shown in equations (41) and (42), the 90-degree rotated and horizontally inverted pattern (z,w,y,x) has a relationship of being rotated by 90 degrees and horizontally inverted with respect to the pattern of the feature information (x,y,z,w). As shown in equations (43) and (44), the positive / negative inverted and 90-degree rotated pattern (-z,-w,-x,-y) has a relationship of being rotated by 90 degrees and horizontally inverted with respect to the pattern of the feature information (x,y,z,w). As shown in equations (45) and (46), the positive / negative inverted, 90-degree rotated, horizontally inverted, vertically inverted pattern (-w, As shown in equations (47) and (48), for the positive / negative inversion, 90-degree rotation, and top / bottom inversion patterns (-w,-z,-x,-y) which have positive / negative inversion, 90-degree rotation, and top / bottom inversion relationships with the pattern of the feature information (x, y, z, w), the prediction residual sum φ and the pattern appearance number m in the prediction adjustment amount table 23 are updated. As shown in equations (49) and (50), for the positive / negative inversion and top / bottom inversion patterns (-x,-y,-w,-z) which have positive / negative inversion and top / bottom inversion relationships with the pattern of the feature information (x, y, z, w), the prediction residual sum φ and the pattern appearance number m in the prediction adjustment amount table 23 are updated.

[0074] φ n+1 (-x,-y,-z,-w)=φ n (-x,-y,-z,-w)-e ···(27) m n+1 (-x,-y,-z,-w)=m n (-x,-y,-z,-w)+1 ···(28)

[0075] φ n+1 (-y,-x,-z,-w)=φ n(-y,-x,-z,-w)+e ···(29) m n+1 (-y,-x,-z,-w)=m n (-y,-x,-z,-w)+1 ···(30)

[0076] ϕ n+1 (z,w,x,y)=ϕ n (z,w,x,y)+e ···(31) m n+1 (z,w,x,y)=m n (z,w,x,y)+1 ···(32)

[0077] φ n+1 (y,x,w,z)=ϕ n (y,x,w,z)+e ···(33) m n+1 (y,x,w,z)=m n (y,x,w,z)+1 ···(34)

[0078] ϕ n+1 (w,z,y,x)=φ n (w,z,y,x)+e ···(35) m n+1 (w,z,y,x)=m n (w,z,y,x)+1 ···(36)

[0079] φ n+1 (w,z,x,y)=φ n (w,z,x,y)-e ···(37) m n+1 (w,z,x,y)=m n (w,z,x,y)+1 ···(38)

[0080] ϕ n+1 (x,y,w,z)=φ n (x,y,w,z)-and ···(39) m n+1 (x,y,w,z)=m n (x,y,w,z)+1 ···(40)

[0081] ϕn+1 (z,w,y,x)=φ n (z,w,y,x)-e ···(41) m n+1 (z,w,y,x)=m n (z,w,y,x)+1 ···(42)

[0082] ϕ n+1 (−z,−w,−x,−y)=φ n (-z,-w,-x,-y)-e ···(43) m n+1 (−z,−w,−x,−y)=m n (-z,-w,-x,-y)+1 ···(44)

[0083] ϕ n+1 (-y,-x,-w,-z)=φ n (-y,-x,-w,-z)-e ···(45) m n+1 (-y,-x,-w,-z)=m n (-y,-x,-w,-z)+1 ···(46)

[0084] ϕ n+1 (-w,-z,-y,-x)=ϕ n (-w,-z,-y,-x)-e ···(47) m n+1 (-w,-z,-y,-x)=m n (-w,-z,-y,-x)+1 ···(48)

[0085] ϕ n+1 (-w,-z,-x,-y)=ϕ n (-w,-z,-x,-y)+e ···(49) m n+1 (-w,-z,-x,-y)=m n (-w,-z,-x,-y)+1 ···(50)

[0086] ϕ n+1(-x,-y,-w,-z)=φ n (-x,-y,-w,-z)+e ···(51) m n+1 (-x,-y,-w,-z)=m n (-x,-y,-w,-z)+1 ···(52)

[0087] φ n+1 (-z,-w,-y,-x)=φ n (-z,-w,-y,-x)+e ···(53) m n+1 (-z,-w,-y,-x)=m n (-z,-w,-y,-x)+1 ···(54)

[0088] It should be noted that, for the plurality of pieces of feature information having spatial correlation in the image described above, one piece of feature information may be registered as a representative in the prediction adjustment amount table 23, and the prediction residual sum φ for the remaining pieces of feature information may be determined based on the sign relationship (same or inverted) with the prediction residual sum φ of the representative piece of feature information. In this way, the memory capacity required for the prediction adjustment amount table 23 can be reduced.

[0089] The compressed data output unit 20 outputs the compressed data generated by the encoding unit 18 to the decoder 30. The compressed data includes the DC component D of each block at the highest level and the prediction residuals e of the AC components α, β, and γ of each block at each layer. α , e β , e γ It contains compressed data for

[0090] The decoder 30 includes a compressed data input unit 31, a decoding unit 32, an AC component prediction unit 33, a feature information identification unit 34, an adjustment amount identification unit 35, an AC component calculation unit 36, an inverse Hadamard transform unit 37, an adjustment amount update unit 38, an image display unit 39, and a prediction adjustment amount table 40.

[0091] 5, the prediction adjustment amount table 40 has a similar configuration to the prediction adjustment amount table 21. That is, the prediction adjustment amount table 40 includes a prediction adjustment amount table 41 for the AC components α and β of the target block, and a prediction adjustment amount table 42 for the AC component γ of the target block.

[0092] The compressed data input unit 31 inputs compressed data transmitted from the encoder 10 via the network 50. The decoding unit 32 decodes the compressed data input by the compressed data input unit 31 by performing decoding corresponding to the encoding by the encoding unit 18. As a result, the DC component D of each top-level block and the prediction residuals e of the AC components α, β, and γ of each block in each layer are obtained. α , e β , e γ The AC component prediction unit 33 executes the same processing as the AC component prediction unit 14.

[0093] The feature information identification unit 34 is an example of a second feature information identification unit, and identifies feature information about the target block. In this embodiment, the feature information about the AC component α of the target block is identified based on the difference between the DC component of the target block and the DC components of partial blocks included in multiple blocks adjacent to the target block B[0,0].

[0094] FIG. 6 is a diagram illustrating calculation of feature information of a target block according to an embodiment.

[0095] The feature information identification unit 34 identifies feature information about the AC component α of the target block based on the difference between the DC component D[0,0] of the target block B[0,0] and the DC components (d[-1,0], d[1,0], d[0,-1]) of the partial blocks (two child blocks adjacent to B[0,0] in each of B[-1,0], B[1,0], and B[0,-1]) of three blocks (B[-1,0], B[1,0], and B[0,-1]) adjacent to the target block B[0,0]. The feature information identification unit 34 calculates feature information (x, y, z) about the AC component α using equation (7).

[0096] Here, when processing is performed on the target block B[0,0], due to the order of processing for each block, DC components are obtained for each child block belonging to block B[0,1] and block B[-1,0], but DC components are not obtained for the child blocks of block B[0,-1] and block B[1,0].

[0097] The feature information identification unit 34 calculates the DC component (d[-1,0], d[0,1]) of a partial block for which a DC component has been obtained based on the DC component of the child block. For example, d[-1,0] is calculated according to equation (9). Furthermore, the feature information identification unit 34 calculates the DC component (d[0,-1], d[1,0]) of a partial block for which a DC component has not been obtained using the DC components of the surrounding blocks, as shown in equation (10), for example.

[0098] The feature information specifying unit 34 calculates feature information for the AC component β of the target block. For the feature information for the AC component β, the same processing is performed as for the AC component α, but with the coordinate system rotated by 90 degrees.

[0099] The feature information identification unit 34 identifies feature information about the AC component γ of the target block based on the difference between the DC component D[0,0] of the target block B[0,0] and the DC components (d[-1,0], d[1,0], d[0,-1], d[0,1]) of the partial blocks (the two child blocks adjacent to B[0,0] in each of B[-1,0], B[1,0], B[0,-1], B[0,1]) of the four blocks (B[-1,0], B[1,0], B[0,-1], B[0,1]) adjacent to the target block B[0,0] vertically and horizontally.

[0100] Specifically, the feature information (x, y, z, w) for the AC component γ is determined by the formula shown in Equation (11). Note that d[-1, 0], d[1, 0], d[0, -1], and d[0, 1] are calculated in the same manner as described when calculating the feature information for the AC component α.

[0101] The adjustment amount specifying section 35 is an example of a second adjustment amount specifying section, and specifies a predicted adjustment amount corresponding to the feature information specified by the feature information specifying section 34 by processing similar to that of the adjustment amount specifying section 16 .

[0102] The AC component calculation unit 36 ​​calculates the horizontal AC component α, the vertical AC component β, and the diagonal AC component γ for the target block. Specifically, the AC component calculation unit 36 ​​calculates the AC components α, β, and γ according to the following equations (55), (56), and (57). AC component α=e α +α'+ψ(x,y,z) / n(x,y,z) ···(55) AC component β=e β +β'+ψ(x,y,z) / n(x,y,z) ···(56) AC component γ=e γ +γ'+φ(x,y,z,w) / m(x,y,z,w) ···(57)

[0103] The inverse Hadamard transform unit 37 is an example of a brightness value calculation unit, and performs an inverse Hadamard transform to calculate the DC component of each child block included in the target block based on the AC components α, β, γ, and DC component D of the target block calculated by the AC component calculation unit 36. Furthermore, if the target block is the lowest-order target block, the inverse Hadamard transform unit 37 calculates the DC component of each pixel included in the target block based on the AC components α, β, γ, and DC component D of the target block. In this way, the DC component of each pixel in the target image is calculated by the processing of the inverse Hadamard transform unit 37.

[0104] The adjustment amount update unit 38 updates the prediction residual e of the AC component obtained by the decoding unit 32. α, e β , e γ The prediction adjustment amount specifying information in the prediction adjustment amount table 40 is updated based on the prediction residual e α , e β , e γ The process of updating the predicted adjustment amount specifying information in the predicted adjustment amount table 40 based on the above is the same as the update process performed by the adjustment amount update unit 19.

[0105] The image display unit 39 displays an image on the display device 108 (see FIG. 9) based on the DC component of the luminance of each pixel in the target image calculated by the inverse Hadamard transform unit 37.

[0106] Next, the processing operation in the image processing system 1 will be described.

[0107] First, the encoding process performed by the encoder 10 will be described.

[0108] FIG. 7 is a flowchart of an encoding process according to an embodiment.

[0109] In the encoding process, first, the target image input unit 11 inputs an image to be encoded (target image), and the DC conversion unit 12 calculates the DC component of the luminance of the target block and calculates the DC component of the luminance of the child block of the target block (step S11). Note that when step S11 is executed for the first time, the process is executed with the topmost block as the target block.

[0110] Next, the Hadamard transform unit 13 performs a Hadamard transform to calculate the horizontal AC component α, vertical AC component β, diagonal AC component γ, and DC component δ of the target block (step S12).

[0111] Next, the AC component prediction unit 14 calculates a predicted value of the AC component for the target block according to known AC component prediction (step S13). In this embodiment, the processing from step S13 onwards is first performed on the AC component α of the target block, then on the AC component β, and then on the AC component γ.

[0112] The feature information identifying unit 15 identifies feature information about the target block (step S14).

[0113] Next, the adjustment amount specifying unit 16 specifies a prediction adjustment amount corresponding to the feature information specified by the feature information specifying unit 15 (step S15). Specifically, when the AC components α and β are the target, the adjustment amount specifying unit 16 specifies an entry in the prediction adjustment amount table 22 corresponding to the feature information (x, y, z) and specifies the prediction adjustment amount by dividing the prediction residual sum ψ of this entry by the pattern appearance number n. When the AC component γ is the target, the adjustment amount specifying unit 16 specifies an entry in the prediction adjustment amount table 23 corresponding to the feature information (x, y, z, w) and specifies the prediction adjustment amount by dividing the prediction residual sum φ of this entry by the pattern appearance number m.

[0114] Next, the prediction residual calculation unit 17 calculates the prediction residual e(e α , e β , or e γ ) (step S16). Specifically, the prediction residual calculation unit 17 calculates the prediction residual e according to the above-mentioned equations (12), (13), and (14).

[0115] Next, the encoding unit 18 performs a lossy transformation, such as quantization, on the prediction residual e calculated by the prediction residual calculation unit 17, and further performs entropy coding, such as Huffman coding or arithmetic coding, to generate compressed data for the prediction residual e of the target block (step S17).

[0116] Next, the adjustment amount update unit 19 updates the prediction adjustment amount specifying information in the prediction adjustment amount table 21 based on the prediction residual e calculated by the prediction residual calculation unit 17 (step S18).

[0117] The DC conversion unit 12 determines whether all blocks in the hierarchical layer being processed have been processed (step S19), and if not all blocks in the hierarchical layer have been processed (step S19: N), the process proceeds to step S11 to process the next target block. On the other hand, if all blocks in the hierarchical layer have been processed (step S19: Y), the DC conversion unit 12 determines whether there is a block in the next lower hierarchical layer (step S20).

[0118] As a result, if there is a block in the next lower layer (step S20: Y), the DC conversion unit 12 changes the layer to be processed to the lower layer (step S21), proceeds to step S11, and executes processing on the block in the lower layer.

[0119] On the other hand, if there is no block in the next lower layer (step S20: N), the DC conversion unit 12 ends the process. As a result, compressed data for the AC component α has been generated.

[0120] Next, the encoder 10 performs the above encoding process on the AC component β to generate compressed data for the AC component β.

[0121] Next, the encoder 10 performs the above encoding process on the AC component γ to generate compressed data for the AC component γ.

[0122] Next, the encoder 10 executes a process of generating compressed data for the DC component D for each block in the highest hierarchy (the DC component δ of the block in the highest hierarchy). The process of generating compressed data for the DC component D can use a known technique.

[0123] After this, the compressed data output unit 20 compiles the compressed data for the AC component α, the AC component β, the AC component γ, and the DC component D for each block in the highest hierarchy, includes it in the compressed data of the target image, and outputs (transmits) it to the decoder 30.

[0124] In this encoding process, a prediction adjustment amount based on the prediction residual is used for blocks with the same feature information for the AC components α, β, and γ, thereby improving prediction accuracy corresponding to luminance values. This reduces the value of the prediction residual calculated using the prediction adjustment amount, thereby improving compression efficiency for the luminance prediction residual.

[0125] Next, the decoding process by the decoder 30 will be described.

[0126] FIG. 8 is a flowchart of a decoding process according to an embodiment.

[0127] The compressed data input unit 31 inputs the compressed data transmitted from the encoder 10 via the network 50 (step S31).

[0128] Next, the decoding unit 32 decodes the compressed data input by the compressed data input unit 31 (step S32). As a result, the DC component D of each block at the top level and the prediction residuals e of the AC components α, β, and γ of each block at each layer are obtained. α , e β , e γ You can get the following.

[0129] Next, the AC component prediction unit 33 calculates a predicted value of the AC component α for the target block according to known AC component prediction (step S33). Here, the target block is selected in the same order as the order in which the processes are executed in the encoder 10.

[0130] Next, the feature information identifying unit 34 identifies feature information about the target block (step S34).

[0131] Next, the adjustment amount specifying unit 35 specifies a prediction adjustment amount corresponding to the feature information specified by the feature information specifying unit 34 (step S35). Specifically, when the adjustment amount specifying unit 35 is to process the AC components α and β, it specifies the entry in the prediction adjustment amount table 41 corresponding to the feature information (x, y, z) and specifies the prediction adjustment amount by dividing the prediction residual sum ψ of this entry by the pattern appearance number n.

[0132] Next, the adjustment amount updating unit 38 updates the prediction adjustment amount identifying information in the prediction adjustment amount table 40 based on the AC component prediction residual e obtained by the decoding unit 32 (step S36). Here, the target block is selected in the same order as the order in which the processes are executed in the encoder 10, so when the target block is processed, the prediction adjustment amount table 40 can be set to the same state as the prediction adjustment amount table 21 when the same target block is processed in the encoder 10.

[0133] Next, the AC component calculation unit 36 ​​calculates the AC component α in the horizontal direction for the target block. Specifically, the AC component calculation unit 36 ​​calculates the AC component α according to equation (55) (step S37).

[0134] Next, the decoder 30 calculates AC components β and γ (step S38). Specifically, the AC component β can be calculated by the same processes as in steps S33 to S37. In step S37, the AC component calculation unit 36 ​​calculates the AC component β according to equation (56). In addition, the AC component γ can be calculated by the same processes as in steps S33 to S37. In step S35, when the adjustment amount specification unit 35 is to process the AC component γ, it specifies an entry in the prediction adjustment amount table 42 corresponding to the feature information (x, y, z, w) and specifies the prediction adjustment amount by dividing the prediction residual sum φ of this entry by the pattern appearance count m. In step S37, the AC component calculation unit 36 ​​calculates the AC component γ according to equation (57).

[0135] Next, the AC component prediction unit 33 determines whether all blocks in the currently executing hierarchical layer have been processed (step S39). If not all blocks in the hierarchical layer have been processed (step S39: N), the process proceeds to step S33, where the next target block is processed. On the other hand, if all blocks in the hierarchical layer have been processed (step S39: Y), the inverse Hadamard transform unit 37 performs an inverse Hadamard transform to calculate the DC component of each child block included in the target block based on the AC component α, AC component β, AC component γ, and DC component D of the target block calculated by the AC component calculation unit 36 ​​(step S40). Note that if the target block is the lowest target block, i.e., a 2-pixel by 2-pixel block, the inverse Hadamard transform unit 37 calculates the DC component of each pixel included in the target block based on the AC component α, AC component β, AC component γ, and DC component D of the target block.

[0136] Next, AC component prediction unit 33 determines whether or not there is a block in the next lower layer (step S41). As a result, if there is a block in the next lower layer (step S41: Y), AC component prediction unit 33 changes the layer to be processed to the lower layer (step S42), and proceeds to step S33 to execute processing on the block in the lower layer.

[0137] On the other hand, if there is no block in the next lower layer (step S41: N), the image display unit 39 displays an image on the display device 108 based on the DC component of the luminance of each pixel in the target image calculated by the inverse Hadamard transform unit 37 (step S43), and ends the processing.

[0138] According to the above-described decoding process, an image can be displayed appropriately based on the compressed data created by the encoder 10.

[0139] The above-mentioned encoder 10 and decoder 30 can each be configured by a computer device.

[0140] 9 is a configuration diagram of a computer device according to one embodiment. In this embodiment, the encoder 10 and the decoder 30 are configured as separate computer devices, but these computer devices may have the same configuration. Therefore, in the following description, for convenience, the computer device shown in FIG. 9 will be used to describe the computer devices that configure the encoder 10 and the decoder 30.

[0141] The computer device 100 includes, for example, a central processing unit (CPU) 101, a main memory 102, a graphics processing unit (GPU) 103, a reader / writer 104, a communication interface (communication I / F) 105, an auxiliary storage device 106, an input / output interface (input / output I / F) 107, a display device 108, and an input device 109. The CPU 101, the main memory 102, the GPU 103, the reader / writer 104, the communication I / F 105, the auxiliary storage device 106, the input / output I / F 107, and the display device 108 are connected via a bus 110. The encoder 10 and the decoder 30 are each configured by appropriately selecting some or all of the components described in the computer device 100.

[0142] Here, at least one of the main memory 102 or the auxiliary storage device 106 functions as an adjustment amount memory unit (first adjustment amount memory unit) that stores the prediction adjustment amount table 21 of the encoder 10, or an adjustment amount memory unit (second adjustment amount memory unit) that stores the prediction adjustment amount table 40 of the decoder 30.

[0143] In the computer device 100 constituting the encoder 10, a CPU 101 executes an image processing program stored in an auxiliary storage device 106 to constitute, for example, a target image input unit 11, a DC conversion unit 12, a Hadamard transform unit 13, an AC component prediction unit 14, a feature information identification unit 15, an adjustment amount identification unit 16, a prediction residual calculation unit 17, an encoding unit 18, an adjustment amount update unit 19, and a compressed data output unit 20.

[0144] In the computer device 100 constituting the decoder 30, the CPU 101 executes an image processing program stored in the auxiliary storage device 106 to constitute, for example, a compressed data input unit 31, a decoding unit 32, an AC component prediction unit 33, a feature information identification unit 34, an adjustment amount identification unit 35, an AC component calculation unit 36, an inverse Hadamard transform unit 37, an adjustment amount update unit 38, and an image display unit 39.

[0145] The main memory 102 is, for example, a RAM, a ROM, etc., and stores programs (processing programs, etc.) executed by the CPU 101 and various information. The auxiliary storage device 106 is, for example, a non-transitory storage device (non-volatile storage device) such as a hard disk drive (HDD) or a solid state drive (SSD), and stores programs executed by the CPU 101 and various information. In the computer device 100 constituting the encoder 10, the main memory 102 stores, for example, a prediction adjustment amount table 21. In the computer device 100 constituting the decoder 30, the main memory 102 stores, for example, a prediction adjustment amount table 40.

[0146] The GPU 103 is a processor suitable for executing specific processes such as image processing, and is suitable for executing processes that are performed in parallel. In this embodiment, the GPU 103 executes predetermined processes in accordance with instructions from the CPU 101.

[0147] The reader / writer 104 is detachably attached to the recording medium 111 and reads and writes data to the recording medium 111. Examples of the recording medium 111 include non-transitory recording media (non-volatile recording media) such as an SD memory card, a floppy disk (FD: registered trademark), a CD, a DVD, a Blu-ray Disc (BD: registered trademark), and a flash memory. In this embodiment, a processing program may be stored in the recording medium 111 and read out and used by the reader / writer 104. In addition, in the computer device 100 constituting the encoder 10, image data to be processed may be stored in the recording medium 111 and read out and used by the reader / writer 104. In addition, in the computer device 100 constituting the encoder 10, the reader / writer 104 may store compressed data in the recording medium 111. In addition, in the computer device 100 constituting the decoder 30, the reader / writer 104 may read and use compressed data from the recording medium 111.

[0148] The communication I / F 105 is connected to the network 50 and transmits and receives data to and from other devices connected to the network 50 .

[0149] The input / output I / F 107 is connected to an input device 109 such as a mouse or a keyboard. In the computer device 100 constituting the encoder 10, the input / output I / F 107 accepts operation input by a user of the encoder 10 using the input device 109. In addition, in the computer device 100 constituting the decoder 30, the input / output I / F 107 accepts operation input by a user of the decoder 30 using the input device 109.

[0150] The display device 108 is, for example, a display device such as a liquid crystal display, and displays and outputs various information. The display device 108 is used, for example, in the decoder 30, for displaying images by the image display unit 39.

[0151] The present invention is not limited to the above-described embodiment and modifications, and can be implemented by making appropriate modifications within the scope of the invention.

[0152] For example, in the above-described embodiment, the adjustment amount update units 19, 38 are configured to always update the prediction adjustment amount tables 21, 40 when a prediction residual is calculated. However, the present invention is not limited to this. For example, the adjustment amount update units 19, 38 may not update the prediction adjustment amount tables 21, 40 when the number of pattern occurrences exceeds a predetermined number. This allows for relatively accurate prediction adjustment values ​​to be obtained and reduces the subsequent processing load. Alternatively, the prediction adjustment amount tables 21, 40 may be created in advance based on adjacent images or similar images of the target image, and the previously created prediction adjustment amount tables 21, 40 may be used as is without updating when processing the target image. In this case, the data of the prediction adjustment amount table 21 used by the encoder 10 may be included in the compressed image data and transmitted to the decoder 30. The decoder 30 may then create a prediction adjustment amount table 40 based on the data of the prediction adjustment amount table 21 in the data and use it for processing.

[0153] Furthermore, in the above embodiment, the feature information of the AC component γ was a pattern indicating the relationship between the DC component of luminance of the target block and some of the partial blocks of each of the four blocks adjacent to the target block in the vertical and horizontal directions. However, the present invention is not limited to this, and may be, for example, a pattern indicating the relationship between the DC component of luminance of the target block and each of the four blocks adjacent to the target block in the vertical and horizontal directions.

[0154] In addition, in the above embodiment, in the encoding and decoding processes, the same processing is performed on blocks of all hierarchical levels to encode or decode them, but the present invention is not limited to this, and it is also possible to encode or decode any of the hierarchical levels using different processing.

[0155] In the above embodiment, an example was shown in which the computer device 100 includes either the encoder 10 or the decoder 30. However, the present invention is not limited to this, and the computer device 100 may include both the encoder 10 and the decoder 30. [Explanation of symbols]

[0156] 1...image processing system, 10...encoder, 11...target image input unit, 12...DC conversion unit, 13...Hadamard transform unit, 14...AC component prediction unit, 15...feature information identification unit, 16...adjustment amount identification unit, 17...prediction residual calculation unit, 18...encoding unit, 19...adjustment amount update unit, 20...compressed data output unit, 21, 22, 23...prediction adjustment amount table, 30...decoder, 31...compressed data input unit, 32...decoding unit, 33...AC component prediction unit, 34...feature information identification unit, 35...adjustment amount identification unit, 36...AC component calculation unit, 37...inverse Hadamard transform unit, 38...adjustment amount update unit, 39...image display unit, 40, 41, 42...prediction adjustment amount table, 50...network, 100...computer device, 101...CPU, 102...main memory, 108...display device, 111...recording medium

Claims

1. an adjustment amount storage unit that stores, in association with each other, feature information for a block in a predetermined image and prediction adjustment amount specifying information that can specify a prediction adjustment amount for adjusting a prediction residual of an AC component of luminance for the block corresponding to the feature information; a feature information identifying unit that identifies feature information about a target block in a target image; an adjustment amount specifying unit that specifies a predicted adjustment amount corresponding to the feature information specified by the feature information specifying unit from the predicted adjustment amount specifying information in the adjustment amount storage unit; Equipped with The feature information is a pattern indicating a relationship between the block and a partial block that is a part of a plurality of adjacent blocks, in terms of a DC component of luminance. Image processing device.

2. an adjustment amount storage unit that stores, in association with each other, feature information for a block in a predetermined image and prediction adjustment amount specifying information that can specify a prediction adjustment amount for adjusting a prediction residual of an AC component of luminance in a diagonal direction for the block corresponding to the feature information; a feature information identifying unit that identifies feature information about a target block in a target image; an adjustment amount specifying unit that specifies a predicted adjustment amount corresponding to the feature information specified by the feature information specifying unit from the predicted adjustment amount specifying information in the adjustment amount storage unit; Equipped with The feature information is a pattern having four values ​​that specify components of a two-dimensional hyperbolic paraboloid that indicates the relationship between the block and four adjacent blocks in the vertical and horizontal directions with respect to the DC components of luminance. Image processing device.

3. The feature information is a pattern indicating a relationship between the block and partial blocks that are parts of each of the four blocks, in terms of the DC component of luminance. The image processing device according to claim 2 .

4. When the characteristic information identification unit is able to grasp the value of the DC component of the luminance of the partial block, it calculates the pattern elements using the value of the DC component. When the characteristic information identification unit is unable to grasp the value of the DC component of the luminance of the partial block, it estimates the value of the DC component of the partial block based on a block including the partial block and its surrounding blocks, and calculates the pattern elements using the estimated value.

4. The image processing device according to claim 1.

5. a prediction residual calculation unit that calculates a prediction residual for the target block by subtracting a prediction value for the target block based on DC components of blocks surrounding the target block and the prediction adjustment amount specified by the adjustment amount specifying unit from an AC component for the target block. Further equipped 3. The image processing device according to claim 1.

6. an adjustment amount updating unit that updates the prediction adjustment amount specifying information associated with feature information corresponding to the target block based on the prediction residual; Further equipped The image processing device according to claim 5 .

7. the prediction adjustment amount specifying information includes the number of occurrences of blocks corresponding to the feature information and a sum of prediction residuals calculated for the blocks corresponding to the feature information; the adjustment amount update unit adds 1 to the number of occurrences of the blocks associated with feature information corresponding to the target block, and adds the calculated prediction residual to a sum of the prediction residuals of the blocks associated with feature information corresponding to the target block; The adjustment amount specifying unit calculates the prediction adjustment amount by dividing the sum of the prediction residuals by the number of occurrences of the block. The image processing device according to claim 6 .

8. an AC component calculation unit that calculates an AC component of the target block based on the prediction adjustment amount specified by the adjustment amount specification unit; a brightness value calculation unit that calculates a brightness value of a child block included in the target block based on the calculated AC component; Further equipped 3. The image processing device according to claim 1.

9. An image processing method by an image processing device, storing, in an adjustment amount storage unit, feature information for a block in a predetermined image and a prediction adjustment amount for adjusting a prediction residual of an AC component of luminance for the block corresponding to the feature information in association with each other; Identifying feature information for a target block in a target image; identifying a predicted adjustment amount corresponding to the identified feature information from the adjustment amount storage unit; The feature information is a pattern indicating a relationship between the block and a partial block that is a part of a plurality of adjacent blocks, in terms of a DC component of luminance. Image processing methods.

10. An image processing program executed by a computer constituting an image processing device, The computer, storing, in an adjustment amount storage unit, feature information for a block in a predetermined image and a prediction adjustment amount for adjusting a prediction residual of an AC component of luminance for the block corresponding to the feature information in association with each other; Identifying feature information for a target block in a target image; specifying a predicted adjustment amount corresponding to the specified feature information from the adjustment amount storage unit; The feature information is a pattern indicating a relationship between the block and a partial block that is a part of a plurality of adjacent blocks, in terms of a DC component of luminance. Image processing program.

11. An image processing system comprising: an encoder that generates compressed data of a target image; and a decoder that decompresses the compressed data of the target image, The encoder comprises: a first adjustment amount storage unit that stores, in association with each other, feature information for a block in a predetermined image and prediction adjustment amount specifying information that can specify a prediction adjustment amount for adjusting a prediction residual of an AC component of luminance for the block corresponding to the feature information; a first feature information identifying unit that identifies feature information about a target block in a target image; a first adjustment amount specifying unit that specifies a predicted adjustment amount corresponding to the feature information specified by the first feature information specifying unit from the predicted adjustment amount specifying information in the first adjustment amount storage unit; a prediction residual calculation unit that calculates a prediction residual for the target block by subtracting a prediction value for the target block based on DC components of blocks surrounding the target block and the prediction adjustment amount specified by the first adjustment amount specifying unit from an AC component for the target block; a compressed data output unit that outputs data including a DC component of luminance of a top target block in the target image and a prediction residual for an AC component of each target block as compressed data; The decoder a compressed data input unit for inputting the compressed data; a second adjustment amount storage unit that stores, in association with each other, feature information for a block in a predetermined image and prediction adjustment amount specifying information that can specify a prediction adjustment amount for adjusting a prediction residual of an AC component of luminance for the block corresponding to the feature information; a second feature information identifying unit that identifies feature information for a target block in a target image based on the compressed data; a second adjustment amount specifying unit that specifies a predicted adjustment amount corresponding to the feature information specified by the second feature information specifying unit from the predicted adjustment amount specifying information in the second adjustment amount storage unit; an AC component calculation unit that calculates an AC component of the target block by adding a prediction residual for an AC component of the target block, a predicted value for the AC component of the target block, and the prediction adjustment amount specified by the second adjustment amount specifying unit; a brightness value calculation unit that calculates a brightness value of a child block included in the target block based on the calculated AC component; Equipped with The feature information is a pattern indicating a relationship between the block and a partial block that is a part of a plurality of adjacent blocks, in terms of a direct current component of luminance. Image processing system.

Images