Coding performed on an uncoded block in an image frame using information from coded-blocks of the image frame
The recursive positioning of coded-blocks relative to uncoded blocks in image frames addresses the inefficiency in modern coding methods, enhancing the coding process by leveraging information from coded-blocks for improved encoding and decoding.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-08-05
- Publication Date
- 2026-03-26
AI Technical Summary
Modern coding operations on uncoded blocks in image frames lack an efficient method to utilize information from coded blocks for effective coding.
An apparatus and method that recursively searches an image frame to position coded-blocks relative to uncoded blocks, using seed coded-blocks and predefined search kernels to gather information for coding operations.
Enhances the efficiency of coding operations by effectively utilizing information from coded-blocks, improving the coding process for both encoding and decoding tasks.
Smart Images

Figure EP2025072486_26032026_PF_FP_ABST
Abstract
Description
[0001] TITLE
[0002] Coding performed on an uncoded block in an image frame using information from coded-blocks of the image frame.
[0003] TECHNOLOGICAL FIELD
[0004] Examples of the disclosure relate to coding performed on an uncoded block in an image frame using information from coded-blocks of the image frame.
[0005] BACKGROUND
[0006] Modern coding operations performed on an uncoded block in an image frame use information from coded-blocks of the image frame.
[0007] It is desirable to have an algorithm that efficiently positions coded blocks in the image frame to be used in a coding operation performed on an uncoded block in the image frame.
[0008] BRIEF SUMMARY
[0009] According to various, but not necessarily all, examples there is provided an apparatus comprising means for: determining an uncoded block of an image frame; searching the image frame to position coded-blocks within the image frame comprising: defining at least one seed coded-block at a position with respect to the uncoded block; defining a process which, when applied to a subject block, results in attempting to visit coded-blocks at pre-determined positions with respect to a position of the subject block; and applying the defined process recursively to all visited coded-blocks as subject blocks, starting from the at least one seed coded-block as an initial subject block; and performing a coding operation on the current uncoded block using information from the positioned coded-blocks of the image frame.
[0010] In some but not necessarily all examples, the at least one seed coded-block is contiguous to the uncoded block.
[0011] In some but not necessarily all examples, three seed coded-blocks are defined. In some but not necessarily all examples the three seed coded blocks contain respectively: a position immediately above a top-left corner of the uncoded block; a position immediately on the left of the top-left corner of the uncoded block; a position immediately above the position immediately on the left of the top-left corner of the uncoded block.
[0012] In some but not necessarily all examples, attempting to visit coded-blocks at predetermined positions with respect to a position of the subject block comprises: for each of the blocks at the pre-determined positions, testing one or more exclusion conditions that prevent visiting a coded-block, wherein in response to a determination from testing that an exclusion condition that prevents visiting a coded-block is satisfied, the block at the predetermined position is not visited, wherein in response to a determination from testing that an exclusion condition that prevents visiting a coded-block is not satisfied, then the block at the predetermined position is visited, wherein the one or more exclusion conditions comprise one or more of: the subject block is outside a defined boundary; the subject block is an uncoded block; and the subject block has already been visited.
[0013] In some but not necessarily all examples, visited coded-blocks at pre-determined positions with respect to a position of the subject block, comprise neighboring blocks contiguous to the subject block.
[0014] In some but not necessarily all examples, the pre-determined positions with respect to a position of the subject block, are located relative to two different edges of the subject block.
[0015] In some but not necessarily all examples, there is a number of pre-determined positions with respect to a position of the subject block, and the number is dependent upon features of the subject block.
[0016] In some but not necessarily all examples, the pre-determined positions with respect to a position of the subject block, comprise positions adjacent different edges of the subject block.
[0017] In some but not necessarily all examples, the pre-determined positions with respect to a position of the subject block, comprise positions adjacent different edges of the subject block that are maximally separated.
[0018] In some but not necessarily all examples, attempting to visit coded-blocks at predetermined positions with respect to a position of the subject block is performed in order based on one or more dimensions of the coded-block. In some but not necessarily all examples, searching the image frame to position coded- blocks within the image frame comprises: defining at multiple seed coded-blocks at a positions with respect to the current uncoded block; for each seed coded-block: defining a process which, when applied to a subject block, results in attempting to visit coded-blocks at pre-determined positions with respect to a position of the subject block; applying the defined process recursively to all visited coded-blocks as subject blocks, starting from the seed coded-block as an initial subject block
[0019] In some but not necessarily all examples, the process which, when applied to a subject block, results in attempting to visit coded-blocks at pre-determined positions with respect to a position of the subject block, is the same process for each seed coded- block.
[0020] In some but not necessarily all examples, the multiple seed coded-blocks are contiguous to the uncoded block.
[0021] In some but not necessarily all examples, the multiple seed coded-blocks are contiguous with a single vertex of the uncoded block.
[0022] In some but not necessarily all examples, a vertex of the uncoded block logically divides the image frame into four quadrants comprising three searchable quadrants that are outside the uncoded block and a fourth quadrant occupied by the uncoded block; wherein searching the image frame to position coded-blocks within the image frame comprises searching each quadrant of the image frame to position coded-blocks within the image frame comprising: defining at least one seed coded-block at a position with respect to the uncoded block in the quadrant; defining a process which, when applied to a subject block, results in attempting to visit coded-blocks at pre-determined positions with respect to a position of the subject block in the quadrant; and applying the defined process recursively to all visited coded-blocks as subject blocks, starting from the at least one seed coded-block as an initial subject block; and wherein the search in each quadrant uses a different search kernel having different pre-determined positions with respect to a position of the subject block.
[0023] In some but not necessarily all examples, the pre-determined positions with respect to a position of the subject block are defined by combinations of x-direction and y- direction unique to that quadrant. In some but not necessarily all examples, the apparatus, comprises means that for the purposes of providing subject blocks, splits a block overlapping quadrants into distinct portions for each quadrant and uses the distinct portions as subject blocks.
[0024] In some but not necessarily all examples, the information from the positioned coded- blocks of the image frame is extracted as a list of blocks visited during the search.
[0025] In some but not necessarily all examples, the image frame is a video frame.
[0026] In some but not necessarily all examples, the apparatus is configured for decoding an encoded image frame, the coding operation being a decoding operation in which an uncoded block of the image frame is a block that has not yet been decoded and a coded block of the image frame is a block that has been decoded.
[0027] In some but not necessarily all examples, the apparatus is configured for encoding an un-encoded image frame, the coding operation being an encoding operation in which an uncoded block of the image frame is a block that has not yet been encoded and a coded block of the image frame is a block that has been encoded.
[0028] According to various, but not necessarily all, examples there is provided a computer program that when run by one or more processors of an apparatus cause the apparatus to perform: determining an uncoded block of an image frame; searching the image frame to position coded-blocks within the image frame comprising: defining at least one seed coded-block at a position with respect to the uncoded block; defining a process which, when applied to a subject block, results in attempting to visit coded-blocks at pre-determined positions with respect to a position of the subject block; and applying the defined process recursively to all visited coded-blocks as subject blocks, starting from the at least one seed coded-block as an initial subject block; and performing a coding operation on the current uncoded block using information from the positioned coded-blocks of the image frame.
[0029] According to various, but not necessarily all, examples there is provided a method comprising: determining an uncoded block of an image frame; searching the image frame to position coded-blocks within the image frame comprising: defining at least one seed coded-block at a position with respect to the uncoded block; defining a process which, when applied to a subject block, results in attempting to visit coded-blocks at pre-determined positions with respect to a position of the subject block; and applying the defined process recursively to visited coded-blocks as subject blocks, starting from the at least one seed coded-block as an initial subject block; and performing a coding operation on the current uncoded block using information from the positioned coded-blocks of the image frame.
[0030] In some examples, the method is an apparatus-implemented method, for example a computer-implemented method.
[0031] According to various, but not necessarily all, embodiments there is provided examples as claimed in the appended claims.
[0032] While the above examples of the disclosure and optional features are described separately, it is to be understood that their provision in all possible combinations and permutations is contained within the disclosure. It is to be understood that various examples of the disclosure can comprise any or all the features described in respect of other examples of the disclosure, and vice versa. Also, it is to be appreciated that any one or more or all the features, in any combination, may be implemented by / comprised in / performable by an apparatus, a method, and / or computer program instructions as desired, and as appropriate. The description of a function should additionally be considered to also disclose any means suitable for performing that function
[0033] BRIEF DESCRIPTION
[0034] Some examples will now be described with reference to the accompanying drawings in which:
[0035] FIG. 1 shows an example of the subject matter described herein;
[0036] FIG. 2 shows another example of the subject matter described herein;
[0037] FIGS. 3A to 3H show another example of the subject matter described herein; FIGS. 4A to 4H show another example of the subject matter described herein; FIG 5 shows another example of the subject matter described herein;
[0038] FIGS 6A and 6B shows another example of the subject matter described herein;
[0039] FIG 7 shows another example of the subject matter described herein;
[0040] FIG 8 shows another example of the subject matter described herein;
[0041] FIG 9 shows another example of the subject matter described herein.
[0042] The figures are not necessarily to scale. Certain features and views of the figures can be shown schematically or exaggerated in scale in the interest of clarity and conciseness. For example, the dimensions of some elements in the figures can be exaggerated relative to other elements to aid explication. Similar reference numerals are used in the figures to designate similar features. For clarity, all reference numerals are not necessarily displayed in all figures.
[0043] DETAILED DESCRIPTION
[0044] In the following, reference is made to blocks of an image frame. A block of an image frame is a continuous portion of the image frame that is coded or to be coded as a block. A block can be an uncoded block 20 (at a position 22) or a coded block 50 (at a position 52). At least some coded blocks 50 can be the subject of a recursive search from an uncoded block. Such coded blocks 50 are referred to as subject blocks 50_i. The first subject block 50_i in the recursive search from an uncoded block 20 is a seed block 40 (at seed position 32) dependent upon the position 22 of the uncoded block 20. The subject blocks 50_i in the subsequent iterations of the recursive search are search blocks 50J (at search positions 62) dependent upon the position 52 of a subject block 50_i. There is an attempt to visit the search blocks 50J. If a search block is successfully visited, it is a visited block 70 used as a subject block 50_i for the next iteration in the recursive search.
[0045] The subject block 50_i, at position 52_i, is a coded block 50 that is a subject of the recursive search at iteration i. The coded-blocks 50 J (search blocks), at position 52 J, are coded blocks 50 at pre-determined positions 62 with respect to the position 52_i of the subject block 50_i for the recursive search at iteration i.
[0046] The following examples relate to an apparatus 300 comprising means for: determining an uncoded block 20 of an image frame 10; searching the image frame 10 to position coded-blocks 50 within the image frame 10 comprising: defining at least one seed coded-block 40 at a position 32 with respect to the uncoded block 20; defining a process which, when applied to a subject block 50_i, results in attempting to visit different coded-blocks 50J at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i; and applying the defined process recursively to all visited coded-blocks 70 as subject blocks, starting from the at least one seed coded-block 40 as an initial subject block; and performing a coding operation on the uncoded block 20 using information from the visited coded-blocks 70 of the image frame 10.
[0047] The apparatus can be alternatively described as an apparatus 300 comprising means for: determining an uncoded block 20 of an image frame 10; searching the image frame 10 to position coded-blocks 50 within the image frame 10 comprising: defining at least one seed coded-block 40 at a position 32 with respect to the uncoded block 20; and applying recursively to all visited coded-blocks 70 as subject blocks, starting from the at least one seed coded-block 40 as an initial subject block 50_i, a process which, when applied to a subject block 50_i, results in attempting to visit coded-blocks 50J at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i; and performing a coding operation on the uncoded block 20 using information from the visited coded-blocks 50 of the image frame 10.
[0048] FIG 1 illustrates an example of a method 100. In at least some examples, the method 100 is a computer-implemented method performed at an apparatus 300 (not illustrated in FIG. 1).
[0049] At method block 102, the method 100 comprises: determining an uncoded block 20 of an image frame 10.
[0050] At method block 104, the method 100 comprises: searching the image frame 10 to position coded-blocks 50 within the image frame 10.
[0051] At method block 112, the method 100 comprises: performing a coding operation on the uncoded block 20 using information from the positioned coded-blocks 50 of the image frame 10.
[0052] The method block 104, of the method 100, comprises: at method block 106, defining at least one seed coded-block 40 at a position 32 with respect to the uncoded block 20; at method block 108, defining a process which, when applied to a subject block 50_i, results in attempting to visit coded-blocks 50J at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i; and at method block 102, applying the defined process recursively to all visited coded- blocks 70 as subject blocks, starting from the at least one seed coded-block 40 as an initial subject block 50_i.
[0053] The recursive search of the image frame 10 positions the visited coded-blocks 50 within the image frame 10.
[0054] In at least some examples, attempting to visit coded-blocks 50J at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i comprises: for each of the blocks 50J at the pre-determined positions 62, testing one or more exclusion conditions that prevent visiting a coded-block, wherein in response to a determination from testing that an exclusion condition that prevents visiting a coded- block is satisfied, the block 50J at the predetermined position 62 is not visited, and wherein in response to a determination from testing that an exclusion condition that prevents visiting a coded-block is not satisfied, then the block 50J at the predetermined position 62 is visited.
[0055] In at least some examples, the one or more exclusion conditions comprise one or more of: the block 50J at the pre-determined position 62 is outside a defined boundary; the block 50J at the pre-determined position 62 is an uncoded block 20; and the block 50J at the pre-determined positions 62 has already been visited in the recursive search.
[0056] In some examples, the defined boundary is an image frame edge or boundary defined with respect to the image frame edge. In some examples, the defined boundary defines a template used to filter out blocks 50J which fully or partially belong to the area inside (or alternatively outside) of the template. As an example, a block 50J is visited only if all pixels within the block 50J belong to the template. As an example, a block 50J is visited only if at least some pixels within the block 50J belong to the template
[0057] In the following examples, the term ‘kernel’ is used to define a template that pre-defines one or more positions (within an image frame 10) with respect to another position (in the same image frame 10). A seed kernel 30 is used to define the kernel used to initiate the search. A search kernel 60 is used to define the kernel used during the recursive search.
[0058] The seed kernel 30 pre-defines one or more seed positions 32 (within an image frame 10) used to find one or more respective seed blocks 40 (within the image frame 10). The pre-defined one or more seed positions 32 are defined with respect to a position 22 (in the image frame 10) of an original uncoded block 20. There is not necessarily a seed block 40 at each of the one or more seed positions 32. There is necessarily one or more seed positions 32 for each seed block 40.
[0059] The search kernel 60 pre-defines multiple search positions 62 (within an image frame 10) used to find one or more respective search blocks 50J (within the image frame 10). The pre-defined search positions 62 (within the image frame 10) are defined with respect to a position 52 (in the image frame 10) of a subject block 50_i (a coded block that is a subject of the recursive search). There is not necessarily a search block 50 J at each of the one or more search positions 62. There is necessarily one or one or more search positions 62 for each search block 50J.
[0060] The recursive search process attempts to visit the search blocks 50J which are coded blocks 50 at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i.
[0061] Initially the subject block 50_i is a seed block 40, which seeds the recursive search. In subsequent iterations of the recursive search, the subject block 50_i is a respective visited block 70, that is a search block 50J that has been successfully visited as a consequence of an attempt to visit the search block 50J.
[0062] FIG 2 illustrates an example of searching an image frame 10 to position coded-blocks 50 within the image frame 10. This corresponds to block 104 of FIG. 1.
[0063] The process 104 comprises:
[0064] At the uncoded block 20, apply a seed kernel 30 (step 202). At the seed kernel 30, identify coded block(s) 50 (step 204). These steps define at least one seed coded- block 40 at a position 32 with respect to the uncoded block 20.
[0065] At each identified coded block (subject block 50_i), apply a search kernel (step 206). At the search kernel, identify coded block(s) 50J for visiting (step 208). Then recursively return to perform step 206, 208 and recursively return to step 206. These steps apply recursively to all visited coded-blocks 70 as subject blocks 50_i, a search process which, when applied to a subject block 50_i, results in attempting to visit coded-blocks 50J at pre-determined positions 62 52J with respect to a position 52_i of the subject block 50_i. The recursive search process starts from the at least one seed coded-block 40 as an initial subject block 50_i.
[0066] A termination condition can be used to halt the recursive process, at step 210, for example at or before all coded blocks 50 have been visited.
[0067] In some examples, the recursive search continues until a termination criterion is met. In some examples the termination criterion is any one of:
[0068] (i) no unvisited coded-blocks remain in the image frame 10;
[0069] (ii) a target number of coded-blocks have been visited;
[0070] (ii) a cumulative area of coded-blocks have been visited (e.g. area adjacent uncoded block 20);
[0071] (iv) a target density (number per area) of coded-blocks have been visited (e.g. number of coded-blocks visited per cumulative area of coded-blocks that have been visited).
[0072] In the following illustrated examples a seed kernel 30 is used to define at least one seed coded-block 40 at a position 32 with respect to an uncoded block 20. A search kernel 60 is used to define pre-determined positions 62 with respect to a position 52_i of a subject block 50_i at which visits to coded-blocks 50J (search blocks) are attempted. The subject block 50_i is originally a seed coded-block 40, and during subsequent iterations of the recursive search, a visited coded-block 70.
[0073] In these illustrated examples, the search kernel 60 is fixed (constant) during the recursive search. The search kernel 60 does not vary or adapt during the recursive search. In other examples (not illustrated), the search kernel 60 is not fixed (constant) during the recursive search and does vary / adapt during the recursive search.
[0074] In these illustrated examples, the search kernel 60 and the seed kernel 30 are different. In other examples (not illustrated), the search kernel 60 and the seed kernel 30 are the same.
[0075] FIG 3A illustrates an example of an image frame 10. The image frame 10 is divided into a grid of pixels or other sub-units. An example of an uncoded block 20 (of one or more sub-units) is illustrated. The uncoded block 20 has a position 22. In this example, the uncoded block 20 is rectangular and the position of the rectangular block is defined by a vertex of the rectangular block. In this example, the position of the rectangular block is defined by a minimum x co-ordinate value (defined within the grid, where x increases from left to right by convention) and a minimum y co-ordinate value (defined within the grid, where y increases from top to bottom by convention) of the uncoded block 20.
[0076] FIG 3B illustrates an example of a seed kernel 30. The seed kernel 30 pre-defines a seed position 32 (within an image frame 10) used to find a seed block 40 (within the image frame 10). The pre-defined seed position 32 is defined with respect to a position (in the image frame 10) of an original block, the uncoded block 20.
[0077] In this example, the pre-defined seed position 32 is contiguous to the uncoded block 20. In this example, the pre-defined seed position 32 is contiguous to the defined position 22 of the uncoded block 20. In this example, if the defined position 22 of the uncoded block 20 is vertex (xc, yc), then the pre-defined seed position 32 is (xc, yc-1) contiguous to the vertex of the uncoded block 20.
[0078] The seed kernel 30 is applied at the uncoded block 20 as illustrated.
[0079] FIG 3C illustrates identification of a coded block 50 (a seed block 40) at the seed kernel 30. The seed block 40 is the coded block 50 that comprises the pre-defined seed position 32 defined by application of the seed kernel 30. Thus a seed coded-block 40 is defined at a seed position 32 with respect to the uncoded block 20. In this example, the at least one seed coded-block 40 is contiguous to the uncoded block 20.
[0080] The recursive search beings. The first subject block 50_i (FIG 3D) in the recursive search is the seed block 40 (FIG 3D). The subsequent subject blocks 50_i (FIG 3G) in the recursive search are visited search blocks 70 (at positions 52) dependent upon the position 52_i of the preceding subject block 50_i in the recursive search. There is an attempt to visit all the search blocks 50J. If a search block 50J is successfully visited, it is a visited search block 70 and is used as a subject block 50_i for the next iteration in the recursive search.
[0081] FIG 3D, illustrates that the seed block 40 is now the subject block 50_i of the recursive search. The position 52_i of the subject block 50_i is illustrated. In this example, the subject block 50_i is rectangular and the position of the rectangular block is defined by a vertex of the rectangular block. In this example, the position 52_i of the rectangular block is defined by a minimum x co-ordinate value (defined within the grid, where x increase from left to right by convention) and a minimum y co-ordinate value (defined within the grid, where y increase from top to bottom by convention) of the subject block 50_i.
[0082] FIG 3E illustrates an example of a search kernel 60. The search kernel 60 pre-defines search positions 62 (within an image frame 10) used to find one or more respective search blocks 50J (within the image frame 10). The pre-defined search positions 62 (within the image frame 10) are defined with respect to a position 52_i (in the image frame 10) of the subject block 50_i (the seed block 40 that is a subject of the recursive search).
[0083] In this example, the pre-defined search positions 62 are contiguous to the subject block 50_i. In this example, the pre-defined search positions 62 are contiguous to vertices of the subject block 50_i and absent from edges of the subject block 50_i.
[0084] In this example, if the defined position 52_i of the subject block 50_i is (x, y) then the pre-defined search positions 62 are at
[0085] (x-1 ,y), (x-1 ,y-1), (x,y-1)
[0086] (x+X, y) (x-1+X,y-1) (x+X,y-1)
[0087] (x+X, y+Y) (x-1 +X, y+Y) (x+X,y-1+Y)
[0088] (x, y+Y) (x-1 , y+Y) (x-1,y-1+Y) where subject block 50_i has dimension X in the x-direction and Y in the y-direction.
[0089] The search kernel 60 is applied at the identified subject block 50_i (the seed block 40) as illustrated.
[0090] In at least some examples, search positions 62 within uncoded blocks (or a previously visited block in the recursive search for the uncoded block 20) can be ignored.
[0091] Referring to FIG 3F, a search block 50J is a coded block 50 that comprises the predefined search position 62 defined by application of the search kernel 60. In this example, the search blocks 50J are contiguous to the subject block 50_i. The recursive search process attempts to visit the search blocks 50J which are blocks 50 at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i. There is an attempt to visit the search blocks 50J. If a search block 50J is successfully visited, it is a visited search block 70 (FIG 3F) and is used as a subject block 50_i for the next iteration in the recursive search, as illustrated in FIG 3G.
[0092] A search block 50J can be successfully visited if, for example, it is valid and has not been previously visited. An invalid search block 50J can be determined using conditions similar to the termination conditions. In some examples, the invalidity criterion is any one of: the search block 50J has been visited; a target number of search blocks 50J have been defined; the search block 50J lies fully or partially outside a boundary; a cumulative area of search blocks 50J have been defined, a target density of search block 50J have been defined.
[0093] FIG 3G, illustrates that the visited search block 70 is now the subject block 50_i of the recursive search. The position 52_i of the subject block 50_i is illustrated. In this example, the subject block 50_i is rectangular and the position 52_i of the rectangular block is defined by a vertex of the rectangular block. In this example, the position of the rectangular block is defined by a minimum x co-ordinate value (defined within the grid, where x increase from left to right by convention) and a minimum y co-ordinate value (defined within the grid, where y increase from top to bottom by convention) of the subject block 50_i.
[0094] FIG 3H illustrates an example of a search kernel 60. The search kernel 60 pre-defines search positions 62 (within an image frame 10) used to find one or more respective search blocks 50J (within the image frame 10). The pre-defined search positions 62 (within the image frame 10) are defined with respect to a position 52_i (in the image frame 10) of the subject block 50_i (a visited coded block 70 that is a subject of the recursive search).
[0095] In this example, the pre-defined search positions 62 are contiguous to the subject block 50_i. In this example, the pre-defined search positions 62 are contiguous to vertices of the subject block 50_i and absent from edges of the subject block 50_i. In this example, the search kernel 60 is the same as that illustrated in FIG 3E.
[0096] The search kernel 60 is applied at the identified subject block 50_i (each of the visited search blocks 70 from the last iteration) as illustrated. The next step (not illustrated) comprises identification of coded blocks 50J (search blocks) at the search kernel 60. The search block 50J is the coded block 50 that comprises a pre-defined search position 62 defined by application of the search kernel 60. The search block(s) 50J can be contiguous to the subject block 50_i. The recursive search process attempts to visit the search blocks 50J which are blocks 50 at predetermined positions 62 with respect to a position 52_i of the subject block 50_i. There is an attempt to visit the search blocks 50J. If a search block 50J is successfully visited, it is a visited search block 70 and is used as a subject block 50_i for the next iteration in the recursive search.
[0097] It will be understood that the process steps illustrated in FIG 3E, 3F, 3G occur recursively: at each subject block 50_i (identified coded block), apply a search kernel 60 (step 206 FIG 2; FIG 3E; FIG 3H) at the applied search kernel 60, identify coded block(s) as the next subject block 50_i (step 208 FIG 2; FIGs 3F & 3G) by attempting to visit.
[0098] FIG 3A to 3H illustrate applying recursively to all visited coded-blocks 70 as subject blocks, starting from the at least one seed coded-block 40 as an initial subject block 50_i, a process which, when applied to a subject block 50_i, results in attempting to visit different coded-blocks 50J at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i.
[0099] FIG 4A illustrates an example of an image frame 10 and uncoded block 20 as previously illustrated in FIG 3A to which has been applied a seed kernel 30. The seed kernel 30 pre-defines seed positions 32 (within an image frame 10) used to find one or more seed blocks 40 (within the image frame 10). The pre-defined seed positions 32 are defined with respect to a position (in the image frame 10) of an original block, the uncoded block 20.
[0100] In this example, the pre-defined seed positions 32 are contiguous to the uncoded block 20. In this example, the pre-defined seed positions 32 are contiguous to the defined position 22 of the uncoded block 20. In this example, if the defined position 22 of the uncoded block 20 is (xc, yc) then the pre-defined seed positions 32 are (xc, yc-1), (xd , yd), (Xc-1 , yc). In this example the pre-defined seed positions 32 are contiguous to a vertex of the uncoded block 20. The seed kernel 30 is applied at the uncoded block 20 as illustrated.
[0101] FIG 4B illustrates identification of one or more coded blocks 50 (seed blocks 40) at the seed kernel 30. A seed block 40 is the coded block 50 that comprises one or more predefined seed positions 32 defined by application of the seed kernel 30. Thus a seed coded-block 40 is defined at a seed position 32 with respect to the uncoded block 20. In this example, the seed coded-blocks 40 are contiguous to the uncoded block 20.
[0102] Three seed coded-blocks 40 are defined contiguous to the uncoded block 20. The three seed coded blocks contain respectively: a first seed position 32_1 immediately above [yc-1] a top-left corner of the uncoded block 20; a second seed position 32_2 immediately on the left [xc-1] of the top-left corner of the uncoded block 20; a third seed position 32_3 immediately above [yc-1] the position immediately on the left [Xc-1] of the top-left corner of the uncoded block 20.
[0103] The first, second, third seed positions 32 are (xc, yc-1), (xc-1 , yc), (xc-1 , yd).
[0104] The recursive search beings. The first subject blocks 50_i (FIG 4G) in the recursive search are the seed blocks 40 (FIG 3D). The subsequent subject blocks 50_i (FIG 4G) in the recursive search are visited search blocks 70 (at positions 52) dependent upon the positions 52_i of the subject blocks 50_i in the preceding iteration of the recursive search. There is an attempt to visit all the search blocks 50J. If a search block 50J is successfully visited, it is a visited search block 70 and is used as a subject block 50_i for the next iteration in the recursive search.
[0105] FIG 4C, illustrates that each seed block 40 is now the subject block 50_i of the recursive search. The position 52_i of the subject block 50_i is illustrated.
[0106] In this example, each subject block 50_i is rectangular and the position of the rectangular block is defined by a vertex of the rectangular block. In this example, the position 52 of the rectangular block is defined by a minimum x co-ordinate value (defined within the grid, where x increase from left to right by convention) and a minimum y co-ordinate value (defined within the grid, where y increase from top to bottom by convention) of the subject block 50_i.
[0107] FIG 4D illustrates an example of a search kernel 60. The search kernel 60 pre-defines search positions 62 (within an image frame 10) used to find one or more respective search blocks 50J (within the image frame 10). The pre-defined search positions 62 (within the image frame 10) are defined with respect to a position 52 (in the image frame 10) of the subject block 50_i (the seed block 40 that is a subject of the recursive search).
[0108] In this example, the pre-defined search positions 62 are contiguous to the subject block 50_i. In this example, the pre-defined search positions 62 are contiguous to vertices of the subject block 50_i and absent from edges of the subject block 50_i.
[0109] A vertex (xc, yc) of the uncoded block 20 logically divides the image frame 10 into four quadrants comprising three searchable quadrants that are outside the uncoded block 20 and a fourth quadrant occupied by the uncoded block 20. The first quadrant Q1 is defined by ( x> Xc, y< yc). The second quadrant Q2 is defined by ( x< Xc, y< yc). The third quadrant Q3 is defined by ( x < xc, y> yc).
[0110] In this example, if the defined position 52_i of the subject block 50_i is (x, y) then the pre-defined search positions 62 are at:
[0111] If (x,y) is in a first quadrant Q1 defined by ( x> xc, y< yc)
[0112] (x,y-1), (x+X,y-1+ Y) where subject block 50_i has dimension X in the x-direction and Y in the y-direction.
[0113] If (x,y) is in a second quadrant Q2 defined by ( x< xc, y< yc) (x-1+X,y-1), (x-1 ,y-1+ Y) where subject block 50_i has dimension X in the x-direction and Y in the y-direction.
[0114] If (x,y) is in a third quadrant Q3 defined by ( x < xc, y> yc) (x-1 ,y), (x-1+X,y+ Y) where subject block 50_i has dimension X in the x-direction and Y in the y-direction.
[0115] The search kernel 60 is applied at the identified subject block 50_i (the seed block 40) as illustrated in FIG 4D.
[0116] In at least some examples, search positions 62 within uncoded blocks (or a previously visited block in the recursive search from the uncoded bock 20) can be ignored.
[0117] FIG 4E illustrates the quadrant-dependent search kernel 60. A first direction is defined parallel to the x-direction and a second direction is defined parallel to the y-direction. Searching the image frame 10 to position coded-blocks within the image frame 10 comprises searching each quadrant of the image frame 10 to position coded-blocks within the image frame 10 comprising: defining at least one seed coded-block 40 at a position with respect to the uncoded block 20 in the quadrant; defining a process which, when applied to a subject block 50_i, results in attempting to visit coded-blocks at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i in the quadrant; and applying the defined process recursively to all visited coded-blocks as subject blocks, starting from the at least one seed coded-block 40 as an initial subject block; and wherein the search in each quadrant uses a different search kernel having different pre-determined positions 62 with respect to a position 52_i of the subject block 50_i. In at least some examples, the pre-determined positions 62 with respect to a position 52_i of the subject block 50_i are defined by combinations of x-direction and y-direction unique to that quadrant
[0118] In this example, the method, for the purposes of providing subject blocks 50_i, splits a block 50 overlapping quadrants into distinct portions for each quadrant and uses the distinct portions as subject blocks 50_i.
[0119] Referring to FIG 4F, a search block 50J is a coded block 50 that comprises a predefined search position 62 defined by application of the search kernel 60. In this example, the search blocks 50J are contiguous to the subject block 50_i. The recursive search process attempts, for each subject block 50_i, to visit the search blocks 50J which are blocks at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i. There is an attempt to visit the search blocks 50J. If a search block 50J is successfully visited, it is a visited search block 70 (FIG 4F) and is used as a subject block 50_i for the next iteration in the recursive search, as illustrated in FIG 4G
[0120] A search block 50J (FIG 4F) can be successfully visited if, for example, it is valid and has not been previously visited. A valid search block 50J can be determined using conditions similar to the termination conditions, as described above.
[0121] FIG 4G, illustrates that the visited search blocks 70 are now the subject blocks 50_i of the recursive search. For each subject block 50_i, the position 52_i of the subject block 50_i is illustrated. In this example, each subject block 50_i is rectangular and the position of the rectangular block is defined by a vertex of the rectangular block. In this example, the position of the rectangular block is defined by a minimum x co-ordinate value (defined within the grid, where x increase from left to right by convention) and a minimum y co-ordinate value (defined within the grid, where y increase from top to bottom by convention) of the subject block 50_i.
[0122] FIG 4H illustrates an example of a search kernel 60. The search kernel 60 pre-defines search positions 62 (within an image frame 10) used to find one or more respective search blocks 50J (within the image frame 10). The pre-defined search positions 62 (within the image frame 10) are defined with respect to a position 52 (in the image frame 10) of the subject block 50_i (a visited coded block 70 that is a subject of the recursive search).
[0123] In this example, the pre-defined search positions 62 are contiguous to the subject block 50_i. In this example, the pre-defined search positions 62 are contiguous to vertices of the subject block 50_i and absent from edges of the subject block 50_i. In this example, the search kernel 60 is the same as that illustrated in FIG 4E.
[0124] The search kernel 60 is applied at the identified subject block 50_i (each of the visited search blocks 70 from the last iteration) as illustrated.
[0125] The next step (not illustrated) comprises identification of coded blocks 50J (search blocks) at the search kernel 60. The search block 50J is the coded block 50 that comprises the pre-defined search position 62 defined by application of the search kernel 60. The search block(s) 40 can be contiguous to the subject block 50_i. The recursive search process attempts to visit the search blocks 50J which are blocks 50 at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i. There is an attempt to visit the search blocks 50J. If a search block 50J is successfully visited, it is a visited search block 70 and is used as a subject block 50_i for the next iteration in the recursive search.
[0126] It will be understood that the process steps illustrated in FIG 4D, 4F, 4G occur recursively: at each subject block 50_i (identified coded block), apply a search kernel 60 (step 206 FIG 2; FIG 4D; FIG 4H) ; at the applied search kernel 60, identify coded block(s) as the next subject block 50_i (step 208 FIG 2; FIGs 4F & 4G) by attempting to visit. FIG 4A to 4H illustrate applying recursively to all visited coded-blocks 70 as subject blocks, starting from the at least one seed coded-block 40 as an initial subject block 50_i, a process which, when applied to a subject block 50_i, results in attempting to visit different coded-blocks 50J at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i.
[0127] This example comprises: determining an uncoded block 20 of an image frame 10; searching the image frame 10 to position coded-blocks 50 within the image frame 10 comprising: defining multiple seed coded-blocks 40 at positions 32 with respect to the uncoded block 20; defining a process which, when applied to a subject block 50_i, results in attempting to visit different coded-blocks 50J at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i; and applying the defined process recursively to all visited coded-blocks 70 as subject blocks, starting from the seed coded-blocks 40 as initial subject blocks; and performing a coding operation on the current uncoded block 20 using information from the visited coded-blocks 70 of the image frame 10.
[0128] The multiple seed coded-blocks 40 can be contiguous to the uncoded block 20, for example, contiguous with a single vertex of the uncoded block 20
[0129] It will be appreciated from the foregoing examples, that the seed kernel 30 in these examples has some sparsity. The seed positions 32 with respect to a position 22 of the uncoded block, where present at an edge of the uncoded block 20 are discontinuous along the edge of the uncoded block 20.
[0130] It will be appreciated from the foregoing examples, that the search kernel 60 in these example has some sparsity. The pre-determined positions 62 with respect to a position 52_i of the subject block 50_i, where present at an edge of the subject block 50_i are discontinuous along the edge of the subject block 50_i.
[0131] In at least some examples, the visited coded-blocks 70 at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i, comprise one or more (but not all) neighboring blocks contiguous to the subject block 50_i. In at least some examples, the visited coded-blocks 70 at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i, comprise only neighboring blocks contiguous to the subject block 50_i.
[0132] In some examples, there is a number of pre-determined positions 62 with respect to a position 52_i of the subject block 50_i, and the number is dependent upon features of the subject block 50_i. As an example, the number of predefined positions 62 to visit depends on features of the subject block 50_i . As an example, the number of predefined positions 62 to visit may depend on the size of the subject block 50_i. As an example, any number of neighboring positions around the seed blocks 30 which contain the initial seed positions 32 can be visited. As an example, any number of neighboring positions around the search blocks 50J which originate from the initial seed positions 32 can be visited.
[0133] In at least some examples, the visited coded-blocks 70 at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i, comprise next nearest neighbor blocks (relative to the subject block 50_i) in two orthogonal directions (x, y) and are contiguous to the subject block 50_i. In some examples, the visited coded- blocks 70 at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i, comprise only next nearest neighbor blocks (relative to the subject block 50_i) in two orthogonal directions (x, y) and are contiguous to the subject block 50_i. In at least some examples, as illustrated in FIG 4G, visited coded-blocks at predetermined positions 62 with respect to a position 52_i of the subject block 50_i, are next nearest neighbor blocks (relative to the subject block 50_i) that are maximally separated in both orthogonal directions (x, y). (e.g. the positions adjacent different orthogonal edges of the subject block 50_i are maximally spaced from a common rightangle vertex of the different orthogonal edges).
[0134] In at least some examples, the pre-determined positions 62 with respect to a position 52_i of the subject block 50_i, are located contiguous to the subject block 50_i. In at least some examples, the pre-determined positions 62 with respect to a position 52_i of the subject block 50_i, are located relative to two different edges of the subject block 50_i. In at least some of these examples, there are one or more pre-determined positions 62 per edge. In at least some examples, the pre-determined positions 62 with respect to a position 52_i of the subject block 50_i, comprise positions adjacent different edges of the subject block 50_i. The positions 62 adjacent different edges of the subject block are contiguous to the subject block 50_i. In some examples, the predetermined positions 62 with respect to a position 52_i of the subject block 50_i, comprise only positions adjacent different edges of the subject block 50_i that are contiguous to the subject block 50_i. In at least some examples, as illustrated in FIG 4E, the pre-determined positions 62 comprise positions adjacent different edges of the subject block 50_i that are maximally separated (e.g. the positions adjacent different edges of the subject block 50_i are maximally spaced from a common vertex of the different edges). In at least some examples, the pre-determined positions 62 with respect to a position 52_i of the subject block 50_i, are adjacent vertexes of the subject block 50_i.
[0135] In at least some examples (FIG 4D, 4E), the pre-determined positions 62 with respect to a position 52_i of the subject block 50_i, are positions offset from diagonally opposing vertices of the subject block 50_i. The first position is offset in at least a distance in one direction (e.g. x-direction) from a first vertex of the subject block 50_i. The second position is offset in at least a distance in another, orthogonal direction (e.g. y-direction), from a second vertex of the subject block 50_i. The first vertex of the subject block is a maximum x distance from the seed position 32 of the seed block 40, and a minimum y distance from the seed position 32 of the seed block 40 (FIG 4D- positions 62 adjacent x=Xc). The second vertex of the subject block is a maximum y distance from the seed block 40, and a minimum x distance from the seed block 40 (FIG 4D- positions 62 adjacent y=yc). In at lest some examples, the first position and the second position are contiguous to the subject block 50_i, outside the subject block 50_i. In at least some examples, the first position is offset only an x distance from first vertex and the second position is offset only a y distance from second vertex.
[0136] In some examples, attempting to visit coded-blocks 50 J at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i is performed in order based on one or more dimensions of the coded-block 50J and / or based on one or more dimensions of the subject block 50_i.
[0137] In some examples, attempting to visit coded-blocks 50 J (search blocks) at predetermined positions 62 with respect to a position 52_i of the subject block 50_i is performed in order based on one or more dimensions of the coded-block 50J, for example, by: visiting the smallest search block 50J first; visiting the search blocks 50J in an order dependent upon one or more dimensions of the search block 50J compared to an adjacent dimension of the subject block 50_i (e.g. visit the relatively thin / short search block 50J first); visiting the search blocks 50J in an order dependent upon density of blocks 50 (e.g. visit the closest packed search block 50J first).
[0138] As an example, the order of visiting a seed block 40 can be determined based on the uncoded block size, or based on a feature originating from the size the uncoded block 20.
[0139] As an example, the order of visiting a search block 50J can be determined based on the subject block size, or based on a feature originating from the size the subject block 50_i that generates the search blocks 50 J.
[0140] FIG 5 illustrates examples of different seed kernels 30. The FIG includes the seed kernels 30 previously described with reference to FIG 3B and FIG 4A.
[0141] A seed kernel 30 pre-defines one or more seed positions 32 (within an image frame 10) used to find a seed block (within the image frame 10). The pre-defined one or more seed positions 32 are defined with respect to a position 22 (in the image frame 10) of an original block, the uncoded block 20.
[0142] In these examples, the pre-defined seed position 32 is contiguous to the uncoded block 20. In these examples, a pre-defined seed position 32 is contiguous to the defined position 22 of the uncoded block 20.
[0143] In these examples, if the defined position 22 of the uncoded block 20 is vertex (xc, yc) and the pre-defined one or more seed positions 32 for the examples given are:
[0144] A) (xc, yc-1)
[0145] B) (xc, yc-1), (Xc-1 , yc-1), (Xc-1 , yc).
[0146] C) (xc, yc-1), (Xc + X / 2, yc-1), (xc+X-1 , yc-1), (Xc+ X, yc-1), (xc-1 , yc), (xc-1 , yc+ Y / 2), (Xc-1 , yc + Y-1), (xc-1 , yc+ Y) where subject block 50_i has dimension X in the x-direction and Y in the y-direction.
[0147] Seed positions 32 can for example be located anywhere adjacent two of the four edges of the uncoded block 20.
[0148] In some examples, seed positions 32 are be spaced (separated) along the edges of the uncoded block 20. In some examples, seed positions 32 are diagonally adjacent at vertexes of the uncoded block 20.
[0149] In some examples, the seed kernel 30 has a fixed spacing between seed positions 32 irrespective of a size of the uncoded block 20. In some examples, the seed kernel 30 has a spacing between seed positions 32 that varies (e.g. scales linearly) with a length of the adjacent edge of the uncoded block 20. In some examples, additional seed positions 32 are introduced as the size of the adjacent edge of the uncoded block 20 increases. For example, an additional seed position 32 can be added at a coordinate located between two seed positions 32 .
[0150] For example, in the example (C) a seed position 32 along an edge is located at xc + X / 2. In other examples, multiple edge seed positions 22 can be located at xc + ( n* X / N), where n=1..N-1 and N>2. For example, in the example (C) a seed position 32 along an edge is located at yc+ Y / 2. In other examples, multiple edge seed positions 22 can be located at yc+ (m* Y / M), where m=1..M-1 and M>2. The values M and N can be the same. The values M and N can change with a size of the uncoded block 20.
[0151] The seed kernel 30 illustrated in FIG 4A & 4B has previously been defined as a specific example: (Xc, yc-1), (xc-1 , yd), (xd , yc) where (xc, yc) is the defined position 22 of the uncoded block 20.
[0152] This can be expressed in a more general form: first seed position 32_1 (xc+ offsetxl, yc+ offsetyl), second seed position 32_2 (xc+ offsetx2, yc+ offsety2), third seed position 32_3 (xc+ offsetx3, yc+ offsety3)...
[0153] In the specific example: offsetxl= 0, offsetyl= -1 (first seed position 32_1 immediately above block position) offsetx2= -1 , offsety2= 0 (second seed position 32_2 immediately to left of block position) offsetx3= -1 , offsety3= -1 (third seed position 32 immediately above and to left of block position)
[0154] Giving seed positions 32 : (xc,yc- 1), (xc- l,yc), (xc- l,yc- 1), where position (xc,yc) corresponds to the top-left corner position 22 of the uncoded block 20. In some examples, the search kernel 60 has a fixed spacing between positions 62 irrespective of a size of the subject block 50_i. In some examples, the search kernel 60 has a spacing between positions 62 that varies (e.g. scales linearly) with a length of the adjacent edge of the subject block 50_i. In some examples, additional positions 62 are introduced as the size of the adjacent edge of the subject block 50_i increases. For example, an additional position 62 can be added at a coordinate located between two positions 62 .
[0155] In FIG 4E, starting from the Q1 subject block 50_i (the seed block 40 which contains the seed position 32 (xc,yc- 1) ), two positions immediately adjacent to the top-left and bottom-right corner of the Q1 subject block 50_i at locations (xnl,ynl- 1) , and (xnl+ wnl,ynl+ hnl- 1) , are visited, where (xnl,ynl) are the coordinates of the topleft corner position of the subject block 50_i (which contains the seed position 32 (xc,yc- 1)) and wnland hnlare the width and the height of said Q1 subject block 50_i, respectively. This process is recursively applied for the subject blocks containing positions at locations (xnl,ynl- 1) and (xnl+ wnl,ynl+ hnl- 1).
[0156] Starting from the subject Q3 subject block 50_i (the seed block 40 which contains the seed position 32 (xc- l,yc) ), two positions immediately adjacent to the top-left and bottom-right corner of the said Q3 subject block 50_i at locations (xn2- l,yn2) and (xn2+ wn2- l,yn2+ hn2) are visited, where (xn2,yn2) are the coordinates of the topleft corner position of the Q3 subject block 50_i ( which contains the seed position 32 (xc- l,yc) ) and wn2and hn2are the width and the height of said Q3 subject block 50_i, respectively. This process is recursively applied for the blocks containing positions at locations (xn2- l,yn2) and (xn2+ wn2- l,yn2+ hn2).
[0157] Starting from the Q2 subject block 50_i (the seed block 40 which contains the seed position 32 (xc- l,yc- 1) ), two positions immediately adjacent to the bottom-left and top-right corner of the said Q2 subject block 50_i at locations (xn3- l,yn3+ hn3- 1) and (xn3+ wn3- l,yn3- 1) are visited, where (xn3,yn3) are the coordinates of the topleft corner position of the Q2 subject block 50_i (which contains the seed position 32 (xc- l,yc- 1) ) and wn3and hn3are the width and the height of said Q2 subject block 50_i, respectively. This process is recursively applied for the blocks containing positions at locations (xn3- l,yn3+ hn3- 1) and (xn3+ wn3- l,yn3- 1).
[0158] FIGs 6A and 6B illustrate an example, where a visited block 70 spans multiple quadrants. In FIG 6A the visited block 70 spans quadrants Q1 and Q2. In FIG 6B the visited block 70 spans quadrants Q2 and Q3. For the purposes of providing subject blocks 50_i for the recursive search, the visited block 70 that spans (overlaps) multiple quadrants is split into distinct portions for each quadrant and the distinct portions are used as subject blocks 50_i to generate search positions 62.
[0159] As an example, an additional check may be performed to ensure that every new recursive search position 62 originating from a specific seed position 32 always belongs to the quadrant corresponding to that seed position 32. As an example, there may be a check to ensure that all search positions 62 originating from the seed position (xc,yc- 1) satisfy the constraint x > xcand y<yc.. Similarly, there may be a check to ensure that all search positions 62 originating from the seed position (xc- l,yc) satisfy the constraint y > ycand x<xc.Finally, there may be a check to ensure that all search positions 62 originating from the seed position 32 (xc- l,yc- 1) satisfy both constraints x < xcand y < yc.
[0160] As an example, if the Q1 and Q2 or the Q3 and Q2 seed positions 32 belong to the same visited block 70, the subsequent search positions 62 which will be checked can be defined in a different way.
[0161] As an example, if the Q1 seed position (xc,yc- 1) and Q2 seed position (xc- l,yc- 1) belong to the same visited block 70 (as illustrated in FIG 6A), the search positions 62 used are (xn- l,yn+ hn- 1), (xc- l,yn- 1), (xc,yn- 1), and (xn+ wn,yn+ hn- 1), where (xn,yn) are the coordinates of the top-left corner position of the visited block 70 which contains the Q1 seed position (xc,yc- 1) and the Q2 seed position (xc- l,yc- 1) and wnand hnare the width and the height of the visited block 70, respectively.
[0162] As an example, if the Q3 seed position (xc- l,yc) and the Q2 seed position (xc- l,yc- 1) belong to the same visited block 70 (as illustrated in FIG 6B), the search positions 62 used are: (xn- l,yc- 1), (xn+ wn- l,yn- 1), (xn- l,yc), and (xn+ wn- 1, yn+ hn), where (xn,yn) are the coordinates of the top-left corner position of the visited block 70 which contains the Q3 seed position (xc- l,yc) and the Q2 seed position (xc- l,yc- 1) and wnand hnare the width and the height of the visited block 70, respectively.
[0163] The coding operation on the current uncoded block 20 uses information from the positioned coded-blocks 70 of the image frame 10. In some examples, the information from the positioned coded-blocks of the image frame 10 is extracted as a list of blocks visited during the search. During the search, a visited block is added to the list only if the same block is not already present in the list. Different lists can be computed for each initial seed block or for each quadrant. As an example, a single list can be computed for blocks found by different neighboring block searches originating from each different seed block 40 (different quadrants). As an example, when a list is computed, the different neighboring block searches can be applied sequentially, where a block from each search is sequentially added to the list. As an example, when a list is computed, the different neighboring block searches can be applied in parallel, where one or more blocks from each search can be added to the list. As an example, the neighboring block list may have a fixed or adaptive size. As an example, the size of the list may be based on a simple block count, but it may be also based on the block area count or some other metric. The block area count may be defined in relative terms based on the size of the current block or in absolute terms based on the area in pixels. As an example, the size of the list may depend on the size of the current block. As an example, the size of the list may depend on the size of neighboring blocks. As an example, whenever a new position is visited, the information about the origin seed may be additionally stored. As an example, the origin seed position 32 for a given position can be estimated based on the coordinates of the tested position.
[0164] Information about the uncoded block 20 can be inferred from information gathered from the visited coded-blocks 70 of the image frame 10. The information gathered from the visited coded-blocks 70 is used when performing a coding operation on the uncoded block 20. In some examples, the information gathered from the visited coded-blocks 70 is used to produce a prediction for the uncoded block 20 by extracting information from the positioned, visited coded-blocks 70. In some examples, the information gathered from the positioned, visited coded-blocks 70 is applied for methods such as include building a histogram of occurrences to compute a prediction for the uncoded block 20. In some examples, the information gathered from the positioned, visited coded-blocks 70 is applied to derive a number of intra-prediction modes, which are then used to compute a number of intra predictors of the current block. A predictor of the current uncoded block can be defined as a block of the same size of the current block, where each sample is a prediction of the collocated sample in the uncoded block. The predicted samples can be computed using previously coded information. The intra predictors can then be blended together to form a final prediction for the uncoded block 20. FIG 7 illustrates examples of apparatus 300 configured to use some or all of the above described processes. In FIG 7 an apparatus 300 is configured to perform encoding 302 of an image frame 10 and an apparatus 300 is configured to perform decoding of an image frame 10.
[0165] The apparatus 300, configured to perform encoding 302, encodes an image frame 10 and transfers the (encoded) image frame 10 to the apparatus 300 that is configured to perform decoding of the (encoded) image frame 10.
[0166] The apparatus 300, configured to perform encoding 302, is configured to perform the above described processes. The apparatus 300 is configured for encoding an image frame 10. The coding operation is an encoding operation. The uncoded block 20 of the image frame 10 is a block that has not yet been encoded. A coded block of the image frame 10 is a block that has been encoded.
[0167] The apparatus 300, configured to perform decoding 304, is configured to perform the above described processes. The apparatus 300 is configured for decoding an encoded image frame 10. The coding operation is a decoding operation. An uncoded block 20 of the image frame 10 is a block that has not yet been decoded. A coded block of the image frame 10 is a block that has been decoded
[0168] In some examples, an apparatus 300 is configured to perform encoding 302 and to perform decoding 304.
[0169] It will be appreciated that the process described above in relation to one uncoded block can be used to code the uncoded block 20, and then be applied to the next uncoded block 20 in the image frame 10. Thus all uncoded blocks 20 in the image frame 10 can be coded.
[0170] It will be appreciated that the process described above in relation to coding one image frame can be used to code multiple image frames, for example video frames.
[0171] Fig 8 illustrates an example of a controller 400 suitable for use in an apparatus 300. Implementation of a controller 400 may be as controller circuitry. The controller 400 may be implemented in hardware alone, have certain aspects in software including firmware alone or can be a combination of hardware and software (including firmware). As illustrated in Fig 8 the controller 400 may be implemented using instructions that enable hardware functionality, for example, by using executable instructions 406 in a general-purpose or special-purpose processor 402 that may be stored on a machine readable storage medium (disk, memory etc.) to be executed by such a processor 402.
[0172] The processor 402 is configured to read from and write to the memory 404. The processor 402 may also comprise an output interface via which data and / or commands are output by the processor 402 and an input interface via which data and / or commands are input to the processor 402.
[0173] The memory 404 stores instructions, program, or code 406 that controls the operation of the apparatus 300 when loaded into the processor 402. The computer program instructions, program or code am 406, provide the logic and routines that enables the apparatus 300 to perform the methods illustrated in the accompanying FIGs. The processor 402 by reading the memory 404 is configured to load and execute the instructions, program, or code 406.
[0174] The apparatus 300 comprises: at least one processor 402; and at least one memory 404 storing instructions that, when executed by the at least one processor 402, cause the apparatus at least to perform: determining an uncoded block 20 of an image frame 10; searching the image frame 10 to position coded-blocks within the image frame 10 comprising: defining at least one seed coded-block 40 at a position with respect to the uncoded block 20; defining a process which, when applied to a subject block 50_i, results in attempting to visit coded-blocks at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i; and applying the defined process recursively to all visited coded-blocks as subject blocks, starting from the at least one seed coded-block 40 as an initial subject block; and performing a coding operation on the current uncoded block 20 using information from the positioned coded-blocks of the image frame 10.
[0175] As illustrated in Fig 9, the instructions, program, or code 406 may arrive at the apparatus 300 via any suitable delivery mechanism 408. The delivery mechanism 408 may be, for example, a machine readable medium, a computer-readable medium, a non-transitory computer-readable storage medium, a computer program product, a memory device, a record medium such as a Compact Disc Read-Only Memory (CD- ROM) or a Digital Versatile Disc (DVD) or a solid-state memory, an article of manufacture that comprises or tangibly embodies the computer program 406. The delivery mechanism may be a signal configured to reliably transfer the computer program 406. The apparatus 300 may propagate or transmit the computer program 406 as a computer data signal.
[0176] The term “non-transitory” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal ) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM).
[0177] Computer program instructions for causing an apparatus to perform at least the following or for performing at least the following: determining an uncoded block 20 of an image frame 10; searching the image frame 10 to position coded-blocks within the image frame 10 comprising: defining at least one seed coded-block 40 at a position with respect to the uncoded block 20; defining a process which, when applied to a subject block 50_i, results in attempting to visit coded-blocks at pre-determined positions 62 with respect to a position 52_i of the subject block 50_i; and applying the defined process recursively to all visited coded-blocks as subject blocks, starting from the at least one seed coded-block 40 as an initial subject block; and performing a coding operation on the current uncoded block 20 using information from the positioned coded-blocks of the image frame 10.
[0178] The computer program instructions may be comprised in a computer program, a non- transitory computer readable medium, a computer program product, a machine readable medium. In some but not necessarily all examples, the computer program instructions may be distributed over more than one computer program.
[0179] Although the memory 404 is illustrated as a single component / circuitry it may be implemented as one or more separate components / circuitry some or all of which may be integrated / removable and / or may provide permanent / semi-permanent / dynamic / cached storage. Although the processor 402 is illustrated as a single component / circuitry it may be implemented as one or more separate components / circuitry some or all of which may be integrated / removable. The processor 402 may be a single core or multi-core processor.
[0180] References to ‘computer-readable storage medium’, ‘computer program product’, ‘tangibly embodied computer program’ etc. or a ‘controller’, ‘computer’, ‘processor’ etc. should be understood to encompass not only computers having different architectures such as single / multi- processor architectures and sequential (Von Neumann) / parallel architectures but also specialized circuits such as field-programmable gate arrays (FPGA), application specific circuits (ASIC), signal processing devices and other processing circuitry. References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device whether instructions for a processor, or configuration settings for a fixed-function device, gate array or programmable logic device etc.
[0181] As used in this application, the term ‘circuitry’ may refer to one or more or all the following:
[0182] (a) hardware-only circuitry implementations (such as implementations in only analog and / or digital circuitry) and
[0183] (b) combinations of hardware circuits and software, such as (as applicable): i. a combination of analog and / or digital hardware circuit(s) with software / firmware and ii. any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory or memories that work together to cause an apparatus, such as a mobile phone or server, to perform various functions and
[0184] (c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (for example, firmware) for operation, but the software may not be present when it is not needed for operation.
[0185] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the claim element, a baseband integrated circuit for a mobile device or a similar integrated circuit in a server, a cellular network device, or other computing or network device.
[0186] The blocks illustrated in the accompanying Figs may represent steps in a method and / or sections of code in the computer program 406. The illustration of a particular order to the blocks does not necessarily imply that there is a required or preferred order for the blocks and the order and arrangement of the block may be varied. Furthermore, it may be possible for some blocks to be omitted.
[0187] As used here ‘module’ refers to a unit or apparatus that excludes certain parts / components that would be added by an end manufacturer or a user. The apparatus 300 can, for example be a module. A controller 400 of the apparatus 300 can, for example be a module.
[0188] Where a structural feature has been described, it may be replaced by means for performing one or more of the functions of the structural feature whether that function or those functions are explicitly or implicitly described.
[0189] The above-described examples find application as enabling components of: automotive systems; telecommunication systems; electronic systems including consumer electronic products; distributed computing systems; media systems for generating or rendering media content including audio, visual and audio visual content and mixed, mediated, virtual and / or augmented reality; personal systems including personal health systems or personal fitness systems; navigation systems; user interfaces also known as human machine interfaces; networks including cellular, non- cellular, and optical networks; ad-hoc networks; the internet; the internet of things; virtualized networks; and related software and services.
[0190] The apparatus can be provided in an electronic device, for example, a mobile terminal, according to an example of the present disclosure. It should be understood, however, that a mobile terminal is merely illustrative of an electronic device that would benefit from examples of implementations of the present disclosure and, therefore, should not be taken to limit the scope of the present disclosure to the same. While in certain implementation examples, the apparatus can be provided in a mobile terminal, other types of electronic devices, such as, but not limited to: mobile communication devices, hand portable electronic devices, wearable computing devices, portable digital assistants (PDAs), pagers, mobile computers, desktop computers, televisions, gaming devices, laptop computers, cameras, video recorders, GPS devices and other types of electronic systems, can readily employ examples of the present disclosure. Furthermore, devices can readily employ examples of the present disclosure regardless of their intent to provide mobility. The term ‘comprise’ is used in this document with an inclusive not an exclusive meaning. That is any reference to X comprising Y indicates that X may comprise only one Y or may comprise more than one Y. If it is intended to use ‘comprise’ with an exclusive meaning then it will be made clear in the context by referring to ‘comprising only one...’ or by using ‘consisting.’
[0191] In this description, the wording ‘connect’, ‘couple’ and ‘communication’ and their derivatives mean operationally connected / coupled / in communication. It should be appreciated that any number or combination of intervening components can exist (including no intervening components), i.e. , to provide direct or indirect connection / coupling / communication. Any such intervening components can include hardware and / or software components.
[0192] As used herein, the term "determine / determining" (and grammatical variants thereof) can include, not least: calculating, computing, processing, deriving, measuring, investigating, identifying, looking up (for example, looking up in a table, a database, or another data structure), ascertaining and the like. Also, "determining" can include receiving (for example, receiving information), accessing (for example, accessing data in a memory), obtaining and the like. Also, " determine / determining" can include resolving, selecting, choosing, establishing, and the like.
[0193] In this description, reference has been made to various examples. The description of features or functions in relation to an example indicates that those features or functions are present in that example. The use of the term ‘example’ or ‘for example’ or ‘can’ or ‘may’ in the text denotes, whether explicitly stated or not, that such features or functions are present in at least the described example, whether described as an example or not, and that they can be, but are not necessarily, present in some of or all other examples. Thus ‘example’, ‘for example’, ‘can’, or ‘may’ refers to a particular instance in a class of examples. A property of the instance can be a property of only that instance or a property of the class or a property of a sub-class of the class that includes some but not all the instances in the class. It is therefore implicitly disclosed that a feature described with reference to one example but not with reference to another example, can where possible be used in that other example as part of a working combination but does not necessarily have to be used in that other example.
[0194] As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or” mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements. Although examples have been described in the preceding paragraphs with reference to various examples, it should be appreciated that modifications to the examples given can be made without departing from the scope of the claims.
[0195] Features described in the preceding description may be used in combinations other than the combinations explicitly described above.
[0196] Although functions have been described with reference to certain features, those functions may be performable by other features whether described or not.
[0197] The description of a feature, such as an apparatus or a component of an apparatus, configured to perform a function, or for performing a function, should additionally be considered to also disclose a method of performing that function. For example, description of an apparatus configured to perform one or more actions, or for performing one or more actions, should additionally be considered to disclose a method of performing those one or more actions with or without the apparatus.
[0198] Although features have been described with reference to certain examples, those features may also be present in other examples whether described or not.
[0199] The term ‘a’, ‘an’ or ‘the’ is used in this document with an inclusive not an exclusive meaning. That is any reference to X comprising a / an / the Y indicates that X may comprise only one Y or may comprise more than one Y unless the context clearly indicates the contrary. If it is intended to use ‘a’, ‘an’ or ‘the’ with an exclusive meaning then it will be made clear in the context. In some circumstances the use of ‘at least one’ or ‘one or more’ may be used to emphasis an inclusive meaning but the absence of these terms should not be taken to infer any exclusive meaning.
[0200] The presence of a feature (or combination of features) in a claim is a reference to that feature or (combination of features) itself and to features that achieve substantially the same technical effect (equivalent features). The equivalent features include, for example, features that are variants and achieve substantially the same result in substantially the same way. The equivalent features include, for example, features that perform substantially the same function, in substantially the same way to achieve substantially the same result.
[0201] In this description, reference has been made to various examples using adjectives or adjectival phrases to describe characteristics of the examples. Such a description of a characteristic in relation to an example indicates that the characteristic is present in some examples exactly as described and is present in other examples substantially as described.
[0202] The above description describes some examples of the present disclosure however those of ordinary skill in the art will be aware of possible alternative structures and method features which offer equivalent functionality to the specific examples of such structures and features described herein above and which for the sake of brevity and clarity have been omitted from the above description. Nonetheless, the above description should be read as implicitly including reference to such alternative structures and method features which provide equivalent functionality unless such alternative structures or method features are explicitly excluded in the above description of the examples of the present disclosure.
[0203] Whilst endeavoring in the foregoing specification to draw attention to those features believed to be of importance the Applicant may seek protection via the claims in respect of any patentable feature or combination of features hereinbefore referred to and / or shown in the drawings whether or not emphasis has been placed thereon. l / we claim:
Claims
35CLAIMS1. An apparatus comprising means for: determining an uncoded block of an image frame; searching the image frame to position coded-blocks within the image frame comprising: defining at least one seed coded-block at a position with respect to the uncoded block; defining a process which, when applied to a subject block, results in attempting to visit coded-blocks at pre-determined positions with respect to a position of the subject block; and applying the defined process recursively to visited coded-blocks as subject blocks, starting from the at least one seed coded-block as an initial subject block; and performing a coding operation on the current uncoded block using information from the positioned coded-blocks of the image frame.
2. An apparatus as claimed in claim 1, wherein the at least one seed coded-block is contiguous to the uncoded block.
3. An apparatus as claimed in claim 2, wherein three seed coded-blocks are defined.
4. An apparatus as claimed in claim 3, wherein the three seed coded-blocks comprise respectively: a first position, for a first seed coded-block, above a top-left corner of the uncoded block; a second position, for a second seed coded-block, on the left of the top-left corner of the uncoded block; a third position, for a third seed coded-block, above the second position.
5. An apparatus as claimed in any preceding claim, wherein attempting to visit coded- blocks at the pre-determined positions with respect to a position of the subject block comprises: for each of the blocks at the pre-determined positions, testing one or more exclusion conditions that prevent visiting a coded-block, wherein in response to a first determination comprising testing that an exclusion condition that prevents visiting the coded-block is satisfied, the block at the predetermined position is not visited,36 wherein in response to a second determination comprising that an exclusion condition that prevents visiting the coded-block is not satisfied, then the block at the predetermined position is visited, wherein the one or more exclusion conditions comprise one or more of: the subject block is outside a defined boundary; the subject block is an uncoded block; and the subject block has already been visited.
6. An apparatus as claimed in any preceding claim, wherein the visited coded-blocks at the pre-determined positions with respect to the position of the subject block, comprise neighboring blocks contiguous to the subject block.
7. An apparatus as claimed in any preceding claim, wherein the pre-determined positions with respect to a position of the subject block, are located relative to two different edges of the subject block.
8. An apparatus as claimed in any preceding claim, wherein there is a number of predetermined positions with respect to the position of the subject block, and the number is dependent upon features of the subject block.
9. An apparatus as claimed in any preceding claim, wherein the pre-determined positions with respect to a position of the subject block, comprise positions adjacent different edges of the subject block.
10. An apparatus as claimed in claim 9, wherein the pre-determined positions with respect to a position of the subject block, comprise positions adjacent different edges of the subject block that are maximally separated.
11. An apparatus as claimed in any preceding claim, wherein attempting to visit coded- blocks at the pre-determined positions with respect to the position of the subject block is performed in order based on one or more dimensions of the coded-block.
12. An apparatus as claimed in any preceding claim, wherein searching the image frame to position coded-blocks within the image frame comprises: defining ? multiple seed coded-blocks at positions with respect to the current uncoded block; for each seed coded-block:defining a process which, when applied to a subject block, results in attempting to visit coded-blocks at pre-determined positions with respect to a position of the subject block; applying the defined process recursively to all visited coded-blocks as subject blocks, starting from the seed coded-block as an initial subject block.
13. An apparatus as claimed in claim 12, wherein the multiple seed coded-blocks are contiguous to the uncoded block.
14. An apparatus as claimed in claim 12 or 13, wherein the multiple seed coded-blocks are contiguous with a single vertex of the uncoded block.
15. An apparatus as claimed in any preceding claim, wherein a vertex of the uncoded block logically divides the image frame into four quadrants comprising three searchable quadrants that are outside the uncoded block and a fourth quadrant occupied by the uncoded block;Wherein searching the image frame to position coded-blocks within the image frame comprises searching each quadrant of the image frame to position coded-blocks within the image frame comprising: defining at least one seed coded-block at a position with respect to the uncoded block in the quadrant; defining a process which, when applied to a subject block, results in attempting to visit coded-blocks at pre-determined positions with respect to a position of the subject block in the quadrant; and applying the defined process recursively to all visited coded-blocks as subject blocks, starting from the at least one seed coded-block as an initial subject block; and wherein the search in each quadrant uses a different search kernel having different pre-determined positions with respect to a position of the subject block.
16. An apparatus as claimed in any preceding claim, wherein the pre-determined positions with respect to a position of the subject block are defined by combinations of x-direction and y-direction unique to that quadrant.
17. An apparatus as claimed in claim 15 or 16, wherein the apparatus, comprises means that for the purposes of providing subject blocks, splits a block overlapping quadrants into distinct portions for each quadrant and uses the distinct portions as the subject blocks.
18. An apparatus as claimed in any preceding claim, wherein the information from the positioned coded-blocks of the image frame is extracted as a list of blocks visited during the search.
19. An apparatus as claimed in any preceding claim, wherein the image frame comprises a video frame.
20. An apparatus as claimed in any preceding claim, wherein the apparatus is configured for decoding an encoded image frame, the coding operation being a decoding operation in which an uncoded block of the image frame is a block that has not yet been decoded and a coded block of the image frame is a block that has been decoded.
21. An apparatus as claimed in any preceding claim, wherein the apparatus is configured for encoding an un-encoded image frame, the coding operation being an encoding operation in which an uncoded block of the image frame is a block that has not yet been encoded and a coded block of the image frame is a block that has been encoded.
22. A computer program that when run by one or more processors of an apparatus cause the apparatus to perform: determining an uncoded block of an image frame; searching the image frame to position coded-blocks within the image frame comprising: defining at least one seed coded-block at a position with respect to the uncoded block; defining a process which, when applied to a subject block, results in attempting to visit coded-blocks at pre-determined positions with respect to a position of the subject block; and applying the defined process recursively to all visited coded-blocks as subject blocks, starting from the at least one seed coded-block as an initial subject block; and performing a coding operation on the current uncoded block using information from the positioned coded-blocks of the image frame.
23. A method comprising: determining an uncoded block of an image frame; searching the image frame to position coded-blocks within the image frame comprising:defining at least one seed coded-block at a position with respect to the uncoded block; defining a process which, when applied to a subject block, results in attempting to visit coded-blocks at pre-determined positions with respect to a position of the subject block; and applying the defined process recursively to visited coded-blocks as subject blocks, starting from the at least one seed coded-block as an initial subject block; and performing a coding operation on the current uncoded block using information from the positioned coded-blocks of the image frame.
Citation Information
Patent Citations
Motion vector prediction
US20180359483A1
History-based motion vector predictor
US20200112741A1
Priority-based non-adjacent merge design
US20210127129A1
Methods and devices for candidate derivation for affine merge mode in video coding
WO2023192335A1