Context and feature based video encoding pre-analysis bit budget

Abstract

Systems, devices, and methods for bit budgeting in video encoding pre-analysis based on context and features are disclosed. A pre-encoder receives a video frame and evaluates each block of the frame for the presence of several context indicators. The context indicators can include skin tone, text, depth of field, and other specific objects. For each context indicator detected, a coefficient is generated and added to other coefficients to generate a final importance value for the block. The coefficients can be adjusted so that only defined portions of a picture are considered important. The final importance value for the block is used to determine the bit budget for the block. The block bit budget is provided to the encoder and used to influence the quantization parameter used to encode the block.

Description

Background Art

[0001] Description of Related Technology

[0002] Various applications perform encoding and decoding of image or video content. For example, video transcoding, desktop sharing, cloud gaming, and game viewing are some of the applications that include support for encoding and decoding of content. An encoder typically has a target bitrate that the encoder attempts to achieve when encoding a given video stream. The target bitrate roughly translates to a target bit size for each frame of the encoded version of a given video stream. For example, in one implementation, the target bitrate is specified in bits per second (e.g., 3 megabits per second (MBPS)), and the frame rate of the video sequence is specified in frames per second (FPS) (e.g., 60FPS, 24FPS). In this example implementation, the preferred bitrate is divided by the frame rate to calculate the preferred bit size of the encoded video frame. It is assumed here that the bitrate trajectory is linear. If it is not linear, a similar approach can be used to roughly estimate the preferred bit size of the encoded frame.

[0003] Each video frame is typically divided into multiple blocks. Examples of blocks include coding tree blocks (CTBs) used with the High Efficiency Video Coding (HEVC) standard or macroblocks used with the H.264 standard. Other types of blocks used with other types of video and image compression standards are also possible. The encoder can adjust how each block of the frame is encoded based on measured characteristics of the encoded block (e.g., level of detail, contrast, etc.). However, if the content of the frame is largely homogeneous, it is disadvantageous to apply adjustments made for one block to the entire frame. For example, the encoder may decide to allocate a higher bit budget for very detailed blocks. However, if most blocks are very detailed, the encoder will quickly run out of available bits in the budget. Natural scenes (e.g., forests, grasslands) and video games are typical examples where the entire picture or a large part of the picture is detailed and / or homogeneous. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Advantages of the methods and mechanisms described herein may be better understood by referring to the following description in conjunction with the accompanying drawings, in which:

[0005] Figure 1 is a block diagram of one implementation of a system for encoding and decoding content.

[0006] Figure 2 This is a block diagram of one implementation of a server.

[0007] Figure 3 is a block diagram of one implementation of a precoder coupled to a table.

[0008] Figure 4is a block diagram of one implementation of a precoder with multiple detectors.

[0009] Figure 5 is a block diagram of one implementation of a precoder that generates a significance table.

[0010] Figure 6 is a block diagram of one implementation of a precoder that generates a bit budget table.

[0011] Figure 7 One possible example of a frame analyzed by the precoder is shown.

[0012] Figure 8 One possible example of a frame analyzed by the precoder is shown.

[0013] Figure 9 is a generalized flow chart illustrating one implementation of a method for a precoder to generate a bit budget per block.

[0014] Figure 10 is a generalized flow chart illustrating one implementation of a method for adjusting context indicator coefficients for blocks of a frame based on the rarity of context indicators for the entire frame. DETAILED DESCRIPTION

[0015] In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, one of ordinary skill in the art will recognize that various implementations may be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques are not shown in detail to avoid obscuring the methods described herein. It should be understood that for the sake of simplicity and clarity of illustration, the elements shown in the figures are not necessarily drawn to scale. For example, the dimensions of some elements may be enlarged relative to other elements.

[0016] Disclosed herein are systems, devices, and methods for performing bit budgeting in video encoding pre-analysis based on context and features. In one implementation, a system includes a precoder and an encoder for encoding a video stream. In one implementation, the precoder receives a video frame and evaluates the presence of several context indicators in each block of the frame. The progressiveness of the blocks can be predefined or adaptively set. For each block, the precoder determines whether any of a number of different types of context indicators are present in the block. Context indicators can include, but are not limited to, memory colors, text, depth of field, and other specific objects. For each detected context indicator, a coefficient is generated and summed with other coefficients to generate a final importance value for the block. The coefficients are adjusted so that only defined portions of the picture are considered important, and blocks deemed important are allocated a specific percentage of the total available bit budget. The final importance value of the block is used to influence the block's bit budget, taking into account the final importance values ​​of other blocks. The block bit budget is provided to the encoder and used to influence, for example, the quantization parameters used to encode the block. Note that bit budgeting based on context indicators can be combined with one or more other techniques to determine the total bit budget to be used for blocks in a frame. In one implementation, the encoder selects a quantization strength (eg, a quantization parameter ("QP")) to use when encoding each block based on the bit budget assigned to the block by a precoder.

[0017] Now refer to Figure 1 , a block diagram illustrating one implementation of a system 100 for encoding and decoding content is shown. The system 100 includes a server 105, a network 110, a client 115, and a display 120. In other implementations, the system 100 includes multiple clients connected to the server 105 via the network 110, wherein the multiple clients receive the same bitstream or different bitstreams generated by the server 105. The system 100 may also include more than one server 105 for generating multiple bitstreams for multiple clients.

[0018] In one implementation, system 100 encodes and decodes video content. In various implementations, system 100 executes different applications, such as video game applications, cloud gaming applications, virtual desktop infrastructure applications, autonomous vehicle applications, online streaming applications, screen sharing applications, or other types of applications. In one implementation, server 105 renders video or image frames and then encodes these frames into a coded bitstream. In one implementation, server 105 includes a pre-encoder and an encoder for managing the encoding process. The pre-encoder may also be referred to herein as a "pre-analysis unit."

[0019] In one implementation, the precoder analyzes blocks of a frame to detect contextual indicators. As used herein, a "contextual indicator" is defined as a feature that is considered to be perceptually important to a particular application being executed by the system. In one implementation, contextual indicators include features such as signs, text, faces, bodies, everyday objects (e.g., cars, streets, streetlights), and memory colors. As used herein, a "memory color" is defined as a familiar color associated with the scene presented in the frame. An example of a "memory color" is skin color. In other implementations, other applications may have other types of contextual indicators.

[0020] For each block, an importance value is generated based on which context indicators are detected in the block and the scores assigned to the context indicators. The importance value is optionally adjusted based on one or more other variables, such as the rarity of the detected context indicators. The precoder assigns a bit budget to each block based on the importance value generated for the block. In one implementation, the importance value of the block is scaled so that the total number of bits assigned to all blocks is within the bit size range calculated based on the coded bitstream that meets the target bitrate. The encoder then encodes the block to match the bit budget assigned to the block by the precoder. In one implementation, the encoder adjusts the quantization parameter (QP) used to encode the block so that the coded block is within a threshold amount of the assigned bit budget. After generating the coded bitstream, the server 105 transmits the coded bitstream to the client 115 via the network 110. The client 115 decodes the coded bitstream and generates a video or image frame to drive to the display 120 or to the display synthesizer.

[0021] The network 110 represents any type of network or combination of networks, including wireless connections, direct local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), intranets, the Internet, wired networks, packet-switched networks, fiber-optic networks, routers, storage area networks, or other types of networks. Examples of LANs include Ethernet, Fiber Distributed Data Interface (FDDI) networks, and token ring networks. In various implementations, the network 110 includes remote direct memory access (RDMA) hardware and / or software, transmission control protocol / Internet protocol (TCP / IP) hardware and / or software, routers, repeaters, switches, power grids, and / or other components.

[0022] The server 105 includes any combination of software and / or hardware for rendering video / image frames and encoding the frames into bitstreams. In one implementation, the server 105 includes one or more software applications executed on one or more processors of one or more servers. The server 105 also includes network communication capabilities, one or more input / output devices and / or other components. The processor of the server 105 includes any number and type of processors (e.g., graphics processing unit (GPU), central processing unit (CPU), digital signal processor (DSP), field programmable gate array (FPGA), application specific integrated circuit (ASIC)). The processor is coupled to one or more memory devices that store program instructions that can be executed by the processor. Similarly, the client 115 includes any combination of software and / or hardware for decoding the bitstream and driving the frames to the display 120. In one implementation, the client 115 includes one or more software applications executed on one or more processors of one or more computing devices. In various implementations, the client 115 is a computing device, a game console, a mobile device, a streaming media player, or other types of devices.

[0023] Now go to Figure 2 , shows a block diagram of one implementation of components of a server 200 for encoding frames of a video. A new frame 210 of a video is received by the server 200 and provided to a pre-encoder 220 and an encoder 230. Each of the pre-encoder 220 and the encoder 230 is implemented using any suitable combination of hardware and / or software. In various implementations, software instructions for implementing the pre-encoder 220 and / or the encoder 230 are stored in a memory 240. The memory 240 represents any number and type of memory devices. In one implementation, the pre-encoder 220 generates a block bit budget 225 for a block of the new frame 210 based on an analysis of the block of the new frame 210. In one implementation, the rate controller 232 adjusts the block bit budget 225 generated by the pre-encoder based on the current budget conditions (i.e., whether the budget trajectory is on track). The encoder 230 then adjusts the encoding parameters when generating the encoded bitstream 235 so that the encoded block meets or approaches the assigned block bit budget 225. In one implementation, the components of the server 200 include ( Figure 1 It should be noted that in other implementations, the server 200 includes other components and / or Figure 2 Other suitable arrangements than those shown.

[0024] In one implementation, precoder 220 processes new frame 210 on a block-by-block basis. For each block, precoder 220 determines which context indicators are present in the block. In one implementation, context indicators include logos, text, and memory colors. In other implementations, precoder 220 searches for other types of context indicators. Precoder 220 generates a relative importance value for the block based on the context indicators detected in the block. The importance value is then used to generate a block bit budget 225, which is transmitted to encoder 230. In another implementation, precoder 220 provides a suggested bit budget to rate controller 232, which then adjusts the suggested bit budget to create a final bit budget for encoding the blocks of new frame 210. In another implementation, precoder 220 transmits the importance value to rate controller 232, which then assigns bit budgets to blocks based on their importance values.

[0025] In various implementations, the precoder 220 and the rate controller 232 work together in various different ways to determine the final bit budget for encoding the blocks of the new frame 210. In one implementation, the precoder 220 assigns block bit budgets 225 to the blocks, and the rate controller 232 does not perform any adjustments to the block bit budgets 225. In another implementation, the precoder 220 assigns block bit budgets 225 to the blocks, and the rate controller 232 then refines these block bit budgets 225 based on the current budget conditions. In yet other implementations, the rate controller 232 generates the bit budgets, and the precoder 220 provides guidance on adjusting the bit budgets for specific individual blocks. In other implementations, other techniques can affect the bit budget based on effects that the human visual system does not perceive well in certain situations, such as fast motion, areas with different motion vectors, etc.

[0026] In one implementation, the encoder 230 determines the quantization strength used to encode each block of the new frame 210 based on the bit budget assigned to the block. In other implementations, the encoder 230 adjusts other settings used when encoding each block of the new frame 210 based on the bit budget assigned to the block. In one implementation, the quantization strength refers to a quantization parameter (QP). It should be understood that when the term QP is used in this document, this term is intended to apply to other types of quantization strength settings used with any type of decoding standard. When encoding a given block, the encoder 230 selects a QP that will result in the bit size of the block closely matching the bit budget calculated by the precoder 220. Matching the bit budget assigned to the block by the precoder 220 allows the encoder 230 to meet the desired bit rate of the encoded bitstream 235.

[0027] Now refer to Figure 3 , shows a block diagram of one implementation of a precoder 305 coupled to a table 310. The precoder 305 is coupled to any number of tables 310 that specify various settings that will affect the generation of a bit budget for various frame blocks being encoded. For example, the context indicator table 310A specifies a plurality of context indicators that have high values ​​and will increase the relative importance of blocks of a frame that contain any of these context indicators. The context indicator table 310A also includes a score to be applied to each context indicator. Alternatively, in another implementation, a separate scoring table is included in the table 310 to specify which score to apply to each context indicator detected within a frame.

[0028] The memory color table 310B identifies specific memory colors that will also increase the relative importance of frame blocks containing more than a threshold amount of pixels of one of these memory colors. For example, in one implementation, skin color is identified as a memory color in one version of the memory color table 310B. In another implementation, blue sky is identified as a memory color in another version of the memory color table 310B. Other types of memory colors may also be specified in other implementations. The memory color table 310B may also specify that a memory color may have a larger value in the presence of a first attribute and / or that a memory color may have a smaller value in the presence of a second attribute. For example, in one implementation, skin color has a larger value in bright areas of the frame. Additionally, in one implementation, skin color has a smaller value in areas of fast motion. In another implementation, grass and trees are less important when affected by depth of field. Other examples of attributes that affect the relative values ​​of memory colors are possible and are contemplated.

[0029] In one implementation, a host computing system (e.g., Figure 1 Each application executed by the system 100 (e.g., a system 100) loads a new set of tables 310 for the precoder 305. For example, an autonomous vehicle application loads a first set of tables 310 for the precoder 305 to use when analyzing frames captured while executing the autonomous vehicle application. Additionally, a video game application loads a second set of tables for the precoder 305 to use when analyzing frames rendered by the video game application. Additionally, a video game streaming application loads a third set of tables for the precoder 305 to use when analyzing frames streamed by the video game streaming application. Other types of applications may also load a specific set of tables 310 for the precoder 305 that is optimized for the type of frames to be generated and / or captured. Additionally, a single application may load different sets of tables 310 for different stages of the application as the video content changes from one stage to another.

[0030] For example, in a video game application or a movie, a first table is loaded for the first scene of the video game or movie. The first scene may have a specific type of contextual indicator that is considered more important than other types of contextual indicators. Then, during the second scene, a second table with a new set of contextual indicators is loaded. This reloading of tables can continue for subsequent scenes of the video game or movie. In another implementation, an autonomous vehicle application loads different tables depending on the current situation. For example, when the vehicle is traveling at a relatively high speed on a highway, a first table of contextual indicators is loaded. Alternatively, a second table of contextual indicators is loaded in a residential area or near a school. Additional tables can be loaded when the autonomous vehicle encounters other situations (e.g., parking lots, highway entrance ramps, gas stations, charging stations, toll booths). To detect use case scenarios (e.g., detecting parking lots or highways), known methods can be used. The methods can be a combination of different analyses, such as analyzing GPS data and data from video analysis.

[0031] Now turn Figure 4 , a block diagram of one implementation of a precoder 410 having a plurality of detectors 415A-N is shown. In one implementation, the precoder 410 receives a frame 405 and performs a pre-analysis process on the frame 405. Figure 4 As shown, precoder 410 includes multiple detectors 415A-N. Note that detectors 415A-N are logical representations of detectors, where detectors 415A-N are implemented using any suitable combination of software and / or hardware. For example, in one implementation, each detector 415A-N is a trained neural network, where each trained neural network is designed to detect a specific type of contextual indicator. Furthermore, it should be understood that in some implementations, a single detector can perform the functions of multiple detectors 415A-N. For example, in another implementation, a single trained neural network is designed to detect multiple different types of contextual indicators.

[0032] In one implementation, each detector 415A-N is responsible for analyzing a block of frame 405 to determine whether the block contains a corresponding context indicator. For example, the first detector 415A searches for a logo in the blocks of frame 405, the second detector 415B searches for text in the blocks of frame 405, the third detector 415N searches for memory colors in the blocks of frame 405, and so on. In other implementations, the detectors 415A-N can search for other types of context indicators in the blocks of frame 405. After performing pre-analysis on frame 405 using the detectors 415A-N, the precoder 410 generates a results table 420 to record which context indicators were found in which blocks of frame 405. For example, in one implementation, the results table 420 includes a row for each block of frame 405, and each column of the results table 420 records that each block of the frame 405 corresponds to a particular detector 415A-N.

[0033] Result table 420 represents an example of the results of the pre-analysis phase for frame 405. As shown in result table 420, block 405A has "No" in columns 415A and 415N and "Yes" in column 415B. This indicates that block 405A of frame 405 contains a context indicator corresponding to detector 415B, but is not found to contain context indicators corresponding to detectors 415A and 415N. In addition, entries for blocks 405B-C are also shown in result table 420. It should be understood that result table 420 only represents one example of a result table. In other implementations, result table 420 can be constructed in other suitable ways. For example, in another implementation, result table 420 can include an importance value or metric in each field instead of "Yes" or "No". Alternatively, another table or matrix can be applied to result table 420 to convert "Yes" and "No" values ​​into importance values. The importance values ​​can then be converted into corresponding bit budgets by precoder 410 or by an encoder (not shown). In some cases, the precoder 410 performs cross-correlations between the columns of the results table 420 to increase or decrease the significance value, as appropriate, if one context indicator is found to be present when another context indicator is present.

[0034] Now refer to Figure 5 , a block diagram illustrating one implementation of a precoder 520 that generates an importance table 530 for a block of a frame. In one implementation, the precoder generates a result table 505 for a block of a frame, as in Figure 4 The precoder 520 then generates the importance table 530 by combining the values ​​retrieved from the score table 510 with the values ​​of the result table 505. For example, in one implementation, each row of the result table 505 has multiple fields, where each field includes "yes" or "no" to indicate whether the corresponding context indicator exists. An example of this type of result table is shown as ( Figure 4)Result table 420.

[0035] In one implementation, the scoring table 510 includes a score field 545 to apply to each "yes" value of a column for an entry of a given block. The scores are then summed to generate the importance values ​​shown for blocks 500A-C of the importance table 530. An example of a scoring table 510 according to one implementation is shown in Figure 5 In one implementation, each contextual indicator has a row with separate columns for entry field 535, contextual indicator field 540, score field 545, increment score if this contextual indicator is present field 550, and decrement score if this contextual indicator is present field 555.

[0036] In one implementation, if the context indicator specified in field 540 is present in the context indicators specified in field 550, the score applied to this context indicator should be increased. For example, if a memory color is found in a bright area of ​​the frame, the score in field 545 should be increased. The amount by which field 545 is increased can be a fixed amount (e.g., 10%), or in another implementation, the amount by which field 545 is increased can be specified in a column of table 510. Alternatively, if the context indicator specified in field 540 is present in the context indicators specified in field 550, the score should be decreased. For example, if a memory color is found in an area of ​​the frame with greater than a threshold amount of motion, the score in field 545 should be decreased for the memory color. The amount by which the score is decreased can be a fixed amount, specified in table 510, or specified in some other manner.

[0037] Blocks 500A-C represent blocks of a frame analyzed by pre-coder 520. In one implementation, importance table 530 is provided to an encoder (e.g., Figure 2 The encoder 230 allocates a bit budget to each block of the frame based on the value in the importance table 530 corresponding to the block. For example, the higher the value in the importance table 530 for a given block, the higher the bit budget allocated to the given block.

[0038] Now turn Figure 6 , shows a block diagram of one implementation of a precoder 620 that generates a bit budget table 630. In one implementation, rather than generating an importance table (e.g., importance table 530), the precoder 620 generates a bit budget table 630 based on the result table 605 and the score table 610. The bit budget table 630 is then provided to the encoder (e.g., Figure 2 The bit budget table 630 is similar to the bit budget table in Figure 5The importance table 530 is generated in the manner described in the discussion of FIG. As shown, blocks 600A-C each have a corresponding number of bits assigned to them based on their importance values. In some cases, the precoder 620 first generates the importance table and then uses the importance table to generate the bit budget table 630. In another implementation, the precoder 620 provides the importance table to the encoder, which then generates the bit budget table 630 based on the importance table.

[0039] Now refer to Figure 7 , shows an example of a frame 700 analyzed by a precoder, according to one implementation. Frame 700 is intended to represent an example of a video frame analyzed by a precoder. For the purposes of this discussion, assume that the precoder is searching for contextual indicators, such as logos and text. In one implementation, the host computing system is executing an autonomous vehicle application. In another implementation, the host computing system is executing a video game application. In other implementations, other types of host applications may be executed that may generate frames similar to frame 700.

[0040] like Figure 7 As shown, frame 700 includes logos 705, 710, and 715. In one implementation, in a first step of analysis by the precoder, logos 705, 710, and 715 are assigned higher importance due to being identified as logos and also identified as containing text. However, due to the presence of a large number of blocks containing logos and / or text, a rarity analysis of frame 700 will result in logos 705, 710, and 715 being assigned lower importance. Further analysis will assign the highest importance to logo 705 based on its focus. The other logos 710 and 715 being out of focus will result in their importance scores being lowered.

[0041] The area outside of the markers 705-715 includes trees and roads. Since the areas of the frame 700 with trees are considered busy (i.e., have a relatively high spatial frequency), this would cause a conventional algorithm to assign a relatively high importance and a relatively large bit budget to the blocks in these areas. However, in terms of where a user is likely to focus their attention in the frame 700, the trees are not as important as the markers 705-715. Therefore, using fewer bits in the bit budget to encode the blocks containing the trees results in less detailed information about the trees that the user is less likely to perceive or notice. Therefore, using more bits for the blocks containing the markers 705-715 would be a better approach. However, the importance of the markers 705-715 is different, so using Figures 4 to 6 The technique described in assigns a higher importance to blocks containing flag 705 than to blocks containing flags 710 - 715. When the encoder encodes the blocks containing flag 705, this higher importance will translate into a larger bit budget allocation for these blocks.

[0042] Now turn Figure 8 , shows an example of a frame 800 analyzed by a pre-encoder according to one implementation. Frame 800 is intended to represent an example of a video frame analyzed by a pre-encoder. As shown, most of frame 800 consists of trees and leaves. Since leaves, branches, etc. change color very frequently, these features tend to be detected as "busy." As a result, a typical encoder assigns a large number of bits to blocks containing trees and leaves. However, for frame 800, this would be a bit allocation error based on where the user is likely looking. The user's attention is likely to be focused on a sign 805. In one implementation, the sign 805 meets the criteria of three separate contextual indicators, where the first criterion is being a sign, the second criterion is containing text, and the third criterion is being in focus. Therefore, when a pre-encoder (e.g., Figure 2 When the precoder 220 of FIG. 80 analyzes the frame 800 using the techniques described herein, the block containing the flag 805 will be assigned a relatively high importance and will receive a relatively high share of the bit budget assigned to the frame 800 .

[0043] Now refer to Figure 9 , shows one implementation of a method 900 for generating a bit budget per block for a precoder. For discussion purposes, the steps and Figure 10 However, it should be noted that in various implementations of the described method, one or more of the described elements may be performed simultaneously, in a different order than shown, or omitted entirely. Other additional elements may also be performed as needed. Any of the various systems or devices described herein may be configured to implement method 900.

[0044] A precoder receives a frame to be encoded (block 905). The precoder analyzes the frame on a block-by-block basis to determine which context indicators are present in each block (block 910). Context indicators may vary depending on the implementation. In one implementation, the precoder uses one or more trained neural networks to detect different context indicators in the blocks. Next, the precoder generates a relative importance value for each block based on whether a specified context indicator is present and based on a score assigned to each context indicator (block 915). In one implementation, the context indicators are identified in a first table that is specific to a particular application executed by the host system. In one implementation, the scores associated with each context indicator are stored in a second table that is also specific to the particular application executed by the host system. The precoder then adjusts the importance value based on one or more other variables (block 920). For example, the other variables may include the rarity of a particular context indicator within the entire frame, the depth of field of the block, and / or other factors.

[0045] Next, the precoder subjects the bit budget assigned to the block to the influence of the importance value of each block (block 925). In one implementation, the precoder calculates a bit budget based on the importance value of each block and assigns the bit budget to the block. In another implementation, the precoder provides the importance value to a rate controller and / or encoder, and the rate controller and / or encoder calculates a bit budget based on the importance value of each block and assigns the bit budget to the block. The encoder then encodes the block to meet the bit budget assigned to the block (block 930). In one implementation, the encoder adjusts the quantization parameter (QP) used to encode the given block based on the bit budget assigned to the given block. In other implementations, the encoder adjusts other parameters so that the given block is encoded with a number of bits that matches or approaches the bit budget assigned to the given block. After block 930, method 900 ends. Note that method 900 can be used in conjunction with one or more other techniques for generating bit budgets or importance values ​​that influence how blocks of a frame are encoded.

[0046] Now turn Figure 10 , illustrates one implementation of a method 1000 for adjusting context indicator coefficients for a block of a frame based on the rarity of context indicators across the entire frame. A precoder detects a first context indicator in a given block of the frame (block 1005). Depending on the implementation, the first context indicator can be a logo, text, a memory color, or other context indicator. Next, the precoder determines whether the first context indicator has been detected in other blocks of the frame (block 1010). The precoder then generates coefficients for the given block that are either directly proportional or inversely proportional to the number of occurrences of the first context indicator in other blocks of the frame (block 1015). If the coefficients are inversely proportional, the less frequently the first context indicator is detected within the frame, the higher the coefficient value generated for the given block. Next, the precoder adds the coefficients to a total score generated for the given block (block 1020). In one implementation, the total score is used as an importance value for the given block. The total score is then used to calculate a bit budget for the given block (block 1025). After block 1025, method 1000 ends. Note that method 1000 may be performed for each context indicator of a plurality of context indicators and for each block of a frame.

[0047] In various implementations, program instructions of a software application are used to implement the methods and / or mechanisms described herein. For example, program instructions that can be executed by a general-purpose processor or a special-purpose processor are envisioned. In various implementations, such program instructions can be represented by a high-level programming language. In other implementations, the program instructions can be compiled from the high-level programming language into a binary form, an intermediate form, or other form. Alternatively, program instructions describing the behavior or design of the hardware can be written. Such program instructions can be represented by a high-level programming language such as C. Alternatively, a hardware design language (HDL) such as VERILOG can be used. In various implementations, the program instructions are stored on any one of a variety of non-transitory computer-readable storage media. During use, a computing system can access the storage medium to provide the program instructions to the computing system for program execution. Generally speaking, such a computing system includes at least one or more memories and one or more processors configured to execute the program instructions.

[0048] It should be emphasized that the above implementation is only a non-limiting example of an implementation. Once the above disclosure is fully understood, many variations and modifications will be apparent to those skilled in the art. The appended claims are intended to be interpreted as covering all such variations and modifications.

Claims

1. A device, comprising: A precoder, wherein the precoder is configured to: receiving a frame to be encoded, wherein the frame comprises a plurality of blocks; for each block of the frame, determining the presence of one or more context indicators, including a first context indicator corresponding to an initial importance value score and a second context indicator for which the initial importance value score is to be changed when the first context indicator and the second context indicator are both present; generating an importance value for each block based on the presence or absence of a specified context indicator within each block; assigning a bit budget to each block based at least in part on the importance value generated for the block; as well as An encoder is configured to encode each block to meet the bit budget assigned to the block.

2. The apparatus of claim 1 , wherein the precoder is further configured to: determining a rarity of each context indicator throughout the frame; and The importance value of each block is adjusted based on the rarity of each detected contextual indicator. The apparatus of claim 1 , wherein the contextual indicator comprises a memory color, text, and a logo. 4 . The apparatus of claim 1 , wherein the precoder is further configured to generate the importance value for each block based on a score assigned to each context indicator detected in the block. 5 . The apparatus of claim 4 , wherein the precoder is further configured to adjust a first score assigned to a first context indicator in response to detecting a second context indicator in the block.

6. The apparatus of claim 1, wherein the precoder is further configured to access a table that specifies a plurality of context indicators to be searched for in the plurality of blocks of the frame.

7. The apparatus of claim 1 , wherein the precoder is further configured to: receiving a first table specifying a first plurality of context indicators corresponding to a first application; In response to the first application generating a first frame to be encoded, searching the first frame for the first plurality of context indicators; receiving a second table specifying a second plurality of context indicators corresponding to a second application; as well as In response to the second application generating a second frame to be encoded, the second frame is searched for the second plurality of context indicators.

8. A method comprising: receiving, by a precoder, a frame to be encoded, wherein the frame comprises a plurality of blocks; for each block of the frame, determining the presence of one or more context indicators, including a first context indicator corresponding to an initial importance value score and a second context indicator for which the initial importance value score is to be changed when the first context indicator and the second context indicator are both present; generating an importance value for each block based on the presence or absence of a specified context indicator within each block; assigning a bit budget to each block based at least in part on the importance value generated for the block; as well as Each block is encoded by the encoder to meet the bit budget assigned to the block.

9. The method of claim 8, further comprising: determining a rarity of each context indicator throughout the frame; as well as The importance value of each block is adjusted based on the rarity of each detected contextual indicator.

10. The method of claim 8, wherein the contextual indicator comprises a memory color, text, and a logo.

11. The method of claim 8, further comprising generating the importance value for each block based on a score assigned to each context indicator detected in the block. 12 . The method of claim 11 , further comprising adjusting a first score assigned to a first context indicator in response to detecting a second context indicator in the block.

13. The method of claim 8, further comprising accessing a table specifying a plurality of context indicators to be searched for in the plurality of blocks of the frame.

14. The method of claim 8, further comprising: receiving a first table specifying a first plurality of context indicators corresponding to a first application; In response to the first application generating a first frame to be encoded, searching the first frame for the first plurality of context indicators; receiving a second table specifying a second plurality of context indicators corresponding to a second application; as well as In response to the second application generating a second frame to be encoded, the second frame is searched for the second plurality of context indicators.

15. A system comprising: Memory; as well as a precoder coupled to the memory, wherein the precoder is configured to: receiving a frame to be encoded, wherein the frame comprises a plurality of blocks; For each block of the frame, determining whether one or more context indicators exist; accessing a first data structure comprising a plurality of entries, wherein each of the plurality of entries identifies a first context indicator, an initial importance value score associated with the first context indicator, and a second context indicator indicating that the initial importance value score is to be changed when the first context indicator and the second context indicator are both present; generating an importance value for each block based on the presence or absence of a specified context indicator within each block; assigning a bit budget to each block based at least in part on the importance value generated for the block; and The bit budget is communicated to an encoder, and each block is encoded to meet the bit budget assigned to the block.

16. The system of claim 15, wherein the precoder is further configured to: determining a rarity of each context indicator throughout the frame; and The importance value of each block is adjusted based on the rarity of each detected contextual indicator. The system of claim 15 , wherein the contextual indicator comprises a memory color, text, and a logo.

18. The system of claim 15, wherein the precoder is further configured to generate the importance value for each block based on a score assigned to each context indicator detected in the block.

19. The system of claim 15, wherein the precoder is further configured to access a table that specifies a plurality of context indicators to be searched for in the plurality of blocks of the frame.

20. The system of claim 15, wherein the precoder is further configured to: receiving a first table specifying a first plurality of context indicators corresponding to a first application; In response to the first application generating a first frame to be encoded, searching the first frame for the first plurality of context indicators; receiving a second table specifying a second plurality of context indicators corresponding to a second application; as well as In response to the second application generating a second frame to be encoded, the second frame is searched for the second plurality of context indicators.

Images