Content adaptive micro coding optimization for video image

Abstract

To optimize a coding parameter for a part of which a coding is identified for an instance of a content.SOLUTION: A method includes steps of: analyzing a location to which a flag is attached from a plurality of locations in a coding of a video image in order to form a cluster of the location; determining a draft micro chunk boundary to the cluster by searching a first start location and a first termination location in a coding; forming the micro chunk within a first searching range before the first start location and a second searching range after the first termination location; searching a second start location in the first searching range and a second termination location in a second searching range; determining a coding parameter on the basis of the micro chunk to be formed on the basis of a content characteristic of the micro chunk; and using the micro chunk for an insertion in a coding of the video image for coding.SELECTED DRAWING: None

Description

[Background technology]

[0001] A video delivery system may provide multiple instances of content (e.g., videos) that can be delivered to client devices. The instances of content are encoded for delivery to the clients. The video delivery system may use an encoding pipeline including an encoder that uses a predefined set of encoding parameters to encode the content. That is, the values ​​of the encoding parameters may be the same for all instances of the content encoded by the encoding pipeline. Typically, the encoding parameters are optimized for characteristics from the most well-known common content that may be encoded by the pipeline. For example, the encoding parameters may be based on characteristics of action movies, where action movies are considered to be the most common content being encoded. However, the video delivery system may have a vast library of content, which may include action movies, animated movies, television programs, nature programs, etc. A problem experienced is that for some content, or more specifically, for some portions of the same instances of content, the use of the same set of encoding parameters may produce sub-optimal encoding of the instances of content or portions of the instances of content.

[0002]

[0002] The included drawings are for illustrative purposes and serve only to provide examples of possible structures and operations for the disclosed inventive systems, apparatus, methods, and computer program products. These drawings in no way limit any changes in form and detail that may be made by those skilled in the art without departing from the spirit and scope of the disclosed implementations. [Brief description of the drawings]

[0003] [Figure 1]

[0003] FIG. 1 illustrates a simplified system for a video encoding pipeline, in accordance with some embodiments. [Diagram 2]

[0004] 1 illustrates an example of video encoding having good and flagged frames according to some embodiments. [Diagram 3]

[0005] 1 illustrates a simplified flowchart of a method for determining draft microchunk boundaries, according to some embodiments. [Figure 4]

[0006] 13 illustrates a second portion of determining microchunk boundaries according to some embodiments. [Diagram 5]

[0007] 1 illustrates an example of a search range for adjusting microchunk boundaries, according to some embodiments. [Figure 6]

[0008] 1 illustrates an example of performing video buffering verifier (VBV) usage-based filtering, according to some embodiments. [Figure 7]

[0009] 1 illustrates an example of scene change based filtering, according to some embodiments. [Figure 8]

[0010] 1 illustrates a first process for selecting an encoding parameter set according to some embodiments. [Figure 9]

[0011] 1 illustrates examples of predefined encoding parameter sets for different labels, according to some embodiments. [Figure 10]

[0012] 1 illustrates an example system for implementing a reinforcement learning process, according to some embodiments. [Figure 11]

[0013] 1 illustrates a video streaming system in communication with multiple client devices over one or more communication networks, according to one embodiment. [Figure 12]

[0014] 1 illustrates a diagram of a device for viewing video content and advertisements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0004]

[0015] Techniques for a video coding system are described herein. In the following description, for purposes of explanation, numerous examples and specific details are set forth to provide a thorough understanding of some embodiments. Some embodiments as defined by the claims may include some or all of the features in these examples, either alone or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.

[0005]

[0016] System Overview

[0006]

[0017] In some embodiments, the system may optimize the encoding parameters for the identified portions of the encoding for the instance of content. In the following, video is used as an example of the instance of content being encoded, but other types of content, such as audio or other content, may also be recognized. The quality control process may identify problematic portions of the encoding that do not meet a quality metric threshold. These frames may be referred to as flagged or "bad" frames in the video. Although the number of flagged frames may be small compared to the number of good frames that meet the quality metric threshold, there is no guarantee as to where the flagged frames may occur prior to encoding, or how the flagged frames may be clustered through the video. For example, the flagged frames may last for several seconds, or the flagged frames may occur for a fraction of a second. To address the problematic portions of the encoding, one solution may be to re-encode the entire video with different encoding parameters. However, this solution may have drawbacks. For example, re-encoding the entire video may waste computational resources. Since the encoding may include a large portion of good frames, re-encoding frames that were good frames may not be needed, wasting computational resources. Also, the quality control process may also need to be re-run for the entire encoding, which wastes additional computational resources. Furthermore, re-encoding the entire video may use the same set of revised encoding parameters, which may not address issues with all local portions where frames were flagged. For example, flagged frames may still fail the quality control process because these frames contain outlier characteristics of content compared to the average characteristics of the rest of the frames. In this case, if the revised encoding parameters are based on average characteristics of the video (as opposed to a general set of encoding parameters used initially for all videos), this would not necessarily guarantee to provide better quality encoding of the flagged frames.

[0007]

[0018] To improve the above, the system may automatically generate optimized encoding parameters for portions of the video that are flagged with quality control or other issues. The system may form clusters from the flagged frames. If multiple clusters of the video are formed, the system may optimize the encoding parameters for each cluster. This improves the encoding of the video because the system targets the encoding parameters for the characteristics of each cluster. In this process, the system searches for locations, such as frame indexes or numbers, of flagged video frames. A micro-chunk optimizer system generates micro-chunk boundaries, such as start and end frames in the video, based on the clusters of flagged frames. An encoding parameter tuner may analyze content characteristics for content associated with the micro-chunk boundaries and generate optimized encoding parameter sets for the micro-chunks. The system may also optimize micro-chunk boundary placement and encoding parameter set generation for the video segment using feedback linking the micro-chunk optimizer system and the encoding parameter tuner. The output may be optimized encoding parameter sets for each micro-chunk in the video. The micro-chunks may then be re-encoded using each respective optimized encoding parameter set. The resulting re-encoded micro-chunks may be inserted into the previously generated encoding using the micro-chunk boundaries to form the final encoding of the video.

[0008]

[0019] Although this discussion uses re-encoding of frames identified by the quality control process for encoding, this process may be used for other processes. For example, this process may be used to insert frames into video, such as in an interoperable master format (IMF) workflow that creates content for distribution. This process may encode the flagged portions of video using optimized encoding parameters for that portion. These portions may not fail the quality control process, but rather optimized encoding parameters are desired for the encoding process. Thus, the following process may be used for any process that requires inserting a portion of content into the encoding.

[0009]

[0020] system

[0010]

[0021] 1 illustrates a simplified system 100 for a video encoding pipeline in accordance with some embodiments. System 100 includes a quality control system 102, a micro-chunk optimizer system 104, an encoding parameter tuner 106, and a micro-encoder system 108. Entities in system 100 may be running on one or more computing devices, such as servers.

[0011]

[0022] The quality control system 102 may receive a video encoding. For example, an encoder may encode the video, and the quality control system 102 may analyze frames of the video encoding using a quality metric. The quality control system 102 may be an automated process that automatically analyzes characteristics of the encoding, or a manual quality control process in which a user analyzes the resulting encoding. The quality control system 102 may determine frames in the encoding that are flagged (e.g., problematic), such as frames that do not meet a threshold (e.g., a peak signal to noise ratio (PSNR) does not meet a threshold). Flagged frames are frames that do not meet a threshold, while other frames that meet the threshold are considered good frames and are not flagged.

[0012]

[0023] The quality control system 102 outputs the flagged frames to the microchunk optimizer system 104. The flagged frames may be identified by a list of frame locations for encoding. The flagged frames may be dispersed or interspersed throughout the duration of the video. The quality control system 102 may also provide information that may be used to optimize the encoding parameter set, for example, the quality control system 102 may indicate which quality control metric failed for each frame or the value of the metric for each frame. Although a frame is flagged, the use of the frame may include other portions of the encoding, such as a block of frames, multiple frames, a segment, etc.

[0013]

[0024] The micro-chunk optimizer system 104 receives the flagged frames as input. The micro-chunk optimizer system 104 may also receive other information related to the video, such as metadata from the original encoding statistics of encoding the video, or other metadata. Original encoding statistics of the frames in the encoding, such as bit rate, resolution, frame rate, etc. The other metadata may be information that may be useful for optimizing encoding parameters, such as type of content, segment duration configuration, group of pictures (GOP) structure, micro-chunk duration limits, etc. The micro-chunk optimizer system 104 uses the input to generate micro-chunk boundaries for clusters of flagged frames. For example, the micro-chunk optimizer system 104 may cluster a set of flagged frames and create micro-chunk boundaries for each cluster. Other optional information that the micro-chunk optimizer system 104 may output may be a target bit rate for each of the micro-chunk boundaries and a list of forced instantaneous decoder refresh (IDR) frames that were used to generate the micro-chunk boundaries. This information may be used to adjust encoding parameters or to re-encode the micro-chunks. The generation of micro-chunk boundaries is described in more detail below.

[0014]

[0025] The encoding parameter tuner 106 receives micro-chunk boundaries, which may be start and end frame indexes in the video or other location information. The encoding parameter tuner 106 also receives the source video and uses the frames associated with the micro-chunks to generate optimal encoding parameter sets for the micro-chunks found within the micro-chunk boundaries. The output may be an optimized encoding parameter set for each micro-chunk. In some embodiments, the encoding parameter tuner 106 may generate the encoding parameter sets using different modes, such as a predefined mode and a continuous learning mode. In the predefined mode, the encoding parameter tuner 106 may select a predefined set of characteristics from multiple predefined sets of characteristics using a content analyzer process that determines characteristics of the micro-chunks. The selected predefined set of characteristics is then used to determine the corresponding encoding parameters from a predefined list of encoding parameter sets. In the continuous learning process, the encoding parameter tuner 106 may use a learning-based process, such as a reinforcement learning-based process, that may automatically discover and learn optimal encoding parameter sets for the micro-chunks. When a revised or optimal coding parameter set is found, the values ​​for the coding parameter set may be revised. For example, the strength of the deblocking filter is a coding parameter that may be revised based on the content type (e.g., the parameters may need to be increased for content such as cartoons compared to live-action content). The coding parameter tuner 106 may also optimize the bit rate, optimize the coding frame type and quantization parameters used for the flagged frames in the micro-chunk, or perform other optimizations.

[0015]

[0026] If updates regarding micro-chunk boundaries are needed, the updates may be sent to the micro-chunk optimizer system 104, which can then generate new micro-chunk start and end frames. The new micro-chunk start and end frames are then provided to the encoding parameter tuner 106, which can then optimize the encoding parameter sets for the new micro-chunk. This iterative process may take multiple iterations until the best set of optimal micro-chunk boundaries and encoding parameter sets is determined.

[0016]

[0027] Once the encoding parameter set for each micro chunk is determined, the micro-encoder system 108 may re-encode the micro chunk according to the respective encoding parameter set. For example, the micro-encoder system 108 may receive a source video and encode a portion of the video according to the micro chunk start and end frames and in the respective encoding parameter set. The encoded micro chunks are then inserted during encoding using the micro chunk boundaries.

[0017]

[0028] The microchunk optimizer system 104 and the encoding parameter tuner 106 are described in more detail below.

[0018]

[0029] Microchunk Optimizer System

[0019]

[0030] The quality control process may output flagged frames, which may be interspersed throughout the video. The microchunk optimizer system 104 may receive the flagged frames and group the flagged frames into clusters. The microchunk optimizer system 104 then determines a start frame and an end frame for each cluster of flagged frames. The start and end frames may be locations, such as frame indexes, in the video and form a microchunk.

[0020]

[0031] FIG. 2 illustrates an example of a video coding 200 with flagged frames, according to some embodiments. The video coding 202 includes good frames at 202 and flagged frames at 204. The flagged frames are indicated with diagonal lines. The flagged frames in each instance may be a different number of frames, such as one frame, two or more frames, etc. Also, each instance of a flagged frame may be a different number of frames, even though the sizes shown in FIG. 2 may be somewhat similar. For example, the flagged frames may be for a fraction of a second, several seconds, etc. As shown, the flagged frames may be interspersed throughout the video coding 200 and may be less than the number of good frames. However, the flagged frames may include a greater number of frames than the good frames.

[0021]

[0032] The micro-chunk optimizer system 104 may generate micro-chunks by grouping together portions of flagged frames. A micro-chunk may be as long as necessary without having a length exceeding a threshold. That is, a micro-chunk may be limited in length to avoid having to re-encode large portions of the video together. A micro-chunk may contain content or video frames that may not share similar video characteristics, such as spatial and temporal characteristics, if the majority are contained in one micro-chunk. Smaller micro-chunks may have a higher probability of having similar characteristics compared to larger chunks.

[0022]

[0033] 3 and 4 illustrate examples of determining micro-chunk boundaries, according to some embodiments. FIG. 3 illustrates a simplified flowchart 300 of a method for determining draft micro-chunk boundaries, according to some embodiments. At 302, the micro-chunk optimizer system 104 receives a flagged frame list. The micro-chunk optimizer system 104 may process the flagged frames to group scattered but otherwise nearby frames into clusters. Different methods for clustering the flagged frames may be used. For example, at 304, the micro-chunk optimizer system 104 determines discontinuities in the flagged frame list to form groups. In some embodiments, the micro-chunk optimizer system 104 receives a discontinuity threshold that is used to determine whether flagged frames belong to the same group. In this process, the micro-chunk optimizer system 104 may start from the current flagged frame and form a boundary using a discontinuity threshold around the identified frame. The current frame may be selected as the first flagged frame from the beginning of the video, may be selected randomly, or may be user selected. Neighboring flagged frames may be included in a cluster if they meet the discontinuity threshold in distance from the current flagged frame. For example, if the current flagged frame is at index 100 and the threshold is 20 frames, the microchunk optimizer system 104 may determine whether there are any neighboring flagged frames within the boundary, which may be forward or backward. For example, any flagged frames within indexes 80 to 120 may be included in the group. The boundary may be extended as frames are added. For example, if a frame is added at an index of 120, the boundary may be extended to an index of 140 by adding the discontinuity threshold to the index of the newly added frame.This process continues until no further frames meet the threshold or the microchunk length is met. Upon forming a cluster, another flagged frame may be selected, such as a flagged frame that is just after the selected cluster. Although the above process is used to form clusters of flagged frames, other processes may be used, such as an automatic clustering process that receives flagged frame indices and automatically forms clusters based on different algorithms.

[0023]

[0034] Once clusters are found, at 306, the micro-chunk optimizer system 104 iterates over the clusters. For example, at 308, for each cluster, the micro-chunk optimizer system 104 determines draft micro-chunk boundaries. The draft micro-chunk boundaries may be based on different requirements. For example, a minimum micro-chunk duration may be required, which may be a minimum number of frames, or a time period, such as seconds. The minimum micro-chunk duration may be required to make the encoder's rate control stable or efficient. Another requirement may be a mandatory IDR frame requirement. For example, the IDR frame requirement may dictate that the start and end frames of a chunk must be at a shot boundary, which is a point of transition or change between consecutive shots in a video sequence. This requirement may be based on different encoding specification requirements. Other requirements may also be used to identify the draft micro-chunk boundaries. Upon determining the draft micro-chunk boundaries, the process continues to FIG. 4.

[0024]

[0035] FIG. 4 illustrates a second part of determining micro-chunk boundaries according to some embodiments. The second part may analyze the draft micro-chunk boundaries to determine whether the draft micro-chunk boundaries should be changed. At 402, the micro-chunk optimizer system 104 determines whether the draft micro-chunk is too long. This requirement may be used due to the desirability of optimizing encoding parameter sets for smaller portions of video that may have common characteristics. If the draft micro-chunk is too long, the characteristics associated with the long micro-chunk may change and initial problems with using the same encoding parameter set for the entire video may arise within the micro-chunk that is considered too long. Therefore, if the draft micro-chunk is too long, at 403, the micro-chunk optimizer system 104 determines a split point within the micro-chunk. Different methods may be used to determine the split point. For example, video buffer verifier (VBV) level, scene change statistics, fixed cadence / threshold based, etc. may be used to split the micro-chunk into multiple micro-chunks. Feedback from content characteristics from 408 may also be used to find splitting points within a micro-chunk. In some embodiments, VBV buffer level data from the original encoding may be used to determine locations to split long micro-chunks. VBV levels are used by the encoder to control bitrate and buffer occupancy of the compressed video stream during encoding and decoding. Areas of low VBV usage may be used as points to split micro-chunks. Scene statistics may be used to split long micro-chunks at scene changes that occur within the long micro-chunk. Other methods may also be used to split long micro-chunks into multiple micro-chunks, such as a set threshold that splits micro-chunks at a set number of frames.

[0025]

[0036] Once the appropriate size micro-chunk is determined, the process proceeds to finalize the micro-chunk start and end frames. The following steps may be performed sequentially, in parallel, or in a different order. A search range before the start frame and after the end frame of the draft micro-chunk boundary is used to determine if a new micro-chunk boundary should be determined. At 404, the micro-chunk optimizer system 104 performs VBV buffer usage based filtering. The micro-chunk optimizer system 104 may use VBV usage from the original encoding statistics to adjust the start or end frames of the micro-chunk boundary. As described in Figures 5 and 6, the micro-chunk optimizer system 104 may attempt to determine micro-chunk boundaries with low VBV usage. A threshold may be used to determine when usage is low (e.g., the VBV level is below a threshold). The VBV usage statistics is a measurement used during video encoding that may prevent buffer underflow or overflow. When the VBV buffer level of the original encoding is already high, any additional bits spent on microchunk encoding may result in a violation of the VBV buffer model, which may result in playback failure on the client. Therefore, it may be desirable to have a low bound on VBV usage to allow for overhead that can increase the bitrate in re-encoding without violating the VBV usage requirements. The output of the VBV buffer usage-based filtering may be a revised search range. Figures 5 and 6 explain the determination of the revised search range using VBV usage-based filtering in more detail.

[0026]

[0037] At 406, the micro-chunk optimizer system 104 performs scene change-based filtering. For example, the micro-chunk optimizer system 104 may search for scene changes within the revised search range to find where the micro-chunk can start or end at a new frame. Another filtering technique may be to ensure that the selected start or end frame is not in a busy action part of the video where many short scenes are expected. This may be measured by calculating the average distance between scene changes and the variance of the duration of these scenes. Selecting a start or end frame at the boundary of a new scene may improve the re-encoding of the micro-chunk since the content before the scene change and the content after the scene change may be different. From this, the optimized encoding parameter set for the content before the scene change may be different for the content after the scene change. The output of the scene-based filtering may be a ranked list of possible start and end frames from within the revised search range, or just one start and one end frame. Scene change-based filtering is explained in more detail in FIG. 7.

[0027]

[0038] At 408, the micro-chunk optimizer system 104 performs content characteristic-based filtering. This process may be optional or required. In this case, the content of the micro-chunk may be analyzed to determine whether the start frame or end frame should be changed. If there is any change in the content characteristics based on the micro-chunk boundaries, the micro-chunk optimizer system 104 may adjust the micro-chunk boundaries accordingly. For example, the micro-chunk optimizer system 104 may determine whether the draft micro-chunk contains more than one characteristic. If so, the micro-chunk optimizer system 104 may determine whether the micro-chunk can be separated and still be larger than the minimum chunk duration threshold. The micro-chunk optimizer system 104 may then separate the micro-chunks where the change in characteristics occurs, and then feedback is sent to perform VBV buffer usage-based filtering at 404 and scene change-based filtering at 406 again with the separated chunks. Also, as discussed below, content feature based filtering may be used to select one of the ranked lists of start and end frames.

[0028]

[0039] At 410, the micro-chunk optimizer system 104 adjusts and finalizes the micro-chunk start and end frames based on the analysis of 404, 406, and 408. At 412, the micro-chunk optimizer system 104 outputs the micro-chunk start and end frames, which can be used by the encoding parameter tuner 106.

[0029]

[0040] Before discussing the coding parameter tuner 106, VBV buffer usage based filtering and content characteristic based filtering will be discussed in more detail.

[0030]

[0041] FIG. 5 illustrates an example of a search range for adjusting microchunk boundaries, according to some embodiments. A draft microchunk is shown at 502. For the microchunk, a start frame boundary is shown at 504 and an end frame boundary is shown at 506. This forms a draft microchunk duration. A first search range 508-1 is found before the start frame and a second search range 508-2 is shown after the end frame. The microchunk optimizer system 104 may search different start or end frames for the microchunk within the search range. The search range may be determined using different methods. For example, a threshold may be used to form the search range, for example, the search range may be 100 frames before the start frame and 100 frames after the end frame. Other methods may also be used, such as a percentage of the draft microchunk duration being used to determine the search range, for example, if the draft microchunk duration is 200 frames, then 20% of the number of frames in the draft chunk duration may be taken as the search range, which may be 40 frames.

[0031]

[0042] Once the search range is determined, VBV usage-based filtering is performed. FIG. 6 illustrates an example of performing VBV usage-based filtering, according to some embodiments. The micro-chunk optimizer system 104 may determine areas of lower VBV usage using original encoding statistics. For example, areas of low VBV usage are shown in 602-1 before the start frame at 504 and 602-2 after the end frame at 506. In some embodiments, the micro-chunk optimizer system 104 defines lower VBV usage via a moving average with a window size of about half the buffer size length. In the example shown in FIG. 6, the specific window size may be set to 1 second (or 24 frames for 24 Hz video). The micro-chunk optimizer system 104 then designates the level of VBV usage for each successive second in categories such as low, medium, and high based on thresholds such as less than 0.5 being high, 0.5-0.85 being medium, and greater than 0.85 being low. The VBV buffer level here means how much of the buffer is still available. In this case, only 15% of the buffer is occupied. The micro-chunk optimizer system 104 may then find the areas where the longest streak of sustained low VBV usage is observed, for example, one area before the start frame and another area after the end frame. Although two areas are described, more than two areas may be found. Furthermore, in the unlikely event that there is no single area of ​​low VBV usage within the search range, the micro-chunk optimizer system 104 may expand the search range and perform a search within the expanded search range.

[0032]

[0043] A graph of VBV usage is shown at 606. The Y-axis is the VBV buffer level from the original encoding. The X-axis corresponds to the encoding of the video. Line 608 shows VBV usage values ​​and draft chunk durations corresponding to frames in the search range. At the time corresponding to the frame in 602-1, the VBV usage meets the low threshold. Also, in the frames found in 602-2, the VBV usage also meets the low threshold. In some embodiments, the microchunk optimizer system 104 determines two areas that contain the longest streaks of sustained low VBV usage, one area before the start frame 504 and one area after the end frame 506. That is, there may be other areas in the search range that contain low VBV usage but with the longest streaks used. In other embodiments, multiple areas of low VBV usage may be used. The streaks may be determined in different ways, such as the VBV usage value must meet a threshold, the average of VBV usage meets a threshold, etc. The micro-chunk optimizer system 104 may then search for boundaries in areas of low VBV usage. For example, the micro-chunk optimizer system 104 may use scene-change-based filtering to search for a start frame boundary in search range 602-1 and an end frame boundary in search range 602-2.

[0033]

[0044] 7 illustrates an example of scene change-based filtering, according to some embodiments. The microchunk optimizer system 104 attempts to search for scene changes within the adjusted search ranges 602-1 and 602-2. It may be preferable to set microchunk boundaries around new scene changes or not place microchunk boundaries around multiple short scene changes.

[0034]

[0045] The microchunk optimizer system 104 may define metrics as indicators of which scene change points can be selected based on inter-scene distance and intra-scene duration statistics (average, variance). In some embodiments, the microchunk optimizer system 104 uses the scene change associated with the longest scene (e.g., looking forward in time, but backward in time can also be used), and at the same time, with the furthest inter-scene distance from other scenes. For a given scene change, the intra-scene duration is defined as the length of forward time between this scene change and the next scene change. When this is the last scene before the end of the search range, the microchunk optimizer system 104 uses the end of the search range to calculate the scene duration. For a given scene change, the inter-scene distance may be defined as the average of the distances to two nearby scene change frames (e.g., backward and forward). For example, if a scene change is detected at frame 100 and two nearby scene changes are detected at frames 70 and 115, respectively, the average distance is calculated at 27.5 frames. When one of the nearby scene changes cannot be found in either direction until the end of the search range, the micro-chunk optimizer system 104 may use the distance to other scene changes directly (e.g., not the average). When there was only one scene change within the search range, the micro-chunk optimizer system 104 uses this scene change directly.

[0035]

[0046] In FIG. 7, a scene change is inserted in the search ranges 602-1 and 602-2 with a line indicated at 702. The search range 602-2 is then expanded to show the scene change. Any scene change detection algorithm may be used, such as frame, pixel, and histogram difference, or any other content analysis algorithm. Note that due to the short ranges typically used, there may be only one or no scene change detected in the search range 602-2. In this case, the micro-chunk optimizer system 104 may return the frame closest to the start or end frame of the draft micro-chunk period as the selected start or end frame if there is no scene change. If there is one scene change, the frames around the scene change may be returned.

[0036]

[0047] In the search range 602-2, the scene change at 704 may be selected. The microchunk optimizer system 104 selects the scene change at 704 because it is farther away from the short scene changes found at 706. Also, the scene change at 704 may maximize the in-scene duration from this scene change to the end of the search range 602-2. The scene change at 708 may not be selected even though it has a longer in-scene duration because it is close to other scene changes indicating that many short scenes are expected and may be a busy area of ​​the video. A busy area of ​​the video may not be a good area to select a scene change because the content may be changing vigorously and more frequently compared to the scene change at 704. A threshold distance from multiple scene changes (e.g., more than one scene change in a period) may be required to select a scene change. Also, instead of selecting a start frame and an end frame, a ranked list of preferred start and end frames may be provided for content feature-based filtering. If content feature based filtering is not performed, then scene change based filtering is used to output selected start and end frames for the microchunk boundaries.

[0037]

[0048] As discussed above, content feature-based filtering may adjust the start and end frames based on an analysis of the content features. For example, content feature-based filtering may first analyze the original draft micro-chunk duration. The micro-chunk optimizer system 104 may determine the content features of the original micro-chunk. If a micro-chunk includes multiple content features and the micro-chunk can be separated and still meet the minimum duration, the micro-chunk optimizer system 104 may separate the micro-chunk. The micro-chunk may then be reanalyzed using VBV usage-based filtering and scene change-based filtering.

[0038]

[0049] If no microchunks are separated, the microchunk optimizer system 104 analyzes the content before and after the original start frame. For example, 100 frames before the original start frame and 100 frames after the original start frame may be used. If it is determined that the content characteristics are different (e.g., the difference between the two portions before and after the start frame does not meet a threshold), the microchunk optimizer system 104 selects the top-ranked scene change point before the start frame. The same process is performed for the original end frame, where the microchunk optimizer system 104 analyzes the content before and after the original end frame. If it is determined that the content characteristics are different (e.g., the difference does not meet a threshold), the microchunk optimizer system 104 selects the top-ranked scene change point after the end frame. The new microchunk formed may be iteratively analyzed using the content characteristics again, and a scene change in the ranked list is selected until it is determined that the content characteristics before and after the boundary are similar. The output is a start frame and an end frame based on the content characteristic analysis. The content analysis checks whether the draft microchunk contains homogenous characteristics. This may mean that a majority of the frames in the draft microchunk (e.g., above a threshold) are identified with the same content label. If the content characteristics vary within a microchunk, the process may attempt to split and / or shorten the microchunk to contain mainly homogenous characteristics only.

[0039]

[0050] As discussed above, the order of VBV buffer usage-based filtering, content characteristic-based filtering, and content characteristic-based filtering may vary. In some embodiments, the filtering may be performed independently, and then a process may use the results to determine the start and end frames of a microchunk. For example, three processes may determine three different start and end frames. Then, the intersection of the three start frames may be used to determine the start frame, and the intersection of the three end frames may be used to determine the end frame. Also, the three start frames may be analyzed to select one of the start frames or a different start frame. The same process may be performed for the end frame.

[0040]

[0051] The following describes the coding parameter tuner operation after the start and end frames have been selected.

[0041]

[0052] Coding Parameter Tuner

[0042]

[0053] The coding parameter tuner 106 may use different methods to select the coding parameters. For example, methods using machine learning or deep learning may directly predict a set of coding parameters, or they may also first extract features (e.g., low-level or high-level features), which are then mapped to coding parameters. Below, two different methods are discussed, but other methods may be recognized. FIG. 8 illustrates a first process for selecting a coding parameter set, according to some embodiments. The coding parameter tuner 106 may use multiple neural networks, such as two neural networks, to determine the coding parameter set. The use of two neural networks is for illustrative purposes, and the method for classifying into different labels may be different, such as using a different number of neural networks. At 802, the coding parameter tuner 106 receives the source video and the microchunk start frame and end frame. Then, at 802, the following process may be performed for the frames in the microchunk from the start frame to the end frame. At 804, a first neural network #1 may be trained to identify whether a frame is within a predefined content category, such as hand-drawn animation, natural content, or other category. The categories are indicated with labels Content Category #1, Content Category #2, and Other Category. If the content cannot be classified into a predefined category, the other category is used. In some embodiments, the neural network may analyze the content of the frame and classify the content into one of the categories. In other embodiments, the first neural network may output a confidence score predicting whether the content falls into each of the respective categories.

[0043]

[0054] A second neural network #2 may be trained to identify different aspects of the characteristics of the content of the frame, such as film grain (e.g., noise). If film grain is used, the film grain aspects may be labeled as film grain category #1, film grain category #2, or other film grain categories. The film grain categories may be based on describing different film grain aspects, such as space-style film grain, TV sitcom-style film grain, etc.

[0044]

[0055] In some embodiments, the neural networks may be designed to work together. For example, if Neural Network #1 does not determine a content category (e.g., other categories are ranked highest), then Neural Network #2 may be run to determine a film grain category. In other embodiments, Neural Network #1 and Neural Network #2 may both be run regardless of classification.

[0045]

[0056] The categories may be associated with a predefined set of encoding parameters. The pooling frame result system 808 may pool the results for the frames in the microchunk to determine a list of labels for the microchunk. For example, Neural Network #1 or Neural Network #2 returns predictions of content characteristics one frame at a time. To obtain the dominant characteristics for the microchunk, the pooling frame result system 808 may use a pooling method, such as a consensus-based pooling method. In some embodiments, for each frame, the pooling frame result system 808 builds a dynamic list of predicted labels. The pooling frame result system 808 may then determine a list of labels for the microchunk based on a confidence level or the like. For example, the pooling frame result system 808 may process the labels for each frame, and when the cumulative confidence of the labels exceeds a threshold, such as 0.8, the pooling frame result system 808 outputs a list of labels. The pooling frame result system 808 then selects one or more of the labels from the list.

[0046]

[0057] In some embodiments, a label for a microchunk may be selected based on weighted votes, where the weight is determined by the reliability. For example, if a frame contains more than one label, the votes may separate reliability. That is, if two labels are chosen for a frame, one with a reliability of 0.6 and the other with a reliability of 0.2, the first label gets 3 / 4 of the votes and the other category gets 1 / 4 of the votes. A predicted label for a chunk may be chosen based on a majority vote. For example, the labels may be content category #1 (score 0.5), film grain category #1 (score 0.6), content category #2 (score 0.1), film grain category #2 (score 0.2), etc. The pooling frame results system 808 may select the film grain category #1 label because it has the highest reliability score.

[0047]

[0058] The coding parameter analyzer 810 then selects a coding parameter set based on the pooling frame result. For example, if one category, such as film grain category #1, is selected, the coding parameter analyzer 810 selects a predefined coding parameter set for film grain category #1.

[0048]

[0059] 9 illustrates examples of predefined encoding parameter sets for different labels, according to some embodiments. Column 902 shows the category labels, and in column 904 shows the encoding parameter sets.

[0049]

[0060] The categories that can be selected include category #1, category #2, category #3, category #4, and others. Each category label can be associated with a coding parameter set, such as coding parameter set #1, coding parameter set #2, coding parameter set #3, coding parameter set #4, and a default set. Other categories are associated with a default set of parameters since the content cannot be classified into a predefined category. The coding parameter sets can be different combinations of parameter values ​​based on the content characteristics of each of the categories. The different combinations can include cutree on and off, high VBV rate, low VBV rate, quantization parameter values, specific block partitioning modes, specific motion estimation modes, etc.

[0050]

[0061] The encoding parameter tuner 106 may also use a method of continuous learning to determine an optimal encoding parameter set. In some embodiments, a learning process (e.g., deep reinforcement learning) may be used. FIG. 10 illustrates an example of a system 1000 for implementing a reinforcement learning process, according to some embodiments. The process includes an agent 1002 and a micro-encoder system 108. In particular, for some encoding parameters, the encoding process may be modeled as a reinforcement learning problem, where the encoder runtime is modeled as the environment, the agent 1002 is fed by the encoder state, and a reward is calculated based on the instantaneous action (or a set of actions that constitute an episode) taken by the agent 1002. In some embodiments, the reward may be modeled by the quality of the encoding of a micro-chunk according to a bitrate target. The actions by the agent 1002 may be specific encoding parameters to be used, such as cuttree on / off, quantization parameter values, VBV usage, a particular block partitioning mode, a particular motion estimation mode, etc. In some embodiments, the agent 1002 uses a mathematical reward to optimize the quality. Different quality metrics (e.g., peak signal-to-noise ratio, structural similarity, or ideally, a more perceptually based metric) may be used to optimize the coding parameters. One example of a possible reward function is as follows:

[0051]

number

[0052]

[0062] Here, the overshoot may be defined by the delta between the actual bitrate and the target bitrate. Based on the reward value, a neural network trained in a reinforcement learning setting can then ideally learn the best encoding parameters for the micro-encoder system 108. In some embodiments, an asynchronous advantage actor critic architecture may be used that allows multiple agents to interact with the reinforcement learning environment. This allows the system to control and optimize multiple encoding parameters simultaneously and reach the optimum faster.

[0053]

[0063] The output of the reinforcement learning process may be a list of frame quantization parameters or other encoding parameters for the micro-chunk. The micro-encoder system 108 may then use the values ​​to encode the frame for the micro-chunk. The reinforcement learning process may continually learn the most optimal parameters based on the encodings encountered. That is, the agent 1002 may continually adjust the parameters in the encoding parameter set based on feedback from the micro-encoder system 108.

[0054]

[0064] Although the above method applies reinforcement learning at the frame level, the reinforcement learning process can be extended to the sub-frame or block level, or across multiple frames.

[0055]

[0065] conclusion

[0056]

[0066] Therefore, the process for re-encoding a portion of a video can be performed more efficiently and optimally, for example, chunk boundaries for the micro-chunks can be optimally determined, and parameters for the micro-chunks can also be optimally determined, which results in re-encoding the portion of a video in a more computationally efficient process with more optimal parameters.

[0057]

[0067] system

[0058]

[0068] Features and aspects as disclosed herein may be implemented in conjunction with a video streaming system 1100 in communication with multiple client devices over one or more communication networks, as shown in Figure 11. Aspects of the video streaming system 1100 are described merely to provide examples of applications for enabling distribution and delivery of content prepared in accordance with the present disclosure. It should be appreciated that the present technology is not limited to streaming video applications and may be adapted to other applications and delivery mechanisms.

[0059]

[0069] In one embodiment, a media programming provider may include a library of media programming. For example, the media programming may be aggregated and provided through a site (e.g., a website), an application, or a browser. A user may access a site or application of a media programming provider and request a media program. A user may be limited to requesting only media programs provided by the media programming provider.

[0060]

[0070] In the system 1100, video data may be obtained from one or more sources, such as from a video source 1110, for use as input to the video content server 1102. The input video data may comprise raw or edited frame-based video data in any suitable digital format, such as Moving Pictures Experts Group (MPEG)-1, MPEG-2, MPEG-4, VC-1, H.264 / Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), or other formats. Alternatively, the video may be provided in a non-digital format and converted to a digital format using a scanner or transcoder. The input video data may comprise various types of video clips or programs, such as television episodes, movies, and other content produced as the primary content of interest to consumers. The video data may also include audio, or only audio may be used.

[0061]

[0071] The video streaming system 1100 may include one or more computer servers or modules 1102, 1104, and 1107 distributed across one or more computers. Each server 1102, 1104, 1107 may include or be operatively coupled to one or more data stores 1109, such as databases, indexes, files, or other data structures. The video content server 1102 may access a data store (not shown) of various video segments. The video content server 1102 may provide the video segments as directed by a user interface controller that communicates with a client device. As used herein, a video segment refers to a distinct portion of frame-based video data, such as may be used in a streaming video session to view a television episode, a movie, a recorded live performance, or other video content.

[0062]

[0072] In some embodiments, the video ad server 1104 may access a data store of relatively short videos (e.g., 10-second, 30-second, or 60-second video ads) that are configured as advertisements for particular advertisers or messages. The advertisements may be offered for advertisers in exchange for some type of payment, or may comprise promotional messages, public service messages, or some other information for the system 1100. The video ad server 1104 may serve video ad segments as directed by a user interface controller (not shown).

[0063]

[0073] The video streaming system 1100 may also include the system 100.

[0064]

[0074] The video streaming system 1100 may further include an integration and streaming component 1107 that integrates video content and video advertisements into streaming video segments. For example, the streaming component 1107 may be a content server or a streaming media server. A controller (not shown) may determine the selection or configuration of advertisements in the streaming video based on any suitable algorithm or process. The video streaming system 1100 may include other modules or units not shown in FIG. 11, such as a management server, a commerce server, a network infrastructure, an advertisement selection engine, etc.

[0065]

[0075] The video streaming system 1100 may be connected to a data communications network 1112. The data communications network 1112 may comprise a local area network (LAN), a wide area network (WAN), such as the Internet, a telephone network, a wireless network 1114 (e.g., a wireless cellular telecommunications network (WCS)), or any combination of these or similar networks.

[0066]

[0076] One or more client devices 1120 may be in communication with the video streaming system 1100 via the data communications network 1112, the wireless network 1114, or another network. Such client devices may include, for example, one or more laptop computers 1120-1, desktop computers 1120-2, “smart” cell phones 1120-3, tablet devices 1120-4, network-enabled televisions 1120-5, or combinations thereof, via a router 1118 for a LAN, via a base station 1117 for the wireless network 1114, or via some other connection. In operation, such client devices 1120 may send or receive data or instructions to the system 1100 in response to user input or other input received from a user input device. In response, the system 1100 may provide video segments and metadata from the data store 1109 to the client device 1120 in response to a selection of a media program. Client device 1120 may output video content from streaming video segments in a media player using a display screen, projector, or other video output device, and may receive user input for interacting with the video content.

[0067]

[0077] Distribution of the audio-visual data may be performed in various ways, for example using streaming, from the streaming component 1107 to remote client devices through computer networks, telecommunications networks, and combinations of such networks. In streaming, a content server continuously streams the audio-visual data to a media player component operating at least partially on the client device, and the media player component may play the audio-visual data simultaneously upon receiving the streaming data from the server. Although streaming has been discussed, other distribution methods may be used. The media player component may begin playing the video data immediately after receiving the first portion of the data from the content provider. Traditional streaming techniques use a single provider that delivers a stream of data to a set of end users. To deliver a single stream to a large audience, high bandwidth and processing power may be required, and as the number of end users increases, the provider's required bandwidth may increase.

[0068]

[0078] Streaming media can be delivered on demand or live. Streaming allows for instant playback at any point in the file. End users can skip a media file to start playback or change playback to any point in the media file. Thus, end users do not have to wait for the file to download incrementally. Typically, streaming media is delivered from a small number of dedicated servers with high bandwidth capabilities via dedicated devices that accept requests for video files and deliver only the amount of data needed to play the video, along with information about the format, bandwidth, and structure of those files, at the speed needed to play it. The streaming media server may also take into account the transmission bandwidth and capabilities of the media player on the destination client. The streaming component 1107 may communicate with the client device 1120 using control and data messages to adapt to changing network conditions as the video is played. These control messages may include commands to enable control functions such as fast forward, fast rewind, pause, or seek to a particular part of the file at the client.

[0069]

[0079] Because the streaming component 1107 transmits video data only as needed and at the rate needed, precise control over the number of streams provided can be maintained. A viewer would not be able to watch high data rate video over a lower data rate transmission medium. However, a streaming media server (1) provides users with random access to video files, (2) allows monitoring of who is watching what video programs and how long they are being watched, (3) uses transmission bandwidth more efficiently because only the amount of data needed to support the viewing experience is transmitted, and (4) video files are not stored in the viewer's computer but discarded by the media player, thus allowing more control over the content.

[0070]

[0080] The streaming component 1107 may use TCP-based protocols such as HyperText Transfer Protocol (HTTP) and Real Time Messaging Protocol (RTMP). The streaming component 1107 may also deliver live webcasts and may perform multicasting, which allows more than one client to tune into a single stream, thus saving bandwidth. A streaming media player may not rely on buffering the entire video to provide random access to any point in the media program. Instead, this is accomplished using control messages sent from the media player to the streaming media server. Other protocols used for streaming are HTTP live streaming (HLS) or Dynamic Adaptive Streaming over HTTP (DASH). The HLS and DASH protocols deliver video over HTTP via a playlist of small segments that are typically made available at various bitrates from one or more content delivery networks (CDNs). This allows the media player to switch both bitrate and content source on a segment-by-segment basis. The switching helps to compensate for network bandwidth variances and infrastructure failures that may occur during the playback of the video.

[0071]

[0081] Delivery of video content through streaming can be accomplished under a variety of models. In one model, a user pays to view a video program, for example, by paying a fee for access to a library of media programs or a limited portion of a media program, or by using a pay-per-view service. In another model, widely adopted by television broadcasting soon after its inception, sponsors pay for the presentation of a media program in exchange for the right to present advertisements during or adjacent to the presentation of the program. In some models, advertisements are inserted at predetermined times during the video program, which may be referred to as "ad slots" or "ad breaks." With streaming video, media players may be configured to prevent a client device from playing the video without also playing predetermined advertisements during designated ad slots.

[0072]

[0082] Referring to Figure 12, a diagram of an apparatus 1200 for viewing video content and advertisements is illustrated. In selected embodiments, the apparatus 1200 may include a processor (CPU) 1202 operably coupled to a processor memory 1204 that holds binary coded functional modules for execution by the processor 1202. Such functional modules may include an operating system 1206 for handling system functions such as input / output and memory access, a browser 1208 for displaying web pages, and a media player 1210 for playing videos. The memory 1204 may hold additional modules not shown in Figure 12, such as modules for performing other operations described elsewhere herein.

[0073]

[0083] The bus 1214 or other communication components may support communication of information within the apparatus 1200. The processor 1202 may be a specialized or dedicated microprocessor configured or operable to perform particular tasks in accordance with the features and aspects disclosed herein by executing machine-readable software code that defines the particular tasks. The processor memory 1204 (e.g., a random access memory (RAM) or other dynamic storage device) may be connected to the bus 1214 or directly to the processor 1202 and may store information and instructions to be executed by the processor 1202. The memory 1204 may also store temporary variables or other intermediate information during the execution of such instructions.

[0074]

[0084] A computer readable medium in storage device 1224 is connected to bus 1214 and may store static information and instructions for processor 1202, for example, storage device (CRM) 1224 may store modules for operating system 1206, browser 1208, and media player 1210 when device 1200 is powered off, from which the modules may be loaded into processor memory 1204 when device 1200 is powered on. Storage device 1224 may include a non-transitory computer readable storage medium that holds information, instructions, or any combination thereof, for example, instructions that, when executed by processor 1202, configure or enable device 1200 to perform one or more operations of the methods as described herein.

[0075]

[0085] A network communication (comm.) interface 1216 may also be connected to the bus 1214. The network communication interface 1216 may optionally provide or support bidirectional data communication between the device 1200 and one or more external devices, such as the streaming system 1100, via a router / modem 1226 and a wired or wireless connection 1225. Alternatively or in addition, the device 1200 may include a transceiver 1218 connected to an antenna 1229, through which the device 1200 may communicate wirelessly with a base station for a wireless communication system or with the router / modem 1226. In the alternative, the device 1200 may communicate with the video streaming system 1100 via a local area network, a virtual private network, or other network. In another alternative, the device 1200 may be incorporated as a module or component of the system 1100 and communicate with other components via the bus 1214 or by some other modality.

[0076]

[0086] Apparatus 1200 may be connected (e.g., via bus 1214 and graphics processing unit 1220) to a display unit 1228. Display 1228 may include any suitable configuration for displaying information to an operator of apparatus 1200. For example, display 1228 may include or utilize a liquid crystal display (LCD), a touchscreen LCD (e.g., a capacitive display), a light emitting diode (LED) display, a projector, or other display device for presenting information to a user of apparatus 1200 in a visual display.

[0077]

[0087] One or more input devices 1230 (e.g., an alphanumeric keyboard, a microphone, a keypad, a remote control, a game controller, a camera, or a camera array) may be connected to the bus 1214 via a user input port 1222 for communicating information and commands to the apparatus 1200. In selected embodiments, the input device 1230 may provide or support control over cursor positioning. Such cursor control devices, also referred to as pointing devices, may be configured as a mouse, a trackball, a trackpad, a touch screen, cursor direction keys, or other devices for receiving or tracking physical movements and converting the movements into electrical signals indicative of cursor movement. The cursor control device may be incorporated into the display unit 1228, for example using a touch-sensitive screen. The cursor control device may communicate directional information and command selections to the processor 1202 to control cursor movement on the display 1228. The cursor control device may have two or more degrees of freedom, for example allowing the device to specify a cursor position in a plane or three-dimensional space.

[0078]

[0088] Some embodiments may be implemented in a non-transitory computer-readable storage medium for use by or in connection with an instruction execution system, apparatus, system, or machine. The computer-readable storage medium includes instructions for controlling a computer system to perform the methods described by some embodiments. The computer system may include one or more computing devices. The instructions, when executed by one or more computer processors, may be configured or operable to perform what is described in some embodiments.

[0079]

[0089] As used in this description and throughout the claims which follow, "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Also, as used in this description and throughout the claims which follow, the meaning of "in" includes "in" and "on," unless the context clearly dictates otherwise.

[0080]

[0090] The above description illustrates various embodiments, along with examples of how aspects of some embodiments may be implemented. The above examples and embodiments should not be considered the only embodiments, but are presented to illustrate the flexibility and advantages of some embodiments as defined by the following claims. Based on the above disclosure and the following claims, other arrangements, embodiments, implementations, and equivalents may be used without departing from the scope of the invention as defined by the claims.

Claims

1. 1. A method comprising: analyzing the flagged locations from a plurality of locations in the encoding of the video to form a cluster of locations, wherein the cluster includes at least one of the flagged locations; determining a draft microchunk boundary for the cluster based on searching a first start location and a first end location in the encoding that contains the cluster; searching within a first search range before the first start location and a second search range after the first end location for a second start location within the first search range and a second end location within the second search range, where the second start location and the second end location form a microchunk; determining a set of encoding parameters for the microchunk formed by the second start location and the second end location based on a content characteristic of the microchunk; using the encoding parameter set for encoding the microchunk for insertion in the encoding of the video; A method comprising:

2. The method of claim 1 , wherein the flagged location is flagged based on a quality metric of the flagged location meeting a threshold.

3. The method of claim 1 , wherein a quality control process analyzes the encoding of the video based on a quality metric and outputs locations for the flagged locations in the encoding of the video.

4. Analyzing the flagged locations includes:

2. The method of claim 1, comprising grouping a portion of flagged locations in the cluster when a distance between locations of the portion of flagged locations in the encoding of the video meets a threshold.

5. generating the first search range and the second search range based on a preset distance from the first start location or the first end location or a length of the cluster; The method of claim 1 further comprising:

6. Determining the draft microchunk boundaries comprises: The method of claim 1 , comprising determining the draft micro-chunk boundaries based on a minimum micro-chunk duration.

7. Searching within the first search range and the second search range includes: determining a first usage area of ​​a video buffer verifier usage satisfying a threshold within the first search range, wherein the video buffer verifier usage is determined during the encoding of the video; determining a second usage area of ​​the video buffer verifier usage that satisfies a threshold within the second search range; The method of claim 1 , comprising:

8. The method of claim 7 , wherein the first usage area and the second usage area have a lower video buffer verifier usage than another area of ​​the first search range or the second search range.

9. Searching within the first search range and the second search range includes: The method of claim 1 , comprising analyzing a scene change to select the second start location or the second end location.

10. Analyzing scene changes for a scene change, calculating an intra-scene distance for the scenes based on the scene change and an inter-scene distance between the scenes; selecting the scene change as the second start location or the second end location based on the intra-scene distance and the inter-scene distance; The method of claim 9 comprising:

11. Analyzing scene changes 10. The method of claim 9, comprising selecting the scene change as the second start location or the second end location based on distance from a plurality of scene changes that are not within a threshold distance from the scene change.

12. Determining the coding parameter set comprises: The method of claim 1 , comprising selecting a predefined category based on a content of the microchunk, wherein the encoding parameter set associated with the predefined category is used for the microchunk.

13. Determining the coding parameter set comprises:

2. The method of claim 1, further comprising: selecting a predefined category based on an aspect of a characteristic from a plurality of aspects of characteristics for the micro-chunk, wherein the encoding parameter set associated with the predefined category is used for the micro-chunk.

14. Determining the coding parameter set comprises: selecting a first predefined category based on the content of the microchunk; selecting a second predefined category based on the characteristic aspects from a plurality of characteristic aspects for the microchunk; 2. The method of claim 1, wherein the encoding parameter set associated with the first predefined category or the second predefined category is used for the micro chunk.

15. Determining the coding parameter set comprises: selecting a plurality of predefined categories for locations in the microchunk, each of the plurality of predefined categories being associated with a predefined encoding parameter set; combining the plurality of predefined categories to select a predefined category from the plurality of predefined categories; using the predefined encoding parameter set as the encoding parameter set for the microchunk; The method of claim 1 , comprising:

16. Determining the coding parameter set comprises:

2. The method of claim 1, comprising using a continuous learning process to select the encoding parameter sets, the continuous learning process continually learning the encoding parameter sets for use when encoding microchunks.

17. The continuous learning process includes: Receiving a state of an encoder encoding a microchunk; generating a reward for use in adjusting the coding parameter set; 20. The method of claim 16 comprising:

18. The method of claim 1 , wherein the flagged locations comprise flagged frames.

19. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a computing device, cause the computing device to: analyzing the flagged locations from a plurality of locations in the encoding of the video to form a cluster of locations, wherein the cluster includes at least one of the flagged locations; determining a draft microchunk boundary for the cluster based on searching a first start location and a first end location in the encoding that contains the cluster; searching within a first search range before the first start location and a second search range after the first end location for a second start location within the first search range and a second end location within the second search range, where the second start location and the second end location form a microchunk; determining a set of encoding parameters for the microchunk formed by the second start location and the second end location based on a content characteristic of the microchunk; using the encoding parameter set for encoding the microchunk for insertion in the encoding of the video; A non-transitory computer-readable storage medium operable to perform the steps of:

20. An apparatus comprising: one or more computer processors; A computer-readable storage medium; The computer-readable storage medium comprises: analyzing the flagged locations from a plurality of locations in the encoding of the video to form a cluster of locations, wherein the cluster includes at least one of the flagged locations; determining a draft microchunk boundary for the cluster based on searching a first start location and a first end location in the encoding that contains the cluster; searching within a first search range before the first start location and a second search range after the first end location for a second start location within the first search range and a second end location within the second search range, where the second start location and the second end location form a microchunk; determining a set of encoding parameters for the microchunk formed by the second start location and the second end location based on a content characteristic of the microchunk; using the encoding parameter set for encoding the microchunk for insertion in the encoding of the video; 23. An apparatus comprising: an apparatus comprising: one or more computer processors;

Images