Video encoding method, apparatus, computing device, medium, and program product

Abstract

The application provides a video coding method, device, computing equipment, medium and program product, and belongs to the field of video processing. The video coding method comprises: performing scalable video coding on a plurality of to-be-coded frames in a to-be-coded video to generate a first coded code stream; for any coded frame in the plurality of coded frames: determining the coding quality of the coding result of each quality level of the coded frame; in response to the coding quality of any coding result in the coding results of the plurality of quality levels satisfying a predetermined coding quality condition, determining the coding result of the quality level as a reference coding result; determining the reference number of layers corresponding to the reference coding result; determining at least one target coding result from the coding results of the plurality of quality levels; and generating a second coded code stream according to at least one target coding result included in the plurality of coded frames respectively. The method can improve the stability of video quality.

Description

Technical Field

[0001] This application relates to the field of video processing, and more particularly to a video encoding method, apparatus, computing device, medium, and program product. Background Technology

[0002] Currently, some video storage and transmission applications have high requirements for visual consistency and video quality. For these applications, constant quality encoding is typically used. Constant quality encoding maintains consistent visual quality throughout the video, rather than forcibly controlling the output bitrate or file size.

[0003] However, when faced with complex scenes or highly dynamic content, it is usually difficult to guarantee absolute stability of video quality, and video quality may fluctuate. Summary of the Invention

[0004] This application aims to at least address the technical problem of unstable video quality in the prior art. Therefore, one objective of this application is to provide a video encoding method to improve video encoding quality and enhance video quality stability.

[0005] An embodiment of the first aspect of this application provides a video encoding method, comprising: performing scalable video encoding on a plurality of frames to be encoded in a video to be encoded to generate a first encoded bitstream, wherein the first encoded bitstream includes a plurality of encoded frames corresponding one-to-one with the plurality of frames to be encoded, any one of the plurality of encoded frames includes encoding results of a plurality of quality levels, the encoding results of the plurality of quality levels including a base layer encoding result and a plurality of enhancement layer encoding results; for any one of the plurality of encoded frames: determining the encoding quality of each of the encoding results of the plurality of quality levels of the encoded frame; responding to the encoding results of the plurality of quality levels... If the coding quality of the coding result at any quality level meets the predetermined coding quality conditions, the coding result at that quality level is determined as the reference coding result; the reference layer number corresponding to the reference coding result is determined, and the reference layer number indicates the number of basic layers and enhancement layers corresponding to the reference coding result; based on the number of basic layers, enhancement layers, and reference layers corresponding to the coding results of multiple quality levels, at least one target coding result is determined from the coding results of multiple quality levels; and based on at least one target coding result included in multiple encoded frames, a second coded bitstream is generated, wherein the second coded bitstream is used to be transmitted to the target client.

[0006] In the technical solution of this application embodiment, by using scalable video coding to obtain basic layer coding results and enhancement layer coding results, and filtering from the coding results according to predetermined coding quality conditions, the bitstream quality output to the client can meet the expected target quality requirements and maintain high stability, thereby improving the video effect.

[0007] In some embodiments, in response to the coding quality of any one of the coding results of multiple quality levels satisfying a predetermined coding quality condition, determining the coding result of that quality level as the baseline coding result includes: determining, from the coding results of multiple quality levels, at least one quality level whose coding quality is greater than or equal to a predetermined coding quality threshold; determining at least one quality difference between the coding quality corresponding to the coding result of the at least one quality level and the coding quality threshold; determining the minimum quality difference from the at least one quality difference; and determining the coding result of the quality level corresponding to the minimum quality difference as the baseline coding result. Using the coding result with the minimum quality difference from the coding quality threshold as the baseline to filter the coding results of multiple quality levels obtained through coding can discard some unnecessary enhancement layers while maintaining video quality, reducing the coding bitrate, and improving coding efficiency.

[0008] In some embodiments, determining at least one target coding result from the coding results of multiple quality levels, based on the number of base layers and enhancement layers and the number of reference layers corresponding to the coding results of multiple quality levels, includes: determining the base layer coding result as the target coding result; and determining the enhancement layer coding result as the target coding result in response to any enhancement layer coding result having an enhancement layer number less than or equal to the number of reference layers. Using the base layer coding result and / or the enhancement layer coding result with a number of layers less than or equal to the number of reference layers as the target coding result can maintain a high level of stability in the quality of the final output encoded bitstream.

[0009] In some embodiments, the encoding quality conditions are predetermined based on the target client. Determining the corresponding encoding quality conditions according to different target clients allows the video encoding method to be applied to different target clients, ensuring that encoded bitstreams that meet the target quality requirements can be obtained for different clients, effectively improving the adaptability and flexibility of the video encoding process.

[0010] In some embodiments, determining the encoding quality of each of the encoding results for multiple quality levels of the encoded frame includes: determining the encoding quality of each of the encoding results for multiple quality levels corresponding to the encoding quality conditions, based on the encoding quality conditions. By determining different types of encoding quality for different clients' different encoding quality conditions, the encoding results can be filtered for different clients to obtain corresponding encoded bitstreams, effectively expanding the applicable scenarios of video encoding methods.

[0011] In some embodiments, the target client includes multiple sub-clients, the encoding quality conditions include multiple sub-encoding quality conditions corresponding to each of the multiple sub-clients, and the second encoded bitstream includes multiple second encoded sub-bitstreams corresponding to each of the multiple sub-clients. For any one of the multiple sub-clients: in response to the encoding quality of the encoding result of any one quality level among the multiple quality levels satisfying a predetermined encoding quality condition, determining the encoding result of that quality level as the baseline encoding result includes: in response to the encoding quality of the encoding result of any one quality level among the multiple quality levels satisfying the sub-encoding quality condition corresponding to that sub-client, determining the encoding result of that quality level as the baseline encoding result corresponding to that sub-client; determining the baseline layer number corresponding to the baseline encoding result includes: The process involves determining the base layer number corresponding to the base encoding result for the sub-client; determining at least one target encoding result from the encoding results of multiple quality levels based on the base layer and enhancement layer numbers and the base layer number corresponding to the encoding results of multiple quality levels; and generating a second encoded bitstream based on the at least one target encoding result included in multiple encoded frames. This process involves generating a second encoded sub-bitstream corresponding to the sub-client based on the at least one target encoding result included in multiple encoded frames, which is then transmitted to the sub-client. Different encoding quality conditions are determined for different clients, ultimately resulting in encoded bitstreams that meet the quality requirements of each client. This approach can be applied to multiple clients, requiring only one SVC encoding to output stable bitstreams for each client.

[0012] In some embodiments, the video coding method further includes: determining a plurality of key reference frames from a plurality of encoded frames, wherein any two adjacent key reference frames include a predetermined number of encoded frames; and for any one of the key reference frames: determining both the base layer coding result and the multiple enhancement layer coding results included in the key reference frame as the target coding result. By introducing key reference frames including all enhancement layers at a certain frequency, coding references can be provided for other coded frames, reducing the risk of encoding / decoding failure due to the lack of reference frames.

[0013] An embodiment of the second aspect of this application provides a video encoding apparatus, comprising: an encoding module, configured to perform scalable video encoding on a plurality of frames to be encoded in a video to be encoded to generate a first encoded bitstream, wherein the first encoded bitstream includes a plurality of encoded frames corresponding one-to-one with the plurality of frames to be encoded, any one of the plurality of encoded frames includes encoding results of a plurality of quality levels, the encoding results of the plurality of quality levels including a base layer encoding result and a plurality of enhancement layer encoding results; a quality determination module, configured to, for any one of the plurality of encoded frames, determine the encoding quality of each of the encoding results of the plurality of quality levels of the encoded frame; and a benchmark evaluation module, configured to, in response to the encoding results of the plurality of quality levels... If the coding quality of the coding result at any quality level meets the predetermined coding quality conditions, the coding result at that quality level is determined as the baseline coding result; the layer number determination module is used to determine the baseline layer number corresponding to the baseline coding result, the baseline layer number indicating the number of base layers and enhancement layers corresponding to the baseline coding result; the quality evaluation module is used to determine at least one target coding result from the coding results of multiple quality levels according to the number of base layers, enhancement layers and baseline layers corresponding to the coding results of multiple quality levels respectively; and the generation module is used to generate a second coded bitstream according to the at least one target coding result included in multiple coded frames respectively, wherein the second coded bitstream is used to be transmitted to the target client.

[0014] An embodiment of the third aspect of this application provides a computing device, including: at least one processor; and at least one memory communicatively connected to the at least one processor, the at least one memory storing instructions that, when executed individually or jointly by the at least one processor, cause the computing device to perform the video encoding method described above.

[0015] An embodiment of the fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed individually or jointly by one or more processors of a computing device, cause the computing device to perform the video encoding method described above.

[0016] An embodiment of the fifth aspect of this application provides a computer program product including instructions that, when executed individually or jointly by one or more processors of a computing device, cause the computing device to perform the video encoding method described above.

[0017] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0018] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0019] Figure 1 This is a flowchart illustrating a video encoding method according to some embodiments of this application;

[0020] Figure 2 This is a flowchart illustrating the process of determining the baseline encoding result in some embodiments of this application;

[0021] Figure 3 This is a flowchart illustrating the process of determining the target encoding result in some embodiments of this application;

[0022] Figure 4 This is a schematic diagram of a first encoded bitstream according to some embodiments of this application;

[0023] Figure 5 This is a schematic diagram of the second encoded bitstream of some embodiments of this application;

[0024] Figure 6 This is a schematic diagram illustrating the transmission of bitstream to the client in some embodiments of this application;

[0025] Figure 7 This is a schematic diagram of a first encoded bitstream according to some embodiments of this application;

[0026] Figure 8 This is a schematic diagram of the second encoded bitstream of some embodiments of this application;

[0027] Figure 9 This is a schematic diagram of a video encoding apparatus according to some embodiments of this application;

[0028] Figure 10 This is a schematic diagram of a computing device according to some embodiments of this application. Detailed Implementation

[0029] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0031] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0032] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0033] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0034] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0035] In the description of the embodiments of this application, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0036] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0037] Currently, some video storage and transmission applications have high requirements for visual consistency and video quality. For these applications, constant quality encoding (CQI) is typically used. CQI maintains consistent visual quality throughout the video, rather than forcibly controlling the output bitrate or file size. CQI is well-suited for preserving high-quality film and television content, educational recordings, or documentaries. In the field of surveillance video encoding, CQI also has unique advantages. Through a constant quality strategy, it ensures that key image areas maintain sufficient recognition quality even in complex scenes or under varying lighting conditions.

[0038] When applying constant quality coding, users can set a constant quality parameter based on the desired image quality. During encoding, the encoder automatically adjusts the bitrate of each frame to maintain the target quality. Constant quality coding is typically implemented using two strategies: Constant Quantization Parameter (CQP) and Constant Rate Factor (CRF). CQP typically uses a fixed quantization parameter to encode all frames, making it suitable for lossless coding or applications with very loose bitrate control. CRF, on the other hand, sets a target quality factor, allowing the encoder to automatically adjust the quantization parameter of each frame during encoding.

[0039] However, both CQP and CRF struggle to maintain high video quality stability in practical applications, especially when dealing with complex scenes or highly dynamic content. CQP encodes all frames using fixed quantization parameters, which can lead to over-encoding (wasting bitrate) in low-complexity scenes and over-compression in high-complexity scenes, resulting in significant quality degradation. CQP's visual quality is difficult to maintain consistently, particularly in dynamic scenes. CRF dynamically adjusts quantization parameters based on image complexity to maintain overall visual quality consistency. However, due to significant differences in complexity across different scenes, the encoder may still experience detail loss or noise in high-complexity regions (such as fast motion or high-detail textures). Furthermore, CRF's "constancy" is based on the encoder's perceptual model, rather than precise video quality evaluation metrics (such as Peak Signal-to-Noise Ratio (PSNR) or Structural Similarity Loss (SSIM),) so subjective video quality may still fluctuate.

[0040] To improve the stability of video encoding quality, Scalable Video Coding (SVC) can be used to encode the video. The core mechanism of SVC is a layered coding structure. Each encoded frame typically includes a base layer and several enhancement layers. The base layer provides the minimum usable video quality and has independent decoding capabilities, allowing video continuity to be maintained even under poor network conditions. The enhancement layers progressively improve video performance, such as resolution, frame rate, or image quality, on top of the base layer. For each encoded frame, there is a base layer encoding result and several enhancement layer encoding results. Each layer's encoding result has a corresponding encoding quality. Therefore, based on predetermined encoding quality conditions, a target encoding result that meets those conditions can be selected from these results to obtain a second encoded bitstream, which is then transmitted to the target client.

[0041] Since the target encoding results included in each frame can meet the encoding quality conditions, the second encoded bitstream transmitted to the target client can have stable video quality while meeting the encoding quality conditions, effectively improving the video effect.

[0042] This application provides a video encoding method. (See also...) Figure 1 The video encoding method 100 includes steps 110 to 160.

[0043] Step 110: Perform scalable video coding on multiple frames to be encoded in the video to be encoded to generate a first encoded bitstream. The first encoded bitstream includes multiple encoded frames that correspond one-to-one with the multiple frames to be encoded. Any one of the multiple encoded frames includes the coding results of multiple quality levels. The coding results of the multiple quality levels include the base layer coding result and multiple enhancement layer coding results.

[0044] For any given encoded frame among multiple encoded frames:

[0045] Step 120: Determine the coding quality of each of the coding results for the multiple quality levels of the encoded frame.

[0046] Step 130: In response to the fact that the coding quality of any one of the coding results of multiple quality levels meets the predetermined coding quality conditions, the coding result of that quality level is determined as the baseline coding result.

[0047] Step 140: Determine the number of reference layers corresponding to the reference coding result. The number of reference layers indicates the number of base layers and enhancement layers corresponding to the reference coding result.

[0048] Step 150: Based on the number of base layers and enhancement layers and the number of reference layers corresponding to the coding results of multiple quality levels, determine at least one target coding result from the coding results of multiple quality levels.

[0049] Step 160: Generate a second encoded bitstream based on at least one target encoding result included in each of the multiple encoded frames. The second encoded bitstream is used to be transmitted to the target client.

[0050] In the embodiments of this application, "coding quality" can be evaluated using different types of metrics, such as peak signal-to-noise ratio (PSNR), structural similarity loss (SSIM), and Visual Multimethod Assessment Fusion (VMAF). This application does not limit the evaluation metrics for coding quality.

[0051] In step 110, each frame in the original video, i.e., the video to be encoded, can be encoded using SVC. The resulting encoded frames will form the first encoded bitstream. As mentioned above, SVC encoding has a layered encoding structure, resulting in a base layer and several enhancement layers, and correspondingly, multiple quality levels of encoding results. Each encoded frame after encoding includes a base layer encoding result and several enhancement layer encoding results. The base layer encoding result has the lowest usable video quality. The enhancement layer encoding results improve the video quality layer by layer based on the base layer. The number of SVC encoding layers can be determined according to different usage requirements. Generally speaking, more layers mean finer video quality division, more stable encoding quality of the final bitstream, and closer to the target encoding quality. However, more layers also mean more computation and a certain loss of encoding efficiency. Therefore, the number of SVC encoding layers can be determined according to different usage requirements, such as different usage scenarios and different video quality requirements. This application does not limit the number of SVC encoding layers. In some embodiments, each layer can be encoded using CRF or CQP methods.

[0052] In step 120, the coding quality corresponding to the coding results of each quality level will be determined based on the selected evaluation metrics. For each encoded frame in the first coded bitstream, the coding quality corresponding to the coding results of each coding layer will be calculated by combining the same frames in the original video.

[0053] Figure 4 The image illustrates an example of the first encoded bitstream. For example... Figure 4 As shown, the first encoded bitstream includes five encoded frames, namely frames 1 to 5. SVC encoding has four layers, consisting of one base layer and three enhancement layers. Each encoded frame includes the encoding results of one base layer and the three enhancement layers. Figure 4 Using PSNR as an example, the coding quality corresponding to each layer of coding results is illustrated. Figure 4 As can be seen, the coding quality of the basic layer coding result is the lowest, while the coding quality of the enhancement layer coding result will increase layer by layer.

[0054] By pre-setting a coding quality condition, the target coding result in each frame must meet this condition, meaning the final output video must satisfy this coding quality condition, resulting in a more stable video quality. In some embodiments, the coding quality condition can be set by setting a coding quality threshold or a target coding quality; for example, each frame after encoding must reach this coding quality threshold or target coding quality.

[0055] exist Figure 4 In the example shown, the coding quality condition is that each frame needs to reach a coding quality threshold of 36.5 PSNR.

[0056] In some embodiments of this application, reference is made to Figure 2 Step 130 includes steps 210 to 240.

[0057] Step 210: Determine the coding result of at least one quality level from the coding results of multiple quality levels whose coding quality is greater than or equal to a predetermined coding quality threshold.

[0058] Step 220: Determine at least one quality difference between the coding quality and the coding quality threshold corresponding to the coding results of at least one quality level.

[0059] Step 230: Determine the minimum quality difference from at least one quality difference.

[0060] Step 240: The encoding result of the quality level corresponding to the minimum quality difference is determined as the baseline encoding result.

[0061] Since the coding quality of the base layer coding result and each enhancement layer coding result increases layer by layer, the difference between the coding quality of each coding result and the coding quality threshold will also increase layer by layer. The process of determining the baseline coding result can also be understood as determining the lowest coding result that reaches the coding quality threshold; that is, the coding result that is greater than or equal to the coding quality threshold and has the smallest difference from the coding quality threshold is taken as the baseline coding result. If the coding quality of the base layer coding result reaches the coding quality threshold, then the base layer coding result can be determined as the baseline coding result; if the coding quality of the base layer coding result does not reach the coding quality threshold, then the baseline coding result is the enhancement layer coding result that is greater than or equal to the coding quality threshold and has the smallest difference from the coding quality threshold.

[0062] By using the encoding result of the quality level with the smallest quality difference from the encoding quality threshold as the benchmark to filter multiple encoding results, some unnecessary enhancement layers can be discarded while maintaining video quality, thereby reducing the encoding bitrate and improving encoding efficiency.

[0063] Continue with Figure 4Let's take an example. For frame 1, the PSNR of the enhancement layer coding results corresponding to enhancement layers 1, 2, and 3 all reach 36.5. The enhancement layer coding result corresponding to the lowest enhancement layer, enhancement layer 1, with the smallest difference from 36.5 is used as the reference coding result for frame 1. For frame 2, the PSNR of the enhancement layer coding results corresponding to enhancement layers 2 and 3 reaches 36.5. The enhancement layer coding result corresponding to the lowest enhancement layer, enhancement layer 2, is used as the reference coding result for frame 2. For frame 3, the PSNR of the enhancement layer coding results corresponding to enhancement layers 1, 2, and 3 reaches 36.5. The enhancement layer coding result corresponding to the lowest enhancement layer, enhancement layer 1, is used as the reference coding result for frame 3. For frame 4, the PSNR of the enhancement layer coding result corresponding to enhancement layer 3 reaches 36.5. The enhancement layer coding result corresponding to enhancement layer 3 is used as the reference coding result for frame 4. For frame 5, the PSNR of the enhancement layer coding results corresponding to enhancement layers 2 and 3 reaches 36.5. The enhancement layer coding result corresponding to the lowest layer, enhancement layer 2, is used as the baseline coding result for frame 5. It should be understood that although in... Figure 4 The basic layer coding results of the five frames shown in the figure did not reach the coding quality threshold of 36.5. However, in other embodiments, if the basic layer coding result of a certain frame reaches 36.5, then the basic layer coding result of that frame is the baseline coding result.

[0064] In step 140, the base layer number corresponding to the base coding result for each encoded frame will be determined. For example... Figure 4 As shown, for frame 1, the reference layer is enhancement layer 1; for frame 2, the reference layer is enhancement layer 2; for frame 3, the reference layer is enhancement layer 1; for frame 4, the reference layer is enhancement layer 3; and for frame 5, the reference layer is enhancement layer 2. In other embodiments, if the reference coding result of a frame is the basic layer coding result, then the reference layer number of that frame is the basic layer.

[0065] In some embodiments of this application, step 150 includes: determining the base layer coding result as the target coding result; and in response to any enhancement layer coding result among a plurality of enhancement layer coding results having an enhancement layer number less than or equal to the base layer number, determining the enhancement layer coding result as the target coding result.

[0066] like Figure 4As shown, a corresponding base coding result is determined for each encoded frame. The base coding result, along with the coding results corresponding to each quality level below that layer, will be determined as the target coding result. For frame 1, the base layer coding result and the enhancement layer coding result corresponding to enhancement layer 1 will be determined as the target coding result. For frame 2, the base layer coding result and the enhancement layer coding results corresponding to enhancement layers 1 and 2 will be determined as the target coding result. For frame 3, the base layer coding result and the enhancement layer coding result corresponding to enhancement layer 1 will be determined as the target coding result. For frame 4, the base layer coding result and the enhancement layer coding results corresponding to enhancement layers 1, 2, and 3 will be determined as the target coding result. For frame 5, the base layer coding result and the enhancement layer coding results corresponding to enhancement layers 1 and 2 will be determined as the target coding result.

[0067] Using the encoding result of a quality level with a layer number less than or equal to the base layer number as the target encoding result can ensure that the quality of the final output encoded bitstream remains highly stable.

[0068] In step 160, a second encoded bitstream is obtained based on the target encoding result corresponding to each encoded frame. This second encoded bitstream is then output to the corresponding target client. Figure 4 The first encoded bitstream shown in the diagram, the resulting second encoded bitstream is as follows: Figure 5 As shown. From Figure 5 As can be seen, for each frame, the corresponding encoding result reaches the predetermined target encoding quality, and the fluctuation range of encoding quality for each frame is small. Therefore, relatively stable video quality can be achieved in the target client, effectively improving the video viewing experience.

[0069] By using scalable video coding to obtain base layer coding results and enhancement layer coding results, and filtering from the coding results according to predetermined coding quality conditions, the bitstream quality output to the client can meet the expected target quality requirements and maintain high stability, thereby improving video quality.

[0070] It should be understood that steps 110 to 160 can be performed after SVC encoding is completed, or they can be performed simultaneously during SVC encoding. If performed after SVC encoding is completed, the encoding results will be filtered according to predetermined video quality conditions, retaining only the target encoding result and discarding the encoding results of other layers. However, if performed simultaneously during SVC encoding, encoding of a frame can be stopped when a certain encoding result reaches the predetermined video quality conditions (i.e., the baseline encoding result is determined), and the obtained encoding result will be determined as the target encoding result for that frame.

[0071] In some embodiments of this application, the encoding quality conditions are predetermined based on the target client.

[0072] In some situations, encoded video may need to be transmitted to different clients. Different clients may have different requirements for encoding quality. For example, a video conferencing system may have multiple clients connected. Since each client may have different network bandwidth, screen size, and primary purpose (live viewing or long-term backup), their encoding quality requirements may also differ. For instance, different clients may use different evaluation metrics to assess video quality, or use the same type of evaluation metric but with different encoding quality thresholds. For example, client 1 might use PSNR for evaluation with a PSNR threshold of 36, client 2 might use SSIM with a threshold of 0.96, and client 3 might use VMAF with a threshold of 95.

[0073] Different encoding quality conditions can be determined for different clients. By determining the appropriate encoding quality conditions for different clients, the video encoding method can be applied to different clients, ensuring that the encoded bitstream meets the target quality requirements for each client. This effectively improves the adaptability and flexibility of the video encoding process.

[0074] In some embodiments of this application, step 120 includes: determining the coding quality of the coding results of multiple quality levels corresponding to the coding quality conditions based on coding quality conditions.

[0075] For different clients, the required encoding quality varies due to different encoding quality conditions. For example, if PSNR is used to evaluate encoding quality, the PSNR of the encoding result for each layer can be determined for each encoded frame; if SSIM is used, the SSIM of the encoding result for each layer can be determined for each encoded frame; and if VMAF is used, the VMAF of the encoding result for each layer can be determined for each encoded frame.

[0076] In some embodiments of this application, the target client includes multiple sub-clients. The encoding quality conditions include multiple sub-encoding quality conditions corresponding to each of the multiple sub-clients. The second encoded bitstream includes multiple second encoded sub-bitstreams corresponding to each of the multiple sub-clients.

[0077] For any one of the multiple sub-clients:

[0078] Step 130 includes: in response to the fact that the encoding quality of the encoding result of any one of the multiple quality levels satisfies the sub-coding quality condition corresponding to the sub-client, the encoding result of that quality level is determined as the baseline encoding result corresponding to the sub-client;

[0079] Step 140 includes: determining the base layer number corresponding to the base encoding result of the sub-client;

[0080] Step 150 includes: determining at least one target encoding result corresponding to the sub-client from the encoding results of multiple quality levels based on the number of base layers and enhancement layers corresponding to the encoding results of multiple quality levels and the number of reference layers corresponding to the sub-client.

[0081] Step 160 includes: generating a second encoded sub-stream corresponding to the sub-client based on at least one target encoding result corresponding to the sub-client included in each of the plurality of encoded frames. The second encoded sub-stream is used to be transmitted to the sub-client.

[0082] As mentioned above, the encoded video may need to be transmitted to different clients, meaning the target clients include multiple different sub-clients, and the encoding quality conditions for each sub-client may also be different. Figure 6 As shown, assuming sub-client 1 uses PSNR for evaluation, sub-client 2 uses SSIM for evaluation, and sub-client 3 uses VMAF for evaluation, then for the first encoded bitstream obtained after SVC encoding, the encoding quality corresponding to each of these three encoding quality conditions can be determined respectively. For sub-client 1, for each encoded frame, the PSNR of the encoding result corresponding to each layer can be determined. Based on the PSNR encoding quality condition, the target encoding result corresponding to sub-client 1 is determined using the method described above, resulting in the second encoded sub-bitstream 1, which is then output to sub-client 1. For sub-client 2, for each encoded frame, the SSIM of the encoding result corresponding to each layer can be determined. Based on the SSIM encoding quality condition, the target encoding result corresponding to sub-client 2 is determined using the method described above, resulting in the second encoded sub-bitstream 2, which is then output to sub-client 2. For sub-client 3, for each encoded frame, the VMAF of the encoding result corresponding to each layer can be determined. Based on the VMAF encoding quality condition, the target encoding result corresponding to sub-client 3 is determined using the method described above, resulting in the second encoded sub-bitstream 3, which is then output to sub-client 3. Using this method, only one SVC encoding is needed to output a stable bitstream for different clients.

[0083] By determining different encoding quality levels for different clients based on their varying encoding quality requirements, the encoding results can be filtered for each client to obtain the corresponding encoded bitstream, effectively expanding the applicable scenarios of video encoding methods.

[0084] In some embodiments of this application, the video encoding method 100 further includes a first process 300. (See reference...) Figure 3 The first process 300 includes steps 310 to 320.

[0085] Step 310: Determine multiple key reference frames from multiple encoded frames. A predetermined number of encoded frames are included between any two adjacent key reference frames.

[0086] Step 320: For any one of the multiple key reference frames: determine the basic layer coding result and multiple enhancement layer coding results included in the key reference frame as the target coding result.

[0087] In some embodiments, for SVC-encoded frames, key reference frames can be introduced at a certain frequency, that is, a key reference frame is determined after a certain number of frame intervals. Figure 7 In the example shown, frame 1 and frame n are key reference frames. Between two key reference frames are a predetermined number of ordinary coded frames, each of which can be encoded with reference to the previous key reference frame. For example... Figure 7 As shown, the basic layer coding results of frames 2, 3, and 4 can be obtained based on the basic layer coding results of frame 1 (the key reference frame). Similarly, the enhancement layer coding results corresponding to enhancement layers 1, 2, and 3 can also be obtained based on the enhancement layer coding results corresponding to enhancement layers 1, 2, and 3 in frame 1. Likewise, the basic layer coding results of frames (n+1), (n+2), and (n+3) can be obtained based on the basic layer coding results of frame n (the key reference frame).

[0088] For key reference frames, the coding results of all quality levels included in the frame are determined as the target coding results, rather than just the base coding results and the coding results of layers below the base layer. In this way, other coded frames can reliably use these key reference frames as reference frames, minimizing the risk of decoding failure due to the lack of reference frames.

[0089] for Figure 7In the example shown, continuing with the PSNR=36.5 encoding quality threshold mentioned above, frames 2, 3, 4, n+1, n+2, and n+3 are not key reference frames, and their corresponding target encoding results can be obtained using the method described above. However, for the two key reference frames, frame 1 and frame n, since the encoding results of other frames may depend on the encoding results of these key reference frames, it is necessary to retain the encoding results of all quality levels of the key reference frames. For example... Figure 7 As shown, although the encoding result of enhancement layer 1 in frame 1 has reached 36.5, and the encoding result of enhancement layer 2 in frame n has also reached 36.5, the basic layer encoding results and all enhancement layer encoding results of these two frames are retained as the target encoding results so that subsequent frames can obtain encoding results based on the key reference frame. For Figure 7 The first encoded bitstream shown in the diagram, the resulting second encoded bitstream is as follows: Figure 8 As shown.

[0090] Based on the same technical concept, embodiments of this application provide a video encoding apparatus. Embodiments of the video encoding apparatus can be referenced to embodiments of the video encoding method; repeated details will not be repeated. Reference Figure 9 The video encoding apparatus 700 includes an encoding module 710, a quality determination module 720, a benchmark evaluation module 730, a layer number determination module 740, a quality evaluation module 750, and a generation module 760.

[0091] Encoding module 710 performs scalable video encoding on multiple frames to be encoded in the video to be encoded, generating a first encoded bitstream. The first encoded bitstream includes multiple encoded frames that correspond one-to-one with the multiple frames to be encoded. Each encoded frame includes encoding results at multiple quality levels. The encoding results at multiple quality levels include base layer encoding results and multiple enhancement layer encoding results.

[0092] The quality determination module 720 is used to determine the coding quality of the coding results of multiple quality levels of any one of a plurality of coded frames.

[0093] The benchmark evaluation module 730 is used to determine the coding result of any quality level as the benchmark coding result in response to the coding quality of the coding result of any one of the coding results of multiple quality levels meeting a predetermined coding quality condition.

[0094] The layer number determination module 740 is used to determine the number of reference layers corresponding to the reference coding result. The reference layer number indicates the number of basic layers and enhancement layers corresponding to the reference coding result.

[0095] The quality assessment module 750 is used to determine at least one target coding result from the coding results of multiple quality levels based on the number of base layers and enhancement layers and the number of reference layers corresponding to the coding results of multiple quality levels respectively.

[0096] The generation module 760 generates a second encoded bitstream based on at least one target encoding result included in each of the multiple encoded frames. The second encoded bitstream is then transmitted to the target client.

[0097] The encoding module 710, quality determination module 720, benchmark evaluation module 730, layer number determination module 740, quality evaluation module 750, and generation module 760 in the video encoding apparatus 700 can correspond to steps 110 to 160 in the video encoding method 100, and will not be described in detail here for the sake of brevity. It should be understood that, corresponding to the embodiments of the video encoding method 100, embodiments of the video encoding apparatus 700 may also include more modules.

[0098] It should be noted that the functions of the modules discussed herein can be divided into multiple modules, and / or at least some functions of multiple modules can be combined into a single module. The specific actions performed by a particular module discussed herein include the specific module itself performing the action, or alternatively, the specific module calling or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, a specific module performing an action can include the specific module performing the action itself and / or another module that performs the action, called or otherwise accessed by the specific module.

[0099] It should also be understood that this article can describe various technologies in the general context of software and hardware components or program modules. The above regarding... Figure 9 The described modules can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuit. Hardware logic / circuit may include integrated circuit chips (which include processors (e.g., Central Processing Unit (CPU), microcontrollers, microprocessors, digital signal processors (DSPs), etc.), memory, one or more communication interfaces, and / or one or more components in other circuitry), and may optionally execute received program code and / or include embedded firmware to perform functions.

[0100] This application provides a computing device 800, such as... Figure 10 As shown. Figure 10 An example configuration of a computing device 800 that can be used to implement the video coding method 100 described herein is shown. For example, the video coding apparatus 700 described above may be implemented wholly or at least partially by the computing device 800 or a similar device or system.

[0101] The computing device 800 may include at least one processor 805 capable of communicating with each other, such as via a bus 804 or other suitable connection, a memory 807, multiple communication interfaces 802, a display device 801, other input / output (I / O) devices 803, and one or more mass storage devices 806. Instructions are stored on the memory 807 that, when executed by the processor 805, cause the processor 805 to perform the video encoding method as described in the above embodiments.

[0102] The computing device 800 can be a variety of different types of devices. Examples of the computing device 800 include, but are not limited to: desktop computers, server computers, laptop or netbook computers, mobile devices (e.g., tablet computers, cellular or other wireless phones (e.g., smartphones), notebook computers, mobile stations), wearable devices (e.g., glasses, watches), entertainment devices (e.g., entertainment appliances, set-top boxes communicatively coupled to a display device, game consoles), televisions or other display devices, automotive computers, and so on.

[0103] Processor 805 may be a single processing unit or multiple processing units, and all processing units may include single or multiple computing units or multiple cores. Processor 805 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Among other capabilities, processor 805 may be configured to acquire and execute computer-readable instructions stored in memory 807, mass storage device 806, or other computer-readable media, such as program code of operating system 808, program code of application program 809, program code of other program 810, etc.

[0104] Memory 807 and mass storage device 806 are examples of computer-readable storage media for storing instructions that are executed by processor 805 to perform the various functions described above. For example, memory 807 may generally include both volatile and non-volatile memory (e.g., RAM, ROM, etc.). Furthermore, mass storage device 806 may generally include hard disk drives, solid-state drives, removable media, including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network-attached storage, storage area networks, etc. Both memory 807 and mass storage device 806 may be collectively referred to herein as memory or computer-readable storage media, and may be non-transitory media capable of storing computer-readable, processor-executable program instructions as computer program code, which may be executed by processor 805 as a specific machine configured to perform the operations and functions described in the examples herein.

[0105] Multiple programs may be stored on mass storage device 806. These programs include operating system 808, one or more application programs 809, other programs 810, and program data 811, and they may be loaded into memory 807 for execution. Examples of such application programs or program modules may include, for example, computer program logic (e.g., computer program code or instructions) for implementing the following components / functions: video encoding apparatus 700 (including encoding module 710, quality determination module 720, benchmark evaluation module 730, layer number determination module 740, quality evaluation module 750, and generation module 760), video encoding method 100 (including any suitable steps of video encoding method 100), and / or other embodiments described herein.

[0106] Although Figure 10 The data is illustrated as being stored in memory 807 of computing device 800, but operating system 808, application program 809, other programs 810 and program data 811 or portions thereof may be implemented using any form of computer-readable medium accessible by computing device 800.

[0107] One or more communication interfaces 802 are used for exchanging data with other devices, such as via a network, direct connection, etc. Such communication interfaces can be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), wired or wireless (such as IEEE 802.11 Wireless LAN (WLAN)) wireless interface, Wi-MAX interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth™ interface, Near Field Communication (NFC) interface, etc. Communication interface 802 can facilitate communication across various network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. Communication interface 802 can also provide communication with external storage devices (not shown), such as storage arrays, network-attached storage, storage area networks, etc.

[0108] In some examples, a display device 801, such as a monitor, may be included for displaying information and images to the user. Other I / O devices 803 may be devices that receive various inputs from the user and provide various outputs to the user, and may include touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input / output devices, and so on.

[0109] The techniques described herein can be supported by these various configurations of computing device 800, and are not limited to specific examples of the techniques described herein. For example, the functionality can also be implemented wholly or partially on a “cloud” using a distributed system. A cloud includes and / or represents a platform for resources. The platform abstracts the underlying functionality of the cloud’s hardware (e.g., servers) and software resources. Resources may include applications and / or data that can be used when performing computational processing on servers remote from computing device 800. Resources may also include services provided via the Internet and / or via subscriber networks such as cellular or Wi-Fi networks. The platform can abstract resources and functionality to connect computing device 800 to other computing devices. Therefore, the implementation of the functionality described herein can be distributed throughout the cloud. For example, the functionality can be implemented partly on computing device 800 and partly through the platform that abstracts the functionality of the cloud.

[0110] This application also provides a computer-readable storage medium storing instructions that, when executed individually or jointly by one or more processors of a computing device, cause the computing device to perform the methods described in any of the above embodiments.

[0111] Computer-readable storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, Digital Universal Disc (DVD) or other optical storage devices, magnetic cassettes, magnetic tapes, disk storage devices or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by computer equipment.

[0112] This application also provides a computer program product including instructions that, when executed individually or jointly by one or more processors of a computing device, cause the computing device to perform the methods as described in any of the above embodiments.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application. In particular, as long as there is no structural conflict, the various technical features mentioned in the embodiments can be combined in any way. This application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A video encoding method, characterized in that, include: Scalable video encoding is performed on multiple frames to be encoded in the video to be encoded to generate a first encoded bitstream, wherein the first encoded bitstream includes multiple encoded frames that correspond one-to-one with the multiple frames to be encoded, and any one of the multiple encoded frames includes encoding results of multiple quality levels, the encoding results of the multiple quality levels including base layer encoding results and multiple enhancement layer encoding results. For any one of the plurality of encoded frames: Determine the coding quality of each of the multiple quality levels of the encoded frame; In response to the fact that the encoding quality of the encoding result of any one of the multiple quality levels meets a predetermined encoding quality condition, the encoding result of that quality level is determined as the baseline encoding result, wherein the encoding quality condition is predetermined according to the target client. Determine the reference layer number corresponding to the reference coding result, wherein the reference layer number indicates the number of base layers and enhancement layers corresponding to the reference coding result; Based on the number of base layers and enhancement layers corresponding to the encoding results of the multiple quality levels, and the number of reference layers, at least one target encoding result is determined from the encoding results of the multiple quality levels; and A second encoded bitstream is generated based on the at least one target encoding result included in each of the plurality of encoded frames. This second encoded bitstream is used to transmit to the target client, which includes a plurality of sub-clients. The encoding quality conditions include a plurality of sub-encoding quality conditions corresponding to each of the plurality of sub-clients. The second encoded bitstream includes a plurality of second encoded sub-bitstreams corresponding to each of the plurality of sub-clients. For any one of the plurality of sub-clients: The step of determining the encoding result of any one of the multiple quality levels as the baseline encoding result if the encoding quality of the encoding result of any one of the multiple quality levels meets a predetermined encoding quality condition includes: determining the encoding result of any one of the multiple quality levels as the baseline encoding result if the encoding quality of the encoding result of any one of the multiple quality levels meets the sub-encoding quality condition corresponding to the sub-client. Determining the base layer number corresponding to the base encoding result includes: determining the base layer number corresponding to the base encoding result of the sub-client; The step of determining at least one target encoding result from the encoding results of the multiple quality levels based on the number of base layers and enhancement layers corresponding to the encoding results of the multiple quality levels and the number of reference layers includes: determining at least one target encoding result corresponding to the sub-client from the encoding results of the multiple quality levels based on the number of base layers and enhancement layers corresponding to the encoding results of the multiple quality levels and the number of reference layers corresponding to the sub-client. The step of generating a second encoded bitstream based on the at least one target encoding result included in the plurality of encoded frames includes: generating a second encoded sub-bitstream corresponding to the sub-client based on the at least one target encoding result included in the plurality of encoded frames corresponding to the sub-client, the second encoded sub-bitstream being used to be transmitted to the sub-client.

2. The video encoding method according to claim 1, characterized in that, The coding quality of the coding result at any one of the multiple quality levels meets a predetermined coding quality condition, and the coding result at that quality level is determined as the baseline coding result, including: From the coding results of the plurality of quality levels, determine the coding result of at least one quality level whose coding quality is greater than or equal to a predetermined coding quality threshold; Determine at least one quality difference between the coding quality corresponding to the coding result of the at least one quality level and the coding quality threshold; Determine the minimum quality difference from the at least one quality difference; and The encoding result of the quality level corresponding to the minimum quality difference is determined as the baseline encoding result.

3. The video encoding method according to claim 1, characterized in that, The step of determining at least one target coding result from the coding results of the multiple quality levels based on the number of base layers and enhancement layers corresponding to the coding results of the multiple quality levels and the number of reference layers includes: The base layer encoding result is determined as the target encoding result; and If the number of enhancement layers corresponding to any of the plurality of enhancement layer encoding results is less than or equal to the reference number, the enhancement layer encoding result is determined as the target encoding result.

4. The video encoding method according to claim 1, characterized in that, The coding quality of each of the multiple quality levels of the coded frame is determined by: Based on the coding quality conditions, the coding quality of each of the multiple quality levels is determined, corresponding to the coding quality conditions.

5. The video encoding method according to any one of claims 1-3, characterized in that, The video encoding method further includes: Multiple key reference frames are determined from the plurality of encoded frames, wherein a predetermined number of encoded frames are included between any two adjacent key reference frames. For any one of the plurality of key reference frames: The basic layer coding results and the multiple enhancement layer coding results included in the key reference frame are all determined as the target coding results.

6. A video encoding device, characterized in that, include: An encoding module is used to perform scalable video encoding on multiple frames to be encoded in a video to be encoded, so as to generate a first encoded bitstream. The first encoded bitstream includes multiple encoded frames that correspond one-to-one with the multiple frames to be encoded. Each of the multiple encoded frames includes encoding results of multiple quality levels. The encoding results of the multiple quality levels include base layer encoding results and multiple enhancement layer encoding results. A quality determination module is used to determine the encoding quality of the encoding results of the multiple quality levels of any one of the multiple encoded frames, wherein the encoding quality conditions are predetermined according to the target client. The benchmark evaluation module is used to determine the coding result of any quality level as the benchmark coding result in response to the coding quality of the coding result of any one of the multiple quality levels meeting a predetermined coding quality condition. The layer number determination module is used to determine the number of reference layers corresponding to the reference coding result, wherein the number of reference layers indicates the number of basic layers and enhancement layers corresponding to the reference coding result; A quality assessment module is used to determine at least one target encoding result from the encoding results of the multiple quality levels based on the number of base layers and enhancement layers corresponding to the encoding results of the multiple quality levels, and the number of reference layers; and A generation module is configured to generate a second encoded bitstream based on the at least one target encoding result included in each of the plurality of encoded frames. The second encoded bitstream is used to be transmitted to the target client, wherein the target client includes multiple sub-clients, the encoding quality conditions include multiple sub-encoding quality conditions corresponding to each of the multiple sub-clients, and the second encoded bitstream includes multiple second encoded sub-bitstreams corresponding to each of the multiple sub-clients. For any one of the plurality of sub-clients: The step of determining the encoding result of any one of the multiple quality levels as the baseline encoding result if the encoding quality of the encoding result of any one of the multiple quality levels meets a predetermined encoding quality condition includes: determining the encoding result of any one of the multiple quality levels as the baseline encoding result if the encoding quality of the encoding result of any one of the multiple quality levels meets the sub-encoding quality condition corresponding to the sub-client. Determining the base layer number corresponding to the base encoding result includes: determining the base layer number corresponding to the base encoding result of the sub-client; The step of determining at least one target encoding result from the encoding results of the multiple quality levels based on the number of base layers and enhancement layers corresponding to the encoding results of the multiple quality levels and the number of reference layers includes: determining at least one target encoding result corresponding to the sub-client from the encoding results of the multiple quality levels based on the number of base layers and enhancement layers corresponding to the encoding results of the multiple quality levels and the number of reference layers corresponding to the sub-client. The step of generating a second encoded bitstream based on the at least one target encoding result included in the plurality of encoded frames includes: generating a second encoded sub-bitstream corresponding to the sub-client based on the at least one target encoding result included in the plurality of encoded frames corresponding to the sub-client, the second encoded sub-bitstream being used to be transmitted to the sub-client.

7. A computing device, characterized in that, include: At least one processor; as well as At least one memory communicatively connected to the at least one processor, the at least one memory storing instructions that, when executed individually or jointly by the at least one processor, cause the computing device to perform the video encoding method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The device stores instructions that, when executed individually or jointly by one or more processors of the computing device, cause the computing device to perform the video encoding method of any one of claims 1 to 5.

9. A computer program product, characterized in that, The instruction includes instructions that, when executed individually or jointly by one or more processors of a computing device, cause the computing device to perform the video encoding method of any one of claims 1 to 5.

Images