A video encoding method, device, computer device and medium

Abstract

Embodiments of the present application disclose a video encoding method and device, computer equipment and a medium, wherein the method comprises: slicing a target video to obtain a plurality of slice videos of the target video; each slice video comprises a plurality of video frames in the target video in sequence; obtaining a region of interest in each slice video, and encoding and processing the corresponding slice video by using the region of interest of each slice video to obtain encoding data of each slice video; and combining the encoding data of each slice video to generate encoding data of the target video, so that the encoding effect of the video can be improved.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a video encoding method, apparatus, computer equipment, and medium. Background Technology

[0002] With the continuous development of computer technology, digital multimedia technology, and encoding / decoding technology, video and images have become increasingly prevalent in various fields and in people's daily lives. Video encoding requires setting specific encoding parameters. Currently, the same encoding parameters are typically used to encode the entire video. However, since different video frames usually contain different content, the required encoding parameters may vary. Using the same encoding parameters may result in poor encoding quality. Therefore, improving encoding quality has become a current research hotspot. Summary of the Invention

[0003] This application provides a video encoding method, apparatus, computer equipment, and medium that can effectively improve the encoding effect of video.

[0004] The first aspect of this application discloses a video encoding method, the method comprising:

[0005] The target video is sliced ​​to obtain multiple sliced ​​video segments for the target video; each sliced ​​video includes multiple consecutive video frames from the target video.

[0006] Obtain the region of interest in each video slice, and use the region of interest of each video slice to encode the corresponding video slice to obtain the encoded data of each video slice;

[0007] The encoded data of each video slice is combined to generate the encoded data of the target video.

[0008] A second aspect of this application discloses a video encoding apparatus, the apparatus comprising:

[0009] A slicing unit is used to slice a target video to obtain multiple sliced ​​videos for the target video; each sliced ​​video includes multiple consecutive video frames in the target video;

[0010] The encoding unit is used to obtain the region of interest in each video slice, and to encode the corresponding video slice using the region of interest of each video slice to obtain the encoded data of each video slice.

[0011] The combining unit is used to combine the encoded data of each video slice to generate the encoded data of the target video.

[0012] A third aspect of this application discloses a computer device including a processor and a memory, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute the method described in the first aspect.

[0013] A fourth aspect of this application discloses a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in the first aspect.

[0014] The fifth aspect of this application discloses a computer program product or computer program, the computer program product or computer program including program instructions, which, when executed by a processor, implement the method of the first aspect described above.

[0015] In this embodiment, the target video can be sliced ​​to obtain multiple sliced ​​videos. A region of interest (ROI) can be obtained from each sliced ​​video, and the ROI of each sliced ​​video can be used to encode the corresponding sliced ​​video, obtaining encoded data for each sliced ​​video. Furthermore, the encoded data of each sliced ​​video can be combined to generate the encoded data of the target video. By implementing the above method, a video can be divided into multiple sliced ​​videos during video encoding, facilitating subsequent encoding of each sliced ​​video. Compared to encoding the entire video, this embodiment provides more granular encoding, thereby improving the encoding effect. In addition to encoding each sliced ​​video, the ROI of each sliced ​​video can be further combined to encode the corresponding sliced ​​video, effectively utilizing the actual features (i.e., ROI) in each sliced ​​video for encoding, ensuring encoding reliability, and thus improving the encoding effect. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the framework of a video encoding system provided in an embodiment of this application;

[0018] Figure 2a This is a flowchart illustrating a video encoding method provided in an embodiment of this application;

[0019] Figure 2b This is a flowchart illustrating another video encoding method provided in an embodiment of this application;

[0020] Figure 3 This is a flowchart illustrating another video encoding method provided in an embodiment of this application;

[0021] Figure 4 This is a flowchart illustrating another video encoding method provided in an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the structure of a video encoding device provided in an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0025] This application proposes a video encoding scheme. The principle of this scheme is as follows: A target video can be sliced ​​to obtain multiple video slices. Each video slice is then encoded to obtain its encoded data. In one embodiment, a Region of Interest (ROI) can be obtained from each video slice. This ROI is then used to encode the corresponding video slice, yielding encoded data for each slice. After obtaining the encoded data for each video slice, these data can be combined to generate the encoded data for the target video. The ROI can generally be understood as the area within a video frame that is most visually appealing to the viewer; simply put, the ROI is the area that the user is most interested in. For example, in a video frame containing a person, the user's focus is on the person in the frame.

[0026] By implementing the above method, a video can be divided into multiple video slices during video encoding, so that each video slice can be encoded subsequently. Compared with encoding the entire video, the embodiments of this application can provide more granular encoding, thereby improving the encoding effect. On the basis of encoding each video slice, the corresponding video slice can be encoded by combining the region of interest in each video slice, so as to effectively utilize the actual features (i.e. region of interest) in each video slice for encoding, ensuring encoding reliability, and thus improving the encoding effect.

[0027] In practical implementation, the execution entity of the video encoding scheme mentioned above can be a computer device, which can be a terminal or a server. The terminal mentioned here can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, etc.

[0028] It should be noted that when the computer device is a server, this application embodiment provides a video encoding system, such as... Figure 1 As shown, the video encoding system includes at least one terminal and at least one server; the terminal can acquire a target video and upload the acquired target video to the server (i.e., computer device), so that the computer device can encode the target video based on the video encoding scheme to obtain the encoded data corresponding to the target video.

[0029] Based on the video encoding scheme provided above, this application embodiment provides a video encoding method that can be executed by the aforementioned computer device. Please refer to... Figure 2a The video encoding method includes, but is not limited to, the following steps:

[0030] S101, slice the target video to obtain multiple slice videos for the target video.

[0031] The target video can refer to the video to be encoded, and it can be of any type or length without limitation. Any video slice can include multiple consecutive video frames from the target video.

[0032] It is understandable that different video frames in the same video usually have multiple different scenes. For example, a video about a school may include scenes such as a playground, classroom, and canteen, while a video about travel may include scenes such as the sea and city. Different scenes also have different effects on the characteristics and complexity of video frames. Based on this factor, the embodiments of this application consider adapting different encoding parameters to video frames of different scenes in order to effectively improve the encoding effect of the video.

[0033] Based on this, step S101 can be specifically implemented as follows: the computer device can obtain the scene type of video frames in the target video. In one embodiment, the scene type of each video frame in the target video can be obtained here, so that the target video can be sliced ​​based on the scene type of each video frame later; in another embodiment, considering that there are many video frames in a video, obtaining the scene type of each video frame may consume a lot of time and computational overhead, resulting in low encoding efficiency. Therefore, only the scene types of some video frames in the target video can be obtained here, so as to ensure the reliability of slicing using scene types while minimizing the time and computational overhead to process the acquisition operation here. In a feasible implementation, the target video frame can be determined from the target video by target frame extraction, and the scene type of the target video frame can be obtained; the target video frame here is a portion of the video frames in the target video. The target frame extraction method can be based on a time interval or a frame interval, etc. The time interval can be 1 second, 3 seconds, etc., and its specific value is not limited. For example, with a time interval of 1 second, video frames are extracted from the target video at 1-second intervals, and the extracted video frames are the target video frames. The frame interval can be 1 frame, 2 frames, etc., and its specific value is not limited. For example, with a frame interval of 2 frames, video frames are extracted from the target video at 2-frame intervals, and the extracted video frames are the target video frames.

[0034] The scene type of the video frame can be obtained by calling a neural network model with scene type recognition function. In one embodiment, the computer device can input the target video into the neural network model to call the neural network model to recognize the scene type of the video frame in the target video and obtain the scene type of the video frame in the target video.

[0035] After obtaining the scene types of the video frames in the target video, the computer device can slice the target video based on these scene types to obtain sliced ​​video. It's also important to understand that in video encoding, the sequence of video frames is typically divided into several GOPs (Groups of Pictures), and each video frame within each GOP is encoded to achieve video encoding. A GOP is a set of pictures in a video frame sequence, representing a group of consecutive video frames. Therefore, a target video can contain one or more coded image groups (picture groups), and any coded image group can contain multiple consecutive video frames from the target video. Thus, slicing the target video can also be done by slicing each coded image group separately. The following explanation uses any coded image group in the target video as an example to illustrate the slicing process.

[0036] In one implementation, for any group of coded images in the target video, the computer device can slice the group of coded images according to a segmentation rule that divides consecutive video frames of the same scene type into a video slice, thereby obtaining a video slice of the group of coded images. In other words, the computer device can divide consecutive video frames of the same scene type into a video slice, meaning that consecutive video frames of the same scene type can be considered as a video slice.

[0037] For example, the video frames in the encoded image group are in the form of: AAAABBBBBBBBCCCCCCCC (A, B, and C represent the scene types of the corresponding video frames). Based on the above division rules, it can be seen that the encoded image group can be divided into: video slices composed of AAAA, video slices composed of BBBBBBB, and video slices composed of CCCCCCCC.

[0038] In one embodiment, to avoid dividing the target video into too many video slices, which would increase computational load and reduce encoding efficiency, this application embodiment may also consider setting some segmentation rules to reduce the number of video slices in the target video and ensure encoding efficiency as much as possible. For example, the principle of such segmentation rules may be one or more of the following: if video frame flickering occurs in consecutive video frames, slicing may not be performed; if the number of video frames of a certain scene type is too small, slicing may not be performed.

[0039] The specific implementation of the slicing process, where the principle of the segmentation rule is that video frame flickering occurs in consecutive video frames and slicing is not required, can be as follows: When slicing any coded image group according to the segmentation rule that divides consecutive video frames of the same scene type into a slice video, if any coded image group includes video frames of a first scene type and video frames of a second scene type, and the video frames of the second scene type are in multiple consecutive video frames of the first scene type, and the number of video frames of the second scene type and the number of video frames of the first scene type satisfy a first preset condition, then any coded image group can be regarded as a slice video.

[0040] The first preset condition can be: the number of video frames of the first scene type is greater than a first quantity, the number of video frames of the second scene type is less than a second quantity, and the first quantity is greater than or equal to the second quantity; or, the number of video frames of the second scene type is less than the number of video frames of the first scene type. When the number of video frames of the first scene type and the number of video frames of the second scene type satisfy the above first preset condition, it can be understood that flickering of video frames of the second scene type occurs in the video frames of the first scene type, that is, flickering of video frames exists in any coded image group. Here, the video frames of the first scene type and the second scene type in the coded image group do not need to be segmented, and the video frames of the first scene type and the second scene type can be directly divided into a slice video.

[0041] For example, the video frames in the encoded image group described above are in the form of: AAAABBAAAA; where A can be understood as a video frame of the first scene type and B can be understood as a video frame of the second scene type. Here, if the number of B frames in the encoded image group is less than the number of A frames, it can be determined that the video frames in the encoded image group are flickering. In this case, the position between A and B can not be set as scene detection, that is, AAAABBAAAA can be divided into a video slice.

[0042] The specific implementation of the slicing process when the principle of the segmentation rule is that the number of video frames of a certain scene type is too small to be sliced ​​can be as follows: When slicing any coded image group according to the segmentation rule of dividing consecutive video frames of the same scene type into a slice video, if the video frames in any coded image group include video frames of multiple scene types, and the number of video frames of the target scene type and the number of video frames of other scene types meet the second preset condition, then the video frames of the target scene type and the video frames of other scene types in any coded image group can be divided into different slice videos.

[0043] The target scene type can be any one of multiple scene types. The second preset condition can be that the number of video frames of the target scene type is greater than the number of video frames of any other scene type; or, the number of video frames of the target scene type is greater than a third condition, the number of video frames of other scene types is less than a fourth condition, and the third condition is greater than or equal to the fourth condition; or, other conditions may be used, which are not limited, only requiring that too few video frames of the same scene type cannot be directly divided into a single video slice.

[0044] For example, the video frames in the encoded image group described above are in the form of AAABBCCDDEEFFFF. Here, the number of frames A, B, C, D, and E in the encoded image group is less than the number of frames F. When segmenting, AAABBCCDDEE can be divided into one video slice, and FFFF can be divided into another video slice. That is, when segmenting the encoded image group, scene segmentation points will not be set between A and B, B and C, C and D, and D and E. Instead, scene detection points will be set between E and F to divide the encoded image group into segmented videos composed of AAABBCCDDEE and segmented videos composed of FFFF.

[0045] S102, obtain the region of interest in each video slice, and use the region of interest of each video slice to encode the corresponding video slice to obtain the encoded data of each video slice.

[0046] The Region of Interest (ROI) can generally be understood as the area of ​​human visual focus, i.e., the area that the user is most interested in. In image or video encoding, the ROI involves region coding parameters (or ROI coding parameters), and optimizing for the ROI is one aspect of user-perceptual encoding optimization. Generally, considering that video clips with higher visual focus are more likely to be encoded with larger ROI coding parameters, larger ROI coding parameters can be used to improve the clarity of the ROI, enhance the perceived image quality, and improve overall video quality. Conversely, for video clips with fewer visually relevant areas, smaller ROI coding parameters can be used to avoid wasting encoding resources. Therefore, in actual encoding, using excessively large ROI coding parameters for non-interested areas can easily cause excessive loss in these areas, making the distortion directly perceptible; while using smaller ROI coding parameters for interested areas results in poor clarity in those areas, leading to poor encoding quality.

[0047] Based on the above considerations, this application embodiment considers adjusting the encoding parameters (such as ROI encoding parameters) required for encoding the sliced ​​video based on the region of interest included in the sliced ​​video, and then using the adjusted encoding parameters to encode the corresponding sliced ​​video to obtain the corresponding encoded data.

[0048] In one implementation, the Region of Interest (ROI) in each video slice can be obtained first, so that each video slice can be encoded separately using its ROI. Considering that the principles for obtaining the ROI of each video slice are similar, the following explanation uses any given video slice as an example. Specifically, for any given video slice, the ROI in each video frame can be determined first; for example, a neural network model with ROI recognition capabilities can be used to identify the ROI of each video frame to obtain the ROI of each video frame; alternatively, other methods can be used to determine the ROI of each video frame, without specific limitations. After obtaining the ROI of each video frame, the ROI for the entire video slice can be further determined based on the ROI of each video frame; for example, the sum of the ROI values ​​of all video frames can be used as the ROI for the entire video slice.

[0049] In one implementation, considering that the principle of encoding each video slice is similar, the encoding process is specifically illustrated here using any one of multiple video slices as an example. After obtaining the region of interest (ROI) in any video slice, the specific implementation of the encoding process in step S102 can be: obtaining the ROI percentage in any video slice, so that the ROI percentage of any video slice can be used to encode the video slice subsequently, obtaining the encoded data of that video slice.

[0050] The region of interest (ROI) percentage can be: the percentage of the ROI in any slice of video relative to the total ROI of the target video; or it can be: the percentage of the ROI in any slice of video relative to the total ROI of that slice of video; there is no specific limitation on this. The principles for determining the ROI percentage are similar, and the former will be used as an example for explanation. In one embodiment, determining the ROI percentage can be achieved by using the ratio between the area of ​​the ROI in any slice of video and the area of ​​the total ROI of the target video as the ROI percentage of the total ROI of the target video.

[0051] In one embodiment, the specific implementation of encoding the video slice using the region of interest (ROI) ratio of any video slice to obtain the encoded data of any video slice can be as follows: adjust the region coding parameters (ROI coding parameters) of the region of interest of any video slice based on the ROI ratio of any video slice, and then use the adjusted ROI coding parameters to encode the video slice to obtain the encoded data of any video slice.

[0052] In one feasible implementation, the above adjustment operation can be specifically implemented by adjusting the region coding parameters of the region of interest in the video slice by different adjustment magnitudes, based on the range of the region of interest's proportion. The specific implementation can be found in the following description:

[0053] If the region of interest (ROI) of any video slice falls within the fourth range, the region coding parameters of that ROI can be adjusted by the fourth adjustment increment. If the ROI of any video slice falls within the fifth range, the region coding parameters of that ROI can be adjusted by the fifth adjustment increment. If the ROI of any video slice falls within the sixth range, the region coding parameters of that ROI can be adjusted by the sixth adjustment increment.

[0054] Specifically, the percentage of the region of interest in the fourth range is higher than that in the fifth range, the percentage of the region of interest in the fifth range is higher than that in the sixth range, the adjustment magnitude in the fourth range is higher than that in the fifth range, and the adjustment magnitude in the fifth range is higher than that in the sixth range. There are no specific limitations on the values ​​corresponding to the fourth, fifth, and sixth ranges, the fourth adjustment magnitude, the fifth adjustment magnitude, and the sixth adjustment magnitude; only that the above relationships are satisfied. For example, the fourth range could be [60%, 100%], the fifth range could be [40%, 60%], the third range could be [30%, 40%], or other ranges; no specific limitations are imposed on these.

[0055] As mentioned earlier, the larger the proportion of the region of interest (ROI) in a video slice, the larger the ROI's region coding parameters need to be to achieve better coding results, and the adjustment range of the ROI's region coding parameters can also be larger. When the ROI's proportion is relatively small, the original ROI's region coding parameters can remain unchanged.

[0056] Based on this, if the region of interest (ROI) of any video slice falls within the fourth range, the ROI region coding parameters can be increased by the fourth adjustment increment. If the ROI of any video slice falls within the fifth range, the ROI region coding parameters can be increased by the fifth adjustment increment. If the ROI of any video slice falls within the sixth range, the ROI region coding parameters can be increased by the sixth adjustment increment.

[0057] In one implementation, for any given video slice, after obtaining the adjusted region coding parameters for that video slice, the adjusted region coding parameters can be used to encode the video slice, thereby obtaining the encoded data of that video slice. This encoding of the video slice can be implemented using a first encoder; that is, the first encoder can be invoked and the adjusted region coding parameters can be used to encode the video slice.

[0058] As described above, after introducing intelligent slicing, the embodiments of this application can further adjust the region coding parameters for the region of interest based on the proportion of the region of interest in the sliced ​​video, so that each sliced ​​video can obtain more suitable coding parameters (such as region coding parameters for the region of interest). Compared with using the same region coding parameters for the region of interest for the entire video, by introducing the intelligent slicing method, the embodiments of this application can provide a more granular allocation of coding parameters, so that the video can achieve higher coding quality while saving coding bitrate.

[0059] S103, combine the encoded data of each video slice to generate the encoded data of the target video.

[0060] In one implementation, after obtaining the encoded data of each video slice in the target video, the encoded data of the target video can be further obtained based on the encoded data of these slices. Specifically, the encoded data of each video slice can be combined to generate the encoded data of the target video; this combination process can refer to splicing the encoded data of each video slice according to the chronological order of each video slice in the target video.

[0061] In specific application scenarios, after generating the encoded data of the target video, the encoded data can be decoded to allow playback of the decoded target video. Therefore, in one implementation, the encoded data of the target video can be sent to a video client, which can then decode the data to obtain the target video; after obtaining the target video, the video client can then play it.

[0062] In this embodiment, the target video can be sliced ​​to obtain multiple sliced ​​videos. Furthermore, the region of interest (ROI) in each sliced ​​video can be obtained, and the ROI of each sliced ​​video can be used to encode the corresponding sliced ​​video, obtaining encoded data for each sliced ​​video. The encoded data of each sliced ​​video can then be combined to generate the encoded data for the target video. Through this method, a video can be divided into multiple sliced ​​videos during video encoding, facilitating subsequent encoding of each sliced ​​video. Compared to encoding the entire video, this embodiment provides more granular encoding, thereby improving the encoding effect. In addition to encoding each sliced ​​video, the ROI in each sliced ​​video can be further combined to encode the corresponding sliced ​​video, effectively utilizing the actual features (i.e., ROI) in each sliced ​​video for encoding, ensuring encoding reliability, and thus improving the encoding effect.

[0063] Please see Figure 2b , Figure 2b This is a flowchart illustrating another video encoding method provided in this application embodiment. The video encoding method described in this embodiment can be applied to the aforementioned computer equipment, such as... Figure 2b As shown, the method includes, but is not limited to, the following steps:

[0064] S201, slice the target video to obtain multiple slice videos for the target video.

[0065] The specific implementation of this step can be found in the description of step S101 above, and will not be repeated here.

[0066] S202, obtain the coding features of each video slice, and predict the coding control parameters of the corresponding video slice based on the coding features of each video slice.

[0067] In one implementation, obtaining the encoding features of each video slice can be achieved by the computer device encoding each video slice to obtain its encoding features. In one embodiment, considering the large processing volume of a single video, to reduce the data computation in subsequent processing and thus improve encoding efficiency, the video slices can be downsampled before encoding to reduce the data volume. Based on this, after obtaining the video slices included in the target video, the computer device can first downsample each video slice to obtain the downsampled video corresponding to each video slice. Furthermore, each downsampled video can be encoded separately to obtain its encoding features, and these features can be used as the encoding features of the corresponding video slice.

[0068] The coding features of each video slice may include frequency information of block division, PSNR (Peak Signal-to-Noise Ratio) values ​​of different frame types, transform coefficients of different block divisions, coding mode information used for coding, etc., without specific limitations.

[0069] The encoding of the downsampled video here can be achieved using a second encoder. That is, the second encoder can encode each downsampled video separately to obtain the encoded features of each downsampled video. This second encoder can refer to an encoder that encodes the video using fixed encoding control parameters, such as an encoder based on the H.264 standard, or other encoders; there is no limitation on this. It can be understood that encoding the target video is equivalent to compressing the target video, and the encoded features obtained here can also be called compression domain features.

[0070] It's important to understand that during video encoding, computer equipment can call the encoder to set the video bitrate. Bitrate control methods generally include CBR (Constant Bit Rate), VBR (Variable Bit Rate), CQP (Constant Quantization Parameter), and CRF (Constant Rate Factor). For example, in short videos and live streaming, CRF is commonly used. This method uses the same CRF value for the entire video, maintaining constant quality, but with a variable bitrate. Typically, the bitrate can change depending on the content of each video frame. In other words, when encoding with CRF, the encoding quality of each video frame is constant, but the bitrate differs. Therefore, while this method maintains a fixed CRF value during video encoding, it can also consume excessive bitrate, leading to wasted encoding resources.

[0071] As mentioned earlier, different video frames in the same video usually have multiple different scenes. For example, a video about a school may include scenes such as the playground, classroom, and cafeteria. Different scenes have different effects on the characteristics and complexity of video frames. Therefore, in order to save more bitrate during video encoding, video frames of complex scenes that are of high visual attention can be encoded with a higher bitrate to ensure higher encoding quality. On the other hand, video frames of flat scenes that are of low visual attention can be encoded with a lower bitrate to ensure encoding quality while occupying as little transmission bandwidth as possible.

[0072] Based on the above considerations, embodiments of this application propose adapting different coding control parameters (such as CRF) to video frames in different scenes within a video. This allows for the control of the encoding bitrate of video frames in different scenes using more suitable coding control parameters, resulting in higher encoding quality while saving bitrate during video encoding processing. Specifically, embodiments of this application introduce an intelligent slicing method. After dividing the video into multiple slices based on the scene of the video frame, different coding control parameters can be predicted for different slices, and then the slices in the video can be encoded using these different coding control parameters.

[0073] The encoding control parameters for the video slices can include any one of CBR, VBR, CQP, CRF, etc. This application embodiment takes CRF (fixed bit rate factor) as the encoding control parameter as an example for related explanation. That is, in this application embodiment, it is necessary to predict the CRF of the video slices under different scene types so as to use different CRFs to encode different video slices. Compared with the traditional method of using the same CRF for encoding throughout the entire video encoding, it can save bit rate while improving video quality.

[0074] In one implementation, the encoding control parameters for each video slice can be obtained by calling a parameter prediction model to predict the encoding features of the video slice. These encoding control parameters can be selected from N reference encoding control parameters, which refer to the encoding control parameters (such as CRF) applicable to the first encoder. The first encoder can refer to the encoder used to encode the video slice, and is different from the second encoder mentioned above; no specific limitation is made in this regard. For example, the first encoder can be an encoder based on the AV1 standard, and the N reference encoding control parameters can include positive integers from 45 to 60.

[0075] Considering that the principles for determining the encoding control parameters of each video slice in the target video are similar, the following will take any video slice in the target video as an example to explain in detail the determination of the encoding control parameters of that video slice.

[0076] In one embodiment, for any video slice in the target video, a parameter prediction model can be invoked to predict the coding flag bits based on N reference coding control parameters and the coding features of that video slice, to obtain the coding quality flag bits of that video slice under each reference coding control parameter. For example, the N reference coding control parameters corresponding to the first encoder and the coding features of that video slice can be input into the parameter prediction model to obtain the coding quality flag bits of that video slice under each reference coding control parameter. To ensure the orderliness of the coding quality flag bits when determining the coding quality flag bits of any video slice under each reference coding control parameter, the reference coding control parameters corresponding to the first encoder and the coding features of that video slice can be input into the parameter prediction model in ascending order. For example, assuming that the N reference coding control parameters include positive integers from 45 to 60, (45, coding feature), (46, coding feature), (47, coding feature)... can be input into the parameter prediction model in ascending order of the values ​​corresponding to the reference coding control parameters to obtain the coding quality flag bits of the video slice under 45, 46, 47... respectively. After obtaining these coding quality flags, the coding control parameters of any slice video can be determined from the N reference coding control parameters based on the coding quality flags of any slice video under each reference coding control parameter.

[0077] The coding quality flag of the video slice under a reference coding control parameter can be used to indicate the quality relationship between the reference coding quality and the target coding quality of the video slice under that reference coding control parameter. The reference coding quality can refer to the coding quality obtained by encoding the video slice using a first encoder and the reference coding control parameter, while the target coding quality can refer to the coding quality obtained by encoding the video slice using a second encoder. The coding quality flag can include a first flag and a second flag. The first flag can be used to indicate that the reference coding quality is greater than the target coding quality, and the second flag can be used to indicate that the reference coding quality is less than or equal to the target coding quality. Mathematically, the first flag can be represented by 1, and the second flag can be represented by 0.

[0078] Specifically, determining the coding control parameters of any given video slice from N reference coding control parameters based on the coding quality flags under each reference coding control parameter can be implemented as follows: Based on the coding quality flags of any given video slice under each reference coding control parameter, candidate coding control parameters are determined from the N reference coding control parameters, and the coding control parameters of the given video slice are then determined based on these candidate parameters. Specifically, determining the candidate coding control parameters from N reference coding control parameters based on the coding quality flags of any given video slice under each reference coding control parameter can be implemented as follows:

[0079] The coding quality flags under the N reference coding control parameters are sorted in ascending order to obtain the sorting result for the coding quality flags.

[0080] If the first to the Mth coding quality flags in the sorting result are all second flags (0), and the (M+1)th coding quality flag is a first flag (1), then the reference coding control parameters corresponding to the Mth and (M+1)th coding quality flags can be used as candidate coding control parameters. In summary, this application embodiment can determine the coding control parameters of any video slice based on the reference coding control parameters corresponding to the Mth and (M+1)th coding quality flags. Here, M is a positive integer greater than or equal to 1 and less than N.

[0081] As mentioned above, the coding quality flags before the Mth coding quality flag are all second flags (0), and the coding quality flags after the (M+1)th coding quality flag are all first flags (1). Therefore, the reference coding control parameters corresponding to the Mth coding quality flag and the reference coding control parameters corresponding to the (M+1)th coding quality flag can be understood as the boundary parameters for coding quality flags 0 and 1.

[0082] For example, assuming a reference coding control parameter (CRF) and N CRFs including positive integers from 45 to 60, if the coding quality flags of a video slice are determined to be 0, 0, 0, 0, 1, and 1 respectively for CRFs of 45, 46, 47, 48, 49, 50, and 51, then it can be determined that when CRF is less than or equal to 49, the coding quality of the video slice in the first encoder is less than the target coding quality; and when CRF is greater than or equal to 50, the coding quality of the video slice in the first encoder is greater than the target coding quality. Here, 49 can be understood as the reference coding control parameter corresponding to the Mth coding quality flag bit mentioned above, and 50 can be understood as the reference coding control parameter corresponding to the (M+1)th coding quality flag bit mentioned above. In other words, 49 and 50 can be used as boundary parameters, meaning that a value can be randomly selected from 49 and 50 as the final required CRF for the video slice.

[0083] In one embodiment, the specific implementation of determining the coding control parameter of any video slice based on the reference coding control parameter corresponding to the Mth coding quality flag bit and the reference coding control parameter corresponding to the M+1th coding quality flag bit can be: randomly selecting a reference coding control parameter from the reference coding control parameter corresponding to the Mth coding quality flag bit and the reference coding control parameter corresponding to the M+1th coding quality flag bit as the coding control parameter of any video slice.

[0084] For example, referring to the example above, assuming the reference coding control parameter corresponding to the Mth coding quality flag and the reference coding control parameter corresponding to the (M+1)th coding quality flag are 49 and 50 respectively, then a value can be randomly selected from 49 and 50 as the final CRF required for the sliced ​​video. For ease of description, the reference coding control parameter corresponding to the Mth coding quality flag and the reference coding control parameter corresponding to the (M+1)th coding quality flag can be referred to as the Mth reference coding control parameter and the (M+1)th reference coding control parameter, respectively.

[0085] In another embodiment, it is important to understand that the smaller the CRF value, the higher the encoding bitrate required for the encoder to encode the video, and the higher the encoding quality of the encoded video. Based on this principle, when determining the encoding control parameters for any video slice based on the Mth reference encoding control parameter and the (M+1)th reference encoding control parameter, the encoding control parameters for any video slice can be selected from the Mth reference encoding control parameter and the (M+1)th reference encoding control parameter based on preset encoding requirements. These preset encoding requirements can be the encoding requirements for the first encoder during encoding, and can include both first and second encoding requirements. The first encoding requirement can refer to a quality requirement that is higher than the bitrate requirement, meaning that when using the first encoder for encoding, the focus is on improving encoding quality. The second encoding requirement can refer to a bitrate requirement that is higher than the quality requirement, meaning that when using the first encoder for encoding, the focus is on reducing the bitrate consumption.

[0086] As mentioned earlier, the reference coding control parameter can refer to the CRF, and the value corresponding to the Mth reference coding control parameter is less than the value corresponding to the (M+1)th reference coding control parameter.

[0087] Based on this, if the preset encoding requirement for the first encoder is the first encoding requirement, i.e., to maximize encoding quality, a smaller reference encoding control parameter can be selected as the final required encoding control parameter. In other words, the Mth reference encoding control parameter can be used as the final required encoding control parameter for the sliced ​​video. Conversely, if the preset encoding requirement for the first encoder is the second encoding requirement, i.e., to minimize bitrate consumption, a larger reference encoding control parameter can be selected as the final required encoding control parameter. In other words, the (M+1)th reference encoding control parameter can be used as the final required encoding control parameter for the sliced ​​video.

[0088] For example, referring to the above example, suppose the Mth reference coding control parameter and the (M+1)th reference coding control parameter are 49 and 50, respectively. If the preset coding requirement of the first encoder is the first coding requirement, then 49 can be used as the final CRF required for the sliced ​​video; if the preset coding requirement of the first encoder is the second coding requirement, then 50 can be used as the final CRF required for the sliced ​​video. In summary, it can be seen that the embodiments of this application can determine the final required coding control parameters from the Mth reference coding control parameter and the (M+1)th reference coding control parameter based on the coding requirements required by the encoder, so that the coding control parameters can meet the corresponding application requirements.

[0089] In summary, this application proposes a video intelligent slicing bitrate parameter prediction method based on compression domain features. Specifically, the video can first be sliced, for example, using a slicing method based on encoder pre-analysis, downsampling the video and analyzing it frame by frame to determine whether the current frame needs slicing. Alternatively, a second encoder can be used to encode the downsampled sliced ​​video to obtain the sliced ​​video's features in the compression domain (i.e., coding features). Then, a parameter prediction model can be used to predict the CRF value of the sliced ​​video. In specific implementation, various CRF values ​​and coding features within a selectable CRF range can be input into the parameter prediction model to obtain the corresponding coding quality flag bit when encoding using the CRF value. This coding quality flag bit can be 1 or 0. Wherein, when the coding quality flag bit is 1, it indicates that the actual coding quality (i.e., the aforementioned reference coding quality) is greater than the target coding quality; when the coding quality flag bit is 0, it indicates that the actual coding quality is less than or equal to the target coding quality. After obtaining the coding quality flags within the CRF range, a binary search method can be used to find the boundary line where the coding quality flags of the first encoder are 0 / 1 in the corresponding CRF and CRF+1, or in other words, to find the boundary parameter, and then determine the final coding control parameters required for the sliced ​​video based on the boundary parameter.

[0090] As can be seen from the above description, the embodiments of this application can slice the video based on different scenes in the video, and can perform pre-analysis for different scenes to allocate different encoding bitrates, thereby achieving more granular encoding bitrate allocation and improving encoding quality while reducing the encoding bitrate.

[0091] S203, obtain the region of interest in each video slice, and encode each video slice using the encoding control parameters and the region of interest to obtain the encoded data of each video slice.

[0092] As mentioned earlier, optimizing for regions of human visual attention is one aspect of user-perceptual encoding optimization. Generally, considering video segments with higher visual attention, a higher bitrate and stronger ROI encoding parameters are preferred. Conversely, for video segments with fewer regions of visual attention, a lower bitrate and smaller ROI encoding parameters are preferred. Therefore, in actual encoding, using excessively high parameters (bitrate and ROI encoding parameters) for non-focused regions can easily cause excessive loss in these areas, making the distortion directly perceptible. Conversely, using insufficient parameters (bitrate and ROI encoding parameters) for focused regions may result in insufficient bitrate allocated to them, leading to poor encoding performance.

[0093] Based on the above considerations, this application embodiment considers adjusting the encoding parameters (such as encoding control parameters, ROI encoding parameters, etc.) required for encoding the sliced ​​video based on the region of interest included in the sliced ​​video, and then using these adjusted encoding parameters to encode the corresponding sliced ​​video and obtain the corresponding encoded data.

[0094] In one implementation, the region of interest (ROI) in each video slice can be obtained first, so that each video slice can be encoded using its encoding control parameters and ROI separately. The specific implementation for obtaining the ROI in each video slice can be found in the description above, and will not be repeated here.

[0095] Considering that the principle of encoding each video slice is similar, this section will specifically illustrate the encoding process using any one of multiple video slices as an example. After obtaining the region of interest (ROI) in any video slice, the specific implementation of the encoding process in step S203 can be: obtaining the ROI percentage in any video slice, so that the encoding control parameters and ROI percentage of any video slice can be used to encode the video slice and obtain its encoded data. The understanding of the ROI percentage can be found in the relevant descriptions above, and will not be repeated here.

[0096] In one embodiment, the specific implementation of encoding the video slice using the encoding control parameters and the region of interest ratio of any video slice to obtain the encoded data of any video slice can be as follows: adjust the encoding control parameters and the region encoding parameters for the region of interest of any video slice based on the region of interest ratio of any video slice, and then use the adjusted encoding control parameters and region encoding parameters to encode the video slice to obtain the encoded data of any video slice.

[0097] The aforementioned encoding control parameters can be a fixed bitrate factor (CRF). In one feasible implementation, the adjustment operation can be implemented by adjusting the fixed bitrate factor and region encoding parameters of the video slice by different adjustment magnitudes based on the range of the region of interest. The specific implementation can be found in the following description:

[0098] If the region of interest (ROI) of any video slice falls within a first range, the fixed bitrate factor and region coding parameters of that video slice can be adjusted by a first adjustment increment. If the ROI of any video slice falls within a second range, the fixed bitrate factor and region coding parameters of that video slice can be adjusted by a second adjustment increment. If the ROI of any video slice falls within a third range, the fixed bitrate factor of that video slice can be adjusted by a third adjustment increment.

[0099] Specifically, the percentage of the region of interest in the first range is higher than that in the second range, the percentage of the region of interest in the second range is higher than that in the third range, the first adjustment magnitude is higher than the second adjustment magnitude, and the second adjustment magnitude is higher than the third adjustment magnitude. The specific values ​​corresponding to the first, second, and third ranges, and the first, second, and third adjustment magnitudes are not specifically limited here; it is only necessary to ensure that the above relationships are satisfied. For example, the first range could be [70%, 100%], the second range could be [50%, 70%], and the third range could be [30%, 50%], or other ranges are also acceptable; no specific limitations are imposed on these.

[0100] As mentioned earlier, the larger the proportion of the region of interest (ROI) in a video slice, the more bitrate and region coding parameters of the ROI need to be allocated to achieve better encoding results. Furthermore, the smaller the CRF value, the higher the bitrate required for the encoder to encode the video. In other words, if the ROI proportion in a video slice is large, the CRF value should be small. Therefore, when adjusting the fixed bitrate factor and region coding parameters based on the ROI proportion, the larger the value corresponding to the ROI proportion, the greater the adjustment range for the fixed bitrate factor and region coding parameters can be. The adjustment direction for the ROI region coding parameters can be towards increasing the value, while the adjustment direction for the fixed bitrate factor can be towards decreasing the value.

[0101] Based on this, if the region of interest (ROI) of any video slice falls within a first range, the fixed bitrate factor of that video slice can be decreased by a first adjustment increment, while the region coding parameters of the ROI can be increased. If the ROI of any video slice falls within a second range, the fixed bitrate factor of that video slice can be decreased by a second adjustment increment, while the region coding parameters of the ROI can be increased. If the ROI of any video slice falls within a third range, the fixed bitrate factor of that video slice can be decreased by a third adjustment increment.

[0102] In summary, after determining the CRF value of the video slice, this embodiment can further adjust the parameters based on the proportion of the Region of Interest (ROI). For example, when the ROI of the video slice exceeds 70% of the overall area of ​​the target video, a higher bitrate can be allocated to the video slice, and the ROI encoding parameters can be increased to achieve joint optimization of bitrate and ROI. When the ROI of the video slice exceeds 50% of the overall area of ​​the target video, a higher bitrate can be allocated to the video slice, and the ROI encoding parameters can be increased. When the ROI of the video slice exceeds 30% of the overall area of ​​the target video, a higher bitrate can be allocated to the video slice.

[0103] In one implementation, for any given video slice, after obtaining the adjusted encoding control parameters and region encoding parameters for that slice, the encoding bitrate of that slice can be controlled using the adjusted constant bitrate factor. Then, the encoding bitrate of that slice and the adjusted region encoding parameters of the region of interest are used to encode the video slice, thereby obtaining the encoded data of that video slice. This encoding of the video slice can be implemented using a first encoder; that is, the first encoder can be invoked, and the adjusted encoding control parameters and region encoding parameters can be used to encode the video slice.

[0104] As described above, the embodiments of this application, after introducing intelligent slicing, can predict different encoding control parameters for different video slices. After obtaining the encoding control parameters, the encoding control parameters and region encoding parameters can be further adjusted based on the proportion of the region of interest in the video slice, so that each video slice can obtain more suitable encoding parameters. Compared with using the same encoding control parameters and region encoding parameters for an entire video, the embodiments of this application, by introducing the intelligent slicing method, can provide a more granular allocation of encoding parameters, so that the video can achieve higher encoding quality while saving encoding bitrate.

[0105] S204 combines the encoded data of each video slice to generate the encoded data of the target video.

[0106] The specific implementation method of this step can be referred to the implementation method in step S203 above, and will not be repeated here.

[0107] To better understand the video coding method proposed in this application, the following is combined with Figure 3 For further explanation, see [link to documentation]. Figure 3 As shown, the video encoding method may include the following steps:

[0108] S1, Video Slicing: For a target video, it can be sliced ​​to obtain multiple slice videos of the target video.

[0109] S2, Obtaining the coding features of the video slices: Each video slice can be downsampled to obtain the downsampled video corresponding to each video slice; then, the second encoder can be used to encode each downsampled video to obtain the features of each video slice in the compression domain from the second encoder, that is, to obtain the coding features of each video slice.

[0110] S3, Prediction of coding control parameters for video slices: For any given video slice, the CRF values ​​within the selectable CRF range and the coding features of the video slice are input into the parameter prediction model to obtain the coding quality flag bit corresponding to the CRF values ​​during encoding. This coding quality flag bit can be 1 or 0. After obtaining the coding quality flag bits within the CRF range, a binary search method can be used to find the boundary line where the coding quality flag bits of the first encoder are 0 / 1 under the corresponding CRF and CRF+1, or in other words, to find the boundary parameter. Then, based on the boundary parameter, the final coding control parameters (CRF) required for the video slice are determined.

[0111] S4, Adjustment (correction) of encoding control parameters and region coding parameters: For any given video slice, the region of interest (ROI) can be detected, and its proportion within that slice can be determined. This ROI proportion can refer to the proportion of the entire target video or the proportion of the entire region of that particular video slice. After determining the ROI proportion, the encoding control parameters and region coding parameters of that video slice can be adjusted based on this proportion to obtain the adjusted encoding control parameters and region coding parameters.

[0112] S5, Encoding of video slices: Encode the corresponding video slices based on the adjusted encoding control parameters and region encoding parameters to obtain the encoded data of each video slice.

[0113] S6, Check if it is the last video slice: The processing of each video slice can be performed serially, that is, for each video slice, steps S2-S5 can be executed sequentially to complete the encoding of each video slice. Therefore, to complete the encoding of the target video, it is necessary to ensure that the encoding of each video slice has been completed. That is, after the encoding of a video slice is completed, it can be further checked whether that video slice is the last video slice. If it is the last video slice, then proceed to step S7; if it is not the last video slice, then steps S2-S5 can be re-executed until each video slice in the target video has been encoded, then proceed to step S7.

[0114] S7, Video Slicing: The encoded data of each video slice is reassembled to generate encoded data for the target video. This encoded data is then returned to the video client so that the client can decode the encoded data to obtain and play the target video.

[0115] In this embodiment, the target video can be preprocessed to obtain multiple video slices and encoding features for each slice. Encoding control parameters for each slice can be predicted based on its encoding features, and regions of interest (ROIs) can be obtained from each slice. Each slice is then encoded using its encoding control parameters and ROIs to obtain encoded data. These encoded data are then combined to generate the encoded data for the target video. This method allows a video to be divided into multiple video slices, each with its own encoding control parameters. Furthermore, the predicted encoding control parameters are combined with the ROIs to encode the corresponding slices, thus combining multiple dimensions of parameters to effectively improve the encoding quality. Practical experience shows that this implementation, compared to using the same encoding control parameters throughout the entire video encoding process, achieves higher encoding quality while saving bitrate.

[0116] Please see Figure 4 , Figure 4 This is a flowchart illustrating another video coding method provided in this application embodiment. The video coding method described in this embodiment can be applied to the aforementioned computer equipment, and this embodiment mainly describes the training process of training an initial prediction model to obtain a parameter prediction model, such as... Figure 4 As shown, the method includes, but is not limited to, the following steps:

[0117] S401, acquire sample videos, preprocess the sample videos to obtain multiple sample slice videos for the sample videos and sample encoding features for each sample slice video.

[0118] It should be noted that the number of sample videos used to train the initial prediction model can be one or more. Considering that the processing of each sample video is similar during training, this embodiment uses a single sample video as an example to specifically illustrate the training of the initial prediction model. The sample video can be any video; its understanding can refer to the understanding of the target video described above, and will not be repeated here.

[0119] In one implementation, after acquiring the sample video, it can be preprocessed to obtain the video slices included in the sample video and the encoding features of each video slice. For ease of description, the video slices and encoding features here can be referred to as sample video slices and sample encoding features, respectively. In one embodiment, the specific implementation of preprocessing the sample video to obtain the corresponding sample video slices and corresponding sample encoding features can be as follows: the computer device can slice the sample video to obtain multiple sample video slices; and downsample each sample video slice to obtain a sample downsampled video corresponding to each sample video slice; then, each sample downsampled video slice can be encoded to obtain the encoding features of each sample downsampled video slice, and the encoding features of each sample downsampled video slice can be used as the sample encoding features of the corresponding sample video slice. The specific implementation of each step here can be referred to the relevant description in step S201 above, and will not be repeated here.

[0120] S402, for any sample slice video among multiple sample slice videos, call the first encoder and encode any sample slice video based on N reference coding control parameters respectively, to obtain the reference coding quality of any sample slice video under each reference coding control parameter.

[0121] It should be noted that, considering that the initial prediction model processes each sample video slice in the sample video in a similar way when training the initial prediction model, the training of the initial prediction model will be explained here using any sample video slice from multiple sample video slices as an example.

[0122] Where N is a positive integer greater than 1, meaning there are multiple reference coding control parameters. These N reference coding control parameters can refer to the coding control parameters applicable to the first encoder. As mentioned earlier, the coding control parameters in this embodiment can refer to the CRF, and the N reference coding control parameters can refer to the CRF applicable to the first encoder. For example, the first encoder can be an encoder based on the AV1 standard, in which case the N reference coding control parameters can include positive integers between 45 and 60.

[0123] In one implementation, for any sample video segment among multiple sample video segments, the computer device can call the first encoder and encode the sample video segment using N reference coding control parameters to obtain the encoding result of the sample video segment under each reference coding control parameter. After obtaining the encoding result under each reference coding control parameter, the coding quality under the corresponding reference coding control parameter can be determined based on the encoding result of the sample video segment under each reference coding control parameter. For example, the coding quality here can be referred to as the reference coding quality.

[0124] The principle of determining the reference coding quality based on a reference coding control parameter is similar. Therefore, this section uses a reference coding control parameter as an example to illustrate the specific implementation of determining the reference coding quality based on the coding result. In one embodiment, the specific implementation can be: using a coding quality evaluation method to evaluate the coding result to obtain the corresponding reference coding quality.

[0125] The coding quality assessment method can be any one of PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity), or MS-SSIM (Multi-scale Structural Similarity), without limitation. PSNR is an objective evaluation method based on pixel values, specifically determining coding quality by comparing the differences in pixel values ​​between the original and processed video or image. SSIM determines coding quality by comparing the correlation between the original and processed video or image; this correlation can be described from three aspects: brightness, contrast, and structure. MS-SSIM, based on SSIM, introduces multiple scales to compare the correlation between the original and processed video or image to determine coding quality.

[0126] S403, call the second encoder to encode any slice of video to obtain the target encoding quality, and based on the reference encoding quality and target encoding quality of any sample slice of video under each reference encoding control parameter, determine the sample encoding quality flag bit of any sample slice of video under each reference encoding control parameter.

[0127] The second encoder differs from the first encoder; for example, it could refer to an encoder based on the H.264 standard. The sample coding quality flag for any sample video slice under a reference coding control parameter can be used to indicate the quality relationship between the reference coding quality and the target coding quality under that parameter. This sample coding quality flag can include a first flag and a second flag. The first flag indicates that the reference coding quality is greater than the target coding quality, and the second flag indicates that the reference coding quality is less than or equal to the target coding quality. Mathematically, the first flag can be represented by 1, and the second flag can be represented by 0.

[0128] Based on this, the specific implementation of determining the sample coding quality flag bit of any sample video slice under the reference coding control parameters can be described as follows. Considering that the principle of determining the sample coding quality flag bit under any of the N reference coding control parameters is similar, this section uses any reference coding control parameter as an example to illustrate the determination of the sample coding quality flag bit. In specific implementation, for any of the N reference coding control parameters, the reference coding quality of any sample video slice under any reference coding control parameter can be compared with the target coding quality.

[0129] If the reference coding quality is greater than the target coding quality, the sample coding quality flag of any sample slice video under any reference coding control parameter can be determined as the first flag (1). If the reference coding quality is less than or equal to the target coding quality, the sample coding quality flag of any sample slice video under any reference coding control parameter can be determined as the second flag (0).

[0130] For example, suppose the reference coding control parameter CRF = 46, and the target coding quality of a sample video slice is 36 when CRF = 46; if the reference coding quality is 34, then the sample coding quality flag of the sample video slice under CRF = 46 is the second flag (0); and if the reference coding quality is 37, then the sample coding quality flag of the sample video slice under CRF = 46 is the first flag (1).

[0131] For example, suppose the reference coding control parameter CRF = 47, and the target coding quality of a sample video slice is 28 when CRF = 47; if the reference coding quality is 31, then the sample coding quality flag of the sample video slice under CRF = 47 is the first flag (1); and if the reference coding quality is 27, then the sample coding quality flag of the sample video slice under CRF = 47 is the second flag (0).

[0132] S404. Based on the sample coding features of each sample video slice in the sample video, N reference coding control parameters, and the sample coding quality flag bits of each sample video slice under the N reference coding control parameters, the initial prediction model is trained to obtain the parameter prediction model.

[0133] In one implementation, based on the above steps, the sample coding quality flags for each sample video slice under N reference coding control parameters can be determined. After obtaining this data, the initial prediction model can be trained based on the sample coding features of each sample video slice, the N reference coding control parameters, and the sample coding quality flags for each sample video slice under the N reference coding control parameters, thus obtaining a parameter prediction model. Specifically, multiple sample data sets for the initial prediction model can be constructed based on the sample coding features of each sample video slice, the N reference coding control parameters, and the sample coding quality flags for each sample video slice under the N reference coding control parameters. Each sample data set can include a training sample and corresponding label data. Specifically, the sample coding features of a sample video slice and a reference coding control parameter can be used to construct a training sample, and the label data corresponding to this training sample can refer to the sample coding quality flags of that sample video slice under the specified reference coding control parameters. Based on this, we can see that for a single sample video segment, we can construct N sample data points about that sample video segment; assuming that the number of sample video segments included in the sample video is K, we can construct N×K sample data points about that sample video.

[0134] Based on this, for any sample data (sample coding features of a sample video slice and a reference coding control parameter), the sample data can be input into the initial prediction model. This initial prediction model then predicts the coding quality flag of the sample video slice under the reference coding control parameter based on the sample data, thus obtaining the predicted coding quality flag of the sample video slice under the reference coding control parameter. After obtaining the predicted coding quality flag of the sample video slice under the reference coding control parameter, the initial prediction model can be trained using both the predicted coding quality flag and the sample coding quality flag (i.e., label data) to obtain the trained initial prediction model, which is the parametric prediction model.

[0135] In one embodiment, the specific implementation of training an initial prediction model based on the predicted coding quality flag and the sample coding quality flag of the sample slice video under the reference coding control parameters to obtain the parameter prediction model can be as follows: call the model loss function to calculate the model loss value based on the predicted coding quality flag and the sample coding quality flag of the sample slice video under the reference coding control parameters, and train the initial prediction model in the direction of reducing the model loss value to obtain the parameter prediction model.

[0136] In summary, this application proposes a video intelligent slicing bitrate parameter prediction method based on compression domain features. Specifically, it can perform pre-analysis and decision slicing based on video quality evaluation indicators, supplemented by encoder's own feature domain information. For video slices, the same deep learning training method is used under different quality evaluation indicators to establish a mapping between input parameters (such as sample coding features of sample slice videos and reference coding control parameters) and coding quality flag bits (i.e., constructing a parameter prediction model). In practical applications, this mapping can be used to determine the coding quality flag bits of the slice video, and then the optimal coding bitrate of the slice video can be obtained based on the coding quality flag bits. In the initial prediction model training, the sample video can be sliced. For example, a slicing method based on encoder pre-analysis can be used to downsample the sample video and analyze it frame by frame to determine whether the current frame needs to be sliced. Alternatively, a second encoder can be used to encode the downsampled sample slice video to obtain the features of the sample slice video in the compression domain (i.e., sample coding features).

[0137] Secondly, for each sample video segment in the sample video, a reference coding quality can be obtained by the first encoder encoding the sample video segment using a coding control parameter (CRF) value within a floating range (e.g., CRF in the range of 45-60). Then, the coding quality flags of the sample video segment at each CRF value can be determined by comparing the reference coding quality of the sample video segment at multiple CRF values ​​with a specified target coding quality. Finally, the CRF values ​​and the sample coding features of the sample video segment can be input into the initial prediction model to obtain the predicted coding quality flags corresponding to encoding using the CRF values. The initial prediction model can then be trained based on the coding quality flags of the sample video segment at the CRF values ​​and the predicted coding quality flags to obtain a parametric prediction model.

[0138] In this embodiment of the application, a parameter prediction model can be trained using a neural network model (i.e., an initial prediction model) and related data to construct a mapping relationship between coding features and coding quality flags corresponding to the first encoder. This allows the parameter prediction model to be directly called and the coding quality flags of a video slice to be predicted based on the coding features of the video slice when it is necessary to determine the coding quality flags of a certain video slice. This effectively improves the intelligence and automation of determining the coding quality flags.

[0139] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a video encoding device provided in an embodiment of this application. The video encoding device described in this embodiment includes:

[0140] The slicing unit 501 is used to slice the target video to obtain multiple sliced ​​videos for the target video; each sliced ​​video includes multiple consecutive video frames in the target video.

[0141] The encoding unit 502 is used to obtain the region of interest in each slice video, and to encode the corresponding slice video using the region of interest of each slice video to obtain the encoded data of each slice video.

[0142] The combination unit 503 is used to combine the encoded data of each video slice to generate the encoded data of the target video.

[0143] In one implementation, the encoding unit 502 is specifically used for:

[0144] Obtain the encoding features of each video slice, and predict the encoding control parameters of each video slice based on the encoding features of each video slice;

[0145] Using the encoding control parameters and region of interest of each video slice, each video slice is encoded to obtain encoded data for each video slice.

[0146] In one implementation, the encoding unit 502 is specifically used for:

[0147] For any one of the multiple video slices, obtain the percentage of the region of interest (ROI) in that one video slice relative to the overall region of the target video.

[0148] The encoding control parameters of any video slice and the region encoding parameters for the region of interest are adjusted based on the proportion of the region of interest in any video slice.

[0149] The adjusted encoding control parameters and region encoding parameters are used to encode any one of the video slices to obtain the encoded data of the video slice.

[0150] In one implementation, the encoding control parameters include a fixed code rate factor (CRF); the encoding unit 502 is specifically used for:

[0151] If the region of interest percentage of any of the video slices is within a first range, then the fixed bitrate factor and region coding parameters of any of the video slices are adjusted by a first adjustment amount.

[0152] If the region of interest percentage of any of the video slices is within the second range, then the fixed bitrate factor and region coding parameters of any of the video slices are adjusted by the second adjustment magnitude.

[0153] If the region of interest of any of the video slices is within the third range, then the fixed bitrate factor of any of the video slices is adjusted by the third adjustment magnitude.

[0154] Among them, the proportion of the region of interest in the first range is higher than that in the second range, the proportion of the region of interest in the second range is higher than that in the third range, the first adjustment range is higher than the second adjustment range, and the second adjustment range is higher than the third adjustment range.

[0155] In one implementation, the encoding unit 502 is specifically used for:

[0156] Each video slice is downsampled to obtain a downsampled video corresponding to each video slice.

[0157] Each downsampled video is encoded separately to obtain the encoding features of each downsampled video, and the encoding features of each downsampled video are used as the encoding features of the corresponding slice video.

[0158] In one implementation, the encoding unit 502 is specifically used for:

[0159] Obtain the scene type of each video frame in the target video;

[0160] The target video is sliced ​​based on the scene type of each video frame in the target video to obtain sliced ​​videos of the target video.

[0161] In one implementation, the target video comprises one or more coded image groups, each coded image group comprising multiple consecutive video frames from the target video; the encoding unit 502 is specifically used for:

[0162] For any coded image group in the target video, the coded image group is sliced ​​according to the segmentation rule of dividing consecutive video frames of the same scene type into a slice video, to obtain the slice video of the coded image group.

[0163] In one implementation, the encoding control parameters for each video slice are obtained by calling a parameter prediction model to predict the encoding features of the corresponding video slice; the encoding unit 502 is specifically used for:

[0164] Acquire sample videos, preprocess the sample videos to obtain multiple sample slice videos for the sample videos and sample encoding features for each sample slice video;

[0165] For any sample slice video among the plurality of sample slice videos, the first encoder is invoked and the sample slice video is encoded based on N reference coding control parameters respectively to obtain the reference coding quality of the sample slice video under each reference coding control parameter; where N is a positive integer greater than 1;

[0166] The second encoder is invoked to encode any of the video slices to obtain the target encoding quality. Based on the reference encoding quality of any sample video slice under each reference encoding control parameter and the target encoding quality, the sample encoding quality flag bit of any sample video slice under each reference encoding control parameter is determined. The sample encoding quality flag bit of any sample video slice under a reference encoding control parameter is used to indicate the quality relationship between the reference encoding quality and the target encoding quality of any sample video slice under that reference encoding control parameter.

[0167] Based on the sample coding features of each sample video slice in the sample video, N reference coding control parameters, and the sample coding quality flag bits of each sample video slice under the N reference coding control parameters, the initial prediction model is trained to obtain the parameter prediction model.

[0168] In one implementation, the encoding control parameters for each video slice are obtained by calling a parameter prediction model to predict the encoding features of the corresponding video slice; the encoding unit 502 is specifically used for:

[0169] For any video slice in the target video, the parameter prediction model is invoked to predict the coding quality flag bit based on each of the N reference coding control parameters and the coding features of the video slice, so as to obtain the coding quality flag bit of the video slice under each reference coding control parameter.

[0170] Sort the coding quality flags under the N reference coding control parameters in ascending order to obtain the sorting result for the coding quality flags.

[0171] If the first to the Mth coding quality flags in the sorting result are all second flags, and the (M+1)th coding quality flag is the first flag, then the coding control parameters of any video slice are determined based on the reference coding control parameters corresponding to the Mth coding quality flag and the reference coding control parameters corresponding to the (M+1)th coding quality flag; where M is a positive integer greater than or equal to 1 and less than N.

[0172] It is understood that the division of units in this embodiment is illustrative and merely a logical functional division; in actual implementation, there may be other division methods. The functional units in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units can be implemented in hardware or as software functional units.

[0173] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device includes a processor 601 and a memory 602. Optionally, the computer device may also include a network interface 603. The processor 601, memory 602, and network interface 603 can exchange data.

[0174] The processor 601 described above can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0175] The aforementioned memory 602 may include read-only memory and random access memory, and provides program instructions and data to the processor 601. A portion of the memory 602 may also include non-volatile random access memory. The processor 601 executes the program instructions by calling them:

[0176] The target video is sliced ​​to obtain multiple sliced ​​video segments for the target video; each sliced ​​video includes multiple consecutive video frames from the target video.

[0177] Obtain the region of interest in each video slice, and use the region of interest of each video slice to encode the corresponding video slice to obtain the encoded data of each video slice;

[0178] The encoded data of each video slice is combined to generate the encoded data of the target video.

[0179] In one implementation, the processor 601 is specifically used for:

[0180] Obtain the encoding features of each video slice, and predict the encoding control parameters of each video slice based on the encoding features of each video slice;

[0181] Using the encoding control parameters and region of interest of each video slice, each video slice is encoded to obtain encoded data for each video slice.

[0182] In one implementation, the processor 601 is specifically used for:

[0183] For any one of the multiple video slices, obtain the percentage of the region of interest (ROI) in that one video slice relative to the overall region of the target video.

[0184] The encoding control parameters of any video slice and the region encoding parameters for the region of interest are adjusted based on the proportion of the region of interest in any video slice.

[0185] The adjusted encoding control parameters and region encoding parameters are used to encode any one of the video slices to obtain the encoded data of the video slice.

[0186] In one implementation, the encoding control parameters include a fixed code rate factor (CRF); the processor 601 is specifically used for:

[0187] If the region of interest of any slice video is within a first range, then the fixed bitrate factor and region coding parameters of any slice video are adjusted by a first adjustment magnitude.

[0188] If the region of interest percentage of any of the video slices is within the second range, then the fixed bitrate factor and region coding parameters of any of the video slices are adjusted by the second adjustment magnitude.

[0189] If the region of interest of any of the video slices is within the third range, then the fixed bitrate factor of any of the video slices is adjusted by the third adjustment magnitude.

[0190] Among them, the proportion of region of interest in the first range is higher than that in the second range, the proportion of region of interest in the second range is higher than that in the third range, the first adjustment range is higher than the second adjustment range, and the second adjustment range is higher than the third adjustment range.

[0191] In one implementation, the processor 601 is specifically used for:

[0192] Each video slice is downsampled to obtain a downsampled video corresponding to each video slice.

[0193] Each downsampled video is encoded separately to obtain the encoding features of each downsampled video, and the encoding features of each downsampled video are used as the encoding features of the corresponding slice video.

[0194] In one implementation, the processor 601 is specifically used for:

[0195] Obtain the scene type of each video frame in the target video;

[0196] The target video is sliced ​​based on the scene type of each video frame in the target video to obtain sliced ​​videos of the target video.

[0197] In one implementation, the target video comprises one or more coded image groups, each coded image group comprising a plurality of consecutive video frames in the target video; the processor 601 is specifically configured to:

[0198] For any coded image group in the target video, the coded image group is sliced ​​according to the segmentation rule of dividing consecutive video frames of the same scene type into a slice video, to obtain the slice video of the coded image group.

[0199] In one implementation, the encoding control parameters for each video slice are obtained by calling a parameter prediction model to predict the encoding features of the corresponding video slice; the processor 601 is specifically used for:

[0200] Acquire sample videos, preprocess the sample videos to obtain multiple sample slice videos for the sample videos and sample encoding features for each sample slice video;

[0201] For any sample slice video among the plurality of sample slice videos, the first encoder is invoked and the sample slice video is encoded based on N reference coding control parameters respectively to obtain the reference coding quality of the sample slice video under each reference coding control parameter; where N is a positive integer greater than 1;

[0202] The second encoder is invoked to encode any of the video slices to obtain the target encoding quality. Based on the reference encoding quality of any sample video slice under each reference encoding control parameter and the target encoding quality, the sample encoding quality flag bit of any sample video slice under each reference encoding control parameter is determined. The sample encoding quality flag bit of any sample video slice under a reference encoding control parameter is used to indicate the quality relationship between the reference encoding quality and the target encoding quality of any sample video slice under that reference encoding control parameter.

[0203] Based on the sample coding features of each sample video slice in the sample video, N reference coding control parameters, and the sample coding quality flag bits of each sample video slice under the N reference coding control parameters, the initial prediction model is trained to obtain the parameter prediction model.

[0204] In one implementation, the encoding control parameters for each video slice are obtained by calling a parameter prediction model to predict the encoding features of the corresponding video slice; the processor 601 is specifically used for:

[0205] For any video slice in the target video, the parameter prediction model is invoked to predict the coding quality flag bit based on each of the N reference coding control parameters and the coding features of the video slice, so as to obtain the coding quality flag bit of the video slice under each reference coding control parameter.

[0206] Sort the coding quality flags under the N reference coding control parameters in ascending order to obtain the sorting result for the coding quality flags.

[0207] If the first to the Mth coding quality flags in the sorting result are all second flags, and the (M+1)th coding quality flag is the first flag, then the coding control parameters of any video slice are determined based on the reference coding control parameters corresponding to the Mth coding quality flag and the reference coding control parameters corresponding to the (M+1)th coding quality flag; where M is a positive integer greater than or equal to 1 and less than N.

[0208] This application also provides a computer storage medium storing program instructions, which, when executed, may include, for example... Figure 2a or Figure 2b or Figure 4 Some or all of the steps of the video encoding method in the corresponding embodiments.

[0209] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0210] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0211] This application also provides a computer program product or computer program that includes program instructions. When executed by a processor, the program instructions can implement some or all of the steps in the above-described methods. For example, the program instructions are stored in a computer-readable storage medium. The processor of a computer device reads the program instructions from the computer-readable storage medium and executes the program instructions, causing the computer device to perform the steps executed in the embodiments of the above-described methods.

[0212] The above provides a detailed description of a video encoding method, apparatus, computer device, and medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A video encoding method, characterized in that, The method includes: The target video is sliced ​​to obtain multiple sliced ​​video segments for the target video; each sliced ​​video includes multiple consecutive video frames from the target video. Extract the region of interest from each video slice; The encoding features of each video slice are obtained. For any video slice in the target video, a parameter prediction model is invoked to predict the encoding quality flag based on each of the N reference encoding control parameters and the encoding features of the video slice, thereby obtaining the encoding quality flag of the video slice under each reference encoding control parameter. Based on the encoding quality flag of the video slice under each reference encoding control parameter, the encoding control parameters of the video slice are determined from the N reference encoding control parameters. Using the encoding control parameters of each video slice and the region of interest, each video slice is encoded to obtain the encoded data of each video slice. The encoded data of each video slice is combined to generate the encoded data of the target video.

2. The method according to claim 1, characterized in that, The process of encoding each video slice using its encoding control parameters and region of interest to obtain encoded data for each video slice includes: For any one of the plurality of video slices, obtain the percentage of the region of interest (ROI) in that one video slice relative to the overall region of the target video. The encoding control parameters of any video slice and the region encoding parameters for the region of interest are adjusted based on the proportion of the region of interest in any video slice. The adjusted encoding control parameters and region encoding parameters are used to encode any one of the video slices to obtain the encoded data of the video slice.

3. The method according to claim 2, characterized in that, The encoding control parameters include a fixed bitrate factor (CRF); the adjustment of the encoding control parameters for any video slice based on the region of interest (ROI) ratio of any video slice, and the region encoding parameters for the ROI, include: If the region of interest of any slice video is within a first range, then the fixed bitrate factor and region coding parameters of any slice video are adjusted by a first adjustment magnitude. If the region of interest percentage of any of the video slices is within the second range, then the fixed bitrate factor and region coding parameters of any of the video slices are adjusted by the second adjustment magnitude. If the region of interest of any of the video slices is within the third range, then the fixed bitrate factor of any of the video slices is adjusted by the third adjustment magnitude. Among them, the proportion of region of interest in the first range is higher than that in the second range, the proportion of region of interest in the second range is higher than that in the third range, the first adjustment range is higher than the second adjustment range, and the second adjustment range is higher than the third adjustment range.

4. The method according to claim 1, characterized in that, The step of obtaining the encoded features of each video slice includes: Each video slice is downsampled to obtain a downsampled video corresponding to each video slice. Each downsampled video is encoded separately to obtain the encoding features of each downsampled video, and the encoding features of each downsampled video are used as the encoding features of the corresponding slice video.

5. The method according to claim 4, characterized in that, The step of slicing the target video to obtain multiple sliced ​​video segments includes: Obtain the scene type of each video frame in the target video; The target video is sliced ​​based on the scene type of each video frame in the target video to obtain sliced ​​videos of the target video.

6. The method according to claim 5, characterized in that, The target video contains one or more coded image groups, and each coded image group contains multiple consecutive video frames from the target video; the step of slicing the target video based on the scene type of the video frames in the target video to obtain sliced ​​video of the target video includes: For any coded image group in the target video, the coded image group is sliced ​​according to the segmentation rule of dividing consecutive video frames of the same scene type into a slice video, to obtain the slice video of the coded image group.

7. The method according to claim 1, characterized in that, The encoding control parameters for each video slice are obtained by calling a parameter prediction model to predict the encoding features of the corresponding video slice. The training process of the parameter prediction model includes: Acquire sample videos, preprocess the sample videos to obtain multiple sample slice videos for the sample videos and sample encoding features for each sample slice video; For any sample slice video among the plurality of sample slice videos, the first encoder is invoked and the sample slice video is encoded based on N reference coding control parameters respectively to obtain the reference coding quality of the sample slice video under each reference coding control parameter; where N is a positive integer greater than 1; The second encoder is invoked to encode any of the video slices to obtain the target encoding quality. Based on the reference encoding quality of any sample video slice under each reference encoding control parameter and the target encoding quality, the sample encoding quality flag bit of any sample video slice under each reference encoding control parameter is determined. The sample encoding quality flag bit of any sample video slice under a reference encoding control parameter is used to indicate the quality relationship between the reference encoding quality and the target encoding quality of any sample video slice under that reference encoding control parameter. Based on the sample coding features of each sample video slice in the sample video, N reference coding control parameters, and the sample coding quality flag bits of each sample video slice under the N reference coding control parameters, the initial prediction model is trained to obtain the parameter prediction model.

8. The method according to claim 1, characterized in that, The step of determining the coding control parameters of any video slice from N reference coding control parameters based on the coding quality flag bits of any video slice under each reference coding control parameter includes: Sort the coding quality flags under the N reference coding control parameters in ascending order to obtain the sorting result for the coding quality flags. If the first to the Mth coding quality flags in the sorting result are all second flags, and the (M+1)th coding quality flag is the first flag, then the coding control parameters of any video slice are determined based on the reference coding control parameters corresponding to the Mth coding quality flag and the reference coding control parameters corresponding to the (M+1)th coding quality flag; where M is a positive integer greater than or equal to 1 and less than N.

9. A video encoding device, characterized in that, include: A slicing unit is used to slice the target video to obtain multiple sliced ​​videos for the target video; Any video slice includes multiple consecutive video frames from the target video; An encoding unit is used to acquire the region of interest (ROI) in each video slice; acquire the encoding features of each video slice; for any video slice in the target video, call a parameter prediction model to predict the encoding quality flag based on each of the N reference encoding control parameters and the encoding features of the video slice, to obtain the encoding quality flag of the video slice under each reference encoding control parameter; based on the encoding quality flag of the video slice under each reference encoding control parameter, determine the encoding control parameters of the video slice from the N reference encoding control parameters; and use the encoding control parameters and the ROI of each video slice to encode each video slice to obtain the encoded data of each video slice. The combining unit is used to combine the encoded data of each video slice to generate the encoded data of the target video.

10. A computer device, characterized in that, The device includes a processor and a memory, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, The computer program product includes program instructions that, when executed by a processor, implement the method of any one of claims 1-8.

Images