A video encoding method, apparatus and device

Abstract

This application discloses a video encoding method, apparatus, and device. The method includes: identifying multiple objects contained in a target video frame; mapping the target video frame into multiple segments according to first encoding parameters corresponding to the multiple objects, wherein the first encoding parameters are determined according to encoding parameters used when encoding historical video frames, and the historical video frames include video frames preceding the target video frame; and encoding the target video frame according to the multiple segments to obtain an encoded video frame corresponding to the target video frame.

Description

Technical Field

[0001] This application relates to the field of video coding technology, and in particular to a video coding method, apparatus and device. Background Technology

[0002] When encoding video, related technologies typically group similar parts of video frames together for encoding to further reduce the amount of data in the encoded video and improve the compression ratio. Current encoding technologies mainly group based on parameter information after encoding (including but not limited to quantization coefficients, motion vectors, filtering parameters, etc.). However, this grouping method is inaccurate and easily groups dissimilar parts together, resulting in a deterioration in the image quality of the encoded video frames. Summary of the Invention

[0003] Therefore, this application discloses the following technical solution:

[0004] The first aspect of this application provides a video encoding method, including:

[0005] Identify multiple objects contained in the target video frame;

[0006] The target video frame is mapped into multiple segments according to the first encoding parameters corresponding to the multiple objects. The first encoding parameters are determined according to the encoding parameters used when encoding historical video frames. The historical video frames include video frames before the target video frame.

[0007] The target video frame is encoded according to the multiple segments to obtain the encoded video frame corresponding to the target video frame.

[0008] A second aspect of this application provides a video encoding apparatus, comprising:

[0009] The segmentation module is used to identify multiple objects contained in the target video frame;

[0010] The classification module is used to map the target video frame into multiple segments according to the first encoding parameters corresponding to the multiple objects. The first encoding parameters are determined according to the encoding parameters used when encoding historical video frames. The historical video frames include video frames before the target video frame.

[0011] The encoding module is used to encode the target video frame according to the multiple segments to obtain the encoded video frame corresponding to the target video frame.

[0012] A third aspect of this application provides an electronic device, including a memory and a processor;

[0013] The memory is used to store computer programs;

[0014] The processor is used to execute the computer program to perform:

[0015] Identify multiple objects contained in the target video frame;

[0016] The target video frame is mapped into multiple segments according to the first encoding parameters corresponding to the multiple objects. The first encoding parameters are determined according to the encoding parameters used when encoding historical video frames. The historical video frames include video frames before the target video frame.

[0017] The target video frame is encoded according to the multiple segments to obtain the encoded video frame corresponding to the target video frame. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 This is a flowchart of a video encoding method provided in an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of the object recognition result of a target video frame provided in an embodiment of this application;

[0021] Figure 3 This is a flowchart illustrating a method for determining first motion coding parameters provided in an embodiment of this application;

[0022] Figure 4 This is a flowchart of a method for mapping a target video frame into multiple segments, provided in an embodiment of this application.

[0023] Figure 5 This is a schematic diagram illustrating the working principle of a video encoding device provided in an embodiment of this application;

[0024] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] This embodiment provides a video encoding method; please refer to [link to relevant documentation]. Figure 1 The method may include the following steps.

[0027] S101, identify multiple objects contained in the target video frame.

[0028] S102, the target video frame is mapped into multiple segments according to the first encoding parameters corresponding to multiple objects. The first encoding parameters are determined according to the encoding parameters used when encoding historical video frames. Historical video frames include video frames before the target video frame.

[0029] S103: Encode the target video frame according to multiple segments to obtain the encoded video frame corresponding to the target video frame.

[0030] The video encoding method of this embodiment can be executed by any electronic device with video encoding function. The type of electronic device is not limited, such as any of mobile phones, desktop computers and tablet computers.

[0031] The beneficial effects of this embodiment are as follows: Compared with the scheme that directly groups based on parameter information, this scheme, when mapping the target video frame into multiple segments, not only considers the encoding parameters, but also further combines the object recognition results of the target video frame, which is conducive to obtaining more accurate segments and avoiding the situation of mapping dissimilar parts to the same segment.

[0032] On the other hand, after mapping the target video frame into multiple segments, the target video frame can be encoded in units of segments. When encoding a segment, the same set of encoding parameters can be shared, which helps to improve encoding efficiency.

[0033] The target video frame in step S101 can be any video frame in the video to be encoded, excluding the first video frame. That is, when encoding the video, starting from the second video frame, the electronic device can treat each video frame as the target video frame, and proceed accordingly. Figure 1 Alternatively, the video encoding method of other embodiments of this application can encode each video frame to obtain the corresponding encoded video frame.

[0034] Optionally, when encoding the first video frame, it is also possible to identify that the first video frame contains multiple objects, then determine the second encoding parameters of each object in the first video frame, map the first video frame into multiple segments according to the second encoding parameters corresponding to the multiple objects, and encode the first video frame according to the multiple segments to obtain the encoded video frame corresponding to the first video frame.

[0035] The method of mapping the first video frame to multiple segments based on the second encoding parameters corresponding to multiple objects is the same as the method of mapping the target video frame to multiple segments based on the first encoding parameters corresponding to multiple objects. The only difference is that the first encoding parameters are replaced with the second encoding parameters.

[0036] The second coding parameters of each object in the first video frame can be specified by the user, or determined in the following way: The coding parameters of each coding block in the first video frame, divided according to a preset size (e.g., 4x4 pixels), are determined using a coding model. For each object, the second coding parameter is determined based on the coding parameters of multiple coding blocks belonging to that object. For example, the average, minimum, maximum, or other statistical values ​​of the coding parameters of multiple coding blocks belonging to that object can be used to determine the second coding parameter of that object. For methods of determining the coding parameters of coding blocks using a coding model, please refer to relevant technologies in the field of video coding.

[0037] Step S101 can be implemented using relevant technologies in the field of video object recognition. For example, the target video frame can be input into a pre-built panoramic segmentation model to obtain the recognition result output by the panoramic segmentation model. The recognition result can indicate the objects present in the target video frame and indicate which object each pixel in the target video frame belongs to.

[0038] A panoramic segmentation model can be a neural network model that can identify objects contained in a video frame. Its model structure and working principle can be found in the relevant technologies in the field of video object recognition, which will not be elaborated here.

[0039] As an example, the recognition results of the target video frame obtained by the panoramic segmentation model can be used... Figure 2 express, Figure 2 The pixels in the target video frame correspond one-to-one with the pixels in the target video frame, through... Figure 2 The color of each pixel can distinguish which pixels belong to the same object and which belong to different objects. If multiple pixels in the target video frame are... Figure 2 The same color indicates that these pixels belong to the same object. The numbers 1 to 7 represent the 7 objects contained in the target video frame, namely object 1 to object 7.

[0040] The target video frame contains multiple objects, which may include the video background of the target video frame, such as... Figure 2 Objects 1 to 3 can also include objects displayed in the target video frame, such as vehicles, pedestrians, buildings, etc. Figure 2 Objects 4 to 7.

[0041] Historical video frames may include the first two video frames of the target video frame in the video to be encoded. For example, if the target video frame is the 10th video frame of the video to be encoded, then historical video frames may include the 9th and 8th video frames of the video to be encoded. Alternatively, historical video frames may only include the video frame preceding the target video frame in the video to be encoded, such as only including the 9th video frame of the video to be encoded in the example above.

[0042] The encoding parameters used when encoding historical video frames can be determined and recorded during the encoding of historical video frames, so that when encoding the target video frame according to the method of this embodiment, the encoding parameters used when encoding historical video frames can be read, and the first encoding parameters of multiple objects contained in the target video frame can be determined based on these encoding parameters.

[0043] The following example uses any object X in the target video frame to illustrate the method for determining the first encoding parameter corresponding to an object in the target video frame.

[0044] Optionally, the first encoding parameter of any object (such as object X mentioned above) includes a first quantization encoding parameter and a first non-quantization encoding parameter;

[0045] Methods for determining the first quantization encoding parameter and the first non-quantization encoding parameter of any object include:

[0046] Multiple coding blocks belonging to the object are identified, and the coding blocks are obtained by dividing the target video frame based on a preset size;

[0047] The average of the historical quantization coding parameters of multiple coding blocks belonging to the object is determined as the first quantization coding parameter of the object;

[0048] The minimum value of the historical unquantized coding parameters of multiple coding blocks belonging to the object is determined as the first unquantized coding parameter of the object;

[0049] Historical quantization coding parameters and historical non-quantization coding parameters are the quantization coding parameters and non-quantization coding parameters used when encoding historical video frames.

[0050] Among them, quantization coding parameters and non-quantization coding parameters are both parameters related to the image quality after video encoding. Therefore, quantization coding parameters and non-quantization coding parameters can also be collectively referred to as image quality parameters.

[0051] Quantization coding parameters refer to the quantization coefficients (denoted by QP) used when encoding video. The role of the quantization coefficients is to allow the encoding of video frames by first processing the frames using a filter designed based on the Discrete Cosine Transform (DCT) algorithm. This transforms the pixel data of the video frame into frequency domain data. The frequency domain data is then divided by the quantization coefficients to obtain compressed quantized frequency domain data. Finally, the encoded video frame is obtained based on the quantized frequency domain data. The structure and working principle of the filter designed based on the Discrete Cosine Transform (DCT) algorithm can be found in relevant technologies in the field of video coding, and will not be elaborated here.

[0052] Non-quantization coding parameters refer to other parameters that are related to the image quality after video encoding and are not part of the quantization coefficients. For example, non-quantization coding parameters may include, but are not limited to, filter parameters (denoted by LR). Filter parameters refer to the parameters used by the aforementioned filter when it is working.

[0053] The first quantization coding parameter of object X refers to the quantization coding parameter determined based on the historical quantization coding parameters of multiple coding blocks belonging to object X. The first non-quantization coding parameter of object X refers to the non-quantization coding parameter determined based on the historical non-quantization coding parameters of multiple coding blocks belonging to object X.

[0054] The preset size for dividing the coding blocks can be determined based on the video coding protocol used by the electronic device, or it can be specified by the user. One coding block is equivalent to a pixel block of a preset size in the target video frame. As an example, if the preset size is 4*4, the target video frame can be divided into multiple coding blocks of 4*4 pixels each, with each coding block containing 4 rows * 4 columns, totaling 16 pixels.

[0055] For any object X, the method to determine multiple coding blocks belonging to object X can be as follows: for each coding block, if at least one pixel in this coding block belongs to object X, then this coding block is determined to belong to object X. Determining coding blocks belonging to object X in this way only requires judging whether there are pixels of object X in the coding block based on the recognition results obtained during previous object recognition. Therefore, it can quickly determine coding blocks belonging to object X, which helps improve the execution efficiency of the method in this embodiment.

[0056] For any object X, the way to determine multiple coding blocks belonging to object X is as follows: if all pixels contained in a coding block belong to object X, then the coding block is determined to belong to object X; if a coding block contains pixels belonging to object X and pixels belonging to other objects, then the number of pixels belonging to different objects in the coding block is compared, and the object with the largest number of corresponding pixels is determined to belong to the coding block.

[0057] For example, if a 4x4 coded block contains 12 pixels belonging to object X and 4 pixels belonging to object Y, then it can be determined that this coded block belongs to object X.

[0058] The advantage of determining the coding block belonging to object X in the above manner is that it can more accurately determine the object to which each coding block belongs, avoiding the situation where a coding block belongs to multiple objects at the same time. This is conducive to encoding the target video frame more accurately and improving the image quality of the obtained coded video frame.

[0059] The historical quantization coding parameters of a coding block refer to the quantization coding parameters used for the coding block at the same position in the historical video frame when encoding historical video frames.

[0060] The historical nonquantized coding parameters of a coding block refer to the nonquantized coding parameters used for the coding block at the same position in the historical video frame when encoding historical video frames.

[0061] For example, the target video frame is divided into m rows and n columns according to a preset size, with a total of n*m coding blocks. When encoding historical video frames, the historical video frames are also divided into m rows and n columns according to a preset size. The historical quantization coding parameters of the coding block in the 2nd row and 3rd column of the target video frame are the same as the quantization coding parameters used in the coding block in the 2nd row and 3rd column of the historical video frame when encoding the historical video frame. The historical non-quantization coding parameters of the coding block in the 4th row and 8th column of the target video frame are the same as the non-quantization coding parameters used in the coding block in the 4th row and 8th column of the historical video frame when encoding the historical video frame.

[0062] Both historical quantized encoding parameters and historical non-quantized encoding parameters can be stored in a designated storage space when encoding historical video frames, and can be read when encoding the target video frame according to the method of this embodiment.

[0063] After identifying multiple coded blocks belonging to object X, the historical quantization coding parameters of all coded blocks belonging to object X can be obtained. The average value of these historical quantization coding parameters is calculated, and this average value is used as the first quantization coding parameter of object X. The calculation process can be expressed by formula (1): QP sX =AVG(QP) （x，y） |Block(x,y)∈X), (1).

[0064] QP sX Let X represent the first quantization coding parameter of object X, AVG() means to calculate the average value, Block(x, y)∈X means the coded block belonging to object X in the x-th row and y-th column of the target video frame, and QP （x，y） This represents the historical quantization encoding parameters of Block(x, y).

[0065] The purpose of using the average value as the first quantization encoding parameter is twofold. First, the smaller the value of the quantization encoding parameter, the larger the amount of data in the encoded video frame may be. By calculating the average value, a larger first quantization encoding parameter can be obtained, thus minimizing the amount of data in the encoded video frame. Second, the historical quantization encoding parameters stored when encoding historical video frames may contain both positive and negative values. Positive quantization encoding parameters are easier to store. By calculating the average value, a positive first quantization encoding parameter can be obtained as much as possible, making it easier to store the first quantization encoding parameter and reducing the storage space occupied by storing the first quantization encoding parameter.

[0066] After identifying multiple coded blocks belonging to object X, we can obtain the historical unquantized coding parameters of all coded blocks belonging to object X, determine the minimum value of these historical unquantized coding parameters, and use this minimum value as the first unquantized coding parameter of object X.

[0067] The process of calculating the first non-quantized coding parameter can be represented by formula (2): FP sX =MIN(FP) （x，y） |Block(x,y)∈X), (2). FP sX This represents the first non-quantized encoding parameter of object X, MIN() represents taking the minimum value, and FP （x，y） This represents the historical unquantized encoding parameters of Block(x,y), such as the historical filtering parameters of Block(x,y).

[0068] The purpose of determining the first non-quantized encoding parameter in the above manner is that the non-quantized encoding parameter is generally a positive value, so there is no need to ensure that the first non-quantized encoding parameter is a positive value by calculating the average value. At the same time, the amount of computation required to determine the minimum value is less than the amount of computation required to calculate the average value. Therefore, determining the minimum value as the first non-quantized encoding parameter is beneficial to reducing the amount of computation generated by executing the method of this embodiment.

[0069] Optionally, considering that the encoding parameters used for the background part in the video frame are basically the same, when determining the first encoding parameters of each object, the objects belonging to the background in the target video frame can be identified, and the same first encoding parameters can be determined for all objects belonging to the background. In other words, all objects belonging to the background in the target video frame (denoted as background objects) share the same set of first encoding parameters.

[0070] Therefore, when determining the first encoding parameters of a background object, all background objects can be regarded as a whole, and the aforementioned method for determining the first encoding parameters of any object can be applied to this whole to obtain a set of first encoding parameters shared by all background objects.

[0071] by Figure 2 For example, objects 1, 2, and 3 all belong to the background; therefore, they share the same first quantization encoding parameter and first non-quantization encoding parameter. Which objects specifically belong to the background can be determined using the aforementioned panoptic segmentation model. The working principle of this model can be found in related technologies and will not be elaborated upon here.

[0072] Determining the first quantization encoding parameter and the first non-quantization encoding parameter of the background object using the above method helps to reduce the amount of computation when executing the method of this embodiment, thereby improving execution efficiency and saving storage space and computing resources.

[0073] Optionally, if the object is a moving object, the object's first encoding parameter may also include a first motion encoding parameter;

[0074] See Figure 3 The method for determining the first motion coding parameters of an object may include the following steps.

[0075] S301, obtain the difference in historical motion coding parameters of the same object in the first two video frames of the target video frame.

[0076] S302, based on the difference, compensate for the historical motion coding parameters of the same object in the previous video frame of the target video frame to obtain the first motion coding parameters of the object.

[0077] If an object in the target video frame is a moving object, it means that the position of this object in the target video frame is different from the position of this object in the historical video frames. In other words, this object has moved relative to the historical video frames.

[0078] Whether each object in the target video frame is a moving object can be determined as follows:

[0079] First, based on the aforementioned panoramic segmentation model, multiple objects in the target video frame are divided into background objects belonging to the background and foreground objects not belonging to the background;

[0080] Then, for each foreground object, the proportion of overlapping instances in that object determines whether it belongs to a moving object or a non-moving object. If the proportion of overlapping instances in an object is greater than a preset threshold S, the object is considered a moving object. Gate If the percentage of overlapping instances in an object is less than or equal to this threshold, then the object is determined to be a non-moving object. If the percentage of overlapping instances in an object is less than or equal to this threshold, then the object is determined to be a moving object.

[0081] In other words, if an object X satisfies the inequality shown in formula (3), it is a non-moving object; if it does not satisfy this inequality, it is a moving object.

[0082]

[0083] Where Sx represents the total number of pixels belonging to object X, and P(i,k) represents whether the pixel (i,k) in the i-th row and k-th column of object X is an instance overlap point. If (i,k) is an instance overlap point, then P(i,k) equals 1, and if (i,k) is not an instance overlap point, then P(i,k) equals 0.

[0084] Threshold value S Gate This threshold represents the lower limit of the percentage of pixels that overlap between a non-moving object in a video frame and a reference frame. If an object in the target video frame is a non-moving object, then the percentage of pixels of this object in the target video frame that overlap with the percentage of pixels of this object in the reference frame should be greater than the threshold value. If the percentage of overlap is not greater than the threshold value, the object can be considered to be moving and thus a moving object. The threshold value can be a real number less than 1 and close to 1, with no specific numerical limit. For example, the threshold value can range from 0.8 to 1, such as 0.86, 0.95, etc.

[0085] In practical applications, the threshold value can be determined manually or in the following way:

[0086] Pre-label at least one non-moving object in at least one video frame. Use a panoramic segmentation model to identify the pixels belonging to the non-moving object in these video frames. For each non-moving object, determine the proportion of instance overlap points among the pixels belonging to this non-moving object. Then determine the threshold value based on the proportion of instance overlap points corresponding to each non-moving object. For example, the average or minimum value of the proportion of these instance overlap points can be used as the threshold value.

[0087] The advantage of determining the threshold value in this way is that, theoretically, all pixels of non-moving objects should belong to instance overlap points. However, in actual applications, due to the recognition error of the panoramic segmentation model, the occlusion of moving objects, and other possible interference factors, pixels of non-moving objects may not all belong to instance overlap points. According to the above method, a threshold value that conforms to the actual situation can be determined under the premise of considering the above interference factors, which is conducive to more accurately distinguishing moving objects and non-moving objects.

[0088] Whether a pixel (i, k) is a point of overlap with an instance can be determined as follows:

[0089] If pixel (i, k) in the target video frame and pixel (i, k) in the corresponding reference frame belong to the same object, for example, both belong to object X, then pixel (i, k) is an instance coincidence point. If pixel (i, k) in the target video frame and pixel (i, k) in the corresponding reference frame do not belong to the same object, for example, it belongs to object X in the target video frame and to object Y in the reference frame, then pixel (i, k) is not an instance coincidence point.

[0090] The reference frame corresponding to the target video frame refers to another video frame in the video to be encoded that is different from the target video frame. It is generally a video frame that precedes the target video frame. For details on how to determine the reference frame, please refer to the relevant technologies in the field of video coding, which will not be elaborated here.

[0091] Motion coding parameters refer to the parameters used in video encoding related to inter-frame prediction. In this embodiment, motion coding parameters may include, but are not limited to, motion vectors (represented by MV) and prediction modes (represented by pred_mode). The motion vector represents the direction and magnitude of the movement of an object in the target video frame relative to the same object in historical video frames, and the prediction mode represents whether inter-frame prediction or intra-frame prediction is used for encoding. Generally, for moving objects, the inter-frame prediction method is fixed for encoding. Therefore, it can be considered that the motion coding parameters only include motion vectors.

[0092] Therefore, for any object X in the target video frame, if object X is a moving object, then it can be determined by... Figure 3 The method involves determining the motion vector of object X in the target video frame based on the historical motion coding parameters of the same object X in the first two video frames, and using this motion vector as the first motion coding parameter of object X in the target video frame. The historical motion coding parameters of object X in the first two video frames refer to the motion vectors of object X determined when encoding the first two video frames.

[0093] The following example uses object X to illustrate this. Figure 3 The implementation process of the method shown.

[0094] When performing step S301, two video frames preceding the target video frame in the video to be encoded can be obtained as historical video frames. For example, if the target video frame is the t-th (e.g., the 20th) video frame, then the (t-1)-th (e.g., the 19th) and (t-2)-th (e.g., the 18th) video frames can be obtained as historical video frames. Then, the motion vector MV of object X in the (t-1)-th historical video frame is obtained. t-1 The motion vector MV of object X in the (t-2)th historical video frame t-2 And obtain the difference MV between the two. t-1 -MV t-2 MV t-1This is equivalent to the historical motion coding parameters of the same object (i.e., object X) when encoding the (t-1)th video frame, MV t-2 This is equivalent to the historical motion coding parameters of the same object when encoding the (t-2)th video frame, MV t-1 -MV t-2 It refers to the difference in historical motion encoding parameters of the same object.

[0095] When performing step S302, the obtained difference can be multiplied by a preset compensation coefficient, and the product is added to the motion vector X of the same object X in the previous video frame, i.e., the (t-1)th video frame mentioned above. The result is the motion vector of object X in the target video frame. The motion vector X of the same object X in the previous video frame is equivalent to the historical motion encoding parameters of the same object when encoding the previous video frame in S302. The compensation coefficient can be a positive number less than 1, and its specific value can be set as needed; for example, a compensation coefficient k can be set. b It can be equal to 0.5, 0.82 or other values, without limitation.

[0096] The above method can be expressed by formula (4), MV pX =MV t-1 +k b (MV) t-1 -MV t-2 (4). Among them, MV pX k represents the motion vector of object X in the target video frame, i.e., the first motion coding parameter of object X obtained in S302. b The above compensation coefficient is given.

[0097] When encoding historical video frames, the motion vector of object X in the historical video frame can be determined based on the motion vectors of the coded blocks belonging to object X in the historical video frame. For example, all coded blocks belonging to object X in the historical video frame can be identified, and the average of the motion vectors of these coded blocks can be used as the motion vector of object X in the historical video frame. Alternatively, a coded block in the historical video frame whose pixels all belong to object X can be identified, and the motion vector of this coded block can be used as the motion vector of object X in the historical video frame.

[0098] The advantage of obtaining the first motion coding parameters of the moving object in the target video frame using the above method is that by combining the difference between the historical motion coding parameters of the same object in the previous two video frames, the historical motion coding parameters of the same object in the previous video frame can be compensated. This can more accurately predict the first motion coding parameters of object X in the target video frame, thereby improving the image quality of the encoded video frame.

[0099] On the other hand, the first motion coding parameters obtained by the above method can more accurately reflect the actual motion of each object in the target video frame (such as motion amplitude, direction, etc.). In this way, when using the first motion coding parameters to determine the segmentation in the future, the accuracy of the segmentation result can be improved, and coding blocks with similar or identical actual motion can be mapped to the same segment, while coding blocks with dissimilar actual motion can be mapped to different segments.

[0100] Alternatively, the first motion coding parameters of the moving object in the target video frame can be determined using other methods, not limited to... Figure 3 The method shown. For example, it is possible to obtain the motion vector of the same object in the previous video frame of the target video frame (e.g., the MV above). t-1 This motion vector is used as the motion vector of the moving object in the target video frame, that is, the MV is... t-1 Assign directly to MV p This allows for faster determination of the first motion coding parameters of the moving object in the target video frame, improving the execution efficiency of the method in this embodiment.

[0101] In summary, the method for determining the first encoding parameter of an object in this embodiment includes, for background objects and non-moving objects, determining the corresponding first quantization encoding parameter and first non-quantization encoding parameter according to the aforementioned method; for moving objects, determining the corresponding first quantization encoding parameter, first non-quantization encoding parameter, and first motion encoding parameter according to the aforementioned method. The first quantization encoding parameter and first non-quantization encoding parameter are the same for multiple background objects. The first encoding parameter includes any one or more of the first quantization encoding parameter, first non-quantization encoding parameter, and first motion encoding parameter.

[0102] An optional method for mapping a target video frame into multiple segments based on a first encoding parameter may include: Figure 4 The steps are shown.

[0103] S401, based on the first encoding parameters corresponding to multiple objects, determine the feature vector of each encoding block of the target video frame, wherein the encoding block is obtained by dividing the target video frame based on a preset size.

[0104] S402, based on the feature vector, cluster each coded block contained in the target video frame to obtain multiple segments mapped to the coded blocks, wherein each segment is mapped to at least one coded block of the target video frame.

[0105] according to Figure 4The advantages of this method of mapping target video frames into multiple segments are as follows: First, by determining the feature vector of the coding block based on the first coding parameters of multiple objects, the identified objects and coding parameters can be combined when mapping each segment, thereby mapping multiple coding blocks to segments more accurately. Second, by first determining the feature vector of the coding block and then performing clustering based on the feature vector of the coding block, it can be ensured that each coding block is mapped to a unique segment after clustering, avoiding the situation where a coding block is mapped to multiple segments at the same time.

[0106] Optionally, the feature vector of each coded block in step S401 can be determined as follows.

[0107] S4011, determine the parameter vector of the object based on the first encoding parameter corresponding to the object.

[0108] Since all background objects share the same first encoding parameter, all background objects can correspond to the same parameter vector.

[0109] S4012, based on the positional relationship of each coded block in the target video frame, determine whether each coded block is located inside an object or at the boundary between different objects.

[0110] S4013, For a coded block located inside an object, determine the feature vector of the coded block based on the parameter vector of the object to which it belongs.

[0111] S4014: For coded blocks located at the boundaries between different objects, the parameter vectors of different objects are fused to obtain the feature vector of the coded block.

[0112] The advantage of determining the feature vector of each coding block in the above way is that, for coding blocks located at the boundaries between different objects, the feature vector obtained by fusing the parameter vectors of different objects can more accurately represent the characteristics of this coding block. Therefore, the feature vector obtained in this way can improve the accuracy of subsequent clustering processing, effectively avoid the situation where inaccurate clustering maps dissimilar coding blocks to the same segment, and thus improve the image quality of the encoded video frame.

[0113] In step S4011, for any object X, the method for determining the object's parameter vector based on the object's corresponding first encoding parameters can be as follows: directly arrange the object's corresponding first quantization encoding parameters, first non-quantization encoding parameters, and first motion encoding parameters in sequence to form an N-dimensional vector, and use this vector as the parameter vector of the object X, where N is the number of first encoding parameters corresponding to the object X, i.e., T. Xn =(QP sX FP sX MV pX ), T Xn QP represents the parameter vector of the nth object Xn in the target video frame.sX FP sX MV pX These represent the first quantization coding parameter, the first non-quantization coding parameter, and the first motion coding parameter of the nth object Xn in the target video frame, respectively.

[0114] For background objects and non-moving objects, the first motion coding parameter in their parameter vector can be set to 0, or the first motion coding parameters of background objects and non-moving objects can be determined in the same way as the method described above for determining the first motion coding parameter of moving objects.

[0115] In step S4012, the edges of each object in the target video frame can be determined based on the recognition results obtained when recognizing multiple objects. Then, the coding blocks located on both sides of the object edge are determined as coding blocks located at the boundary between different objects, and other coding blocks are determined as coding blocks located inside the object.

[0116] Alternatively, in step S4012, for each coding block, it can be determined whether the pixels contained in the coding block belong to the same object. If all the pixels contained in a coding block belong to the same object, such as object X, then it is determined that the coding block is located inside the object, for example, inside object X. If all the pixels contained in a coding block belong to two or more different objects, for example, some belong to object X1, some to object X2, and some to object X3, then it is determined that the coding block is located at the boundary between different objects.

[0117] An alternative method for obtaining the feature vector of a coding block by fusing parameter vectors is to sequentially concatenate the parameter vectors of multiple different objects corresponding to the boundary of the coding block, and use the concatenated vector as the feature vector of the coding block.

[0118] The specific object parameter vectors to be merged can be determined based on the pixels contained in the coding block. For example, if the pixels in the coding block belong to object X1 and object X2 respectively, then the parameter vector T of object X1 is concatenated. X1 and the parameter vector T of object X2 X2 Obtain the feature vector of the coded block. If the pixels contained in the coded block belong to objects X1, X2, and X3 respectively, then concatenate the parameter vector T of object X1. X1 The parameter vector T of object X2 X2 and the parameter vector T of object X3 X2 The feature vector of the coded block is obtained.

[0119] Because of the small size, in most cases a coding block contains pixels belonging to a maximum of three different objects. Therefore, the feature vector of a coding block can be set to be formed by concatenating three parameter vectors.

[0120] For a small number of encoded blocks containing pixels belonging to four or more objects, the parameter vectors of the three objects with the most corresponding pixels can be concatenated to form a feature vector, while the parameter vectors of other objects are ignored. For example, in a 4x4 encoded block, if 5 pixels belong to object X1, 5 pixels belong to object X2, 4 pixels belong to object X3, and 2 pixels belong to object X4, the parameter vector T of object X1 can be concatenated. X1 The parameter vector T of object X2 X2 and the parameter vector T of object X3 X2 Obtain the feature vector of the coded block, ignoring the parameter vector T of object X4. X4 .

[0121] Limiting the number of parameter vectors concatenated has the advantage of avoiding excessively large feature vector data, thereby saving resources consumed by the method in this embodiment and improving execution efficiency.

[0122] When the pixels in a coded block belong to two different objects, two of the three parameter vectors used for concatenation can be the parameter vector of the object with more corresponding pixels in the coded block, and one can be the parameter vector of the other object. For example, if a 4x4 coded block has 11 pixels belonging to object X1 and 5 pixels belonging to object X2, then the two T... X1 And a T X2 The feature vector of this coding block is formed by concatenating the features.

[0123] The advantage of using the above method for concatenation is that the number of parameter vectors of each object in the feature vector can represent the number of pixels belonging to different objects, making the obtained feature vector more accurately represent the features of the coding block located at the boundary.

[0124] When fusing parameter vectors using the above method, to ensure that the feature vectors of different coding blocks all have the same dimension, the method for determining the feature vector of a coding block located inside an object in S4013 can be to copy the parameter vector of the object to which it belongs three times, and then concatenate these three identical parameter vectors to form the feature vector of the coding block located inside the object. For example, if all pixels contained in a coding block belong to object X1, that is, this coding block is located inside object X1, then the three T... X1 The feature vector of this coding block is formed by concatenating the features.

[0125] Based on the above method, for any coded block B1, the feature vector of the coded block can be any one of the following three cases:

[0126] First, if coded block B1 is located inside object X1, and all its pixels belong to object X1, then the feature vector V of coded block B1... B1 = (T X1 T X1 T X1);

[0127] Second, if the encoding block B1 is located at the boundary of different objects, and some pixels of B1 belong to object X1, while other pixels belong to object X2, and more pixels belong to X2, then the feature vector V of the encoding block B1... B1 = (T X1 T X2 T X2 );

[0128] Second, the coding block B1 is located at the boundary of different objects. The pixels of B1 are divided into three parts, belonging to object X1, object X2, and object X3 respectively. Then the feature vector V of the coding block B1 is... B1 = (T X1 T X2 T X3 ).

[0129] The advantage of obtaining the feature vector using the above method is that after obtaining the parameter vector, there is no need to perform secondary calculations on the parameter vector. The feature vector of the coding block can be obtained simply by concatenating the parameter vectors according to the above rules, thereby improving the execution efficiency of the method in this embodiment.

[0130] Another alternative method to obtain the feature vector of a coding block by fusing parameter vectors is to weight and sum the parameter vectors of the objects to which each pixel belongs in the coding block according to the proportion of pixels belonging to each object, and use the result as the feature vector of this coding block. For example, if a 4*4 coding block B1 has 11 pixels belonging to object X1 and 5 pixels belonging to object X2, then the feature vector V of coding block B1 is... B1 =11 / 16*T X1 +5 / 16*T X2 .

[0131] When the feature vector is obtained by fusing using the above method, the feature vector of the coded block located inside the object in S4013 can be determined by using the parameter vector of the object as the feature vector of the coded block located inside the object. That is, if the coded block B1 is located inside the object X1, and all pixels contained in B1 belong to the object X1, then the feature vector V of the coded block B1 is... B1 =T X1 .

[0132] After obtaining the feature vectors of each coded block in the target video frame, step S402 can be executed based on these feature vectors. In step S402, any clustering algorithm (such as K-means clustering) can be used to process the feature vectors of each coded block in the target video frame. Coded blocks with similar feature vectors are grouped into the same class, and coded blocks with dissimilar feature vectors are grouped into different classes. This divides all coded blocks of the target video frame into K classes. The set of coded blocks in each class is equivalent to a segment (denoted as a segment) mapped to the target video frame, resulting in K segments. Here, K is a preset integer, and its specific value can be preset as needed, such as 5, 10, 12, or other values. A larger K value can be set when a higher quality coded video frame is required, and a smaller K value can be set when higher encoding efficiency is needed. For methods of clustering the feature vectors of each coded block, please refer to relevant techniques in the field of clustering algorithms.

[0133] After mapping multiple coding blocks into K segments by clustering, the subsequent encoding processing of the target video frame can be performed on a segment-by-segment basis, instead of on a block-by-block basis. Since the number of segments obtained by clustering is obviously less than the number of coding blocks, mapping multiple segments as described above can reduce the amount of computation in subsequent encoding stages, thereby improving encoding efficiency and saving computational resources.

[0134] Optionally, there may be multiple first encoded parameters corresponding to an object, and the parameter vector of the object determined in step S4011 based on the first encoded parameters corresponding to the object may also be:

[0135] The parameter vector of the object is determined based on the multiple first coding parameters corresponding to the object and the influence factor of each first coding parameter;

[0136] The influence factor of the first encoding parameter is determined based on the degree of influence of the first encoding parameter on the amount of data in the encoded video frame.

[0137] In this embodiment, for any object, each of the first coding parameters corresponding to this object can be multiplied by the influence factor corresponding to each of the first coding parameters, and the resulting product is used as the scaled first coding parameter. Then, the scaled first coding parameters are arranged sequentially to form an N-dimensional vector, and this vector is used as the parameter vector of this object X, where N is the number of first coding parameters corresponding to object X, i.e., T. Xn =(Ks1*QP sX Ks2*FP sX Ks3*MV pX ), T Xn QP represents the parameter vector of the nth object Xn in the target video frame. sX FPsX MV pX Let Ks1 represent the first quantization coding parameter, the first non-quantization coding parameter, and the first motion coding parameter of the nth object Xn in the target video frame, respectively. Let Ks1 represent the influence factor corresponding to the first quantization coding parameter, Ks2 represent the influence factor corresponding to the first non-quantization coding parameter, and Ks3 represent the influence factor corresponding to the first motion coding parameter; Ks1*QP sX Ks2*FP sX and Ks3*MV pX These represent the first quantized encoding parameter after scaling, the first non-quantized encoding parameter after scaling, and the first motion encoding parameter after scaling, respectively.

[0138] The influence factors corresponding to different first coding parameters can be the same or different. When encoding different videos, the influence factors corresponding to the first coding parameters can be the same or different. For example, when encoding video frames of video 1, Ks1 can be equal to 0.6, and when encoding video frames of video 2, Ks1 can be equal to 0.7. Video 1 and Video 2 represent two different videos.

[0139] When encoding a video, the influence factor of the first encoding parameter can be determined based on the degree of influence of the first encoding parameter on the data volume of the encoded video frame. One possible method for determining the influence factor based on the degree of influence includes:

[0140] For each first encoding parameter, multiple candidate influence factors of different sizes are preset based on experience. For example, the candidate influence factors corresponding to the first quantization encoding parameter are determined to be 0.35, 0.4, 0.45, 0.5 and 0.6. Then, each candidate influence factor is substituted into the aforementioned method for determining the parameter vector based on the influence factor to determine different parameter vectors. The encoding method of this embodiment is applied to encode the same target video frame based on different parameter vectors to obtain multiple different candidate encoded video frames. The data volume of multiple candidate encoded video frames is compared, and the candidate encoded video frame with the smallest data volume is determined. The candidate influence factor applied to this candidate encoded video frame is determined as the influence factor when encoding this video.

[0141] For example, after comparison, it was found that the candidate video frame with the smallest data size was obtained when Ks1 equals 0.5, Ks2 equals 0.6, and Ks3 equals 0.55. Therefore, Ks1 equals 0.5, Ks2 equals 0.6, and Ks3 equals 0.55 were used as image factors to encode each video frame of the video to be encoded.

[0142] The advantage of determining the parameter vector of the object using the above method is that: by dynamically setting the influence factor applicable to a specific video to be encoded based on the degree of influence on the amount of data, the residual of the quantized frequency domain data obtained based on the filter and quantization coefficients relative to the original target video frame can be reduced as much as possible, thereby reducing the amount of residual data in the encoded video frame, and ultimately reducing the amount of data in the encoded video frame, thus improving the compression ratio of the encoding method in this embodiment.

[0143] Another alternative method for mapping the target video frame into multiple segments based on the first encoding parameters may include the following steps:

[0144] Based on the first encoding parameters corresponding to multiple objects, the multiple objects are clustered to obtain multiple slices mapped to multiple objects, wherein each slice is mapped to at least one object;

[0145] The coded blocks contained in the object are mapped to the corresponding fragments of the object to obtain multiple fragments mapped to the coded blocks. Each fragment is mapped to at least one coded block of the target video frame. The coded blocks are obtained by dividing the target video frame based on a preset size.

[0146] In this embodiment, for each object in the target video frame, the parameter vector corresponding to the object can be determined based on the first encoding parameter corresponding to the object, in accordance with the method for determining the parameter vector in the previous embodiment, thereby obtaining multiple parameter vectors corresponding to multiple objects contained in the target video frame.

[0147] Then, clustering algorithms (such as K-means) can be used to process the multiple parameter vectors corresponding to multiple objects. Objects with similar parameter vectors are grouped into the same class, and objects with dissimilar parameter vectors are grouped into different classes. The multiple objects contained in the target video frame are divided into K categories. The set of objects in the same class can be regarded as a slice, thus obtaining multiple slices mapped to multiple objects. K is a preset integer, and its specific value can be preset as needed.

[0148] Combination Figure 2 For example, using the above method, object 1, object 2, and object 3 can be mapped to the same shard, denoted as shard 1; object 4, object 6, and object 7 can be mapped to the same shard, denoted as shard 2; and object 5 can be mapped to a shard, denoted as shard 3.

[0149] Then, for each object, the coded blocks contained in the object can be mapped to the corresponding fragments of the object. In this way, multiple fragments can be mapped to each coded block of the target video frame, achieving the effect of mapping the target video frame into multiple fragments.

[0150] Based on the previous example, the coded blocks contained in objects 1, 2, and 3 can be mapped to fragment 1. In other words, the set of coded blocks contained in objects 1, 2, and 3 can be used as fragment 1. Similarly, the set of coded blocks contained in objects 4, 6, and 7 can be used as fragment 2, and the set of coded blocks contained in object 5 can be used as fragment 3.

[0151] The advantage of mapping to multiple segments using the above method is that when performing clustering, it is not necessary to process the feature vectors corresponding to the coded blocks in the target video frame, but only the parameter vectors of each object. The number of coded blocks in a video frame obviously far exceeds the number of objects, so the above method can reduce the number of vectors to be processed during clustering, thereby improving the execution efficiency of the method in this embodiment.

[0152] Optionally, when clustering objects using the method described above, the coded blocks located at the boundaries between different objects cannot be mapped to any specific slice because they contain pixels from different objects simultaneously. Therefore, such coded blocks can be mapped to specific slices as follows:

[0153] For a coding block located at the boundary between different objects, the target object corresponding to the coding block is determined based on the proportion of pixels belonging to different objects in the coding block, and the coding block is mapped to the fragment corresponding to the target object.

[0154] When the pixels in a coding block belong to two or more different objects, the number of pixels belonging to each object in the coding block can be determined, the object with the largest number of corresponding pixels is identified as the target object, and then the coding block is mapped to the fragment corresponding to the target object.

[0155] Combination Figure 2 For example, suppose there is a 4*4 coded block B2, in which 10 pixels belong to object 2 and 6 pixels belong to object 4. Then, for B2, object 2 can be identified as the target object, and B2 can be mapped to the fragment 1 corresponding to object 2, that is, B2 is included in the set of fragment 1.

[0156] Optionally, if there are multiple objects with the largest number of pixels in a coding block, one of them can be randomly selected as the target object. For example, suppose there is a 4*4 coding block B3, in which 6 pixels belong to object 6, 6 pixels belong to object 7, and 4 pixels belong to object 3. Then, one of object 6 and object 7 can be randomly selected as the target object corresponding to coding block B3.

[0157] Optionally, if there are multiple objects with the largest number of corresponding pixels in a coding block, the target object can be uncertain. Instead, the coding block can be mapped to a specified slice based on the mapping relationship between slices and pixels. Specifically, for each pixel in the coding block, this pixel can be mapped to the slice corresponding to its object. For example, if pixel (1,1) belongs to object 1, and object 1 is mapped to slice 1 through clustering, then pixel (1,1) is mapped to slice 1, and so on. In this way, the number of pixels in the coding block mapped to different slices can be determined, and the coding block can be mapped to the slice with the largest number of corresponding pixels.

[0158] Combining the encoding block B3 in the previous example, after mapping each pixel of encoding block B3 to the corresponding slice of the object, it is found that 6 pixels belonging to object 6 and 6 pixels belonging to object 7, a total of 12 pixels, are mapped to slice 2, and 4 pixels belonging to object 3 are mapped to slice 1. Therefore, it can be determined that encoding block B3 is mapped to slice 2.

[0159] Optionally, the target video frame is encoded based on multiple segments to obtain the encoded video frame corresponding to the target video frame, including:

[0160] For each segment, determine the target encoding parameters corresponding to the segment;

[0161] The coding block mapped to each segment is encoded according to the target coding parameters corresponding to each segment to obtain the coded video frame corresponding to the target video frame.

[0162] For each segment, the target coding parameters of this segment can be determined based on the coding parameters corresponding to one or more coding blocks mapped to this segment.

[0163] One method for determining the target coding parameters corresponding to this segment is to determine any coding block mapped to this segment, and determine the first coding parameter of the object to which this coding block belongs as the target coding parameter corresponding to this segment.

[0164] Based on the above method, the target coding parameters of a segment can be obtained directly without additional calculations, thus reducing computational load. Furthermore, since segmentation is achieved by mapping similar coding blocks using a clustering algorithm, the coding parameters corresponding to the coding blocks within the same segment are highly similar. Using the first coding parameter of any coding block as the target coding parameter for the segment can generally meet the requirements for encoding that segment, thereby obtaining encoded video frames with better image quality. Therefore, the above method can effectively reduce the computational load of this scheme while maintaining image quality. Alternatively, the method for determining the target coding parameters corresponding to a segment can be: determine which objects all the coding blocks mapped to this segment belong to, and fuse the first coding parameters of these objects to obtain the target coding parameters corresponding to this segment. For example, if the coding blocks mapped to segment 2 belong to objects 4, 6, and 7, then the first coding parameters of object 4, object 6, and object 7 can be fused to obtain the target coding parameters for segment 2. The fusion method can be averaging, weighted summation, or other methods.

[0165] The advantage of determining the target coding parameters using the above method is that the target coding parameters obtained by fusing the first coding parameters of multiple objects can better match the needs of each object corresponding to the segment during encoding, thereby further improving the image quality of the encoded video frame obtained based on the target coding parameters. After obtaining the target coding parameters corresponding to each segment, the mapping relationship between the coded blocks and segments in the target video frame, as well as the target coding parameters of each segment, can be input into the video encoder. The video encoder can be a software encoder or a hardware encoder implemented based on any encoding protocol in the field of video encoding, such as an encoder implemented based on the AV1 encoding protocol (or VP9, ​​HEVC, H264, etc., without limitation). The working principle of the video encoder can be found in relevant technologies in the field of video encoding. The AV1 encoding protocol is an open-source video encoding protocol jointly developed by the Open Multimedia Industry Consortium (AOMedia).

[0166] After obtaining the above input, the video encoder can encode all coding blocks mapped to each segment of the target video frame according to the target coding parameters corresponding to that segment, thereby obtaining encoded data corresponding to multiple segments. The set of encoded data corresponding to multiple segments is the aforementioned encoded video frame. In other words, when the video encoder encodes the target video frame, multiple coding blocks mapped to the same segment share the same set of target coding parameters.

[0167] The above encoding method can reduce the number of encoding parameters used during encoding. Only a few sets (equal to the number of segments) of target encoding parameters are needed to encode a large number of encoding blocks in the target video frame, thus improving encoding efficiency. On the other hand, since the object to which the encoding block belongs is comprehensively considered when mapping each encoding block to a segment, as well as the encoding parameters applicable to each object when encoding historical video frames, even if multiple encoding blocks of the same segment share a set of target encoding parameters, a high-quality encoded video frame can still be obtained.

[0168] Optionally, when encoding the coding blocks of the target video frame, the video encoder can adjust the target coding parameters for each segment to obtain adjusted coding parameters. The coding blocks mapped to this segment are then encoded based on these adjusted parameters, instead of directly encoding according to the target coding parameters corresponding to this segment. For methods of adjusting the target coding parameters, please refer to relevant technologies in the field of video coding concerning video encoders.

[0169] The advantage of doing this is that the target coding parameters corresponding to the segment are determined based on the aforementioned first coding parameters, which in turn are determined based on the coding parameters used when encoding historical video frames. The target coding parameters determined in this way may not fully match the characteristics of the target video frame. Directly using the target coding parameters may result in poor image quality of the encoded video frame. Adjusting the target coding parameters is beneficial to obtaining adjusted coding parameters that better match the characteristics of the target video frame, thereby improving the image quality of the encoded video frame.

[0170] After encoding the target video frame into an encoded video frame, the encoding parameters of each encoding block used to encode the target video frame can be recorded so that subsequent video frames can be mapped into fragments based on these encoding parameters when encoding subsequent video frames.

[0171] The implementation process of the video encoding method in this embodiment is described below in conjunction with a practical application scenario:

[0172] Obtain the first video frame of the video to be encoded, and use the panoramic segmentation model to identify the objects in the first video frame;

[0173] Since this is the first video frame, the first encoding parameters cannot be determined based on historical video frames. Therefore, the video encoder can be called to process the first video frame to determine the encoding parameters of each coded block in the first video frame. Then, the second encoding parameters corresponding to each object can be determined based on the encoding parameters of the coded blocks contained in each object. The method for determining the second encoding parameters can be the same as the method for determining the first encoding parameters based on historical encoding parameters in the aforementioned embodiment, and will not be described in detail here. Historical encoding parameters include the aforementioned historical quantized encoding parameters, historical non-quantized encoding parameters, and historical motion encoding parameters.

[0174] Based on the second encoding parameters corresponding to each object, the first video frame is mapped into multiple segments, that is, the first video frame is divided into segments. After the mapping is completed, the segment map, segment ID, and target encoding parameters corresponding to the segment can be output. The method of mapping into multiple segments based on the second encoding parameters is the same as the method of mapping into multiple segments based on the first encoding parameters.

[0175] The fragment map represents the mapping relationship between fragments and coded blocks in video frames, that is, it indicates which fragment each coded block is specifically mapped to. Fragment identifiers can be fragment 1, fragment 2, fragment 3 to fragment K, which are used to distinguish different fragments.

[0176] Then, using a video encoder, for each segment, according to the target encoding parameters corresponding to this segment, each encoding block mapped to this segment is encoded, thereby obtaining the first encoded video frame corresponding to the first video frame;

[0177] Starting from the second video frame, at least one reference frame can be specified by the video encoder or the user to distinguish moving objects from non-moving objects based on the reference frame;

[0178] Starting from the second video frame, the panoramic segmentation model sequentially obtains and identifies the objects contained in each video frame so that the encoding method of the aforementioned embodiment can map each video frame into multiple segments based on the objects contained in each video frame and its first encoding parameters. Then, the video encoder encodes each video frame according to the multiple segments obtained by mapping, and obtains the encoded video frame corresponding to each video frame until all video frames of the video to be encoded are encoded.

[0179] Meanwhile, after each video frame is encoded, the video encoder feeds back the encoding parameters used during encoding (i.e., the aforementioned historical encoding parameters) to determine the first encoding parameters of objects in subsequent video frames based on these historical encoding parameters.

[0180] This application also provides a video encoding device; please refer to [link to relevant documentation]. Figure 5 This is a schematic diagram illustrating the working principle of the device, which may include the following modules.

[0181] The segmentation module 501 is used to identify multiple objects contained in the target video frame;

[0182] The classification module 502 is used to map the target video frame into multiple segments according to the first encoding parameters corresponding to multiple objects. The first encoding parameters are determined according to the encoding parameters used when encoding historical video frames. The historical video frames include video frames before the target video frame.

[0183] The encoding module 503 is used to encode the target video frame according to multiple segments to obtain the encoded video frame corresponding to the target video frame.

[0184] like Figure 5 As shown, the segmentation module 501 obtains the target video frame as input, calls the panoramic segmentation model or other models capable of recognizing objects to identify the objects contained in the target video frame, and obtains the recognition results indicating the objects to which each pixel in the target video frame belongs, for example, obtaining... Figure 2 The recognition results are shown below;

[0185] The classification module 502 obtains the recognition result as input, distinguishes moving objects and non-moving objects based on the reference frame, and determines the first encoding parameters of each object in the target video frame in the recognition result based on the historical encoding parameters used in the encoding historical video frame fed back by the encoding module 503. Then, it classifies each encoding block contained in the target video frame according to the first encoding parameters of each object, that is, it maps each encoding block of the target video frame into multiple segments.

[0186] Among them, after the classification module 502 maps to obtain multiple segments, it can also determine the target encoding parameters of each segment;

[0187] Finally, the encoding module 503 obtains the mapping relationship (e.g., the aforementioned segmentation map) and the target encoding parameters for each segment output by the classification module 502. Based on the mapping relationship and the target encoding parameters, it encodes the coded blocks mapped to each segment, obtaining the encoded video frame. The encoding parameters used to encode each coded block are then fed back to the classification module 502 as historical encoding parameters to continue processing the next video frame. The encoding module 503 can implement the above functions using either the aforementioned hardware encoder or software encoder.

[0188] This application also provides an electronic device; please refer to [link to relevant documentation]. Figure 6 The electronic device includes a memory 601 and a processor 602. The memory 601 stores computer programs. The processor 602 executes the computer programs to perform:

[0189] Identify multiple objects contained in the target video frame;

[0190] Based on the first encoding parameters corresponding to multiple objects, the target video frame is mapped into multiple segments. The first encoding parameters are determined according to the encoding parameters used when encoding historical video frames. Historical video frames include video frames before the target video frame.

[0191] The target video frame is encoded by multiple segments to obtain the encoded video frame corresponding to the target video frame.

[0192] The working principles of the above electronic devices and video encoding devices can be found in the relevant steps of the video encoding method in the foregoing embodiments, and will not be repeated here.

[0193] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The technical solutions of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or certain parts of the embodiments of this application.

[0194] In this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The above descriptions are merely preferred embodiments of this application. Those skilled in the art can make various improvements and modifications without departing from the principles of this application, and such improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A video encoding method, comprising: Identify multiple objects contained in the target video frame; The target video frame is mapped into multiple segments according to the first encoding parameters corresponding to the multiple objects. The first encoding parameters are determined according to the encoding parameters used when encoding historical video frames. The historical video frames include video frames before the target video frame. The target video frame is encoded according to the multiple segments to obtain the encoded video frame corresponding to the target video frame.

2. The method according to claim 1, wherein mapping the target video frame into multiple segments according to the first encoding parameters corresponding to the plurality of objects comprises: Based on the first encoding parameters corresponding to the plurality of objects, the feature vector of each encoding block of the target video frame is determined, wherein the encoding block is obtained by dividing the target video frame based on a preset size; Based on the feature vector, each coded block contained in the target video frame is clustered to obtain multiple segments mapped to the coded blocks, wherein each segment is mapped to at least one coded block of the target video frame.

3. The method according to claim 2, wherein determining the feature vector of each coded block of the target video frame based on the first coding parameters corresponding to the plurality of objects comprises: The parameter vector of the object is determined based on the first encoding parameter corresponding to the object; Based on the positional relationship of each coded block in the target video frame, it is determined whether each coded block is located inside the object or at the boundary between different objects; For a coded block located inside the object, the feature vector of the coded block is determined based on the parameter vector of the object to which it belongs; For a coding block located at the boundary between different objects, the parameter vectors of the different objects are fused to obtain the feature vector of the coding block.

4. The method according to claim 3, wherein the object corresponds to multiple first encoding parameters; Determining the parameter vector of the object based on the first encoding parameter corresponding to the object includes: The parameter vector of the object is determined based on the multiple first encoding parameters corresponding to the object and the influence factor of each first encoding parameter; The influence factor of the first encoding parameter is determined based on the degree of influence of the first encoding parameter on the data volume of the encoded video frame.

5. The method according to claim 2, wherein encoding the target video frame according to the plurality of segments to obtain the encoded video frame corresponding to the target video frame comprises: For each of the aforementioned segments, determine the target encoding parameters corresponding to the segment; The coding block mapped to each segment is encoded according to the target coding parameters corresponding to each segment to obtain the coded video frame corresponding to the target video frame.

6. The method according to claim 1, wherein mapping the target video frame into multiple segments according to the first encoding parameters corresponding to the plurality of objects comprises: Based on the first encoding parameters corresponding to the multiple objects, the multiple objects are clustered to obtain multiple slices mapped to the multiple objects, wherein each slice is mapped to at least one of the objects; The coded blocks contained in the object are mapped to the fragments corresponding to the object to obtain multiple fragments mapped to the coded blocks, wherein each fragment is mapped to at least one coded block of the target video frame, and the coded block is obtained by dividing the target video frame based on a preset size.

7. The method according to claim 6, further comprising: For a coding block located at the boundary between different objects, the target object corresponding to the coding block is determined according to the proportion of pixels belonging to the different objects in the coding block, and the coding block is mapped to the fragment corresponding to the target object.

8. The method according to claim 1, wherein the first encoding parameter of any of the objects includes a first quantization encoding parameter and a first non-quantization encoding parameter; The method for determining the first quantization encoding parameter and the first non-quantization encoding parameter of any of the objects includes: Multiple coded blocks belonging to the object are identified, and the coded blocks are obtained by dividing the target video frame based on a preset size; The average value of the historical quantization coding parameters of multiple coding blocks belonging to the object is determined as the first quantization coding parameter of the object; The minimum value of the historical non-quantized coding parameters of multiple coding blocks belonging to the object is determined as the first non-quantized coding parameter of the object; The historical quantization coding parameters and the historical non-quantization coding parameters are the quantization coding parameters and non-quantization coding parameters used when encoding the historical video frames.

9. A video encoding apparatus, comprising: The segmentation module is used to identify multiple objects contained in the target video frame; The classification module is used to map the target video frame into multiple segments according to the first encoding parameters corresponding to the multiple objects. The first encoding parameters are determined according to the encoding parameters used when encoding historical video frames. The historical video frames include video frames before the target video frame. The encoding module is used to encode the target video frame according to the multiple segments to obtain the encoded video frame corresponding to the target video frame.

10. An electronic device, comprising a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program to perform: Identify multiple objects contained in the target video frame; The target video frame is mapped into multiple segments according to the first encoding parameters corresponding to the multiple objects. The first encoding parameters are determined according to the encoding parameters used when encoding historical video frames. The historical video frames include video frames before the target video frame. The target video frame is encoded according to the multiple segments to obtain the encoded video frame corresponding to the target video frame.

Images