Video decoding method, video encoding method, device, storage medium and equipment

Abstract

Embodiments of the present application disclose a video decoding method, a video encoding method, a device, a storage medium and equipment, and can be applied to cloud technology, intelligent transportation, auxiliary driving and the like. The video encoding method comprises: acquiring a media application scene and a video content feature of original video data to be encoded; determining a target sampling parameter used for sampling processing of the original video data according to the media application scene and the video content feature; performing sampling processing on the original video data according to the target sampling parameter to obtain sampled video data; and encoding the sampled video data to obtain video encoding data corresponding to the original video data. The encoding efficiency of the original video data can be improved by the present application.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a video decoding method, a video encoding method, a device, a storage medium and equipment. BACKGROUND

[0002] With the development of digital media technology and computer technology, video is applied in various fields, such as mobile communication, network identification, network television, etc., which brings great convenience to people's entertainment and life. Under the condition of limited bandwidth, the traditional encoder encodes the video frame without distinction, so that there is a large amount of redundant information in the video code stream, resulting in a relatively low coding efficiency of video data. SUMMARY

[0003] The technical problem to be solved by the embodiments of the present application is to provide a video decoding method, a video encoding method, a device, a storage medium and equipment, which can improve the coding efficiency of original video data.

[0004] In one aspect, the embodiments of the present application provide a video encoding method, comprising:

[0005] Obtaining a media application scenario and a video content feature of original video data to be encoded;

[0006] Determining a target sampling parameter for sampling processing of the original video data according to the media application scenario and the video content feature;

[0007] Performing sampling processing on the original video data according to the target sampling parameter to obtain sampled video data;

[0008] Encoding the sampled video data to obtain video encoding data corresponding to the original video data.

[0009] In one aspect, the embodiments of the present application provide a video decoding method, comprising:

[0010] Obtaining video encoding data to be decoded and a target sampling parameter corresponding to the video encoding data; the video encoding data is obtained by encoding sampled video data, the sampled video data is obtained by sampling processing original video data corresponding to the video encoding data based on the target sampling parameter, and the target sampling parameter is determined according to a media application scenario and a video content feature of the original video data;

[0011] Decoding the video encoding data to obtain the sampled video data;

[0012] Performing sampling recovery processing on the sampled video data according to the target sampling parameter to obtain the original video data corresponding to the video encoding data.

[0013] In one aspect, the embodiments of the present application provide a video decoding device, comprising:

[0014] a first obtaining module, configured to obtain video coding data to be decoded and target sampling parameters corresponding to the video coding data, wherein the video coding data is obtained by coding sampled video data, the sampled video data is obtained by sampling original video data corresponding to the video coding data based on the target sampling parameters, and the target sampling parameters are determined according to a media application scenario and video content features of the original video data;

[0015] a decoding module, configured to decode the video coding data to obtain the sampled video data;

[0016] a sampling recovery module, configured to perform sampling recovery processing on the sampled video data based on the target sampling parameters to obtain the original video data corresponding to the video coding data.

[0017] An embodiment of the present application provides a video coding device, which comprises:

[0018] a second obtaining module, configured to obtain a media application scenario and video content features of original video data to be coded;

[0019] a second determining module, configured to determine target sampling parameters used for sampling processing on the original video data based on the media application scenario and the video content features;

[0020] a sampling processing module, configured to perform sampling processing on the original video data based on the target sampling parameters to obtain sampled video data;

[0021] a coding module, configured to code the sampled video data to obtain video coding data corresponding to the original video data.

[0022] An embodiment of the present application provides a computer device, which comprises a processor and a memory.

[0023] The processor is connected with the memory, and the memory is configured to store a computer program, and the computer program is configured to be executed by the processor, so that the computer device executes the method provided in the embodiments of the present application.

[0024] An embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device with the processor executes the method provided in the embodiments of the present application.

[0025] One embodiment of this application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in this application embodiment.

[0026] In this embodiment, on the encoding device side, the target sampling parameters of the original video data are adaptively determined based on the media application scenario and video content. Sampling processing is then performed on the original video data based on these target sampling parameters to obtain sampled video data. This improves the sampling accuracy of the original video data and effectively reduces the redundancy of the video encoding data while ensuring video viewing quality. Furthermore, the sampled video data is encoded to obtain video encoded data. Only this video encoded data needs to be sent to the decoding device, reducing the data volume of the video encoded data and thus improving the transmission efficiency and encoding efficiency of the original video data. On the decoding device side, the video encoded data is decoded to obtain the sampled video data. Since the video encoded data is obtained by encoding the sampled video data (i.e., encoding a portion of the original video content), only the encoded data of a portion of the video content needs to be decoded during the decoding process, improving the decoding efficiency. Simultaneously, sampling recovery processing is performed on the sampled video data based on the aforementioned target sampling parameters, which can, to some extent, recover the original video data and improve the quality of the video data. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of a video data processing process provided in an embodiment of this application;

[0029] Figure 2 This is a schematic diagram of an encoding unit provided in an embodiment of this application;

[0030] Figure 3 This is a flowchart illustrating a video encoding method provided in an embodiment of this application;

[0031] Figure 4 This is a schematic diagram of a time sampling method provided in an embodiment of this application;

[0032] Figure 5 This is a schematic diagram of a spatial sampling method provided in an embodiment of this application;

[0033] Figure 6 This is a schematic diagram of a video decoding method provided in an embodiment of this application;

[0034] Figure 7 This is a schematic diagram of the structure of a video decoding device provided in an embodiment of this application;

[0035] Figure 8 This is a schematic diagram of the structure of a video encoding device provided in an embodiment of this application;

[0036] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application;

[0037] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] This application relates to the field of cloud technology. Specifically, it relates to cloud computing, a computing model that distributes computing tasks across a resource pool composed of numerous computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." Resources in the "cloud" appear infinitely scalable to the user and can be accessed, used on demand, and expanded at any time. This application utilizes cloud computing to encode and decode video data.

[0040] This application relates to video data processing technology. The complete video data processing process specifically includes: video acquisition, video encoding, video file encapsulation, video transmission, video file decapsulation, video decoding, and final video presentation. Video acquisition converts analog video into digital video and saves it in a digital video file format; that is, video acquisition converts video signals into binary digital information. The binary information converted from the video signal is a binary data stream, also known as the video signal's bitstream. Video encoding uses compression technology to convert the original video format file into another video format file. The generation of video media content mentioned in this application embodiment includes real scenes captured by cameras and screen content scenes generated by computers. From the perspective of video signal acquisition methods, video signals can be divided into two types: those captured by cameras and those generated by computers. Due to differences in statistical characteristics, their corresponding compression coding methods may also differ. Modern mainstream video coding technologies, taking international video coding standards HEVC (High Efficiency Video Coding, HEVC / H.265), VVC (versatile video coding, VVC / H.266), and AVS (Audio Video Coding Standard), or AVS3 (the third-generation video coding standard introduced by the AVS standards group), as examples, employ a hybrid coding framework, performing the following series of operations and processing on the input raw video signal, such as... Figure 1 As shown, Figure 1 This is a schematic diagram of a video data processing process provided in an embodiment of this application, which can be specifically described as follows: Figure 1 As shown:

[0041] ① Block partition structure: The input image (such as a video frame in video data) is divided into several non-overlapping processing units according to its size. Each processing unit will undergo a similar compression operation. This processing unit is called a CTU (Coding Tree Unit) or LCU (Largest Coding Unit). The coding tree unit generally starts from the largest coding unit and continues downwards. Further subdivisions can be made below the CTU to obtain one or more basic coding units, called CUs (Coding Units). Each CU is the most basic element in a coding process. The following describes the various coding methods that can be used for each CU, such as... Figure 2As shown, Figure 2 This is a schematic diagram of an encoding unit provided in an embodiment of this application. The relationship between LCU (or CTU) and CU can be as follows: Figure 2 As shown.

[0042] ② Predictive Coding: This includes intra-frame prediction and inter-frame prediction. The original video signal is predicted using a selected reconstructed video signal to obtain the residual video signal. The encoder needs to determine the most suitable predictive coding mode from among many possible modes for the current CU and inform the decoder.

[0043] a. Intra(picture)Prediction: The predicted signal comes from a region within the same image that has already been encoded and reconstructed.

[0044] b. Inter-frame prediction: The predicted signal comes from other encoded images that are different from the current image (called reference images).

[0045] ③ Transform Coding and Quantization: The residual video signal undergoes transform operations such as DFT (Discrete Fourier Transform) and DCT (Discrete Cosine Transform, a subset of DFT) to transform the signal into the transform domain, which are called transform coefficients. The signal in the transform domain is then further subjected to lossy quantization, losing some information, making the quantized signal more suitable for compression representation.

[0046] In some video coding standards, there may be more than one transform option to choose from. Therefore, the encoder needs to select one of the transforms for the current encoding CU and inform the decoder. The fineness of quantization is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a wider range of values ​​will be quantized into the same output, which usually results in greater distortion and a lower bitrate. Conversely, a smaller QP value means that coefficients with a smaller range of values ​​will be quantized into the same output, which usually results in less distortion and a higher bitrate.

[0047] ④ Entropy Coding or Statistical Coding: The quantized transform domain signal is statistically compressed and encoded based on the frequency of each value, finally outputting a binary (0 or 1) compressed bitstream. Simultaneously, other information generated during encoding, such as the selected mode and motion vectors, also requires entropy coding to reduce the bit rate.

[0048] Statistical coding is a lossless coding method that can effectively reduce the bit rate required to represent the same signal. Common statistical coding methods include variable length coding (VLC) or context-adaptive binary arithmetic coding (CABAC).

[0049] ⑤ Loop Filtering: After the encoded image undergoes inverse quantization, inverse transform, and prediction compensation operations (the reverse of operations ② to ④ above), a reconstructed decoded image can be obtained. Compared with the original image, the reconstructed image differs in some information due to the influence of quantization, resulting in distortion. Filtering the reconstructed image, such as deblocking, SAO (Sample Adaptive Offset), or ALF (Adaptive Loop Filter), can effectively reduce the distortion caused by quantization. Since these filtered reconstructed images will be used as a reference for subsequent encoded images to predict future signals, the above filtering operations are also called loop filtering, or filtering operations within the coding loop.

[0050] Figure 1 The document illustrates the basic process of a video encoder. Figure 1 The k-th CU (labeled as S) k Taking [×, y] as an example, where k is a positive integer greater than or equal to 1 and less than or equal to the number of CUs in the current input image, and S k [×, y] represents the pixel with coordinates [×, y] in the k-th CU, where × represents the x-coordinate of the pixel and y represents the mid-coordinate of the pixel. k The predicted signal is obtained after [×, y] undergoes a preferred processing method such as motion compensation or intra-frame prediction. S k [×, y] and Subtraction yields the residual signal U k [×, y], and then the residual signal U k The [×, y] transformation and quantization are performed, and the quantized output data goes to two different places: one is sent to an entropy encoder for entropy encoding, and the encoded bitstream is output to a buffer for storage, waiting to be transmitted; the other application is to perform inverse quantization and inverse transformation to obtain the signal U'. k [×, y]. Signal U' k [×, y] and The summation yields a new prediction signal S* k [×, y], and S* k [×, y] are saved in the buffer of the current image. S* k [×, y] is obtained by intra-image prediction f(S*). k [×,y]), S* k [×, y] is filtered through a loop to obtain S'. k [×, y], and S' k [×, y] are sent to the decoded image buffer for use in generating the reconstructed video. k [×, y] is obtained after motion-compensation prediction, resulting in S'. r [×+m × y+m y ], S' r [×+m × y+m y ] represents the reference block, m × and m y These represent the horizontal and vertical components of the motion vector, respectively.

[0051] After encoding the video data, the resulting data stream needs to be encapsulated and transmitted to the user. Video file encapsulation refers to storing the encoded and compressed video and audio in a single file according to a specific encapsulation format (or container, or file container). Common encapsulation formats include AVI (Audio Video Interleaved) or ISOBMFF (ISO Based Media File Format). ISOBMFF is the media file encapsulation standard, and the most typical ISOBMFF file is MP4 (Moving Picture Experts Group 4) file. The encapsulated file is then transmitted to a decoding device, which performs decapsulation and decoding operations to present the final video content.

[0052] It is understandable that the file decapsulation process of the decoding device is the reverse of the file encapsulation process described above. The decoding device can decapsulate the encapsulated file according to the file format requirements at the time of encapsulation to obtain the video bitstream. The decoding process of the decoding device is also the reverse of the encoding process. The decoding device can decode the video bitstream to restore the video data. According to the above encoding process, at the decoding end, for each CU, after the decoder obtains the compressed bitstream, it first performs entropy decoding to obtain various mode information and quantized transform coefficients. Each coefficient undergoes inverse quantization and inverse transform to obtain the residual video signal. On the other hand, based on the known encoding mode information, the prediction signal corresponding to the CU can be obtained. After adding the two, the reconstructed signal can be obtained. Finally, the reconstructed value of the decoded image needs to undergo a loop filtering operation to produce the final output signal.

[0053] like Figure 3 As shown, Figure 3 This is a flowchart illustrating a video encoding method provided in an embodiment of this application. This method can be executed by a computer device, which may refer to an encoding device. Figure 3 As shown, the method may specifically include, but is not limited to, the following steps:

[0054] S101, Obtain the media application scenarios and video content characteristics of the raw video data to be encoded.

[0055] Specifically, after acquiring the raw video data to be encoded, the encoding device can determine the media application scenario of the raw video data. This scenario can include user viewing scenarios, machine recognition scenarios, etc. Machine recognition scenarios can include scenarios where a machine interprets the video data and completes related tasks (such as detection and recognition). Because the video perception characteristics of the target object differ in different media application scenarios—for example, the video perception characteristics of the target user in a user viewing scenario are different from those of the target machine in a machine recognition scenario—the quality and resolution requirements for video data differ between user viewing scenarios and machine recognition scenarios. The encoding device can also acquire the video content characteristics of the raw video data, which can include the rate of change of video content, the amount of information in the video content, the video resolution of video frames in the raw video data, and the number of video frames played per unit time in the raw video data.

[0056] S102, Based on the media application scenario and video content characteristics, determine the target sampling parameters used for sampling processing of the original video data.

[0057] Specifically, media application scenarios can reflect the quality requirements of the video data needed by the target object (such as content change rate requirements and resolution requirements), while the video content characteristics of the original video data can reflect the video content change rate and the amount of information in the video content. Encoding equipment can determine the target sampling parameters for sampling processing the original video data based on the media application scenario and video content characteristics. These target sampling parameters can include the target sampling method and the target sampling rate under that method. Specifically, the target sampling method can include temporal sampling and spatial sampling. Temporal sampling refers to sampling video frames, while spatial sampling refers to sampling video data based on its resolution. The target sampling rate under the target sampling mode can include the target sampling rate under the temporal sampling mode and the target sampling rate under the spatial sampling mode. For example, the target sampling rate under the temporal sampling mode can refer to 2x frame subtraction (i.e., sampling one frame at a time interval of one frame), 3x frame subtraction (i.e., sampling one frame at a time interval of two frames), etc.; while the target sampling rate under the spatial sampling mode can be any value greater than 0, such as 0.5x (i.e., resolution reduced by 0.5 times), 0.75x (i.e., resolution reduced by 0.75 times), 2x (i.e., resolution increased by 2 times), etc.

[0058] Optionally, the encoding device may determine the specific method for using the target sampling parameters for sampling processing of the raw video data, including: determining the target sampling method for sampling processing of the raw video data based on video content characteristics; determining the video perception characteristics of the target object in the media application scenario; the target object being the object for perceptual processing of the raw video data; determining the target sampling rate under the target sampling method based on the video perception characteristics and video content characteristics; and determining the target sampling rate and target sampling method as the target sampling parameters for sampling processing of the raw video data.

[0059] Specifically, the encoding device can determine the target sampling method for sampling the original video data based on the characteristics of the video content. This adaptive determination of the target sampling method improves the accuracy of the original video data sampling. The encoding device can determine the video perception characteristics of the target object in the media application scenario of the original video data. This target object is the object to be perceptually processed in the original video data, and these video perception characteristics can reflect information such as the quality and resolution requirements of the target object for the video data. Furthermore, the encoding device can determine the target sampling rate under the target sampling method based on the video perception characteristics and video content characteristics. The target sampling rate and target sampling method are then defined as the target sampling parameters for sampling the original video data. This adaptive determination of the target sampling method and target sampling rate under the target sampling method, based on the media application scenario and video content characteristics, improves the sampling accuracy of the original video data. It ensures that the decoding device does not affect the application (such as user viewing or machine recognition) when restoring the original video data from the encoded video data, while also reducing the amount of data in the encoded video data. In other words, by sampling the original video data using target sampling parameters, we can ensure that the decoding device can restore the viewing quality of the original video data based on the video encoding data, while also reducing the amount of video encoding data.

[0060] The target sampling method here can include one of temporal sampling, spatial sampling, or a combination of both. Temporal sampling refers to frame-by-frame sampling of the original video data, while spatial sampling refers to video resolution sampling of the original video data. The target sampling rate under temporal sampling refers to the ratio of the number of extracted video frames to the number of original video frames when performing frame-by-frame sampling. The target sampling rate under spatial sampling refers to the ratio of the sampled video resolution to the original video resolution when performing video resolution sampling of the original video data.

[0061] Optionally, the encoding device may determine the specific method for selecting the target sampling method by: determining the repetition rate of video content in the original video data based on the rate of change of video content included in the video content features; and determining the target sampling method for sampling the original video data based on the repetition rate of video content in the original video data.

[0062] Specifically, video content characteristics include the rate of change of the video content in the original video data (i.e., the rate of change of the content in the video frame). This rate of change can refer to the speed of movement of a moving object or the rate of change of pixels within the video content. The encoding device can determine the repetition rate of the video content in the original video data based on this rate of change. This repetition rate can refer to the repetition rate between two adjacent frames in any playback order. Furthermore, the encoding device can determine the target sampling method for sampling the original video data based on the repetition rate. If the repetition rate of the video content in the original video data is too low, frame sampling is not recommended. If frame sampling is performed, it will affect the display effect of the original video data reconstructed by the decoding device (e.g., problems such as discontinuous video content or large jumps in video content). In this way, determining the target sampling method based on the repetition rate of video content in the original video data can improve the accuracy of the target sampling method. This ensures that the display effect of the original video data restored by the decoding device based on the video encoding data is not affected, while reducing the amount of video encoded data obtained by encoding the original video data.

[0063] Optionally, the encoding device may determine the target sampling method based on the repetition rate in the following ways: if the repetition rate of the video content in the original video data is greater than a first repetition rate threshold, then the temporal sampling method and the spatial sampling method are determined as the target sampling methods for sampling the original video data. If the repetition rate of the video content in the original video data is less than or equal to the first repetition rate threshold and greater than a second repetition rate threshold, then the temporal sampling method is determined as the target sampling method for sampling the original video data; the second repetition rate threshold is less than the first repetition rate threshold. If the repetition rate of the video content in the original video data is less than or equal to the second repetition rate threshold, then the spatial sampling method is determined as the target sampling method for sampling the original video data. The first and second repetition rate thresholds can be set according to the perceptual requirements of the target object or according to specific circumstances. This embodiment does not limit the first and second repetition rate thresholds.

[0064] Specifically, if the encoding device determines that the repetition rate of the video content in the original video data is greater than a first repetition rate threshold, temporal and spatial sampling of the original video data will not affect the display effect of the sampled video data. Therefore, temporal and spatial sampling methods can be determined as the target sampling methods for sampling the original video data. This can significantly reduce the amount of data in the encoded video data obtained by encoding the original video data. Specifically, if the encoding device determines that the repetition rate of the video content in the original video data is less than or equal to a first repetition rate threshold and greater than a second repetition rate threshold, then temporal sampling can be determined as the target sampling method for sampling the original video data, where the second repetition rate threshold is less than the first repetition rate threshold. In this way, using only temporal sampling to sample the original video data can reduce the amount of data in the encoded video data while avoiding the significant information loss that occurs when temporal and spatial sampling methods are used for temporal sampling, thus preventing any impact on the display effect of the original video data reconstructed from the sampled video data.

[0065] Of course, if the encoding device determines that the repetition rate of the video content in the original video data is less than or equal to a first repetition rate threshold and greater than a second repetition rate threshold, then spatial sampling can be determined as the target sampling method for sampling the original video data. In other words, if the repetition rate of the video content in the original video data is less than or equal to a first repetition rate threshold and greater than a second repetition rate threshold, either temporal sampling or spatial sampling can be used as the target sampling method for sampling the original video data. Specifically, if the encoding device determines that the repetition rate of the video content in the original video data is less than or equal to a second repetition rate threshold, then using temporal sampling would result in the loss of a large amount of information. In this way, spatial sampling can be determined as the target sampling method for sampling the original video data. Thus, based on the repetition rate of the video content in the original video data, a suitable target sampling method can be determined, which can improve sampling accuracy. Of course, the method by which the encoding device determines the target sampling method based on the repetition rate can be applied to video frames in the original video data. For example, the target sampling method for sampling the current video frame can be determined based on the repetition rate between the current video frame and the reference video frame (which can be the video frame that is in the previous frame in the playback order or the video frame that is in the next frame in the playback order).

[0066] Optionally, the encoding device may further determine the target sampling method by: determining the complexity of the video content in the original video data based on the amount of video content information included in the video content features; and determining the target sampling method for sampling processing the original video data based on the complexity of the video content in the original video data.

[0067] Optionally, the video content features of the original video data include the amount of video content information. This amount of information reflects the complexity of the original video data; that is, the higher the amount of information, the more complex the video content, and vice versa. The encoding device can determine the complexity of the video content in the original video data based on the amount of information included in the video content features. Furthermore, the encoding device can determine the target sampling method for sampling the original video data based on the complexity of the video content. For example, if the complexity of the video content in the original video data is high, spatial sampling may cause the loss of key information, resulting in chaotic video content in the sampled video data. Therefore, spatial sampling is not suitable for sampling the original video data. Thus, determining the target sampling method for sampling the original video data based on its complexity improves sampling accuracy by identifying a suitable method. Of course, the method by which the encoding device determines the target sampling method based on complexity can be applied to video frames in the original video data. For example, the target sampling method for sampling the current video frame can be determined based on the complexity of the current video frame in the original video data.

[0068] Optionally, the encoding device may determine the target sampling method based on complexity in the following ways: if the complexity of the video content in the original video data is less than a first complexity threshold, then the temporal sampling method and the spatial sampling method are determined as the target sampling methods for sampling processing the original video data. If the complexity of the video content in the original video data is greater than or equal to the first complexity threshold and less than a second complexity threshold, then the spatial sampling method is determined as the target sampling method for sampling processing the original video data; the second complexity threshold is greater than the first complexity threshold. If the complexity of the video content in the original video data is greater than the second complexity threshold, then the temporal sampling method is determined as the target sampling method for sampling processing the original video data. The first complexity threshold and the second complexity threshold can be set according to the perceptual requirements of the target object or according to specific circumstances. This embodiment does not limit the first complexity threshold and the second complexity threshold.

[0069] Specifically, if the encoding device determines that the complexity of the original video data's content is less than a first complexity threshold, indicating that the original video data's content is relatively simple, then temporal sampling and spatial sampling methods can be used to sample the original video data. This will not affect the display effect of the original video data reconstructed by the decoding device based on the sampled video, and it can also significantly reduce the amount of data in the resulting encoded video data. If the complexity of the video content in the original video data is greater than or equal to the first complexity threshold, but less than the second complexity threshold, then spatial sampling is determined as the target sampling method for sampling the original video data, with the second complexity threshold being greater than the first complexity threshold. In this way, using only spatial sampling to sample the original video data can reduce the amount of data in the corresponding encoded video data while avoiding the significant information loss that occurs when temporal sampling and spatial sampling methods are used for temporal sampling of the original video data, thus preventing any impact on the display effect of the original video data reconstructed from the sampled video data.

[0070] Of course, if the complexity of the video content in the original video data is greater than or equal to the first complexity threshold and less than the second complexity threshold, then temporal sampling can be determined as the target sampling method for sampling the original video data. In other words, if the complexity of the video content in the original video data is greater than or equal to the first complexity threshold and less than the second complexity threshold, either temporal or spatial sampling can be used as the target sampling method for sampling the original video data. Furthermore, if the complexity of the video content in the original video data is greater than the second complexity threshold, it indicates that the complexity of the video content in the original video data is high. Using spatial sampling may cause the original video data to lose key information, resulting in chaotic video content in the sampled video data. Therefore, spatial sampling is not suitable for sampling the original video data, and temporal sampling can be determined as the target sampling method for sampling the original video data.

[0071] Optionally, the specific method by which the encoding device determines the target sampling method may include: the encoding device can determine the repetition rate of video content in the original video data based on the rate of change of video content included in the video content features, and determine the complexity of video content in the original video data based on the amount of information in the video content included in the video content features. Further, the encoding device can determine the target sampling method for sampling processing the original video data based on the repetition rate and complexity of the video content in the original video data. Specifically, the encoding device can detect whether to use temporal sampling to sample the original video data based on the repetition rate, and detect whether to use spatial sampling based on the complexity of the video content in the original video data. Specifically, if the repetition rate of video content in the original video data is greater than a third repetition rate threshold, then temporal sampling is used as the target sampling method for sampling processing the original video data; if the repetition rate of video content in the original video data is less than or equal to the third repetition rate threshold, then temporal sampling is prohibited as the target sampling method for sampling processing the original video data. If the complexity of the video content in the original video data is less than the third complexity threshold, then spatial sampling is used as the target sampling method for sampling processing the original video data. If the repetition rate of the video content in the original video data is greater than or equal to the third complexity threshold, then temporal sampling is prohibited as the target sampling method for sampling processing the original video data. The third repetition rate threshold can be set according to the perceptual needs of the target object or according to specific circumstances. This application embodiment does not impose any restrictions on the third repetition rate threshold. Similarly, the third complexity threshold can be set according to the perceptual needs of the target object or according to specific circumstances. This application embodiment does not impose any restrictions on the third complexity threshold.

[0072] Optionally, the encoding device may determine the target sampling rate under the target sampling mode in the following specific way: if the target sampling mode is time sampling mode, then based on the video perception characteristics, determine the limited number of video frames corresponding to the perceived video data of the target object per unit time. The target sampling rate under the time sampling mode is determined based on the ratio between the limited number of video frames and the number of video frames played per unit time in the original video data indicated by the video content characteristics.

[0073] Specifically, if the encoding device determines to use time sampling to sample the original video data, it can determine the limit number of video frames that the target object can perceive per unit time based on video perception characteristics. Here, "unit time" can refer to per second, per minute, etc. It's understandable that there's a limit to the number of video frames a user or machine can perceive per unit time. For example, if the user's eye can perceive a video frame rate of 55 frames per second, the human eye cannot distinguish between a video with a frame rate exceeding 55 frames per second and a video with a frame rate of 55 frames per second. Only when the frame rate is too low can the human eye perceive the choppy quality of the video. Since different objects have different video perception characteristics, the limit number of video frames corresponding to different objects is different. The limit number of video frames that the target object can perceive per unit time can be less than or equal to the target object's perceptible video frame rate; this limit number of video frames can refer to the minimum number of frames required to meet the target object's perception needs. Specifically, the encoding device can determine the limited number of video frames that the target object can perceive per unit time based on the video perception features and the video content features of the original video data. After sampling the original video data according to the limited number of video frames, the original video data with video quality and resolution that meet the perception requirements of the target object can be restored based on the sampled video data.

[0074] Furthermore, the encoding device can determine the target sampling rate in the time-based sampling mode based on the ratio between the limited number of video frames and the number of video frames played per unit time in the original video data indicated by the video content features. Specifically, since the sampling rate of video frame sampling needs to be a positive integer, the encoding device can obtain the ratio between the limited number of video frames and the number of video frames played per unit time in the original video data indicated by the video content features. If the ratio is a positive integer, then this ratio is used to determine the target sampling rate in the time-based sampling mode. If the ratio is not a positive integer, then the ratio is rounded down to obtain the rounded ratio, which is then used as the target sampling rate in the time-based sampling mode. In this way, since different target objects can perceive different limited number of video frames, the target sampling rate in the time-based sampling mode is adaptively determined according to the limited number of video frames corresponding to the target object. This ensures that when the original video data is reconstructed from the sampled video data, the quality and resolution of the reconstructed original video data meet the perceptual requirements of the target object.

[0075] Optionally, the encoding device may determine the target sampling rate under the target sampling mode by means of video perceptual features and video content features, including: if the target sampling mode is spatial sampling mode, then determining the restricted video resolution associated with the target object based on the video perceptual features. The ratio between the restricted video resolution and the video resolution of the video frames in the original video data indicated by the video content features is determined as the target sampling rate under the spatial sampling mode.

[0076] Specifically, if the target sampling method is spatial sampling, the encoding device can determine the limiting video resolution associated with the target object based on video perceptual features. The limiting video resolution associated with different target objects is different; this limiting video resolution can refer to the minimum resolution required to meet the perceptual needs of the target object. For example, the video resolution required by a user (i.e., the human eye) when viewing video data is different from the video resolution required by a machine when processing a recognition task. Because users require rich visual effects when viewing video data, they need a higher video resolution. When processing a recognition task, the machine only needs to identify the relevant information of the object to be recognized, requiring a lower video resolution. Furthermore, the encoding device can determine the target sampling rate under spatial sampling by the ratio between the limiting video resolution and the video resolution of the video frames in the original video data indicated by the video content features. In this way, since different target objects require different limiting video resolutions, the target sampling rate under spatial sampling can be adaptively determined based on the limiting video resolution of the target object. This ensures that when reconstructing the original video data from the sampled video data, the quality and resolution of the reconstructed original video data meet the perceptual needs of the target object.

[0077] Optionally, the encoding device may determine the target sampling rate under the target sampling mode based on video perception features and video content features in the following ways: If the target sampling mode is temporal sampling mode and spatial sampling mode, then based on the video perception features, determine the limited number of video frames corresponding to the perceived video data of the target object per unit time, and determine the limited video resolution associated with the target object. The target sampling rate under the temporal sampling mode is determined based on the ratio between the limited number of video frames and the number of video frames played per unit time in the original video data indicated by the video content features. The target sampling rate under the spatial sampling mode is determined by the ratio between the limited video resolution and the video resolution of the video frames in the original video data indicated by the video content features.

[0078] Specifically, if the target sampling method is temporal sampling and spatial sampling, the encoding device can determine the limited number of video frames that the target object can perceive per unit time based on video perception characteristics. Since different objects have different video perception characteristics, the limited number of video frames corresponding to different objects will be different. Further, the encoding device can determine the target sampling rate under temporal sampling based on the ratio between the limited number of video frames and the number of video frames played per unit time in the original video data indicated by the video content characteristics. Specifically, since the sampling rate of video frame sampling needs to be a positive integer, the encoding device can obtain the ratio between the limited number of video frames and the number of video frames played per unit time in the original video data indicated by the video content characteristics. If the ratio is a positive integer, it is used as the target sampling rate under temporal sampling. If the ratio is not a positive integer, it is rounded down to the nearest integer, and this rounded ratio is used as the target sampling rate under temporal sampling. In this way, since different target objects can perceive different number of limited video frames, the target sampling rate is adaptively determined according to the limited video frame number corresponding to the target object. This allows the quality and resolution of the restored original video data to meet the perception requirements of the target object when restoring the original video data from the sampled video data.

[0079] Furthermore, the encoding device can determine the limiting video resolution associated with the target object based on video perceptual features. The limiting video resolution associated with different target objects is different, and this limiting video resolution can refer to the minimum resolution required to meet the perceptual needs of the target object. The encoding device can determine the target sampling rate in spatial sampling mode as the ratio between the limiting video resolution and the video resolution of the video frames in the original video data indicated by the video content features. Thus, since different target objects require different limiting video resolutions, adaptively determining the target sampling rate in spatial sampling mode based on the limiting video resolution of the target object ensures that the quality and resolution of the reconstructed original video data meet the perceptual needs of the target object when reconstructing the original video data from the sampled video data.

[0080] S103, based on the target sampling parameters, the original video data is sampled to obtain sampled video data.

[0081] Specifically, the encoding device can sample the raw video data according to the target sampling parameters to obtain sampled video data. The target sampling parameters include the target sampling method and the target sampling rate under that method. The encoding device can then sample the raw video data based on these parameters to obtain the sampled video data. This process of sampling the raw video data to obtain sampled video data, followed by encoding, yields the corresponding encoded video data. This reduces the amount of encoded video data, thereby improving transmission efficiency and reducing storage space.

[0082] Optionally, the encoding device samples the original video data according to the target sampling parameters. The specific method for obtaining the sampled video data may include: if the target sampling method is time-based sampling, obtaining the playback sequence number of the video frames in the original video data, and the total number of video frames contained in the original video data. Based on the target sampling rate and the total number of video frames under the time-based sampling method, determining the number of video frames to be extracted from the original video data, as the first number of video frames. Extracting the first number of video frames from the original video data according to the playback sequence number of the video frames in the original video data, as the sampled video data.

[0083] Specifically, if the target sampling method is time-based sampling, the encoding device can obtain the playback sequence number of the video frames in the original video data, as well as the total number of video frames contained in the original video data. Based on the target sampling rate under time-based sampling and the total number of video frames, the number of video frames to be extracted from the original video data is determined as the first number of video frames. Specifically, the encoding device can obtain the ratio between the total number of video frames and the target sampling rate under time-based sampling (i.e., total number of video frames / target sampling rate under time-based sampling), as the first number of video frames. For example, if the total number of video frames in the original video data is 100, and the target sampling rate under time-based sampling is 2x, then the first number of video frames is 100 / 2 = 50. Further, the encoding device can extract the first number of video frames from the original video data according to the playback sequence number of the video frames in the original video data, as the sampled video data.

[0084] Specifically, the encoding device can extract video frames from the original video data at intervals according to the playback sequence number of the video frames in the original video data and the target sampling rate under the time sampling method. These extracted video frames are then used as sampled video data. For example... Figure 4 As shown, Figure 4 This is a schematic diagram of a time sampling method provided in an embodiment of this application, as shown below. Figure 4As shown, the original video data contains a total of 10 video frames. The target sampling rate in time-based sampling mode is 2x. The original video data includes video frames 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ... The encoding device can extract one video frame at a time interval from the original video data, i.e., extract video frames 0, 2, 4, 6, 8, ... as sampled video data. In other words, based on the 2x sampling rate in time-based sampling mode, the original video data including video frames 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ... is sampled to obtain the sampled video data, i.e., video frames 0, 2, 4, 6, 8, ...

[0085] Optionally, after the encoding device samples the original video data using a time-based sampling method and a target sampling rate under the time-based sampling method to obtain sampled video data, to ensure that the decoding device can reconstruct the total number of video frames of the original video data, the encoding device can send the total number of video frames and the target sampling rate under the time-based sampling method to the decoding device. The decoding device then performs sampling recovery processing on the sampled video data corresponding to the video encoding data based on the total number of video frames and the target sampling rate under the time-based sampling method to reconstruct the number of frames of the original video data.

[0086] Optionally, when the encoding device samples the original video data using time sampling, to ensure that the decoding device can reconstruct the total number of video frames in the original video data, the encoding device can send the number of discarded frames at the end of the sampling process and the target sampling rate under time sampling to the decoding device. The decoding device then uses this information to perform sampling recovery processing on the sampled video data corresponding to the encoded video data, based on the number of discarded frames and the target sampling rate under time sampling, to reconstruct the number of frames in the original video data. The number of discarded frames can refer to the number of video frames discarded at the end of the original video data after time sampling.

[0087] Specifically, when the encoding device sends the number of dropped frames at the end of the sampling process and the target sampling rate under time-sampling mode to the decoding device, it can generate TemporalScaleFlag (time sampling tag), TemporalRatio (target sampling rate tag under time-sampling mode), and DroppedFrameNumber (dropped frame number tag). TemporalScaleFlag can be 0 or 1. A TemporalScaleFlag value of 1 indicates that the encoding device used time-sampling mode to sample the original video data, while a TemporalScaleFlag value of 0 indicates that the encoding device did not use time-sampling mode. When TemporalScaleFlag is 1, TemporalRatio can be set to the target sampling rate under time-sampling mode; for example, TemporalRatio can be 2, 3, 4, etc. When TemporalScaleFlag is 1, DroppedFrameNumber can be set to the number of video frames dropped at the end after time-sampling the original video data. For example, using time sampling and a target sampling rate of 2, after sampling video frames 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9 of the original video data, the number of video frames discarded at the end is 1 (i.e., video frame 9 is discarded at the end).

[0088] Optionally, the encoding device may perform sampling processing on the original video data according to the target sampling parameters. The specific method for obtaining the sampled video data may include: if the target sampling method is spatial sampling, then obtaining video frame M from the original video data. i The original video resolution; i is a positive integer less than or equal to M, where M is the number of video frames in the original video data. Based on the target sampling rate under the spatial sampling method and the number of video frames M... i The original video resolution, for video frames M with the original video resolution i Perform resolution conversion to obtain video frames M with the target video resolution. i After performing resolution sampling on all video frames in the original video data, the original video data after resolution conversion is determined as the sampled video data.

[0089] Specifically, if the target sampling method is spatial sampling, the encoding device can acquire video frame M from the original video data. i The original video resolution, which reflects the number of pixels in the original video data. Specifically, if the original video data has M video frames... iThe higher the original video resolution, the more video frames per second (M). i The more pixels a video contains, the more frames it has (M). i The clearer the image; if the original video data contains video frames M i The lower the original video resolution, the more video frames M i The fewer pixels it contains, the higher the video frame rate (M). i The more blurred the image, the better. For example, a video frame with a resolution of 1920×1080 contains more pixels than a video frame with a resolution of 720×480, but the amount of data encoded from a 1920×1080 video frame is greater than the amount of data encoded from a 720×480 video frame. Here, M is the number of video frames in the original video data, and M is a positive integer, such as M can take values ​​of 1, 2, 3…, while i can be a positive integer less than or equal to M.

[0090] Furthermore, the encoding device can adjust the target sampling rate and video frame M based on the spatial sampling method. i The original video resolution, for video frames M with the original video resolution i Perform resolution conversion to obtain video frames M with the target video resolution. i After sampling the resolution of all video frames in the original video data, the original video data after resolution conversion is determined as the sampled video data. In this way, by sampling the video resolution of the original video data through spatial sampling and a target sampling rate under spatial sampling, it is possible to reduce the amount of video encoded data corresponding to the original video data while ensuring that the perceptual requirements of the target object are met. This improves the transmission efficiency of the video encoded data, enabling the decoding device to quickly acquire and decode the video encoded data, thus improving decoding efficiency.

[0091] Specifically, the encoding device can employ any of the following spatial sampling methods: nearest neighbor interpolation, resampling filtering, bilinear interpolation, or sampling model prediction (such as video or image super-resolution neural networks) to sample video frames M with the original video resolution. i Perform resolution conversion to obtain video frames M with the target video resolution. i Among them, video frames M with the original video resolution i This includes Q original pixels and their corresponding pixel values, where Q is a positive integer. The encoding device can use the nearest neighbor interpolation method to interpolate the video frame M with the original video resolution. i Perform resolution conversion to obtain video frames M with the target video resolution. iSpecifically, this can be achieved by multiplying the target sampling rate in spatial sampling mode by the original video resolution, using this product as the initial video resolution. This initial video resolution comprises P sampled pixels, where P is a positive integer. Further, the encoding device can determine the sampled pixels P from Q original pixels. j The corresponding reference pixel, the sampled pixel P j It belongs to P sampled pixels, where j is a positive integer less than or equal to P. The pixel value of the reference pixel is used as the sampled pixel P. j Given the pixel values ​​of P sampled pixels, a video frame M with the initial video resolution is generated based on the P sampled pixels and their corresponding pixel values. i .

[0092] Optionally, the encoding device obtains video frames M with the target video resolution. i Specific methods may include: matching the target sampling rate under spatial sampling with the video frame M i The product of the original video resolutions is used as the initial video resolution. For video frames M with the original video resolution... i Perform resolution conversion to obtain video frame M with the initial video resolution. i If video frames M have an initial video resolution i If the encoding conditions are not met, then for video frame M with the initial video resolution... i Perform pixel filling, and then fill the video frame M. i The existing video resolution is determined as the target video resolution, and the padded video frame M is... i The video frame M with the target video resolution was determined. i If video frames M have an initial video resolution i If the encoding conditions are met, the initial video resolution is determined as the target video resolution, and video frames M with the initial video resolution are... i The video frame M with the target video resolution was determined. i .

[0093] Specifically, the encoding device can match the target sampling rate in spatial sampling mode with the video frame rate (M). i The product of the original video resolution and the resolution of the video frame is used as the initial video resolution. Further, the resolution of the video frame M is the original video resolution. i Perform resolution conversion to obtain video frame M with the initial video resolution. i Encoding devices can convert video frames into M... iThe original video resolution is denoted as width × height (i.e., width × height), and the target sampling rate under spatial sampling mode is denoted as q. Therefore, the initial video resolution is width * q × height * q. Figure 5 As shown, Figure 5 This is a schematic diagram of a spatial sampling method provided in an embodiment of this application, as shown below. Figure 5 As shown, the target sampling rate in spatial sampling mode is marked as 1 / 2. When the original resolution of video frame 50a is width × height, the encoding device can obtain the product between the 1 / 2 sampling rate in spatial sampling mode and the original resolution of video frame 50a as the initial video resolution width / 2 × height / 2. Further, the encoding device can use any one of the following spatial sampling methods—nearest neighbor interpolation, resampling filtering, bilinear interpolation, or sampling model prediction—to perform resolution conversion on video frame 50a with the original video resolution width × height, resulting in video frame 50b with the initial video resolution width / 2 × height / 2. Since the encoder in the encoding device can only encode video frames with a fixed resolution format, the encoding device can detect video frames M with the initial video resolution. i Does the encoding condition meet? This encoding condition can refer to a resolution that is a multiple of 8, meaning the width and height in the video resolution are both multiples of 8. For example, when the initial video frame M... i When the resolution is 720×480, 720 is a multiple of 8, and 480 is also a multiple of 8. Therefore, the encoding device can determine the video frame M with the initial video resolution. i The encoding conditions are met; when the initial video resolution of the video frame M... i When the resolution is 727×483, 727 is not a multiple of 8, and 483 is not a multiple of 8. Therefore, the encoding device can determine the video frame M with the initial video resolution. i The encoding conditions are not met.

[0094] Furthermore, if the encoding device determines that video frame M has an initial video resolution... i If the encoding conditions are not met, the encoding device can encode video frames M with the initial video resolution. i Pixel padding is performed to fill the initial video resolution with a resolution that meets the encoding requirements, resulting in the padded video frame M. i Encoding devices can convert the padded video frames into M... i The video resolution is determined as the target video rate, and the padded video frame M is... i The video frame M with the target video resolution was determined. i Specifically, the encoding device encodes video frames M with an initial video resolution. iWhen performing pixel padding, the video encoding resolution that satisfies the encoding conditions and has the smallest difference from the initial video resolution can be used as the target video resolution. For example, the initial video resolution is the video frame M. i When the resolution is 717×479, the difference between the video resolution 720×480 and the initial video resolution 714×475 is minimized. Therefore, the video frame M with a resolution of 717×479 can be optimized. i Pixel filling is performed to obtain video frame M with a resolution of 720×480. i .

[0095] Specifically, the encoding device can use the target pixel value to assign values ​​to video frames M. i Pixel padding is performed, and the target pixel value can be any pixel value, such as 0 or 255. This application embodiment does not limit the target pixel value. For example, the encoding device can fill the initial video resolution with three pixels of 0 in the horizontal direction (i.e., width direction) and one pixel of 0 in the vertical direction (i.e., height direction), resulting in a video frame M with a padded resolution of 720×480. i If video frames M have an initial video resolution i If the encoding conditions are met, the initial video resolution is determined as the target video resolution, and video frames M with the initial video resolution are... i The video frame M with the target video resolution was determined. i .

[0096] Optionally, after the encoding device samples the original video data using spatial sampling, if the video frame M has the target video resolution... i If it is obtained after pixel padding, then obtain the video frame M with the target video resolution. i The pixel filling position information during pixel filling is sent to the decoding device along with the target sampling rate under spatial sampling mode. The decoding device then performs sampling restoration processing on the sampled video data decoded from the video encoded data based on the target sampling rate and pixel filling position information under spatial sampling mode, in order to restore the original video data. If the video frame M has the target video resolution... i If the data is obtained without pixel padding, the target sampling rate under spatial sampling mode is sent to the decoding device. The decoding device is used to perform sampling recovery processing on the sampled video data corresponding to the video encoded data according to the target sampling rate under spatial sampling mode, so as to restore the original video data.

[0097] Specifically, the encoding device can generate SpatialScaleFlag (temporal spatial sampling label), SpatialScaleRatio (target sampling rate label under spatial sampling mode), PaddingFlag (pixel padding label), PaddingX (horizontal padding pixel values, i.e., width), and PaddingY (vertical padding pixel values, i.e., height). SpatialScaleFlag can be 0 or 1. A SpatialScaleFlag value of 1 indicates that the encoding device used spatial sampling to sample the original video data, while a SpatialScaleFlag value of 0 indicates that the encoding device did not use spatial sampling to sample the original video data. When SpatialScaleFlag is 1, SpatialScaleRatio can be set to the target sampling rate under spatial sampling mode, such as 0.5, 0.75, 2, or any value greater than 0. When SpatialScaleFlag is 1, PaddingFlag can be set to 0 or 1. A PaddingFlag value of 1 indicates that pixel padding occurred during the spatial sampling of the original video data (i.e., video frames M with the target video resolution). i (It is obtained after pixel padding); when the value of PaddingFlag is 0, it is used to indicate that there is no pixel padding operation during the spatial sampling process of the original video data (i.e., the video frame M with the target video resolution). i (This is obtained without pixel padding). Furthermore, when the value of PaddingFlag is 1, the encoding device can be set to PaddingX (horizontal padding pixel values) and PaddingY (vertical padding pixel values).

[0098] Optionally, the specific method by which the encoding device samples the original video data may include: if the target sampling method is temporal sampling and spatial sampling, then based on the target sampling rate under temporal sampling and the total number of video frames in the original video data, determine the number of video frames to be extracted from the original video data as the second number of video frames. According to the playback sequence number of the video frames in the original video data, extract the second number of video frames from the original video data as the initial sampled video data. Obtain the video frame N from the initial sampled video data. j The original video resolution; j is a positive integer less than or equal to N, where N is the number of video frames in the initial sampled video data. Based on the target sampling rate under the spatial sampling method and the number of video frames N... j The original video resolution, for video frames N with the original video resolution. jPerform resolution conversion to obtain video frames N with the target video resolution. j After performing resolution sampling on all video frames in the initial sampled video data, the resolution-sampled initial sampled video data is determined as the sampled video data. For details, please refer to the above content regarding the target sampling method being temporal sampling and spatial sampling; these will not be repeated here in the embodiments of this application.

[0099] Specifically, when the target sampling method is temporal sampling or spatial sampling, the encoding device can send the total number of video frames in the original video data and the target sampling rate under temporal sampling to the decoding device. Simultaneously, if N video frames have the target video resolution... j If it is obtained through pixel padding, then the encoding device needs to combine the target sampling rate under spatial sampling mode with the video frame N with the target video resolution. j The pixel fill position information in the video frame is sent to the decoding device; if the video frame N has the target video resolution... j If the data is obtained through pixel padding, the encoding device needs to send the target sampling rate under spatial sampling mode to the decoding device so that the decoding device can recover the video frame count and video resolution of the original video data.

[0100] S104, Encode the sampled video data to obtain the video encoded data corresponding to the original video data.

[0101] Specifically, the encoding device can use prediction methods such as intra-frame prediction or inter-frame prediction to predict the sampled video data and obtain the residual video signal of the sampled video data. Further, the encoding device can transform the residual video signal of the sampled video data to obtain the transform domain signal of the sampled video data, and quantize the transform domain signal of the sampled video data to obtain the quantized transform domain signal. Further, the encoding device can perform entropy encoding on the quantized transform domain signal to output binarized (0 or 1) video encoded data. Of course, the encoding device can also perform entropy encoding on the target sampling parameters and parameters such as the total number of video frames and pixel padding position information of the original video data to reduce the bit rate. Therefore, the embodiments of this application can reduce the bit rate of the video encoded data and improve the transmission efficiency of the video encoded data by encoding the sampled video data, thereby reducing the storage space of the video encoded data.

[0102] Optionally, the encoding device can determine key video region information (such as Region of Interest, ROI) about the original video data and send this key video region information along with the encoded video data to the decoding device. The key video region information is used to instruct the decoding device to perform image enhancement processing on the key video regions in the original video data. In this way, the decoding device performs image enhancement processing on the key video regions in the original video data to enrich the display effect of the recovered original video data, thereby improving the accuracy of the recovered original video data. Simultaneously, it can improve the accuracy of object recognition tasks when subsequently performing object recognition on the recovered original video data.

[0103] Optionally, the encoding device may determine key video region information about the original video data in the following ways: inputting the original video data into an object detection model, performing embedding vector transformation on the original video data through the embedding layer of the object detection model to obtain the media embedding vector of the original video data. Then, extracting objects from the media embedding vector through the object extraction layer of the object detection model to obtain the video objects in the original video data. The region to which the video object belongs in the original video data is determined as the key video region, and key video region information describing the location of the key video region in the original video data is generated. Of course, the encoding device may also extract the key video regions from the original video data through other key video region determination methods (such as object detection, object recognition, etc.), and this embodiment of the application does not impose any limitations on this. Specifically, after extracting the key video regions from the original video data, the encoding device can generate ROINumber (region quantity label) and ROIInformation (region feature information label, such as region coordinate information). ROINumber indicates the number of key video regions in the current original video data or the current video frame. If ROINumber is greater than 0, ROIInformation of ROINumber is transmitted. This ROIInformation indicates information about the key video regions (i.e., ROI regions of interest), such as coordinates and the types of video objects included. As shown in Table 1, after temporal and spatial sampling of the original video data, the encoding device can transmit the parameters in Table 1 so that the decoding device can recover the original video data based on the parameters in Table 1.

[0104]

[0105]

[0106] Table 1

[0107] In this embodiment, by adaptively determining the target sampling parameters of the original video data based on the media application scenario and video content, and then sampling the original video data based on the target sampling parameters to obtain sampled video data, the sampling accuracy of the original video data can be improved. This effectively reduces the redundancy of the video encoding data while ensuring video viewing quality. Furthermore, the sampled video data is encoded to obtain video encoded data. Only this video encoded data needs to be sent to the decoding device, which reduces the data volume of the video encoded data and improves the transmission efficiency. This allows the decoding device to quickly obtain the video encoded data and also improves the encoding efficiency of the original video data.

[0108] like Figure 6 As shown, Figure 6 This application provides a video decoding method, which will be described below in conjunction with... Figure 6 This application provides a detailed description of the video decoding method proposed in its embodiments. This method can be executed by a computer device, which may refer to a decoding device. Figure 6 As shown, the method may specifically include, but is not limited to, the following steps:

[0109] S201, Obtain the video encoding data to be decoded, and the target sampling parameters corresponding to the video encoding data.

[0110] Specifically, the decoding device can acquire the video encoding data to be decoded, as well as the target sampling parameters corresponding to the video encoding data. The video encoding data is obtained by encoding the sampled video data, and the sampled video data is obtained by sampling the original video data corresponding to the video encoding data based on the target sampling parameters. The target sampling parameters are determined according to the media application scenario and video content characteristics of the original video data.

[0111] Optionally, the target sampling parameters are sent by the encoding device. These parameters include the target sampling method and the target sampling rate under that method. The target sampling method is determined based on the video content characteristics of the original video data. The target sampling rate under the target sampling method is determined based on video perception features and video content features. Video perception features refer to the perception characteristics of the target object in the media application scenario regarding the video data; the target object is the object that performs perception processing on the original video data. The target sampling method can include a temporal sampling method and a target sampling rate under that method, as well as a spatial sampling method and a target sampling rate under that method. The determination of the target sampling parameters can refer to the above. Figure 3 The content of step S102 will not be repeated here in the embodiments of this application.

[0112] S202, decode the video encoded data to be decoded to obtain sampled video data.

[0113] Specifically, the decoding process of the decoding device is the reverse of the encoding process of the encoding device. The decoding device acquires the video encoded data to be decoded sent by the encoding device, performs entropy decoding on the video encoded data, and obtains various parameters and quantized transform coefficients. The quantized transform coefficients undergo inverse quantization and inverse transform to obtain the residual video signal. On the other hand, based on the known encoding mode information sent by the encoding device, the corresponding video prediction signal can be obtained. After adding the video prediction signal and the residual video signal, the reconstructed video signal can be obtained. Finally, the reconstructed video signal needs to undergo a loop filtering operation to generate sampled video data. This sampled video data can refer to the sampled video data obtained by sampling the original video data in the encoding device.

[0114] S203, based on the target sampling parameters, perform sampling recovery processing on the sampled video data to obtain the original video data corresponding to the video encoded data.

[0115] Specifically, the decoding device can perform sampling and restoration processing on the sampled video data according to the target sampling parameters to obtain the original video data corresponding to the video encoded data. It is evident that by quickly acquiring low-bitrate video encoded data and decoding and sampling it to obtain the original video data, the decoding device can improve the decoding efficiency of video encoded data and reduce the storage space required for the video encoded data.

[0116] Optionally, the specific method by which the decoding device performs sampling recovery processing on the sampled video data may include: if the target sampling method is time sampling, then based on the target sampling rate under time sampling, determining the number of third video frames to be recovered between the first and second decoded video frames; the first and second decoded video frames are video frames in the sampled video data whose playback sequence numbers are adjacent. A third number of recovered video frames is inserted between the first and second decoded video frames. After the recovered video frames are inserted between any two adjacent decoded video frames in the sampled video data, the original video data corresponding to the video encoded data is determined based on the recovered sampled video data.

[0117] Specifically, if the decoding device receives parameters from the encoding device instructing it to sample the original video data using a time-based sampling method, the decoding device can determine that the target sampling method is time-based sampling, obtain the target sampling rate under this method, and determine the number of third video frames to be recovered between the first and second decoded video frames based on the target sampling rate. Specifically, if the target sampling rate under the time-based sampling method is n times the target rate (i.e., one video frame is sampled every other video frame), then the number of third video frames to be recovered between the first and second decoded video frames is n-1. For example, if the target sampling rate under the time-based sampling method is 2 times the target rate (i.e., one video frame is sampled every other video frame), then the number of third video frames to be recovered between the first and second decoded video frames is 1. If the target sampling rate under the time-based sampling method is 3 times the target rate (i.e., one video frame is sampled every two video frames), then the number of third video frames to be recovered between the first and second decoded video frames is 2. Furthermore, the encoding device can insert a third number of recovered video frames between the first and second decoded video frames. After the recovered video frames are inserted between any two adjacent decoded video frames in the sampled video data, the original video data corresponding to the video encoded data is determined based on the recovered sampled video data.

[0118] Optionally, the recovered video frames of the third video frame number are generated based on the first decoded video frame; or, the recovered video frames of the third video frame number are generated based on the second decoded video frame; or, the recovered video frames of the third video frame number are generated based on the first decoded video frame and the second decoded video frame.

[0119] Specifically, the recovered video frame of the third video frame count can be the same as the first decoded video frame, that is, the first decoded video frame of the third video frame count is inserted between the first decoded video frame and the second decoded video frame. Alternatively, the recovered video frame of the third video frame count can be the same as the second decoded video frame, that is, the second decoded video frame of the third video frame count is inserted between the first decoded video frame and the second decoded video frame. In other words, a repeated video frame of the third video frame count (which can be either the first or second decoded video frame) is inserted between the first and second decoded video frames. Of course, the recovered video frame of the third video frame count can refer to the one obtained by network prediction based on the first and second decoded video frames. For example, the decoding device can predict the object movement information in the recovered video frame between the first and second decoded video frames based on the object movement information in the first and second decoded video frames.

[0120] Optionally, the specific method by which the decoding device determines the original video data corresponding to the video encoded data based on the recovered sampled video data may include: after inserting recovered video frames between any two adjacent decoded video frames in the sampled video data, obtaining the fourth video frame number contained in the recovered sampled video data, and the total number of video frames contained in the original video data. If the fourth video frame number is the same as the total number of video frames, then the recovered sampled video data is determined as the original video data corresponding to the video encoded data. If the fourth video frame number is different from the total number of video frames, then the difference between the fourth video frame number and the total number of video frames is obtained as the fifth video frame number, and the recovered video frame of the fifth video frame number is inserted after the recovered sampled video data to obtain the original video data corresponding to the video encoded data.

[0121] Specifically, if the parameters received by the decoding device from the encoding device include the total number of video frames in the original video data, then after inserting recovered video frames between any two adjacent decoded video frames in the sampled video data, the decoding device can obtain the number of video frames in the recovered sampled video data as the fourth video frame number. Further, the decoding device can check whether the fourth video frame number in the recovered sampled video data is the same as the total number of video frames in the original video data. If the fourth video frame number is different from the total number of video frames, the difference between the fourth video frame number and the total number of video frames is obtained as the fifth video frame number. After the recovered sampled video data, the recovered video frame of the fifth video frame number is inserted to obtain the original video data corresponding to the video encoded data. In this way, the number of video frames in the original video data can be accurately recovered. The recovered video frame for the fifth video frame can be determined based on the latest-played video frame in the recovered sampled video data. For example, the recovered video frame for the fifth video frame can be the latest-played video frame in the recovered sampled video data, or it can be predicted by the network based on the latest-played video frame in the recovered sampled video data. If the fourth video frame number is the same as the total number of video frames, then the recovered sampled video data can be determined as the original video data corresponding to the video encoded data.

[0122] Specifically, if the parameters received by the decoding device from the encoding device include the number of discarded video frames at the end of the original video data, then the recovered video frames with the discarded video frames can be directly inserted after the recovered sampled video data to obtain the original video data corresponding to the video encoded data. This allows for accurate recovery of the number of video frames in the original video data. Alternatively, the recovered video frames with the discarded video frames can also be determined based on the latest-played video frame in the recovered sampled video data.

[0123] Optionally, the specific method by which the decoding device performs sampling restoration processing on the sampled video data may include: if the target sampling method is spatial sampling, then obtaining the current target video resolution of the third decoded video frame in the sampled video data. Based on the target sampling rate and target video resolution under spatial sampling, performing resolution restoration processing on the third decoded video frame with the target video resolution to obtain a third decoded video frame with the original video resolution. After completing the restoration of all decoded video frames in the sampled video data, the restored sampled video data is determined as the original video data corresponding to the sampled video data.

[0124] Specifically, if the target sampling method is spatial sampling, taking the third video frame in the sampled video data as an example, this third decoded video frame belongs to any video frame in the sampled video data. The encoding device can obtain the current target video resolution of the third decoded video frame in the sampled video data, as well as the target sampling rate under spatial sampling. Further, the decoding device can obtain the initial video resolution by the ratio between the current target video resolution of the third decoded video frame and the target sampling rate under spatial sampling (i.e., target video resolution / target sampling rate under spatial sampling). The encoding device can use a spatial sampling recovery method to recover the third decoded video frame with the target video resolution, obtaining a third decoded video frame with the initial video resolution. Further, based on the third decoded video frame with the initial video resolution, a third decoded video frame with the original video resolution is generated. The spatial sampling recovery method can be any one of the nearest neighbor interpolation method, resampling filtering method, bilinear interpolation method, or sampling model prediction method. It can be the same as or different from the spatial sampling method in the encoding device. After recovering all decoded video frames from the sampled video data, the recovered sampled video data is identified as the original video data corresponding to the video encoded data. This allows for accurate recovery to the original video resolution, improving decoding accuracy.

[0125] Optionally, the specific method by which the decoding device obtains a third decoded video frame with the original video resolution may include: if pixel padding position information about the third decoded video frame is received, then according to the pixel padding position information, the third decoded video frame with the target video resolution is cropped to obtain a third decoded video frame with the initial video resolution; the ratio between the initial video resolution and the target sampling rate under spatial sampling mode is determined as the original video resolution to be recovered for the third decoded video frame; resolution recovery processing is performed on the third decoded video frame with the initial video resolution to obtain a third decoded video frame with the original video resolution. If pixel padding position information about the third decoded video frame is not received, then the ratio between the target sampling rate and the target video resolution under spatial sampling mode is determined as the original video resolution to be recovered for the third decoded video frame; resolution recovery processing is performed on the third decoded video frame with the target video resolution to obtain a third decoded video frame with the original video resolution.

[0126] For example, with a target sampling rate of 0.5 in spatial sampling mode and a target video resolution of 960×544 for the third decoded video frame, if the received pixel filling position information for the third decoded video frame indicates that 4 pixels are filled in the vertical direction, the decoding device can crop 4 pixels from the vertical pixel values ​​of the target video resolution of 960×544 to obtain a third decoded video frame with an initial video resolution of 960×540. Further, the decoding device can obtain the ratio between the initial video resolution of 960×540 and 0.5 (i.e., the target sampling rate in spatial sampling mode) (i.e., (960×540) / 0.5) to obtain the original video resolution to be restored from the third decoded video frame, which is 1920×1080. Further, the decoding device can perform resolution restoration processing on the third decoded video frame with the target video resolution using any of the nearest neighbor interpolation method, resampling filtering method, bilinear interpolation method, or sampling model prediction method to obtain a third decoded video frame with the original video resolution. This allows for accurate restoration to the original video resolution of the original video data, improving decoding accuracy.

[0127] Optionally, the specific method by which the decoding device performs sampling restoration processing on the sampled video data may include: if the target sampling method is temporal sampling and spatial sampling, then the target video resolution of the third decoded video frame in the sampled video data is obtained. Based on the target sampling rate and target video resolution under spatial sampling, resolution restoration processing is performed on the target video resolution of the third decoded video frame to obtain a third decoded video frame with the original video resolution. After restoring all decoded video frames in the sampled video data, the restored sampled video data is determined as the initial video data corresponding to the video encoded data. Based on the target sampling rate under temporal sampling, the sixth video frame number to be restored between the first and second initial video frames is determined; the first and second initial video frames are video frames with adjacent playback sequence numbers in the initial video data. The restored video frame number six is ​​inserted between the first and second initial video frames. After the restored video frames are inserted between any two adjacent initial video frames in the initial video data, the original video data corresponding to the video encoded data is determined based on the restored initial video data.

[0128] Specifically, if the decoding device receives parameters from the encoding device, including TemporalScaleFlag (temporal sampling label), TemporalRatio (target sampling rate label in temporal sampling mode), DroppedFrameNumber (number of dropped frames at the end label), SpatialScaleFlag (temporal-spatial sampling label), SpatialScaleRatio (target sampling rate label in spatial sampling mode), PaddingFlag (pixel padding label), PaddingX (horizontal padding pixel value, i.e., width direction), and PaddingY (vertical padding pixel value, i.e., height direction), the decoding device can determine whether the decoded sampled video data has undergone spatial sampling based on the SpatialScaleFlag parameter. If the SpatialScaleFlag value is 1, it continues to obtain SpatialScaleRatio and PaddingFlag from the parameters. If the PaddingFlag value is 1, it continues to obtain PaddingX and PaddingY, and clips the horizontal and vertical padding pixels in the sampled video data. The original video resolution is calculated based on SpatialScaleRatio and the target video resolution of the cropped sampled video data. Spatial sampling methods (such as nearest neighbor interpolation, bilinear interpolation, video or image super-resolution neural networks, etc.) are then used to restore the cropped sampled video data, resulting in the initial video data corresponding to the sampled video data.

[0129] Furthermore, the decoding device can determine whether the sampled video data has undergone time sampling based on the TemporalScaleFlag parameter. If the TemporalScaleFlag value is 1, it continues to obtain the TemporalRatio and DroppedFrameNumber from the parameters. The decoding device uses the TemporalRatio to determine the sixth video frame number of the recovered video frames between the first and second initial video frames in the initial video data. The first and second initial video frames are video frames with adjacent playback sequence numbers in the initial video data. The video frames to be recovered between the first and second initial video frames can be determined using methods such as repeated frames or video frame interpolation networks. For details, refer to the above content on determining the video frames to be recovered between the first and second decoded video frames. A recovered video frame numbering the sixth video frame is inserted between the first and second initial video frames. After the recovered video frames are inserted between any two adjacent initial video frames in the initial video data, the missing video frames in the recovered initial video data are filled in using the DroppedFrameNumber to obtain the corresponding cropped original video data.

[0130] Optionally, the decoding device can receive key video region information about the video encoded data sent by the encoding device, and determine the key video regions in the original video data based on the key video region information. Image enhancement is then performed on the key video regions in the original video data to obtain the image-enhanced original video data. Specifically, if the decoding device obtains ROINumber (region quantity label) and ROIInformation (region feature information label, such as region coordinate information), the decoding device can use ROINumber and ROIInformation to perform image enhancement processing on all key video regions in the original video data. This improves the display effect of the recovered original video data and also increases the accuracy of the original video data recovery. Furthermore, it improves the accuracy of object recognition tasks when performing subsequent object recognition tasks on the recovered original video data.

[0131] Optionally, the decoding device performs image enhancement on key video regions in the original video data to obtain enhanced original video data. Specific methods may include: inputting the key video regions from the original video data into an image enhancement model; generating image enhancement coefficients for the key video regions in the original video data through an enhancement coefficient generation layer in the image enhancement model; and then performing image enhancement on the key video regions in the original video data through the image enhancement layer in the image enhancement model, based on the image enhancement coefficients, to obtain the enhanced original video data. In this way, performing image enhancement processing on key video regions in the original video data through an image enhancement model can improve image enhancement efficiency and accuracy. Of course, the decoding device may also use other image enhancement methods to perform image enhancement processing on key video regions in the original video data; this embodiment does not impose limitations on these methods.

[0132] In this embodiment, sampled video data is obtained by decoding the video encoded data to be decoded. The target sampling parameters are determined based on the media application scenario and video content characteristics of the original video data corresponding to the video encoded data. Therefore, the video encoded data is obtained by sampling and encoding the original video data using the target sampling parameters, which are determined based on the media application scenario and video content characteristics of the original video data corresponding to the video encoded data. Since the video encoded data is obtained by encoding the sampled video data, that is, by encoding a portion of the video content in the original video data, only the encoded data of a portion of the video content needs to be decoded during the decoding process, improving the decoding efficiency of the video data. Simultaneously, by performing sampling recovery processing on the sampled video data based on the aforementioned target sampling parameters, the original video data can be recovered to a certain extent, improving the quality of the video data.

[0133] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a video decoding device provided in an embodiment of this application. The aforementioned video decoding device can be a computer program (including program code) running on a computer device; for example, the video decoding device is an application software. The video decoding device can be used to execute corresponding steps in the video decoding method provided in the embodiments of this application. Figure 7 As shown, the video decoding device may include: a first acquisition module 11, a decoding module 12, a sampling recovery module 13, a receiving module 14, a first determination module 15, and an image enhancement module 16.

[0134] The first acquisition module 11 is used to acquire the video encoding data to be decoded and the target sampling parameters corresponding to the video encoding data; the video encoding data is obtained by encoding the sampled video data, and the sampled video data is obtained by sampling the original video data corresponding to the video encoding data based on the target sampling parameters. The target sampling parameters are determined according to the media application scenario and video content characteristics of the original video data.

[0135] Decoding module 12 is used to decode the video encoded data to obtain sampled video data;

[0136] The sampling restoration module 13 is used to perform sampling restoration processing on the sampled video data according to the target sampling parameters to obtain the original video data corresponding to the video encoded data.

[0137] The target sampling parameters are sent by the encoding device and include the target sampling method and the target sampling rate under the target sampling method.

[0138] The target sampling method is determined based on the video content characteristics of the original video data;

[0139] The target sampling rate under the target sampling method is determined based on video perception features and video content features; video perception features are the perception features of the target object on the video data in the media application scenario, and the target object is the object that performs perception processing on the original video data.

[0140] The sampling recovery module 13 includes:

[0141] The first determining unit 1301 is used to determine the number of third video frames to be recovered between the first decoded video frame and the second decoded video frame, based on the target sampling rate under the time sampling method if the target sampling method is time sampling method; the first decoded video frame and the second decoded video frame are video frames with adjacent playback sequence numbers in the sampled video data.

[0142] The insertion unit 1302 is used to insert a third number of restored video frames between the first decoded video frame and the second decoded video frame.

[0143] The second determining unit 1303 is used to determine the original video data corresponding to the video encoded data based on the recovered sampled video data after inserting and restoring video frames between any two adjacent decoded video frames in the sampled video data.

[0144] The recovered video frame for the third video frame number is generated based on the first decoded video frame; or,

[0145] The recovered video frames for the third video frame number are generated based on the second decoded video frames; or,

[0146] The recovered video frames for the third video frame number are generated based on the first and second decoded video frames.

[0147] Specifically, the second determining unit 1303 is used for:

[0148] After inserting and recovering video frames between any two adjacent decoded video frames in the sampled video data, the fourth video frame number of the video frames contained in the recovered sampled video data and the total number of video frames contained in the original video data are obtained.

[0149] If the fourth video frame number is the same as the total video frame number, then the recovered sampled video data will be determined as the original video data corresponding to the video encoded data;

[0150] If the fourth video frame number is different from the total video frame number, the difference between the fourth video frame number and the total video frame number is obtained as the fifth video frame number. After recovering the sampled video data, the recovered video frame of the fifth video frame number is inserted to obtain the original video data corresponding to the video encoded data.

[0151] The sampling recovery module 13 also includes:

[0152] The first acquisition unit 1304 is used to acquire the current target video resolution of the third decoded video frame in the sampled video data if the target sampling method is spatial sampling method.

[0153] The resolution restoration unit 1305 is used to perform resolution restoration processing on the third decoded video frame with the target video resolution according to the target sampling rate and the target video resolution in the spatial sampling mode, so as to obtain the third decoded video frame with the original video resolution.

[0154] The third determining unit 1306 is used to determine the recovered sampled video data as the original video data corresponding to the video encoded data after completing the recovery of all decoded video frames in the sampled video data.

[0155] Specifically, the resolution restoration unit 1305 is used for:

[0156] If pixel padding position information about the third decoded video frame is received, then the third decoded video frame with the target video resolution is cropped according to the pixel padding position information to obtain the third decoded video frame with the initial video resolution. The ratio between the initial video resolution and the target sampling rate under the spatial sampling mode is determined as the original video resolution to be restored for the third decoded video frame. The resolution restoration processing is performed on the third decoded video frame with the initial video resolution to obtain the third decoded video frame with the original video resolution.

[0157] If no pixel filling position information for the third decoded video frame is received, the ratio between the target sampling rate and the target video resolution in spatial sampling mode will be used as the original video resolution to be recovered for the third decoded video frame. Resolution recovery processing will be performed on the third decoded video frame with the target video resolution to obtain the third decoded video frame with the original video resolution.

[0158] The sampling recovery module 13 also includes:

[0159] The fourth determining unit 1307 is used to obtain the target video resolution of the third decoded video frame in the sampled video data if the target sampling method is time sampling method and spatial sampling method, and to perform resolution restoration processing on the target video resolution of the third decoded video frame according to the target sampling rate and target video resolution under spatial sampling method, so as to obtain the third decoded video frame with the original video resolution.

[0160] The fifth determining unit 1308 is used to determine the recovered sampled video data as the initial video data corresponding to the video encoded data after completing the recovery of all decoded video frames in the sampled video data;

[0161] The sixth determining unit 1309 is used to determine the sixth video frame number of the recovered video frame between the first initial video frame and the second initial video frame according to the target sampling rate under the time sampling mode; the first initial video frame and the second initial video frame are video frames with adjacent playback sequence numbers in the initial video data;

[0162] The seventh determining unit 1310 is used to insert a restored video frame of the sixth video frame number between the first initial video frame and the second initial video frame. After the restored video frame is inserted between any two adjacent initial video frames in the initial video data, the original video data corresponding to the video encoded data is determined based on the restored initial video data.

[0163] The video decoding device also includes:

[0164] The receiving module 14 is used to receive key video region information about the video encoding data sent by the encoding device;

[0165] The first determining module 15 is used to determine the key video region in the original video data based on the key video region information;

[0166] The image enhancement module 16 is used to enhance key video regions in the original video data to obtain the image-enhanced original video data.

[0167] The image enhancement module 16 includes:

[0168] The first generation unit 1601 is used to input key video regions in the original video data into the image enhancement model, and generate image enhancement coefficients for the key video regions in the original video data through the enhancement coefficient generation layer in the image enhancement model.

[0169] The image enhancement unit 1602 is used to enhance key video regions in the original video data according to the image enhancement coefficient through the image enhancement layer in the image enhancement model, so as to obtain the image-enhanced original video data.

[0170] According to one embodiment of this application, Figure 7 The various modules in the video decoding device shown can be individually or entirely combined into one or more units, or some of these units can be further divided into multiple functionally smaller sub-units to achieve the same operation without affecting the technical effects of the embodiments of this application. The above modules are based on logical functional division; in practical applications, the function of one module can be implemented by multiple units, or the function of multiple modules can be implemented by one unit. In other embodiments of this application, the video decoding device may also include other units; in practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0171] On the decoding device side, the video encoded data is decoded to obtain sampled video data. Since the video encoded data is obtained by encoding the sampled video data, that is, by encoding a portion of the original video data, the decoding process only needs to decode the encoded data of a portion of the video content, thus improving the decoding efficiency. Simultaneously, based on the aforementioned target sampling parameters, sampling restoration processing is performed on the sampled video data, which can, to some extent, recover the original video data and improve the quality of the video data.

[0172] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a video encoding apparatus provided in an embodiment of this application. The aforementioned video encoding apparatus can be a computer program (including program code) running on a computer device; for example, the video encoding apparatus is an application software. The video encoding apparatus can be used to execute corresponding steps in the video encoding method provided in the embodiments of this application. Figure 8 As shown, the video encoding device may include: a second acquisition module 21, a second determination module 22, a sampling processing module 23, an encoding module 24, a first transmission module 25, a second transmission module 26, a third transmission module 27, a fourth determination module 28, and a fourth transmission module 29.

[0173] The second acquisition module 21 is used to acquire the media application scenarios and video content characteristics of the raw video data to be encoded;

[0174] The second determining module 22 is used to determine the target sampling parameters for sampling processing the original video data based on the media application scenario and video content characteristics.

[0175] The sampling processing module 23 is used to perform sampling processing on the original video data according to the target sampling parameters to obtain sampled video data;

[0176] The encoding module 24 is used to encode the sampled video data to obtain the video encoded data corresponding to the original video data.

[0177] The second determining module 22 includes:

[0178] The eighth determining unit 2201 is used to determine the target sampling method for sampling processing the original video data based on the characteristics of the video content.

[0179] The ninth determining unit 2202 is used to determine the video perception features of the target object in the video data in the media application scenario; the target object is the object that performs perception processing on the original video data;

[0180] The tenth determining unit 2203 is used to determine the target sampling rate under the target sampling method based on the video perception features and video content features;

[0181] The eleventh determining unit 2204 is used to determine the target sampling rate and target sampling method as target sampling parameters for sampling processing of the original video data.

[0182] Specifically, the eighth determining unit 2201 is used for:

[0183] The repetition rate of video content in the original video data is determined based on the rate of change of video content included in the video content features.

[0184] Based on the repetition rate of video content in the original video data, determine the target sampling method for sampling processing the original video data.

[0185] Specifically, based on the repetition rate of video content in the original video data, a target sampling method is determined for sampling processing of the original video data, including:

[0186] If the repetition rate of video content in the original video data is greater than the first repetition rate threshold, then the temporal sampling method and the spatial sampling method are determined as the target sampling methods for sampling processing the original video data.

[0187] If the repetition rate of the video content in the original video data is less than or equal to the first repetition rate threshold and greater than the second repetition rate threshold, then the time sampling method is determined as the target sampling method for sampling processing the original video data; the second repetition rate threshold is less than the first repetition rate threshold.

[0188] If the repetition rate of the video content in the original video data is less than or equal to the second repetition rate threshold, then the spatial sampling method is determined as the target sampling method for sampling processing the original video data.

[0189] Specifically, the eighth determining unit 2201 is used for:

[0190] The complexity of the video content in the original video data is determined based on the amount of video content information included in the video content features.

[0191] Based on the complexity of the video content in the original video data, determine the target sampling method for sampling processing the original video data.

[0192] Specifically, based on the complexity of the video content in the original video data, the target sampling method for sampling processing the original video data is determined, including:

[0193] If the complexity of the video content of the original video data is less than the first complexity threshold, then the temporal sampling method and the spatial sampling method are determined as the target sampling methods for sampling processing the original video data.

[0194] If the complexity of the video content in the original video data is greater than or equal to the first complexity threshold and less than the second complexity threshold, then the spatial sampling method is determined as the target sampling method for sampling processing the original video data; the second complexity threshold is greater than the first complexity threshold.

[0195] If the complexity of the video content in the original video data exceeds the second complexity threshold, then the temporal sampling method will be determined as the target sampling method for sampling and processing the original video data.

[0196] Specifically, the tenth determining unit 2203 is used for:

[0197] If the target sampling method is time sampling, then based on the video perception characteristics, determine the limit number of video frames corresponding to the perceived video data of the target object per unit time.

[0198] The target sampling rate under the time sampling method is determined by the ratio between the limited number of video frames and the number of video frames played per unit time in the original video data indicated by the video content characteristics.

[0199] Specifically, the tenth determining unit 2203 is used for:

[0200] If the target sampling method is spatial sampling, then the limiting video resolution associated with the target object is determined based on the video perception features;

[0201] The target sampling rate under spatial sampling mode is determined by the ratio between the restricted video resolution and the video resolution of the video frames in the original video data indicated by the video content features.

[0202] Specifically, the tenth determining unit 2203 is used for:

[0203] If the target sampling method is time sampling and spatial sampling, then based on the video perception characteristics, determine the limited number of video frames corresponding to the perceived video data of the target object per unit time, and determine the limited video resolution associated with the target object;

[0204] The target sampling rate under the time sampling method is determined by the ratio between the limited number of video frames and the number of video frames played per unit time in the original video data indicated by the video content characteristics.

[0205] The target sampling rate under spatial sampling mode is determined by the ratio between the restricted video resolution and the video resolution of the video frames in the original video data indicated by the video content features.

[0206] The sampling processing module 23 includes:

[0207] The second acquisition unit 2301 is used to acquire the playback sequence number of the video frame in the original video data and the total number of video frames contained in the original video data if the target sampling method is time sampling method.

[0208] The twelfth determining unit 2302 is used to determine the number of video frames to be extracted from the original video data based on the target sampling rate and the total number of video frames under the time sampling method, and use it as the first video frame number.

[0209] The first extraction unit 2303 is used to extract a video frame of the first video frame number from the original video data according to the playback sequence number of the video frame in the original video data, and use it as sampled video data.

[0210] The sampling processing module 23 includes:

[0211] The third acquisition unit 2304 is used to acquire video frame M from the original video data if the target sampling method is spatial sampling method. i The original video resolution; i is a positive integer less than or equal to M, where M is the number of video frames in the original video data;

[0212] The first resolution conversion unit 2305 is used to convert the target sampling rate under the spatial sampling mode and the video frame M. i The original video resolution, for video frames M with the original video resolution i Perform resolution conversion to obtain video frames M with the target video resolution. i ;

[0213] The thirteenth determining unit 2306 is used to determine the original video data after resolution sampling of all video frames in the original video data as the sampled video data.

[0214] The first resolution conversion unit 2305 is specifically used for:

[0215] The target sampling rate under spatial sampling method and the video frame M i The product of the original video resolutions is used as the initial video resolution;

[0216] For video frames M with original video resolution i Perform resolution conversion to obtain video frame M with the initial video resolution. i ;

[0217] If video frame M has an initial video resolution i If the encoding conditions are not met, then for video frame M with the initial video resolution... i Perform pixel filling, and then fill the video frame M. i The existing video resolution is determined as the target video resolution, and the padded video frame M is... i The video frame M with the target video resolution was determined. i ;

[0218] If video frame M has an initial video resolution i If the encoding conditions are met, the initial video resolution is determined as the target video resolution, and video frames M with the initial video resolution are... i The video frame M with the target video resolution was determined. i .

[0219] The sampling processing module 23 includes:

[0220] The fourteenth determining unit 2307 is used to determine the number of video frames to be extracted from the original video data as the second video frame number, based on the target sampling rate under the time sampling method and the total number of video frames in the original video data if the target sampling method is a time sampling method and a spatial sampling method.

[0221] The second extraction unit 2308 is used to extract a second video frame from the original video data according to the playback sequence number of the video frame in the original video data, and use it as the initial sampled video data.

[0222] The fourth acquisition unit 2309 is used to acquire video frame N in the initial sampled video data. j The original video resolution; j is a positive integer less than or equal to N, where N is the number of video frames in the initial sampled video data;

[0223] The second resolution conversion unit 2310 is used to convert the target sampling rate under the spatial sampling mode and the video frame N. j The original video resolution, for video frames N with the original video resolution. j Perform resolution conversion to obtain video frames N with the target video resolution. j ;

[0224] The fifteenth determining unit 2311 is used to determine the initial sampled video data after resolution sampling of all video frames in the initial sampled video data as the sampled video data.

[0225] The video encoding device also includes:

[0226] The first sending module 25 is used to send the total number of video frames and the target sampling rate under the time sampling method to the decoding device. The decoding device is used to perform sampling recovery processing on the sampled video data corresponding to the video encoded data according to the total number of video frames and the target sampling rate under the time sampling method.

[0227] The video encoding device also includes:

[0228] The second transmitting module 26 is used to transmit video frames M with a target video resolution. i If it is obtained after pixel padding, then obtain the video frame M with the target video resolution. i The pixel filling position information for pixel filling in the spatial sampling mode is sent to the decoding device. The decoding device is used to perform sampling recovery processing on the sampled video data corresponding to the video encoded data according to the target sampling rate and pixel filling position information under the spatial sampling mode.

[0229] The third transmitting module 27 is used to transmit video frames M with a target video resolution. i If the data is obtained without pixel padding, the target sampling rate under the spatial sampling method is sent to the decoding device. The decoding device is used to perform sampling recovery processing on the sampled video data corresponding to the video encoded data according to the target sampling rate under the spatial sampling method.

[0230] The video encoding device also includes:

[0231] The fourth determining module 28 is used to determine key video region information about the original video data;

[0232] The fourth sending module 29 is used to send key video region information and video encoding data to the decoding device; the key video region information is used to instruct the decoding device to perform image enhancement processing on key video regions in the original video data.

[0233] The fourth determining module 28 includes:

[0234] The vector transformation unit 2801 is used to input the original video data into the target detection model, and perform embedding vector transformation on the original video data through the embedding layer in the target detection model to obtain the media embedding vector of the original video data.

[0235] The object extraction unit 2802 is used to extract objects from the media embedding vector through the object extraction layer in the object detection model to obtain video objects in the original video data;

[0236] The second generation unit 2803 is used to determine the region to which the video object belongs in the original video data as the key video region, and generate key video region information to describe the location of the key video region in the original video data.

[0237] According to one embodiment of this application, Figure 8 The various modules in the video encoding apparatus shown can be individually or entirely combined into one or more units, or some of these units can be further divided into multiple functionally smaller sub-units to achieve the same operation without affecting the technical effects of the embodiments of this application. The above modules are based on logical functional division; in practical applications, the function of one module can be implemented by multiple units, or the function of multiple modules can be implemented by one unit. In other embodiments of this application, the video encoding apparatus may also include other units; in practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0238] On the encoding device side, by adaptively determining the target sampling parameters of the original video data based on the media application scenario and video content, and sampling the original video data based on the target sampling parameters to obtain sampled video data, the sampling accuracy of the original video data can be improved. This can effectively reduce the redundancy of the video encoding data while ensuring video viewing quality. Furthermore, the sampled video data is encoded to obtain encoded video data. Only this encoded video data needs to be sent to the decoding device, which can reduce the data volume of the encoded video data, thereby improving the transmission efficiency of the encoded video data. This allows the decoding device to quickly obtain the encoded video data, and also improves the encoding efficiency of the original video data.

[0239] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 9 As shown, the computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. Optionally, the network interface 1004 may include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the processor 1001. Figure 9 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application.

[0240] In such Figure 9 In the computer device 1000 shown, the network interface 1004 provides network communication functionality; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0241] The process involves obtaining the video encoding data to be decoded, as well as the target sampling parameters corresponding to the video encoding data. The video encoding data is obtained by encoding the sampled video data, and the sampled video data is obtained by sampling the original video data corresponding to the video encoding data based on the target sampling parameters. The target sampling parameters are determined according to the media application scenario and video content characteristics of the original video data.

[0242] Decode the encoded video data to obtain the sampled video data;

[0243] Based on the target sampling parameters, the sampled video data is processed to recover the original video data corresponding to the video encoded data.

[0244] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 6 The description of the video decoding method in the corresponding embodiments can also be performed as described above. Figure 7 The description of the video decoding device in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0245] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 10 As shown, the computer device 2000 may include a processor 2001, a network interface 2004, and a memory 2005. Furthermore, the computer device 2000 may also include a user interface 2003 and at least one communication bus 2002. The communication bus 2002 is used to enable communication between these components. The user interface 2003 may include a display screen and a keyboard; optionally, the user interface 2003 may also include a standard wired interface or a wireless interface. Optionally, the network interface 2004 may include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 2005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 2005 may also be at least one storage device located remotely from the processor 2001. Figure 10 As shown, the memory 2005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

[0246] In such Figure 10In the computer device 2000 shown, the network interface 2004 provides network communication functionality; the user interface 2003 is mainly used to provide an input interface for the user; and the processor 2001 can be used to call the device control application program stored in the memory 2005 to achieve:

[0247] The media application scenarios and video content characteristics of the raw video data to be encoded;

[0248] Based on the media application scenarios and video content characteristics, determine the target sampling parameters used for sampling processing of the raw video data;

[0249] Based on the target sampling parameters, the original video data is sampled to obtain sampled video data;

[0250] The sampled video data is encoded to obtain the video encoded data corresponding to the original video data.

[0251] It should be understood that the computer device 2000 described in the embodiments of this application can execute the foregoing text. Figure 3 The description of the video encoding method in the corresponding embodiments can also be performed as described above. Figure 8 The description of the video encoding apparatus in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0252] Furthermore, it should be noted that this application embodiment also provides a computer-readable storage medium, which stores a computer program executed by the aforementioned video decoding device. The computer program includes program instructions, and when the processor executes the program instructions, it can execute the aforementioned... Figure 6 The corresponding embodiment describes the video decoding method, or... Figure 3 The video encoding method described in the corresponding embodiments will not be repeated here.

[0253] Furthermore, the beneficial effects of using the same method will not be repeated here. For technical details not disclosed in the computer-readable storage medium embodiments involved in this application, please refer to the description of the method embodiments of this application. As an example, program instructions may be deployed and executed on a computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed in multiple locations and interconnected through a communication network. Multiple computer devices distributed in multiple locations and interconnected through a communication network can constitute a blockchain system.

[0254] Furthermore, it should be noted that this application also provides a computer program product or computer program, which may include computer instructions, which may be stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor may execute the computer instructions, causing the computer device to perform the aforementioned actions. Figure 6 The corresponding embodiment describes the video decoding method, or... Figure 3 The description of the video encoding method in the corresponding embodiments is already provided and will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer program products or computer program embodiments related to this application, please refer to the description of the method embodiments of this application.

[0255] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0256] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0257] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0258] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0259] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A video encoding method, characterized in that, include: The media application scenarios and video content characteristics of the raw video data to be encoded; The media application scenarios include user viewing scenarios or machine recognition scenarios; Based on the rate of change of video content included in the video content features, the repetition rate of video content in the original video data is determined; If the repetition rate of the video content in the original video data is less than or equal to a first repetition rate threshold and greater than a second repetition rate threshold, then the time sampling method is determined as the target sampling method for sampling the original video data; the second repetition rate threshold is less than the first repetition rate threshold, and the time sampling method refers to frame sampling of the original video data; If the repetition rate of the video content in the original video data is less than or equal to the second repetition rate threshold, then the spatial sampling method is determined as the target sampling method for sampling the original video data. The spatial sampling method refers to sampling the original video data at video resolution. Determine the video perception features of the target object in the media application scenario based on the video data; the target object is the object to which the original video data is perceptually processed. Based on the video perception features and the video content features, determine the target sampling rate under the target sampling method; The target sampling rate and the target sampling method are determined as the target sampling parameters for sampling processing the original video data; Based on the target sampling parameters, the original video data is sampled to obtain sampled video data; The sampled video data is encoded to obtain the video encoded data corresponding to the original video data.

2. The method according to claim 1, characterized in that, The step of determining the target sampling method for sampling the original video data based on the repetition rate of the video content in the original video data includes: If the repetition rate of the video content in the original video data is greater than the first repetition rate threshold, then the temporal sampling method and the spatial sampling method are determined as the target sampling methods for sampling the original video data.

3. The method according to claim 2, characterized in that, The method further includes: The complexity of the video content in the original video data is determined based on the amount of video content information included in the video content features. Based on the complexity of the video content in the original video data, a target sampling method is determined for sampling processing of the original video data.

4. The method according to claim 3, characterized in that, The step of determining the target sampling method for sampling processing the original video data based on the complexity of the video content in the original video data includes: If the complexity of the video content of the original video data is less than the first complexity threshold, then the temporal sampling method and the spatial sampling method are determined as the target sampling methods for sampling processing the original video data. If the complexity of the video content in the original video data is greater than or equal to the first complexity threshold and less than the second complexity threshold, then the spatial sampling method is determined as the target sampling method for sampling processing the original video data; the second complexity threshold is greater than the first complexity threshold. If the complexity of the video content in the original video data is greater than the second complexity threshold, then the time sampling method is determined as the target sampling method for sampling the original video data.

5. The method according to claim 1, characterized in that, Determining the target sampling rate under the target sampling method based on the video perception features and the video content features includes: If the target sampling method is a time sampling method, then based on the video perception features, the limited number of video frames corresponding to the perceived video data of the target object per unit time time is determined; The target sampling rate under the time sampling method is determined based on the ratio between the limited number of video frames and the number of video frames played per unit time in the original video data indicated by the video content features.

6. The method according to claim 1, characterized in that, Determining the target sampling rate under the target sampling method based on the video perception features and the video content features includes: If the target sampling method is spatial sampling, then the restricted video resolution associated with the target object is determined based on the video perception features; The ratio between the restricted video resolution and the video resolution of the video frames in the original video data indicated by the video content features is determined as the target sampling rate under the spatial sampling method.

7. The method according to claim 1, characterized in that, Determining the target sampling rate under the target sampling method based on the video perception features and the video content features includes: If the target sampling method is a time sampling method and a spatial sampling method, then based on the video perception features, the limited number of video frames corresponding to the perceived video data of the target object per unit time time is determined, and the limited video resolution associated with the target object is determined; The target sampling rate under the time sampling method is determined based on the ratio between the limited number of video frames and the number of video frames played per unit time in the original video data indicated by the video content features. The ratio between the restricted video resolution and the video resolution of the video frames in the original video data indicated by the video content features is determined as the target sampling rate under the spatial sampling method.

8. The method according to claim 1, characterized in that, The step of sampling the original video data according to the target sampling parameters to obtain sampled video data includes: If the target sampling method is time sampling method, then obtain the playback sequence number of the video frame in the original video data, and the total number of video frames contained in the original video data; Based on the target sampling rate under the time sampling method and the total number of video frames, determine the number of video frames to be extracted from the original video data, and use it as the first video frame number; According to the playback sequence number of the video frames in the original video data, the first number of video frames are extracted from the original video data and used as the sampled video data.

9. The method according to claim 1, characterized in that, The step of sampling the original video data according to the target sampling parameters to obtain sampled video data includes: If the target sampling method is spatial sampling, then obtain video frame M from the original video data. i The original video resolution; i is a positive integer less than or equal to M, where M is the number of video frames in the original video data; Based on the target sampling rate under the spatial sampling method and the video frame M i The original video resolution, for video frames M with said original video resolution i Perform resolution conversion to obtain video frames M with the target video resolution. i ; After performing resolution sampling on all video frames in the original video data, the original video data after resolution conversion is determined as the sampled video data.

10. The method according to claim 9, characterized in that, The target sampling rate under the spatial sampling method and the video frame M are used. i The original video resolution, for video frames M with said original video resolution i Perform resolution conversion to obtain video frames M with the target video resolution. i ,include: The target sampling rate under the spatial sampling method is compared with the video frame M. i The product of the original video resolutions is used as the initial video resolution; For video frames M with the original video resolution i Perform resolution conversion to obtain video frame M with the initial video resolution. i ; If video frame M has the initial video resolution i If the encoding conditions are not met, then for video frame M with the initial video resolution... i Perform pixel filling, and then fill the video frame M. i The video resolution is determined as the target video resolution, and the padded video frame M is... i The video frame M with the target video resolution was determined. i ; If video frame M has the initial video resolution i If the encoding conditions are met, then the initial video resolution is determined as the target video resolution, and video frames M with the initial video resolution are... i The video frame M with the target video resolution is determined. i .

11. The method according to claim 1, characterized in that, The step of sampling the original video data according to the target sampling parameters to obtain sampled video data includes: If the target sampling method is a time sampling method and a spatial sampling method, then the number of video frames to be extracted from the original video data is determined based on the target sampling rate under the time sampling method and the total number of video frames in the original video data, and is used as the second number of video frames. According to the playback sequence number of the video frames in the original video data, the video frames of the second video frame number are extracted from the original video data as the initial sampled video data; Obtain video frame N from the initial sampled video data j The original video resolution; j is a positive integer less than or equal to N, where N is the number of video frames in the initial sampled video data; Based on the target sampling rate under the spatial sampling method and the video frame N j The original video resolution, for video frames N having the original video resolution. j Perform resolution conversion to obtain video frames N with the target video resolution. j ; After performing resolution sampling on all video frames in the initial sampled video data, the initial sampled video data after resolution sampling is determined as the sampled video data.

12. A video decoding method, characterized in that, include: Obtain the video encoding data to be decoded, and the target sampling parameters corresponding to the video encoding data; The video encoding data is obtained by encoding sampled video data. The sampled video data is obtained by sampling the original video data corresponding to the video encoding data based on the target sampling parameters. The target sampling parameters include a target sampling rate and a target sampling method. The target sampling rate is determined based on video perception features and video content features of the original video data. The video perception features are the perception features of the target object for the video data in the media application scenario. The target object is the object that performs perception processing on the original video data. The media application scenario includes user viewing scenarios or machine recognition scenarios. If the repetition rate of the video content in the original video data is less than or equal to a first repetition rate threshold and greater than a second repetition rate threshold, then the target sampling method is a temporal sampling method. If the repetition rate of the video content in the original video data is less than or equal to the second repetition rate threshold, then the target sampling method is a spatial sampling method. The repetition rate of the video content in the original video data is determined by the rate of change of the video content included in the video content features. The video encoded data is decoded to obtain the sampled video data; Based on the target sampling parameters, the sampled video data is subjected to sampling recovery processing to obtain the original video data corresponding to the video encoded data.

13. The method according to claim 12, characterized in that, The step of performing sampling recovery processing on the sampled video data according to the target sampling parameters to obtain the original video data corresponding to the video encoded data includes: If the target sampling method is a time sampling method, then the third video frame number of the recovery video frame to be recovered between the first decoded video frame and the second decoded video frame is determined according to the target sampling rate under the time sampling method; the first decoded video frame and the second decoded video frame belong to the video frames in the sampled video data that have an adjacent playback sequence number. A third number of restored video frames is inserted between the first decoded video frame and the second decoded video frame. After inserting and recovering video frames between any two adjacent decoded video frames in the sampled video data, the original video data corresponding to the video encoded data is determined based on the recovered sampled video data.

14. The method according to claim 13, characterized in that, After inserting and recovering video frames between any two adjacent decoded video frames in the sampled video data, the original video data corresponding to the video encoded data is determined based on the recovered sampled video data, including: After inserting and restoring video frames between any two adjacent decoded video frames in the sampled video data, the fourth video frame number of the video frames contained in the restored sampled video data and the total number of video frames contained in the original video data are obtained. If the fourth video frame number is the same as the total video frame number, then the recovered sampled video data is determined as the original video data corresponding to the video encoded data; If the fourth video frame number is not the same as the total video frame number, the difference between the fourth video frame number and the total video frame number is obtained as the fifth video frame number. After the recovered sampled video data, the recovered video frame of the fifth video frame number is inserted to obtain the original video data corresponding to the video encoded data.

15. The method according to claim 12, characterized in that, The step of performing sampling recovery processing on the sampled video data according to the target sampling parameters to obtain the original video data corresponding to the video encoded data includes: If the target sampling method is spatial sampling, then obtain the current target video resolution of the third decoded video frame in the sampled video data; Based on the target sampling rate under the spatial sampling method and the target video resolution, the third decoded video frame with the target video resolution is subjected to resolution restoration processing to obtain the third decoded video frame with the original video resolution. After recovering all decoded video frames from the sampled video data, the recovered sampled video data is identified as the original video data corresponding to the video encoded data.

16. The method according to claim 15, characterized in that, The step of performing resolution restoration processing on the target video resolution of the third decoded video frame based on the target sampling rate under the spatial sampling mode and the target video resolution to obtain a third decoded video frame with the original video resolution includes: If pixel padding position information about the third decoded video frame is received, then according to the pixel padding position information, the third decoded video frame with the target video resolution is cropped to obtain a third decoded video frame with an initial video resolution. The ratio between the initial video resolution and the target sampling rate under the spatial sampling mode is determined as the original video resolution to be restored for the third decoded video frame. The third decoded video frame with the initial video resolution is subjected to resolution restoration processing to obtain a third decoded video frame with the original video resolution. If no pixel filling position information for the third decoded video frame is received, the ratio between the target sampling rate and the target video resolution under the spatial sampling mode is determined as the original video resolution to be restored for the third decoded video frame. Resolution restoration processing is performed on the third decoded video frame with the target video resolution to obtain a third decoded video frame with the original video resolution.

17. The method according to claim 12, characterized in that, The step of performing sampling recovery processing on the sampled video data according to the target sampling parameters to obtain the original video data corresponding to the video encoded data includes: If the target sampling method is a time sampling method and a spatial sampling method, then the target video resolution of the third decoded video frame in the sampled video data is obtained. Based on the target sampling rate under the spatial sampling method and the target video resolution, the target video resolution of the third decoded video frame is restored to obtain a third decoded video frame with the original video resolution. After recovering all decoded video frames from the sampled video data, the recovered sampled video data is determined as the initial video data corresponding to the video encoded data. Based on the target sampling rate under the time sampling method, the sixth video frame number of the video frames to be recovered between the first initial video frame and the second initial video frame is determined; the first initial video frame and the second initial video frame belong to video frames with adjacent playback sequence numbers in the initial video data; A sixth number of restored video frames is inserted between the first initial video frame and the second initial video frame. After the restored video frames are inserted between any two adjacent initial video frames in the initial video data, the original video data corresponding to the video encoded data is determined based on the restored initial video data.

18. A video decoding device, characterized in that, include: The first acquisition module is used to acquire the video encoding data to be decoded, and the target sampling parameters corresponding to the video encoding data; The video encoded data is obtained by encoding the sampled video data. The sampled video data is obtained by sampling the original video data corresponding to the video encoded data based on the target sampling parameters. The target sampling parameters are determined according to the media application scenario and video content characteristics of the original video data. This includes a target sampling rate and a target sampling method. The target sampling rate is determined based on video perception features and video content features of the original video data. The video perception features are the perception features of the target object in the media application scenario regarding the video data. The target object is the object that performs perception processing on the original video data. The media application scenario includes user viewing scenarios or machine recognition scenarios. If the repetition rate of the video content in the original video data is less than or equal to a first repetition rate threshold and greater than a second repetition rate threshold, then the target sampling method is a temporal sampling method. If the repetition rate of the video content in the original video data is less than or equal to the second repetition rate threshold, then the target sampling method is a spatial sampling method. The repetition rate of the video content in the original video data is determined by the rate of change of the video content included in the video content features. The decoding module is used to decode the video encoded data to obtain the sampled video data; The sampling recovery module is used to perform sampling recovery processing on the sampled video data according to the target sampling parameters to obtain the original video data corresponding to the video encoded data.

19. A video encoding apparatus, characterized in that, include: The second acquisition module is used to acquire the media application scenarios and video content characteristics of the raw video data to be encoded; The media application scenarios include user viewing scenarios or machine recognition scenarios; The second determining module includes: the eighth determining unit, the ninth determining unit, the tenth determining unit, and the eleventh determining unit; The eighth determining unit is configured to determine the repetition rate of video content in the original video data based on the rate of change of video content included in the video content features; if the repetition rate of video content in the original video data is less than or equal to a first repetition rate threshold and greater than a second repetition rate threshold, then the temporal sampling method is determined as the target sampling method for sampling the original video data; the second repetition rate threshold is less than the first repetition rate threshold, and the temporal sampling method refers to frame sampling of the original video data; if the repetition rate of video content in the original video data is less than or equal to the second repetition rate threshold, then the spatial sampling method is determined as the target sampling method for sampling the original video data; the spatial sampling method refers to video resolution sampling of the original video data. The ninth determining unit is used to determine the video perception features of the target object in the media application scenario in relation to the video data; the target object is the object that performs perception processing on the original video data; The tenth determining unit is used to determine the target sampling rate under the target sampling method based on the video perception features and the video content features; The eleventh determining unit is used to determine the target sampling rate and the target sampling method as target sampling parameters for sampling processing of the original video data; The sampling processing module is used to perform sampling processing on the original video data according to the target sampling parameters to obtain sampled video data; The encoding module is used to encode the sampled video data to obtain the video encoded data corresponding to the original video data.

20. A computer device, characterized in that, include: Processor and memory; The processor is connected to a memory, wherein the memory is used to store a computer program, and the processor is used to invoke the computer program to cause the computer device to perform the method according to any one of claims 1-17.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-17.

Images