Data processing method and device, equipment and storage medium

Abstract

The invention provides a signal data processing method and device, equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: determining whether a time domain recovery flag bit is set in code stream data, determining the number of residual frames under the condition that the time domain recovery flag bit is set, and carrying out the frame compensation based on the number of the residual frames, and obtaining a first complementary frame segment. Coding and decoding based on variable up-sampling multiplying power are realized.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority to Chinese Patent Application No. 202410460640.4, filed on April 16, 2024, entitled “Data processing method, device, equipment and storage medium”, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates to the technical field of image processing, and particularly relates to a data processing method, device, equipment and storage medium.

[0004] With the development of the times, the rate of information transmission is accelerating. Among the current transmitted information, video information accounts for a large proportion.

[0005] In the process of information transmission, frame extraction coding technology is a commonly used technology in the field of signal coding technology. The current frame extraction coding technology usually extracts frames from the video according to a fixed frame extraction ratio, and then encodes the video obtained by frame extraction. However, with the development of technology, the single up-sampling ratio coding and decoding method has been difficult to meet the needs of users. How to realize the coding and decoding of variable up-sampling ratio is a problem to be solved in the field. SUMMARY

[0006] The present disclosure provides a data processing method, device, equipment and storage medium, which at least to some extent realizes the coding and decoding of variable up-sampling ratio.

[0007] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0008] According to one aspect of the present disclosure, a data processing method is provided, applied to a decoding end, comprising: determining whether a time domain recovery flag bit is set in code stream data; in the case that the time domain recovery flag bit is set, determining the number of remaining frames; based on the number of remaining frames, performing frame filling to obtain a first frame filling segment.

[0009] In one embodiment of the present disclosure, the method further comprises: determining a plurality of sampling segments, a number of original frames corresponding to each sampling segment and a sampling ratio corresponding to each sampling segment based on the code stream data; based on the number of original frames corresponding to each sampling segment and the sampling ratio corresponding to each sampling segment, performing frame filling to obtain a frame filling segment corresponding to each sampling segment.

[0010] In one embodiment of the present disclosure, the method further comprises: in a case where the current sampling frame is not the last sampling frame, obtaining a sampling rate corresponding to the current sampling frame; in a case where the sampling rate corresponding to the current sampling frame is not one, performing frame interpolation based on the sampling rate, the current sampling frame and a neighboring sampling frame of the current sampling frame to obtain a second frame-interpolated segment corresponding to the current sampling frame.

[0011] In one embodiment of the present disclosure, the method further comprises: in a case where the current sampling frame is not the last sampling frame, obtaining a sampling rate corresponding to the current sampling frame; in a case where the sampling rate corresponding to the current sampling frame is one, determining a third frame-interpolated segment based on the current sampling frame.

[0012] In one embodiment of the present disclosure, the method further comprises: determining a decoded video based on the first frame-interpolated segment, the second frame-interpolated segment and the third frame-interpolated segment.

[0013] In one embodiment of the present disclosure, the method further comprises: in a case where the time domain recovery flag is set, determining whether the time domain sampling rate corresponding to the current sampling frame is switched; in a case where the time domain sampling rate corresponding to the current sampling frame is switched, obtaining a sampling rate corresponding to the current sampling frame; and performing frame interpolation based on the sampling rate.

[0014] In one embodiment of the present disclosure, the method further comprises: in a case where the time domain sampling rate corresponding to the current sampling frame is not switched, obtaining a sampling rate corresponding to a sampling frame before the current sampling frame; and performing frame interpolation based on the sampling rate corresponding to the sampling frame before the current sampling frame.

[0015] In one embodiment of the present disclosure, the method further comprises: in a case where the time domain recovery flag is set, determining whether the current sampling frame corresponds to a sampling rate maintaining flag; in a case where the current sampling frame corresponds to the sampling rate maintaining flag, determining whether a sampling rate maintaining flag bit is set; in a case where the sampling rate maintaining flag bit is not set, determining whether a sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag; in a case where the sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag, determining whether a sampling rate maintaining flag bit corresponding to the sampling frame after the current sampling frame is set; and in a case where the sampling rate maintaining flag bit corresponding to the sampling frame after the current sampling frame is set, performing frame interpolation based on a sampling rate corresponding to a last sampling frame of the sampling frame after the current sampling frame.

[0016] In one embodiment of the present disclosure, the method further comprises: in the case that the time domain recovery flag is set, determining whether the current sampling frame corresponds to a sampling rate maintaining flag; in the case that the current sampling frame corresponds to the sampling rate maintaining flag, determining whether a sampling rate maintaining flag bit is set; in the case that the sampling rate maintaining flag bit is set, determining the sampling rate corresponding to the current sampling frame; and performing frame interpolation on the sampling frame after the current sampling frame based on the sampling rate.

[0017] In one embodiment of the present disclosure, the method further comprises: in the case that the time domain recovery flag is set, determining whether the current sampling frame corresponds to a sampling rate maintaining flag; in the case that the current sampling frame corresponds to the sampling rate maintaining flag, determining whether a sampling rate maintaining flag bit is set; in the case that the sampling rate maintaining flag bit is set, determining the sampling rate corresponding to the current sampling frame; and performing frame interpolation on the sampling frame after the current sampling frame based on the sampling rate.

[0018] In one embodiment of the present disclosure, the method further comprises: in the case that the time domain recovery flag is set, determining whether the current sampling frame corresponds to a sampling rate maintaining flag; in the case that the current sampling frame corresponds to the sampling rate maintaining flag, determining whether a sampling rate maintaining flag bit is set; in the case that the sampling rate maintaining flag bit is set, determining whether the sampling frame after the current sampling frame corresponds to a sampling rate maintaining flag; in the case that the sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag, determining whether a sampling rate maintaining flag bit corresponding to the sampling frame after the current sampling frame is set; in the case that the sampling rate maintaining flag bit corresponding to the sampling frame after the current sampling frame is set, determining the sampling rate corresponding to the sampling frame after the current sampling frame; and performing frame interpolation on the sampling frame after the current sampling frame based on the sampling rate.

[0019] In one embodiment of the present disclosure, the method further comprises: in the case that the time domain recovery flag is set, determining whether the current sampling frame corresponds to a sampling rate maintaining flag; in the case that the current sampling frame corresponds to the sampling rate maintaining flag, determining whether a sampling rate maintaining flag bit is set; in the case that the sampling rate maintaining flag bit is set, obtaining the sampling rate corresponding to the current sampling frame and the number of maintained frames; and performing frame interpolation on the number of sampling frames including the current frame and maintaining the number of frames at the sampling rate.

[0020] In an embodiment of the present disclosure, the method further comprises: in the case that the time domain recovery flag is set, determining whether the current sampling frame corresponds to a sampling rate keeping flag; in the case that the current sampling frame corresponds to the sampling rate keeping flag, determining whether a sampling rate keeping flag bit is set; in the case that the sampling rate keeping flag is set, obtaining a sampling rate corresponding to the current sampling frame and a keeping quantity; determining whether a sampling frame after the keeping quantity corresponds to the sampling rate keeping flag; in the case that the sampling frame after the keeping quantity corresponds to the sampling rate keeping flag, determining whether a sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is set; in the case that the sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is not set, determining a sampling rate corresponding to the sampling frame after the keeping quantity; and performing frame filling on the sampling frame after the keeping quantity based on the sampling rate corresponding to the sampling frame after the keeping quantity.

[0021] In an embodiment of the present disclosure, the method further comprises: in the case that the time domain recovery flag is set, determining whether the current sampling frame corresponds to a sampling rate keeping flag; in the case that the current sampling frame corresponds to the sampling rate keeping flag, determining whether a sampling rate keeping flag bit is set; in the case that the sampling rate keeping flag is set, obtaining a sampling rate corresponding to the current sampling frame and a keeping quantity; determining whether a sampling frame after the keeping quantity corresponds to the sampling rate keeping flag; in the case that the sampling frame after the keeping quantity corresponds to the sampling rate keeping flag, determining whether a sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is set; in the case that the sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is not set, determining a sampling rate corresponding to the sampling frame after the keeping quantity; and performing frame filling on the sampling frame after the keeping quantity based on the sampling rate corresponding to the current sampling frame in the case that the sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is set.

[0022] In an embodiment of the present disclosure, the keeping quantity is determined by a quantity of sampling frames between target frames, wherein the target frames comprise key frames.

[0023] In an embodiment of the present disclosure, the method further comprises: in the case that the time domain recovery flag is set, determining whether the current sampling frame is a key frame; in the case that the current sampling frame is a key frame, determining whether a sampling rate keeping flag bit corresponding to the current sampling frame is set; in the case that the sampling rate keeping flag bit corresponding to the current sampling frame is set, determining whether a sampling frame after the current sampling frame is a key frame; in the case that the sampling frame after the current sampling frame is a key frame, determining a sampling rate corresponding to the sampling frame after the current sampling frame; and performing frame filling based on the sampling rate.

[0024] In an embodiment of the present disclosure, the method further comprises: in the case that the time domain recovery flag bit is set, determining whether the current sampling frame is a key frame; in the case that the current sampling frame is a key frame, determining whether a sampling rate maintaining flag bit corresponding to the current sampling frame is set; in the case that the sampling rate maintaining flag bit corresponding to the current sampling frame is not set, determining whether a sampling frame after the current sampling frame is a key frame; in the case that the sampling frame after the current sampling frame is not a key frame, determining whether the sampling frame after the current sampling frame corresponds to a sampling rate; in the case that the sampling frame after the current sampling frame corresponds to the sampling rate, performing frame interpolation based on the sampling rate.

[0025] In an embodiment of the present disclosure, the method further comprises: in the case that the time domain recovery flag bit is set, determining whether the current sampling frame is a key frame; in the case that the current sampling frame is a key frame, determining whether a sampling rate maintaining flag bit corresponding to the current sampling frame is set; in the case that the sampling rate maintaining flag bit corresponding to the current sampling frame is not set, determining whether a sampling frame after the current sampling frame is a key frame; in the case that the sampling frame after the current sampling frame is not a key frame, determining whether the sampling frame after the current sampling frame corresponds to a sampling rate; in the case that the sampling frame after the current sampling frame corresponds to the sampling rate, performing frame interpolation based on the sampling rate.

[0026] In an embodiment of the present disclosure, the method further comprises: in the case that the time domain recovery flag bit is set, determining whether the current sampling frame is a key frame; in the case that the current sampling frame is a key frame, determining whether a sampling rate maintaining flag bit corresponding to the current sampling frame is set; in the case that the sampling rate maintaining flag bit corresponding to the current sampling frame is not set, determining whether a sampling frame after the current sampling frame is a key frame; in the case that the sampling frame after the current sampling frame is not a key frame, determining whether the sampling frame after the current sampling frame corresponds to a sampling rate; in the case that the sampling frame after the current sampling frame does not correspond to the sampling rate, performing frame interpolation based on a sampling rate corresponding to the current sampling frame.

[0027] In an embodiment of the present disclosure, the code stream data comprises a sequence parameter set and end information;

[0028] The determining, in the code stream data, whether the time domain recovery flag bit is set comprises: determining, according to the sequence parameter set, whether the time domain recovery flag bit is set;

[0029] The determining, in the case that the time domain recovery flag bit is set, the number of remaining frames comprises:

[0030] The determining, in the case that the time domain recovery flag bit is set, whether the end information has target data comprises:

[0031] In the case that the end information has target data, the method further comprises: acquiring, according to the target data, remaining frame information, and determining the number of the remaining frames according to the remaining frame information.

[0032] In one embodiment of the present disclosure, the code stream data comprises a sequence parameter set and end information.

[0033] In the case that the time domain recovery flag bit is set, the number of remaining frames is determined.

[0034] It is determined whether the end information has target data.

[0035] In the case that the target data exists, it is determined whether the time domain recovery flag bit is set.

[0036] In the case that the time domain recovery flag bit is set, the remaining frame information is obtained according to the target data, and the number of the remaining frames is determined according to the remaining frame information.

[0037] In one embodiment of the present disclosure, the image information comprises the sampling rate maintaining flag bit and time domain sampling rate change information; the time domain sampling rate change information comprises a time domain sampling rate change index.

[0038] According to another aspect of the present disclosure, a data processing method applied to a decoding end is provided, comprising: performing frame padding according to a sampling rate corresponding to a sampling frame, wherein code stream data comprises at least two different sampling rates.

[0039] In one embodiment of the present disclosure, the performing frame padding according to the sampling rate corresponding to the sampling frame comprises:

[0040] The sampling rate corresponding to a current sampling frame is obtained.

[0041] In the case that the sampling rate corresponding to the current sampling frame is not one, frame padding is performed based on the sampling rate, the current sampling frame and a neighboring sampling frame of the current sampling frame, to obtain a second frame padding segment corresponding to the current sampling frame.

[0042] In one embodiment of the present disclosure, the performing frame padding according to the sampling rate corresponding to the sampling frame comprises:

[0043] The sampling rate corresponding to the current sampling frame is obtained.

[0044] In the case that the sampling rate corresponding to the current sampling frame is one, a third frame padding segment is determined based on the current sampling frame.

[0045] In one embodiment of the present disclosure, the method further comprises:

[0046] The number of remaining frames is determined.

[0047] Frame padding is performed based on the number of the remaining frames, to obtain a first frame padding segment.

[0048] According to still another aspect of the present disclosure, a data processing apparatus applied to an encoding end is provided, comprising:

[0049] A fifteenth frame supplementing module is configured to supplement frames according to a sampling rate corresponding to the sampling frames, wherein the code stream data comprises at least two different sampling rates.

[0050] According to another aspect of the present disclosure, a data processing method applied to an encoding end is provided, comprising: encoding a video to obtain code stream data, so that a decoding end implements any one of the data processing methods in the decoding end based on the code stream data.

[0051] According to still another aspect of the present disclosure, a data processing apparatus applied to a decoding end is provided, comprising: a first determining module configured to determine whether a time domain recovery flag bit is set in the code stream data; a second determining module configured to determine a number of remaining frames in a case where the time domain recovery flag bit is set; and a first frame supplementing module configured to supplement frames based on the number of remaining frames to obtain a first frame supplemented segment.

[0052] In an embodiment of the present disclosure, the apparatus further comprises: a fiftieth determining module configured to determine a plurality of sampling segments, a number of original frames corresponding to each sampling segment and a sampling rate corresponding to each sampling segment based on the code stream data; and a sixteenth frame supplementing module configured to supplement frames based on the number of original frames corresponding to each sampling segment and the sampling rate corresponding to each sampling segment to obtain a frame supplemented segment corresponding to each sampling segment.

[0053] In an embodiment of the present disclosure, the apparatus further comprises: a first obtaining module configured to obtain a sampling rate corresponding to a current sampling frame in a case where the current sampling frame is not the last sampling frame; and a second frame supplementing module configured to supplement frames based on the sampling rate, the current sampling frame and a neighboring sampling frame of the current sampling frame to obtain a second frame supplemented segment corresponding to the current sampling frame in a case where the sampling rate corresponding to the current sampling frame is not one.

[0054] In an embodiment of the present disclosure, the apparatus further comprises: a second obtaining module configured to obtain a sampling rate corresponding to a current sampling frame in a case where the current sampling frame is not the last sampling frame; and a third determining module configured to determine a third frame supplemented segment based on the current sampling frame in a case where the sampling rate corresponding to the current sampling frame is one.

[0055] In an embodiment of the present disclosure, the apparatus further comprises: a fourth determining module configured to determine a decoded video based on the first frame supplemented segment, the second frame supplemented segment and the third frame supplemented segment.

[0056] In one embodiment of the present disclosure, the apparatus further comprises: a fifth determining module, configured to determine whether the time domain sampling rate corresponding to the current sampling frame is switched in the case that the time domain recovery flag is set; a third acquiring module, configured to acquire the sampling rate corresponding to the current sampling frame in the case that the time domain sampling rate corresponding to the current sampling frame is switched; and a third frame supplementing module, configured to perform frame supplementing based on the sampling rate.

[0057] In one embodiment of the present disclosure, the apparatus further comprises: a fourth acquiring module, configured to acquire the sampling rate corresponding to the sampling frame before the current sampling frame in the case that the time domain sampling rate corresponding to the current sampling frame is not switched; and a fourth frame supplementing module, configured to perform frame supplementing based on the sampling rate corresponding to the sampling frame before the current sampling frame.

[0058] In one embodiment of the present disclosure, the apparatus further comprises: a sixth determining module, configured to determine whether the sampling rate maintaining flag corresponding to the current sampling frame exists in the case that the time domain recovery flag is set; a seventh determining module, configured to determine whether the sampling rate maintaining flag bit is set in the case that the sampling rate maintaining flag corresponding to the current sampling frame exists; an eighth determining module, configured to determine whether the sampling rate maintaining flag corresponding to the sampling frame after the current sampling frame exists in the case that the sampling rate maintaining flag bit is not set; a ninth determining module, configured to determine whether the sampling rate maintaining flag bit corresponding to the sampling rate maintaining flag corresponding to the sampling frame after the current sampling frame is set in the case that the sampling rate maintaining flag corresponding to the sampling frame after the current sampling frame exists; and a fifth frame supplementing module, configured to perform frame supplementing based on the sampling rate corresponding to the last sampling frame of the sampling frame after the current sampling frame in the case that the sampling rate maintaining flag bit corresponding to the sampling rate maintaining flag corresponding to the sampling frame after the current sampling frame is set.

[0059] In one embodiment of the present disclosure, the apparatus further comprises: a tenth determining module, configured to determine whether the sampling rate maintaining flag corresponding to the current sampling frame exists in the case that the time domain recovery flag is set; an eleventh determining module, configured to determine whether the sampling rate maintaining flag bit is set in the case that the sampling rate maintaining flag corresponding to the current sampling frame exists; a twelfth determining module, configured to determine whether the sampling rate maintaining flag corresponding to the sampling frame after the current sampling frame exists in the case that the sampling rate maintaining flag bit is not set; a thirteenth determining module, configured to determine whether the sampling rate maintaining flag bit corresponding to the sampling rate maintaining flag corresponding to the sampling frame after the current sampling frame is set in the case that the sampling rate maintaining flag corresponding to the sampling frame after the current sampling frame exists; a fourteenth determining module, configured to acquire the sampling rate corresponding to the sampling frame after the current sampling frame in the case that the sampling rate maintaining flag bit corresponding to the sampling rate maintaining flag corresponding to the sampling frame after the current sampling frame is not set; and a sixth frame supplementing module, configured to perform frame supplementing based on the sampling rate.

[0060] In one embodiment of the present disclosure, the apparatus further comprises: a fifteenth determining module configured to determine whether the current sampling frame corresponds to a sampling rate maintaining flag in a case where the time domain recovery flag is set; a sixteenth determining module configured to determine whether a sampling rate maintaining flag bit is set in a case where the current sampling frame corresponds to the sampling rate maintaining flag; a seventeenth determining module configured to determine a sampling rate corresponding to the current sampling frame in a case where the sampling rate maintaining flag bit is set; and a seventh frame supplementing module configured to supplement frames after the current sampling frame based on the sampling rate.

[0061] In one embodiment of the present disclosure, the apparatus further comprises: an eighteenth determining module configured to determine whether the current sampling frame corresponds to a sampling rate maintaining flag in a case where the time domain recovery flag is set; a nineteenth determining module configured to determine whether a sampling rate maintaining flag bit is set in a case where the current sampling frame corresponds to the sampling rate maintaining flag; a twentieth determining module configured to determine whether the frames after the current sampling frame correspond to the sampling rate maintaining flag in a case where the sampling rate maintaining flag bit is set; a twenty-first determining module configured to determine whether a sampling rate maintaining flag bit corresponding to the frames after the current sampling frame is set in a case where the frames after the current sampling frame correspond to the sampling rate maintaining flag; a twenty-second determining module configured to determine a sampling rate corresponding to the frames after the current sampling frame in a case where the sampling rate maintaining flag bit corresponding to the frames after the current sampling frame is set; and an eighth frame supplementing module configured to supplement the frames after the current sampling frame based on the sampling rate.

[0062] In one embodiment of the present disclosure, the apparatus further comprises: a twenty-third determining module configured to determine whether the current sampling frame corresponds to a sampling rate maintaining flag in a case where the time domain recovery flag is set; a twenty-fourth determining module configured to determine whether a sampling rate maintaining flag bit is set in a case where the current sampling frame corresponds to the sampling rate maintaining flag; a fifth obtaining module configured to obtain a sampling rate corresponding to the current sampling frame and a maintaining number in a case where the sampling rate maintaining flag bit is set; and a ninth frame supplementing module configured to supplement the sampling frames containing the current frame and the maintaining number by the sampling rate.

[0063] In one embodiment of the present disclosure, the apparatus further comprises: a twenty-fifth determining module configured to determine whether the current sampling frame corresponds to a sampling rate keeping flag in the case that the time domain recovery flag is set; a twenty-sixth determining module configured to determine whether a sampling rate keeping flag bit is set in the case that the current sampling frame corresponds to the sampling rate keeping flag; a sixth acquiring module configured to acquire a sampling rate corresponding to the current sampling frame and a keeping quantity in the case that the sampling rate keeping flag is set; a twenty-seventh determining module configured to determine whether a sampling frame after the keeping quantity corresponds to the sampling rate keeping flag; a twenty-eighth determining module configured to determine whether a sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is set in the case that the sampling frame after the keeping quantity corresponds to the sampling rate keeping flag; a twenty-ninth determining module configured to determine a sampling rate corresponding to the sampling frame after the keeping quantity in the case that the sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is not set; and a tenth frame supplementing module configured to supplement the sampling frame after the keeping quantity based on the sampling rate corresponding to the sampling frame after the keeping quantity.

[0064] In one embodiment of the present disclosure, the apparatus further comprises: a thirtieth determining module configured to determine whether the current sampling frame corresponds to a sampling rate keeping flag in the case that the time domain recovery flag is set; a thirty-first determining module configured to determine whether a sampling rate keeping flag bit is set in the case that the current sampling frame corresponds to the sampling rate keeping flag; a seventh acquiring module configured to acquire a sampling rate corresponding to the current sampling frame and a keeping quantity in the case that the sampling rate keeping flag is set; a thirty-second determining module configured to determine whether a sampling frame after the keeping quantity corresponds to the sampling rate keeping flag; a thirty-third determining module configured to determine whether a sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is set in the case that the sampling frame after the keeping quantity corresponds to the sampling rate keeping flag; and an eleventh frame supplementing module configured to supplement the sampling frame after the keeping quantity based on the sampling rate in the case that the sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is set.

[0065] In one embodiment of the present disclosure, the keeping quantity is determined by a quantity of sampling frames between target frames, wherein the target frames include key frames.

[0066] In one embodiment of the present disclosure, the apparatus further comprises a thirty-fourth determining module configured to determine whether the current sampling frame is a key frame if the time domain recovery flag is set; a thirty-fifth determining module configured to determine whether a sampling rate maintaining flag corresponding to the current sampling frame is set if the current sampling frame is a key frame; a thirty-sixth determining module configured to determine whether a sampling frame after the current sampling frame is a key frame if the sampling rate maintaining flag corresponding to the current sampling frame is set; a thirty-seventh determining module configured to determine a sampling rate of the sampling frame after the current sampling frame if the sampling frame after the current sampling frame is a key frame; and a twelfth frame supplementing module configured to supplement frames based on the sampling rate.

[0067] In one embodiment of the present disclosure, the apparatus further comprises a thirty-eighth determining module configured to determine whether the current sampling frame is a key frame if the time domain recovery flag is set; a thirty-ninth determining module configured to determine whether a sampling rate maintaining flag corresponding to the current sampling frame is set if the current sampling frame is a key frame; a fortieth determining module configured to determine whether a sampling frame after the current sampling frame is a key frame if the sampling rate maintaining flag corresponding to the current sampling frame is set; and a thirteenth frame supplementing module configured to supplement frames based on a sampling rate corresponding to the current sampling frame if the sampling frame after the current sampling frame is not a key frame.

[0068] In one embodiment of the present disclosure, the apparatus further comprises a forty-first determining module configured to determine whether the current sampling frame is a key frame if the time domain recovery flag is set; a forty-second determining module configured to determine whether a sampling rate maintaining flag corresponding to the current sampling frame is set if the current sampling frame is a key frame; a forty-third determining module configured to determine whether a sampling frame after the current sampling frame is a key frame if the sampling rate maintaining flag corresponding to the current sampling frame is not set; a forty-fourth determining module configured to determine whether the sampling frame after the current sampling frame corresponds to a sampling rate if the sampling frame after the current sampling frame is not a key frame; and a forty-fifth determining module configured to supplement frames based on the sampling rate if the sampling frame after the current sampling frame corresponds to the sampling rate.

[0069] In one embodiment of the present disclosure, the apparatus further comprises: a forty-sixth determining module configured to determine whether the current sample frame is a key frame if the time domain recovery flag bit is set; a forty-seventh determining module configured to determine whether a sample rate maintaining flag bit corresponding to the current sample frame is set if the current sample frame is a key frame; a forty-eighth determining module configured to determine whether a sample frame after the current sample frame is a key frame if the sample rate maintaining flag bit corresponding to the current sample frame is not set; a forty-ninth determining module configured to determine whether the sample frame after the current sample frame corresponds to a sample rate if the sample frame after the current sample frame is not a key frame; and a fourteenth frame supplementing module configured to supplement a frame based on the sample rate corresponding to the current sample frame if the sample frame after the current sample frame does not correspond to the sample rate.

[0070] According to still another aspect of the present disclosure, there is provided a data processing apparatus applied to an encoding end, comprising: an encoding module configured to encode a video to obtain code stream data, so that a decoding end implements any one of the data processing methods in the decoding end based on the code stream data.

[0071] According to still another aspect of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the above-mentioned data processing method by executing the executable instructions.

[0072] According to still another aspect of the present disclosure, there is provided a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the above-mentioned data processing method.

[0073] The data processing method, apparatus, device and storage medium provided by the embodiments of the present disclosure determine whether the time domain recovery flag bit is set in the code stream data, determine the number of remaining frames if the time domain recovery flag bit is set, and supplement a frame based on the number of remaining frames to obtain a first frame supplement segment. The variable up-sampling rate based coding and decoding is realized.

[0074] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0075] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure. It is readily apparent to one skilled in the art that the following description in the drawings is merely exemplary and explanatory, and other drawings can be obtained by one skilled in the art without creative effort based on these drawings.

[0076] Figure 1A flowchart of a data processing method in an embodiment of the present disclosure is shown;

[0077] Figure 2 A flowchart of another data processing method in an embodiment of the present disclosure is shown;

[0078] Figure 3 A flowchart of still another data processing method in an embodiment of the present disclosure is shown;

[0079] Figure 4 A flowchart of still another data processing method in an embodiment of the present disclosure is shown;

[0080] Figure 5 A flowchart of still another data processing method in an embodiment of the present disclosure is shown;

[0081] Figure 6 A flowchart of still another data processing method in an embodiment of the present disclosure is shown;

[0082] Figure 7 A flowchart of still another data processing method in an embodiment of the present disclosure is shown;

[0083] Figure 8 A flowchart of still another data processing method in an embodiment of the present disclosure is shown;

[0084] Figure 9 A flowchart of a decoding instance in an embodiment of the present disclosure is shown;

[0085] Figure 10 A flowchart of another decoding instance in an embodiment of the present disclosure is shown;

[0086] Figure 11 A flowchart of still another decoding instance in an embodiment of the present disclosure is shown;

[0087] Figure 12 A flowchart of still another data processing method in an embodiment of the present disclosure is shown;

[0088] Figure 13 A flowchart of still another decoding instance in an embodiment of the present disclosure is shown;

[0089] Figure 14 A flowchart of still another decoding instance in an embodiment of the present disclosure is shown;

[0090] Figure 15 A flowchart of still another data processing method in an embodiment of the present disclosure is shown;

[0091] Figure 16 A flowchart of still another data processing method in an embodiment of the present disclosure is shown;

[0092] Figure 17 A flowchart of still another data processing method in an embodiment of the present disclosure is shown;

[0093] Figure 18 Fig. 1 shows a schematic diagram of a data processing apparatus according to an embodiment of the present disclosure;

[0094] Figure 19 Fig. 2 shows a schematic diagram of another data processing apparatus according to an embodiment of the present disclosure;

[0095] Figure 20 Fig. 3 shows a structure block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0096] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations can be implemented in any

[0097] In addition, the drawings are to be considered in all respects as illustrative and not restrictive; similar reference characters denote similar elements throughout the drawings. Like elements in the figures are indicated by like reference numbers throughout the various figures. Where certain elements of the present disclosure can be shown as discrete entities, it will be appreciated that other embodiments of the elements can be implemented as software entities that are integrated into a single software program or application. In addition, some of the blocks in the diagrams can represent functional entities that can be implemented in software, hardware, or a combination of both.

[0098] It should be understood that various steps in the methods of the present disclosure can be performed in a different order and / or concurrently with each other. Furthermore, various additional or alternative steps can be performed, and / or certain illustrated steps can be omitted. The scope of the present disclosure is not limited in this regard.

[0099] It should be noted that the terms "first", "second", and the like, used in the present disclosure are used only to distinguish different devices, modules, or units, and do not imply the order or sequence of the functions performed by the devices, modules, or units.

[0100] It should be noted that the terms "one", "multiple", etc. used in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise explicitly stated in the context, it should be understood as "one or more".

[0101] To solve the above problem, the embodiments of the present disclosure provide a data processing method, apparatus, device, and storage medium.

[0102] To describe the present disclosure in detail, Figure 1A flowchart of a data processing method in the embodiment of the present disclosure is shown. It should be noted that the data processing method in the embodiment of the present disclosure is applied to a decoding end.

[0103] As shown in Figure 1 , the data processing method can include the following steps.

[0104] S110, determining whether a time domain recovery flag bit is set in the code stream data. In some embodiments, the time domain recovery flag bit can be temproal_restoration_flag. In some embodiments, the time domain flag recovery set position can be temproal_restoration_flag=1, and the time domain flag recovery bit not set can be temproal_restoration_flag=0. In some embodiments, after obtaining the code stream data, the code stream data can be parsed first, and then it is determined whether the time domain recovery flag bit is set in the code stream data.

[0105] S120, determining the number of remaining frames in the case where the time domain recovery flag bit is set. In some embodiments, after obtaining the code stream data, the code stream data can be parsed to obtain the parsed code stream data. In some embodiments, the remaining frames can be the frames in the original video that are not sampled and located after the last sampled frame. For example, the number of remaining frames can be num_temporal_remain. For example, if the original video contains 1, 2, 3, 4, 5 frames in total, and 1, 3 are sampled frames, then 4, 5 are remaining frames, and the number of remaining frames is 2.

[0106] S130, performing frame interpolation based on the number of remaining frames to obtain a first frame-interpolated segment. In some embodiments, after determining the number of remaining frames, the last sampled frame can be repeated to perform frame interpolation. In some embodiments, after determining the number of remaining frames, at least one sampled frame can be input into an intelligent model to perform frame interpolation. In some embodiments, the first frame-interpolated segment can include a segment composed of the remaining frames.

[0107] The data processing method provided by the embodiment of the present disclosure determines whether a time domain recovery flag bit is set in the code stream data, determines the number of remaining frames in the case where the time domain recovery flag bit is set, and performs frame interpolation based on the number of remaining frames to obtain a first frame-interpolated segment. The decoding based on variable up-sampling ratio is realized.

[0108] In order to describe the present disclosure in detail, Figure 2 A flowchart of another data processing method in the embodiment of the present disclosure is shown. It should be noted that the data processing method in the embodiment of the present disclosure is applied to a decoding end.

[0109] As shown in Figure 2As shown, the data processing method can include:

[0110] S210, acquiring a sampling ratio corresponding to the current sampling frame. In some embodiments, before acquiring the sampling ratio corresponding to the previous sampling frame, it can be first determined whether the time domain recovery flag is set. In the case where the time domain recovery flag is set, S210 is executed. In some embodiments, the sampling ratio corresponding to the sampling segment can be determined based on the time domain sampling ratio index. In some embodiments, the sampling ratio corresponding to any sampling frame can be determined based on the time domain sampling ratio index. In some embodiments, the sampling ratio corresponding to a non-sampling frame can be determined based on the time domain sampling ratio index. The non-sampling frame can be a frame other than the sampling frame in the code stream data. For example, the time domain sampling ratio index can be temporal_ratio_idx. In some embodiments, the sampling ratio corresponding to temporal_ratio_idx can be between 0 and 3.

[0111] S220, in the case where the sampling ratio corresponding to the current sampling frame is not one, performing frame interpolation based on the sampling ratio, the current sampling frame and the adjacent sampling frame of the current sampling frame to obtain a second frame interpolation segment corresponding to the current sampling frame. In some embodiments, after determining the sampling ratio, the number of frames between the current sampling frame and the adjacent sampling frame of the current sampling frame is determined based on the sampling ratio. For example, TemporaryResamplingRatio-1 can be used to represent the number of frames between the current sampling frame and the adjacent sampling frame of the current sampling frame. For example, TemporaryResamplingRatio = 2^(temporal_resampling_ratio_idx)-1. In some embodiments, temporal_resampling_ratio_idx can be the sampling ratio between the current sampling frame and the adjacent sampling frame of the current sampling frame. In some embodiments, temporal_resampling_ratio_idx can be the time domain sampling ratio index between the current sampling frame and the adjacent sampling frame of the current sampling frame, and the sampling ratio between the current sampling frame and the adjacent sampling frame of the current sampling frame can be determined based on the time domain sampling ratio index between the current sampling frame and the adjacent sampling frame of the current sampling frame.

[0112] The data processing method provided by the embodiments of the present disclosure determines whether the time domain recovery flag is set in the code stream data, determines the number of remaining frames in the case where the time domain recovery flag is set, performs frame interpolation based on the number of remaining frames to obtain a first frame interpolation segment. The decoding based on the variable up-sampling ratio is realized.

[0113] Figure 3Another data processing method flowchart in the embodiments of the present disclosure is shown. It should be noted that the data processing method in the embodiments of the present disclosure is applied to the decoding end.

[0114] As shown in Figure 3 , the data processing method can include:

[0115] S310, acquiring a sampling ratio corresponding to a current sampling frame;

[0116] S320, in a case where the sampling ratio corresponding to the current sampling frame is one, determining a third frame filling segment based on the current sampling frame. In some embodiments, the sampling ratio corresponding to the current sampling frame can be determined first, and then it is determined whether the sampling ratio corresponding to the sampling ratio corresponding to the current sampling frame is zero. In some embodiments, a decoded video can be determined based on the first frame filling segment, the second frame filling segment, and the third frame filling segment. In some embodiments, the length of a frame of the decoded video can be TotalNumberOfFrames.

[0117]

[0118] wherein num_temporal_remain is the number of remaining frames, N is the number of sampling frames, i is a sampling frame, and TemporalResamplingratio is a sampling ratio between the i th sampling frame and the i+1 th sampling frame. In some embodiments, the first frame filling segment, the second frame filling segment, and the third frame filling segment can all be empty segments, i.e., the frame filling segment does not exist.

[0119] The data processing method provided by the embodiments of the present disclosure determines whether the time domain recovery flag bit is set in the code stream data, determines the number of remaining frames in a case where the time domain recovery flag bit is set, performs frame filling based on the number of remaining frames, and obtains a first frame filling segment. Variable upsampling ratio based decoding is achieved.

[0120] In order to make the present disclosure more detailed, Figure 4 Another data processing method flowchart in the embodiments of the present disclosure is shown. It should be noted that the data processing method in the embodiments of the present disclosure is applied to the decoding end.

[0121] As shown in Figure 4 , the data processing method can include:

[0122] S410, acquiring the number of sampling frames. In some embodiments, before S410, it can be determined whether the time domain recovery flag bit is set. In some embodiments, the number of sampling frames can be determined based on all output frames, wherein the all output frames can be outpulist.

[0123] S420, determining whether the current sampling frame is the last sampling frame. In some embodiments, the current sampling frame can be determined as the ith sampling frame, and then a determination is made based on whether i is less than the number of sampling frames, where i is counted from 0. Illustratively, determining whether the current sampling frame is the last sampling frame can be a judgment: i

[0124] S430, in the case where the current sampling frame is not the last sampling frame, determining whether the sampling rate corresponding to the current sampling frame is one. In some embodiments, it can be determined based on the statement TemporalResamplingRatio[i] == 1 whether the sampling rate corresponding to the current sampling frame is one.

[0125] S440, in the case where the sampling rate corresponding to the current sampling frame is not one, determining the sampling rate of the current sampling frame. In some embodiments, the sampling rate is TemporalResamplingRatio[i]. In some embodiments, TemporalResamplingRatio[i] returns Interpolated frames with a length of TemporalResamplingRatio[i] - 1 based on interpolation(inputpics[i], inputpics[i+1]).

[0126] In some embodiments, in the case where the sampling rate corresponding to the current sampling frame is not one, determining the sampling rate of the current sampling frame. In some embodiments, the sampling rate is TemporalResamplingRatio[i]. In some embodiments, TemporalResamplingRatio[i] returns Interpolated frames with a length of TemporalResamplingRatio[i] - 1 based on interpolation(inputpics[i-1], inputpics[i]).

[0127] S450, in the case where the sampling rate corresponding to the current sampling frame is one, determining the frame filling segment based on the current sampling frame.

[0128] S460, determining the frame filling segment based on the current sampling frame, the next frame continuous to the current sampling frame, and the sampling rate of the current sampling frame. In some embodiments, the above steps are looped until the current sampling frame is the last sampling frame. In some embodiments, i can be increased by 1 in each loop, illustratively, i = i + 1.

[0129] S470, determining the number of remaining frames.

[0130] S480, performing padding based on the number of remaining frames to obtain a first padded segment.

[0131] The data processing method provided by the embodiment of the present disclosure determines whether the time domain recovery flag bit is set in the code stream data, determines the number of remaining frames in the case where the time domain recovery flag bit is set, performs padding based on the number of remaining frames to obtain a first padded segment. The decoding based on variable up-sampling rate is realized.

[0132] Figure 5 A flowchart of another data processing method in the embodiment of the present disclosure is shown. It should be noted that the data processing method in the embodiment of the present disclosure is applied to the decoding end.

[0133] As shown in Figure 5 , the data processing method can include:

[0134] S510, determining a plurality of sampling segments, a number of original frames corresponding to each sampling segment and a sampling rate corresponding to each sampling segment based on the code stream data. In some embodiments, the sampling rate of each sampling segment is different. In some embodiments, before S510, it can be determined whether the time domain recovery flag bit is set, and in the case where the time domain recovery flag bit is set, S510 is executed.

[0135] S520, performing padding based on the number of original frames corresponding to each sampling segment and the sampling rate corresponding to each sampling segment to obtain a padded segment corresponding to each sampling segment. In some embodiments, after obtaining the padded segment corresponding to each sampling segment, a decoded video can be determined based on the padded segment corresponding to each sampling segment and the first padded segment.

[0136] The data processing method provided by the embodiment of the present disclosure determines whether the time domain recovery flag bit is set in the code stream data, determines the number of remaining frames in the case where the time domain recovery flag bit is set, performs padding based on the number of remaining frames to obtain a first padded segment. The decoding based on variable up-sampling rate is realized.

[0137] Figure 6 A flowchart of another data processing method in the embodiment of the present disclosure is shown. It should be noted that the data processing method in the embodiment of the present disclosure is applied to the decoding end.

[0138] As shown in Figure 6 , the data processing method can include:

[0139] S610, in the case that the time domain recovery flag bit is set, determining whether the time domain sampling rate corresponding to the current sampling frame is switched. In some embodiments, the current sampling frame can be an I frame. The I frame, also known as an internal picture or key frame, is a complete picture and can be decoded without reference to other frames. In video compression encoding, the I frame is usually the first frame of each group of pictures (GOP), and after moderate compression, it can be used as a reference point for random access. Due to its independence, the I frame can ensure the integrity and accuracy of the video data during transmission. In some embodiments, before S610, it can be determined whether the time domain recovery flag bit is set, and in the case that the time domain recovery flag bit is set, S610 is executed.

[0140] S620, in the case that the time domain sampling rate corresponding to the current sampling frame is switched, obtaining the sampling rate corresponding to the current sampling frame.

[0141] S630, performing frame interpolation based on the sampling rate. In some embodiments, the time domain sampling rate index can be determined first, and after determining the time domain sampling rate index, the sampling rate can be determined based on the time domain sampling rate index.

[0142] The data processing method provided by the embodiments of the present disclosure determines whether the time domain recovery flag bit is set in the code stream data, determines the number of remaining frames in the case that the time domain recovery flag bit is set, performs frame interpolation based on the number of remaining frames, and obtains a first frame interpolation segment. The decoding based on the variable upsampling rate is realized.

[0143] Figure 7 A flowchart of another data processing method in the embodiments of the present disclosure is shown. It should be noted that the data processing method in the embodiments of the present disclosure is applied to a decoding end.

[0144] As shown in Figure 7 , the data processing method can include:

[0145] S710, in the case that the time domain sampling rate corresponding to the current sampling frame is not switched, obtaining the sampling rate corresponding to the sampling frame before the current sampling frame;

[0146] S720, performing frame interpolation based on the sampling rate corresponding to the sampling frame before the current sampling frame. In some embodiments, the method of frame interpolation has been described in detail in the above embodiments, and after determining the sampling rate, the frame interpolation can be performed based on the above method, which will not be described here.

[0147] The data processing method provided by the embodiment of the present disclosure determines whether the time domain recovery flag bit is set in the code stream data, determines the number of remaining frames in the case where the time domain recovery flag bit is set, performs frame filling based on the number of remaining frames, and obtains a first frame filling segment. The decoding based on the variable upsampling ratio is realized.

[0148] Figure 8 A flowchart of another data processing method in the embodiment of the present disclosure is shown. It should be noted that the data processing method in the embodiment of the present disclosure is applied to a decoding end.

[0149] As shown in Figure 8 , the data processing method can include the following steps.

[0150] S810, in the case where the time domain recovery flag bit is set, it is determined whether the current sampling frame corresponds to a sampling ratio keeping flag. In some embodiments, the sampling ratio keeping flag can be a flag bit, where the sampling ratio keeping flag can be in a set or unset state. Exemplarily, in the case where the sampling ratio keeping flag is set, it can be represented as ratio_consisent_flag=1 or ratio_changed_flag=0. In the case where the sampling ratio keeping flag is not set, it can be represented as ratio_consisent_flag=0 or ratio_changed_flag=1. Exemplarily, ratio_consisent_flag=1 or ratio_changed_flag=0 can also be in the case where the sampling ratio switching flag is not set; ratio_consisent_flag=0 or ratio_changed_flag=1 can also be in the case where the sampling ratio switching flag is set. In some embodiments, in the case where the sampling ratio switching flag is set, i.e., temporal resampling ratio changed flag equal to 1, the subsequent sampling ratio uses the changed sampling ratio. In the case where the sampling ratio switching flag is not set, i.e., temporal resampling ratio changed flag equal to 0, the subsequent sampling ratio uses the sampling ratio corresponding to the previous sampling frame.

[0151] S820, in the case where the current sampling frame corresponds to the sampling ratio keeping flag, it is determined whether the sampling ratio keeping flag bit is set.

[0152] S830, in the case where the sampling ratio keeping flag bit is not set, it is determined whether the sampling frame after the current sampling frame corresponds to the sampling ratio keeping flag. In some embodiments, the sampling frame after the current sampling frame can be any frame after the current sampling frame.

[0153] S840, in the case that the sampling frame after the current sampling frame corresponds to the sampling rate keeping flag, determining whether the sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set;

[0154] S850, in the case that the sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set, performing frame interpolation based on the sampling rate corresponding to the last sampling frame of the sampling frame after the current sampling frame. In some embodiments, the sampling rate can be ratio_idx. In some embodiments, the time domain sampling rate index can be ratio_idx. In some embodiments, the last sampling frame of the sampling frame after the current sampling frame can be the sampling frame continuous with the sampling frame after the current sampling frame. In some embodiments, in the case that the sampling rate keeping flag bit corresponding to the current sampling frame is not set, it can be determined whether the sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set. In the case that the sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is not set, determining the sampling rate corresponding to the sampling frame after the current sampling frame; performing frame interpolation based on the sampling rate.

[0155] To make a detailed description of the above embodiments, Figure 9 A decoding instance diagram in an embodiment of the present disclosure is shown.

[0156] As Figure 9 shown, the sampling rate keeping flag bit of the first sampling frame is not set, the sampling rate keeping flag bits of the second to ninth and eleventh and twelfth sampling frames are set, and the sampling rate keeping flag bit of the tenth sampling frame is not set. At this time, frame interpolation is performed based on the sampling rate corresponding to the first sampling frame between the first to tenth sampling frames, and frame interpolation is performed based on the sampling rate corresponding to the tenth sampling frame between the tenth to twelfth sampling frames.

[0157] To make a detailed description of the above embodiments, Figure 10 Another decoding instance diagram in an embodiment of the present disclosure is shown. As Figure 10 shown, the sampling rate keeping flag bits of the first and ninth sampling frames are not set, and the sampling rate keeping flag bits of the second to eighth and tenth sampling frames are set. At this time, frame interpolation is performed based on the sampling rate corresponding to the first sampling frame between the first to ninth sampling frames, and frame interpolation is performed based on the sampling rate corresponding to the ninth sampling frame between the ninth to eleventh sampling frames.

[0158] To make a detailed description of the above embodiments, Figure 11 Another decoding instance diagram in an embodiment of the present disclosure is shown. As Figure 11As shown, the sampling rate maintaining flag of the 1st, 6th and 7th sampling frames is not set, and the sampling rate maintaining flag of the 2nd-5th, 8th and 9th sampling frames is set. At this time, the 1st-6th sampling frames are filled based on the sampling rate corresponding to the 1st sampling frame, the 6th-7th sampling frames are filled based on the sampling rate corresponding to the 6th sampling frame, and the 7th-9th sampling frames are filled based on the sampling rate corresponding to the 7th sampling frame. The 11th sampling frame is the last frame.

[0159] Figure 12 A flow chart of another data processing method in the embodiment of the present disclosure is shown. It should be noted that the data processing method in the embodiment of the present disclosure is applied to a decoding end. As shown in the figure, Figure 12 The data processing method can include:

[0160] S1210, in the case where the time domain recovery flag is set, determining whether the current sampling frame corresponds to a sampling rate maintaining flag;

[0161] S1220, in the case where the current sampling frame corresponds to the sampling rate maintaining flag, determining whether the sampling rate maintaining flag is set;

[0162] S1230, in the case where the sampling rate maintaining flag is set, determining the sampling rate corresponding to the current sampling frame;

[0163] S1240, filling the sampling frame after the current sampling frame based on the sampling rate. In some embodiments, in the case where the sampling rate maintaining flag corresponding to the current sampling frame is set, it is determined whether the sampling rate maintaining flag corresponding to the sampling frame after the current sampling frame is set; in the case where the sampling rate maintaining flag corresponding to the sampling frame after the current sampling frame is set, the sampling rate corresponding to the sampling frame after the current sampling frame is determined; and the sampling frame after the current sampling frame is filled based on the sampling rate. In some embodiments, in the case where the sampling rate maintaining flag corresponding to the current sampling frame is set, it is determined whether the sampling rate maintaining flag corresponding to the sampling frame after the current sampling frame is set; in the case where the sampling rate maintaining flag corresponding to the sampling frame after the current sampling frame is not set, the sampling rate corresponding to the sampling frame after the current sampling frame is determined; and the sampling frame after the current sampling frame is filled based on the sampling rate.

[0164] In order to describe the above embodiments in detail, Figure 13 A decoding instance diagram in the embodiment of the present disclosure is shown. As shown in the figure, Figure 13As shown, the sampling rate keeping flag of the 1st sampling frame is set, the sampling rate keeping flag of the 9th sampling frame is not set, and the sampling rate keeping flag of the 10th sampling frame is not set. The 1st sampling frame is used for frame interpolation between the 1st and 9th sampling frames, the 9th sampling frame is used for frame interpolation between the 9th and 10th sampling frames, and the 10th sampling frame is used for frame interpolation between the 10th and 12th sampling frames.

[0165] To make the above-mentioned embodiments more detailed, Figure 14 Another decoding instance diagram in the embodiments of the present disclosure is shown. As shown in Figure 14 As shown, the sampling rate keeping flags of the 1st to 11th sampling frames are not set. In this case, the sampling rate of the previous sampling frame is used for frame interpolation between any two sampling frames.

[0166] The data processing method provided by the embodiments of the present disclosure determines whether the time domain recovery flag is set in the code stream data. In the case where the time domain recovery flag is set, the number of remaining frames is determined, frame interpolation is performed based on the number of remaining frames, and a first frame interpolation segment is obtained. The decoding based on variable upsampling rate is realized.

[0167] Figure 15 Another data processing method flowchart in the embodiments of the present disclosure is shown. It should be noted that the data processing method in the embodiments of the present disclosure is applied to the decoding end. As shown in Figure 15 The data processing method can include the following steps.

[0168] S1510, in the case where the time domain recovery flag is set, determining whether the current sampling frame corresponds to the sampling rate keeping flag.

[0169] S1520, in the case where the current sampling frame corresponds to the sampling rate keeping flag, determining whether the sampling rate keeping flag is set.

[0170] S1530, in the case where the sampling rate keeping flag is set, obtaining the sampling rate corresponding to the current sampling frame and the number of kept frames.

[0171] S1540, frame interpolating the sampling frames containing the current frame and the number of kept frames at the sampling rate. In some embodiments, the number of kept frames is determined by the number of sampling frames between target frames. The target frame can be a key frame. In some embodiments, the target frame can also include a frame marked by an encoder. In some embodiments, as shown in Figure 13As shown, the 1st sampling frame and the 9th sampling frame are target frames. At this time, it can be determined that the retention number is 8, that is, the sampling magnification between the 1st sampling frame and the 9th sampling frame and the sampling magnification corresponding to the 1st sampling frame are supplemented. In some embodiments, after S1540, the method may further include: determining whether the sampling frames after the retention number correspond to the sampling magnification retention flag; if the sampling frames after the retention number correspond to the sampling magnification retention flag, determining whether the sampling magnification retention flag corresponding to the sampling frames after the retention number is set; if the sampling magnification retention flag corresponding to the sampling frames after the retention number is not set, determining the sampling magnification corresponding to the sampling frames after the retention number; and supplementing the sampling frames after the retention number based on the sampling magnification corresponding to the sampling frames after the retention number.

[0172] like Figure 13 As shown, the sampling rate holding flag corresponding to the 9th sampling frame is not set. In this case, after the 9th sampling frame, a frame is added based on the sampling rate corresponding to the 9th sampling frame. In some embodiments, after S1540, the method may further include: determining whether the sampling frames after the retained number correspond to the sampling rate holding flag; if the sampling frames after the retained number correspond to the sampling rate holding flag, determining whether the sampling rate holding flag corresponding to the sampling frames after the retained number is set; if the sampling rate holding flag corresponding to the sampling frames after the retained number is set, adding the sampling frames after the retained number based on the sampling rate corresponding to the current sampling frame.

[0173] The data processing method provided by the embodiments of the present disclosure determines whether a time domain recovery flag is set in the bitstream data. If the time domain recovery flag is set, the method determines the number of remaining frames and performs frame insertion based on the number of remaining frames to obtain a first inserted frame segment. This method implements decoding based on a variable upsampling ratio.

[0174] Figure 16 FIG1 shows another flow chart of a data processing method in an embodiment of the present disclosure. It should be noted that the data processing method in an embodiment of the present disclosure is applied to a decoding end. Figure 16 As shown, the data processing method may include:

[0175] S1610, in the case of the time domain recovery flag bit being set, determine whether the current sampling frame is a key frame. In some embodiments, the time domain sampling information such as the sampling rate retention flag bit whether it is set, the time domain sampling rate index, etc. is not limited to the high-level syntax structure such as SEI / AU / PPS. For example, a new ID of picture parameter set (PPS) can also be added, and the PPS under the new ID contains temporal_ratio_idx. The I frame and the frame with ratio_changed_flag being 1 use the PPS structure of the new ID, and the rest of the frames use the existing PPS structure. The frames using the existing PPS structure are associated with the temporal_ratio_idx in the PPS corresponding to the previous I frame or the frame with ratio_changed_flag being 1. Therefore, the judgment of ratio_consistent_flag or ratio_changed_flag is hidden in the judgment of the ID of PPS. Similar operations can also be added to the NAL header of a new ID or the SEI of a new ID.

[0176] Wherein, the NAL unit is the basic syntax structure of NAL, which contains a byte of header information and a series of byte streams from VCL called raw byte sequence payload (target). The NAL header is one byte, of which the 7th bit is a forbidden bit, which is 0 by default in the encoding, and can be set to 1 when the network identifies that there is a bit error in the unit, so that the recipient discards the unit. The 6th to 5th bits are priority, with a value range of 0 to 3, and the higher the value, the more important the current NAL, which needs to be protected first. The 4th to 0th bits represent the type of NAL, such as slice in non-IDR image and slice not partitioned, slice in IDR image, supplemental enhancement information SEI, sequence parameter set, picture parameter set PPS, etc.

[0177] Wherein, SEI (Supplemental Enhancement Information) is an important component in the H.264 / AVC video coding standard. SEI is a kind of additional enhancement information, which can contain a series of parameters and data related to video coding and transmission.

[0178] In some embodiments, whether the current sampling frame is a key frame can be determined based on the codec.

[0179] S1620, in the case of the current sampling frame being a key frame, determine whether the sampling rate retention flag bit corresponding to the current sampling frame is set.

[0180] S1630, in the case of the sampling rate retention flag bit corresponding to the current sampling frame being set, determine whether the sampling frame after the current sampling frame is a key frame.

[0181] S1640, in the case that the sampling frame after the current sampling frame is a key frame, determining a sampling rate corresponding to the sampling frame after the current sampling frame;

[0182] S1650, performing frame interpolation based on the sampling rate. In some embodiments, after S1650, the method can further comprise: in the case that the sampling frame after the current sampling frame is not a key frame, performing frame interpolation based on the sampling rate corresponding to the current sampling frame. In some embodiments, in the case that the sampling frame after the current sampling frame is not a key frame, it can be further judged whether the sampling frame after the current sampling frame corresponds to a sampling rate maintaining flag. If the sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag and the sampling rate maintaining flag is set, frame interpolation can be performed based on the sampling rate of the previous sampling frame of the sampling frame after the current sampling frame. In some embodiments, in the case that the sampling frame after the current sampling frame is not a key frame, it can be further judged whether the sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag. If the sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag and the sampling rate maintaining flag is not set, it is determined whether the sampling frame after the current sampling frame corresponds to the sampling rate; in the case that the sampling frame after the current sampling frame corresponds to the sampling rate, performing frame interpolation based on the sampling rate.

[0183] In some embodiments, as Figure 13As shown, the 1st and 9th sampling frames are key frames, and the 2nd-8th sampling frames are non-key frames. In this case, the 2nd-9th sampling frames are interpolated based on the sampling rate corresponding to the 1st sampling frame. After the 9th sampling frame, the frames are interpolated based on the sampling rate corresponding to the 9th sampling frame. The 10th sampling frame is a non-key frame, and the rate maintaining flag corresponding to the 10th sampling frame is not set. Therefore, the sampling rate corresponding to the 10th sampling frame is obtained, and the frames between the 10th and 11th sampling frames are interpolated based on the sampling rate corresponding to the 10th sampling frame. The rate maintaining flag corresponding to the 11th sampling frame is set. Therefore, the frames between the 11th and 12th sampling frames are also interpolated based on the sampling rate corresponding to the 10th sampling frame. In some embodiments, after S1650, the method can further include: in the case that the sampling frame after the current sampling frame is not a key frame, determining whether the sampling frame after the current sampling frame corresponds to a sampling rate; in the case that the sampling frame after the current sampling frame corresponds to a sampling rate, determining the sampling rate corresponding to the sampling frame after the current sampling frame; and interpolating based on the sampling rate. In some embodiments, before determining whether the sampling frame after the current sampling frame corresponds to a sampling rate, it is further required to determine whether the sampling frame after the current sampling frame corresponds to a sampling rate maintaining flag. If the sampling frame after the current sampling frame corresponds to a sampling rate maintaining flag and the sampling rate maintaining flag is set, the interpolation is performed based on the sampling rate of the previous sampling frame of the sampling frame after the current sampling frame. In some embodiments, before determining whether the sampling frame after the current sampling frame corresponds to a sampling rate, it is further required to determine whether the sampling frame after the current sampling frame corresponds to a sampling rate maintaining flag. If the sampling frame after the current sampling frame corresponds to a sampling rate maintaining flag and the sampling rate maintaining flag is not set, it is determined whether the sampling frame after the current sampling frame corresponds to a sampling rate; and in the case that the sampling frame after the current sampling frame corresponds to a sampling rate, the interpolation is performed based on the sampling rate.

[0184] Figure 17 A flowchart of another data processing method in the embodiments of the present disclosure is shown. It should be noted that the data processing method in the embodiments of the present disclosure is applied to a decoding end.

[0185] As shown in the flowchart of the data processing method, the data processing method can include the following steps. Figure 17

[0186] S1710, in the case that the time domain recovery flag is set, determining whether the current sampling frame is a key frame;

[0187] S1720, in the case that the current sampling frame is a key frame, determining whether the sampling rate maintaining flag corresponding to the current sampling frame is set;

[0188] S1730, in the case that the sampling rate maintaining flag corresponding to the current sampling frame is not set, determining whether the sampling frame after the current sampling frame is a key frame; ​

[0189] S1740, in the case that the sampling frame after the current sampling frame is not a key frame, determining whether the sampling frame after the current sampling frame corresponds to a sampling rate. In some embodiments, in the case that the sampling frame after the current sampling frame is not a key frame, it can also be determined whether the sampling frame after the current sampling frame corresponds to a sampling rate maintaining flag. If it corresponds to a sampling rate maintaining flag and the sampling rate maintaining flag is not set, it is determined whether the sampling frame after the current sampling frame corresponds to a sampling rate, and in the case that the sampling frame after the current sampling frame corresponds to a sampling rate, the frame is filled based on the sampling rate.

[0190] S1750, in the case that the sampling frame after the current sampling frame corresponds to a sampling rate, the frame is filled based on the sampling rate. In some embodiments, after S1740, the method further comprises: in the case that the sampling frame after the current sampling frame does not correspond to a sampling rate, the frame is filled based on the sampling rate corresponding to the current sampling frame. In some embodiments, before determining whether the sampling frame after the current sampling frame corresponds to a time domain sampling rate, it is also necessary to determine whether the sampling frame after the current sampling frame corresponds to a sampling rate maintaining flag. If it corresponds to a sampling rate maintaining flag and the sampling rate maintaining flag is set, the frame can be filled based on the sampling rate of the previous sampling frame of the sampling frame after the current sampling frame.

[0191] The data processing method provided by the embodiments of the present disclosure determines whether a time domain recovery flag in code stream data is set, in the case that the time domain recovery flag is set, determines the number of remaining frames, and fills the frame based on the number of remaining frames to obtain a first filled frame segment. The decoding based on variable upsampling rate is realized.

[0192] In some embodiments, the method further comprises: determining a plurality of sampling segments, the number of original frames corresponding to each sampling segment, and the sampling rate corresponding to each sampling segment based on the code stream data;

[0193] Filling the frame based on the number of original frames corresponding to each sampling segment and the sampling rate corresponding to each sampling segment to obtain a filled frame segment corresponding to each sampling segment.

[0194] In some embodiments, the method further comprises: the code stream data comprises a sequence parameter set and end information; the determining whether a time domain recovery flag in code stream data is set comprises: determining whether a time domain recovery flag is set according to the sequence parameter set; the determining the number of remaining frames in the case that the time domain recovery flag is set comprises: in the case that the time domain recovery flag is set, determining whether there is target data in the end information; in the case that there is target data, obtaining remaining frame information according to the target data, and determining the number of remaining frames according to the remaining frame information.

[0195] In some embodiments, the method further comprises: the code stream data comprises a sequence parameter set and end information; and the determining the number of remaining frames when the time domain recovery flag bit is set comprises: determining whether the end information has target data; when the target data exists, determining whether the time domain recovery flag bit is set; and when the time domain recovery flag bit is set, obtaining the remaining frame information according to the target data, and determining the number of remaining frames according to the remaining frame information.

[0196] In some embodiments, the sequence parameter set comprises a SPS, the target data comprises rbsp data, and the end information comprises an EOS or an EOB.

[0197] In some embodiments, when the target data does not exist in the end information, the number of remaining frames is 0, and the last frame sample frame is the last frame of the original sequence.

[0198] In some embodiments, some frames belong to the sampling rate change flag bit which is always set, such as I frames.

[0199] In some embodiments, the image information comprises the sampling rate maintaining flag bit and the time domain sampling change rate information; and the time domain sampling change rate information comprises a time domain sampling change rate index.

[0200] In some embodiments, the sequence level parameter comprises a time domain recovery flag bit and a sequence level time domain sampling rate; when the time domain recovery flag bit is 1, the sequence level time domain sampling rate is obtained; and if the PH does not update the time domain sampling information, the sequence level time domain sampling rate is used.

[0201] The sequence level parameter comprises a time domain recovery flag bit and a sequence level time domain sampling rate; when the time domain recovery flag bit is 1, the sequence level time domain sampling rate is obtained; and if the PH does not update the time domain sampling information, the sequence level time domain sampling rate is used.

[0202] The flag bit being 1 can be a flag bit set

[0203] sps_temporal_restoration_ratio_index specifies the sequence level temporal resampling scaling factor, each frame initialize the temporal resampling ratio idx with sps_temporal_restoration_ratio_index. sps_temporal_restoration_flag equal to 1 specifies that temporal restoration is enabled. sps_temporal_restoration_flag equal to 0 specifies that temporal restoration is disabled.

[0204] sps_temporal_restoration_ratio_index specifies the sequence level temporal resampling scaling factor, the temporary variable of the historical temporal resampling ratio initialize the temporal resampling ratio idx with sps_temporal_restoration_ratio_index.

[0205] ph_temporal_resampling_ratio_changed_flag equal to 1 specifies that the temporal restoration is with the changed temporal resampling ratio. ph_temporal_resampling_ratio_changed_flag equal to 0 specifies that the temporal restoration is with the historical temporal resampling ratio. This historical temporal resampling ratio is a temporary variable in decoder, which will be updated with the changed temporal resampling ratio.

[0206] ph_temporal_resampling_ratio_idx specifies the temporal resampling scaling factor when sps_temporal restoration is enabled and ph_temporal_resampling_ratio_changed_flag is active.

[0207] In some embodiments, the picture information comprises a picture header (PH).

[0208] In some embodiments, the target data comprises raw byte sequence payload (RBSP) data.

[0209] Based on the same inventive concept, the present disclosure also provides a data processing method in the embodiments, as follows. Since the principle of the method embodiments to solve the problem is similar to the above-mentioned method embodiments, the implementation of the method embodiments can be referred to the implementation of the above-mentioned method embodiments, and the repeated parts will not be described here.

[0210] The method comprises: performing frame interpolation according to a sampling rate corresponding to a sampling frame.

[0211] In some embodiments, the performing frame interpolation according to the sampling rate corresponding to the sampling frame comprises:

[0212] obtaining a sampling rate corresponding to the current sampling frame;

[0213] In a case where the sampling rate corresponding to the current sampling frame is not one, performing frame interpolation based on the sampling rate, the current sampling frame and a neighboring sampling frame of the current sampling frame to obtain a second frame-interpolated segment corresponding to the current sampling frame.

[0214] In some embodiments, the performing frame interpolation according to the sampling rate corresponding to the sampling frame comprises:

[0215] obtaining a sampling rate corresponding to the current sampling frame;

[0216] In a case where the sampling rate corresponding to the current sampling frame is one, determining a third frame-interpolated segment based on the current sampling frame.

[0217] In some embodiments, the method further comprises:

[0218] determining a number of remaining frames;

[0219] performing frame interpolation based on the number of remaining frames to obtain a first frame-interpolated segment.

[0220] In some embodiments, the sequence-level parameter comprises a time domain recovery flag and a time domain preservation flag, and in a case where both the two flags are set, a sequence-level sampling rate is obtained, and a subsequent sequence other than the remaining frames is frame-interpolated using the sampling rate. In some embodiments, the above-mentioned setting of the two flags can comprise that both the two flags are 1.

[0221] Based on the same inventive concept, the embodiments of the present disclosure further provide a data processing method, as follows. Since the principle of the method embodiments to solve the problem is similar to the above-mentioned method embodiments, the implementation of the method embodiments can be referred to the implementation of the above-mentioned method embodiments, and the repeated parts will not be repeated.

[0222] It should be noted that the data processing method in the embodiments of the present disclosure is applied to the encoding end to encode the video to obtain the code stream data, so that the decoding end implements any one of the data processing methods in the above embodiments based on the code stream data.

[0223] Based on the same inventive concept, the embodiments of the present disclosure further provide a data processing apparatus, as follows. Since the principle of the apparatus embodiments to solve the problem is similar to the above-mentioned method embodiments, the implementation of the apparatus embodiments can be referred to the implementation of the above-mentioned method embodiments, and the repeated parts will not be repeated.

[0224] Figure 18 A schematic diagram of another data processing apparatus in the embodiments of the present disclosure is shown. It should be noted that the data processing apparatus in the embodiments of the present disclosure is applied to the decoding end.

[0225] As Figure 18 shown, the data processing apparatus can include: a first determination module 1810 configured to determine whether a time domain recovery flag is set in the code stream data; a second determination module 1820 configured to determine a number of remaining frames in a case where the time domain recovery flag is set; and a first frame filling module 1830 configured to fill frames based on the number of remaining frames to obtain a first frame filling segment.

[0226] In some embodiments, the apparatus further includes: a fiftieth determination module configured to determine, based on the code stream data, a plurality of sampling segments, a number of original frames corresponding to each sampling segment, and a sampling rate corresponding to each sampling segment; and a sixteenth frame filling module configured to fill frames based on the number of original frames corresponding to each sampling segment and the sampling rate corresponding to each sampling segment to obtain a frame filling segment corresponding to each sampling segment.

[0227] In some embodiments, the apparatus further includes: a first acquisition module configured to acquire a sampling rate corresponding to a current sampling frame in a case where the current sampling frame is not a last sampling frame; and a second frame filling module configured to fill frames based on the sampling rate, the current sampling frame, and a neighboring sampling frame of the current sampling frame to obtain a second frame filling segment corresponding to the current sampling frame in a case where the sampling rate corresponding to the current sampling frame is not one.

[0228] In some embodiments, the apparatus further includes: a second acquisition module configured to acquire a sampling rate corresponding to a current sampling frame in a case where the current sampling frame is not a last sampling frame; and a third determination module configured to determine a third frame filling segment based on the current sampling frame in a case where the sampling rate corresponding to the current sampling frame is one. In some embodiments, the apparatus further includes: a fourth determination module configured to determine a decoded video based on the first frame filling segment, the second frame filling segment, and the third frame filling segment.

[0229] In some embodiments, the apparatus further includes: a fifth determination module configured to determine whether a time domain sampling rate corresponding to a current sampling frame is switched in a case where the time domain recovery flag is set; a third acquisition module configured to acquire a sampling rate corresponding to the current sampling frame in a case where the time domain sampling rate corresponding to the current sampling frame is switched; and a third frame filling module configured to fill frames based on the sampling rate.

[0230] In some embodiments, the apparatus further includes: a fourth acquisition module configured to acquire a sampling rate corresponding to a sampling frame before the current sampling frame in a case where the time domain sampling rate corresponding to the current sampling frame is not switched; and a fourth frame filling module configured to fill frames based on the sampling rate.

[0231] In some embodiments, the apparatus further comprises: a sixth determining module, configured to determine whether the current sampling frame corresponds to a sampling rate keeping flag in the case that the time domain recovery flag is set; a seventh determining module, configured to determine whether a sampling rate keeping flag bit is set in the case that the current sampling frame corresponds to the sampling rate keeping flag; an eighth determining module, configured to determine whether a sampling frame after the current sampling frame corresponds to the sampling rate keeping flag in the case that the sampling rate keeping flag bit is not set; a ninth determining module, configured to determine whether a sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set in the case that the sampling frame after the current sampling frame corresponds to the sampling rate keeping flag; and a fifth frame supplementing module, configured to supplement a frame based on a sampling rate corresponding to a previous sampling frame of the sampling frame after the current sampling frame in the case that the sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set.

[0232] In some embodiments, the apparatus further comprises: a tenth determining module, configured to determine whether the current sampling frame corresponds to a sampling rate keeping flag in the case that the time domain recovery flag is set; an eleventh determining module, configured to determine whether a sampling rate keeping flag bit is set in the case that the current sampling frame corresponds to the sampling rate keeping flag; a twelfth determining module, configured to determine whether a sampling frame after the current sampling frame corresponds to the sampling rate keeping flag in the case that the sampling rate keeping flag bit is not set; a thirteenth determining module, configured to determine whether a sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set in the case that the sampling frame after the current sampling frame corresponds to the sampling rate keeping flag; a fourteenth determining module, configured to determine a sampling rate corresponding to the sampling frame after the current sampling frame in the case that the sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is not set; and a sixth frame supplementing module, configured to supplement a frame based on the sampling rate.

[0233] In some embodiments, the apparatus further comprises: a fifteenth determining module, configured to determine whether the current sampling frame corresponds to a sampling rate keeping flag in the case that the time domain recovery flag is set; a sixteenth determining module, configured to determine whether a sampling rate keeping flag bit is set in the case that the current sampling frame corresponds to the sampling rate keeping flag; a seventeenth determining module, configured to determine a sampling rate corresponding to the current sampling frame in the case that the sampling rate keeping flag bit is set; and a seventh frame supplementing module, configured to supplement a frame after the current sampling frame based on the sampling rate.

[0234] In some embodiments, the apparatus further comprises: an eighteenth determining module, configured to determine whether the current sampling frame corresponds to a sampling rate keeping flag in the case that the time domain recovery flag is set; a nineteenth determining module, configured to determine whether a sampling rate keeping flag bit is set in the case that the current sampling frame corresponds to the sampling rate keeping flag; a twentieth determining module, configured to determine whether a sampling frame after the current sampling frame corresponds to the sampling rate keeping flag in the case that the sampling rate keeping flag bit is set; a twenty-first determining module, configured to determine whether a sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set in the case that the sampling frame after the current sampling frame corresponds to the sampling rate keeping flag; a twenty-second determining module, configured to determine a sampling rate corresponding to the sampling frame after the current sampling frame in the case that the sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set; and an eighth frame supplementing module, configured to supplement the sampling frame after the current sampling frame based on the sampling rate.

[0235] In some embodiments, the apparatus further comprises: a twenty-third determining module, configured to determine whether the current sampling frame corresponds to a sampling rate keeping flag in the case that the time domain recovery flag is set; a twenty-fourth determining module, configured to determine whether a sampling rate keeping flag bit is set in the case that the current sampling frame corresponds to the sampling rate keeping flag; a fifth acquiring module, configured to acquire a sampling rate corresponding to the current sampling frame and a keeping number in the case that the sampling rate keeping flag bit is set; and a ninth frame supplementing module, configured to supplement the sampling frame containing the current frame and the keeping number by the sampling rate.

[0236] In some embodiments, the apparatus further comprises: a twenty-fifth determining module, configured to determine whether the current sampling frame corresponds to a sampling rate keeping flag in the case that the time domain recovery flag is set; a twenty-sixth determining module, configured to determine whether a sampling rate keeping flag bit is set in the case that the current sampling frame corresponds to the sampling rate keeping flag; a sixth acquiring module, configured to acquire a sampling rate corresponding to the current sampling frame and a keeping number in the case that the sampling rate keeping flag bit is set; a twenty-seventh determining module, configured to determine whether a sampling frame after the keeping number corresponds to the sampling rate keeping flag; a twenty-eighth determining module, configured to determine whether a sampling rate keeping flag bit corresponding to the sampling frame after the keeping number is set in the case that the sampling frame after the keeping number corresponds to the sampling rate keeping flag; a twenty-ninth determining module, configured to determine a sampling rate corresponding to the sampling frame after the keeping number in the case that the sampling rate keeping flag bit corresponding to the sampling frame after the keeping number is not set; and a tenth frame supplementing module, configured to supplement the sampling frame after the keeping number based on the sampling rate corresponding to the sampling frame after the keeping number.

[0237] In some embodiments, the apparatus further includes: a 30th determining module configured to determine whether the current sampling frame corresponds to a sampling rate keeping flag in the case that the time domain recovery flag is set; a 31st determining module configured to determine whether a sampling rate keeping flag bit of the current sampling frame is set in the case that the current sampling frame corresponds to the sampling rate keeping flag; a 7th obtaining module configured to obtain a sampling rate corresponding to the current sampling frame and a keeping number in the case that the sampling rate keeping flag is set; a 32nd determining module configured to determine whether a sampling frame after the keeping number corresponds to the sampling rate keeping flag; a 33rd determining module configured to determine whether a sampling rate keeping flag bit of the sampling frame after the keeping number is set in the case that the sampling frame after the keeping number corresponds to the sampling rate keeping flag; and an 11th frame supplementing module configured to supplement frames based on the sampling rate for the sampling frame after the keeping number in the case that the sampling rate keeping flag bit of the sampling frame after the keeping number is set.

[0238] In some embodiments, the keeping number is determined by a number of sampling frames between target frames, wherein the target frames include key frames.

[0239] In some embodiments, the apparatus further includes: a 34th determining module configured to determine whether the current sampling frame is a key frame in the case that the time domain recovery flag is set; a 35th determining module configured to determine whether a sampling rate keeping flag bit of the current sampling frame is set in the case that the current sampling frame is a key frame; a 36th determining module configured to determine whether a sampling frame after the current sampling frame is a key frame in the case that the sampling rate keeping flag bit of the current sampling frame is set; a 37th determining module configured to determine a sampling rate of the sampling frame after the current sampling frame in the case that the sampling frame after the current sampling frame is a key frame; and a 12th frame supplementing module configured to supplement frames based on the sampling rate.

[0240] In some embodiments, the apparatus further includes: a 38th determining module configured to determine whether the current sampling frame is a key frame in the case that the time domain recovery flag is set; a 39th determining module configured to determine whether a sampling rate keeping flag bit of the current sampling frame is set in the case that the current sampling frame is a key frame; a 40th determining module configured to determine whether a sampling frame after the current sampling frame is a key frame in the case that the sampling rate keeping flag bit of the current sampling frame is set; and a 13th frame supplementing module configured to supplement frames based on a sampling rate corresponding to the current sampling frame in the case that the sampling frame after the current sampling frame is not a key frame.

[0241] In some embodiments, the apparatus further includes a forty-first determining module configured to determine whether the current sample frame is a key frame if the time domain recovery flag bit is set; a forty-second determining module configured to determine whether a sample rate maintaining flag bit corresponding to the current sample frame is set if the current sample frame is a key frame; a forty-third determining module configured to determine whether a sample frame after the current sample frame is a key frame if the sample rate maintaining flag bit corresponding to the current sample frame is not set; a forty-fourth determining module configured to determine whether the sample frame after the current sample frame corresponds to a sample rate if the sample frame after the current sample frame is not a key frame; and a forty-fifth determining module configured to perform frame interpolation based on the sample rate if the sample frame after the current sample frame corresponds to the sample rate.

[0242] In some embodiments, the apparatus further includes a forty-sixth determining module configured to determine whether the current sample frame is a key frame if the time domain recovery flag bit is set; a forty-seventh determining module configured to determine whether a sample rate maintaining flag bit corresponding to the current sample frame is set if the current sample frame is a key frame; a forty-eighth determining module configured to determine whether a sample frame after the current sample frame is a key frame if the sample rate maintaining flag bit corresponding to the current sample frame is not set; a forty-ninth determining module configured to determine whether the sample frame after the current sample frame corresponds to a sample rate if the sample frame after the current sample frame is not a key frame; and a fourteenth frame interpolation module configured to perform frame interpolation based on the sample rate corresponding to the current sample frame if the sample frame after the current sample frame does not correspond to the sample rate.

[0243] The data processing apparatus provided by the embodiments of the present disclosure determines whether the time domain recovery flag bit is set in the code stream data, determines the number of remaining frames if the time domain recovery flag bit is set, and performs frame interpolation based on the number of remaining frames to obtain a first frame-interpolated segment. The variable up-sampling rate-based coding and decoding is achieved.

[0244] Based on the same inventive concept, the present disclosure further provides a data processing apparatus in the embodiments as follows. Since the principle of the apparatus embodiments to solve the problem is similar to the above-mentioned method embodiments, the implementation of the apparatus embodiments can be referred to the implementation of the above-mentioned method embodiments, and the repeated parts will not be described herein.

[0245] Figure 19 Another schematic diagram of a data processing apparatus in the embodiments of the present disclosure is shown. It should be noted that the data processing apparatus in the embodiments of the present disclosure is applied to the encoding end. As shown in FIG. 19, the data processing apparatus 1900 can include an encoding module 1910 configured to encode a video to obtain code stream data, so that the decoding end implements any one of the data processing methods in the decoding end based on the code stream data. Figure 19 ​

[0246] The data processing apparatus provided by the embodiments of the present disclosure determines whether the time domain recovery flag bit is set in the code stream data, determines the number of remaining frames in the case where the time domain recovery flag bit is set, performs frame padding based on the number of remaining frames, and obtains a first frame padding segment. The variable up-sampling ratio based coding and decoding is realized.

[0247] The data processing apparatus provided by the embodiments of the present disclosure can be used to perform the positioning method provided by each of the above method embodiments, and the implementation principles and technical effects are similar. For the sake of simplicity, they will not be described here again.

[0248] Those skilled in the art can understand that each aspect of the present disclosure can be implemented as a system, a method or a program product. Therefore, each aspect of the present disclosure can be specifically implemented as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0249] The electronic device 2000 according to this embodiment of the present disclosure will be described below with reference to Figure 20 Figure 20 The displayed electronic device 2000 is only an example and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0250] As shown in Figure 20 The electronic device 2000 is in the form of a general computing device. The components of the electronic device 2000 can include but are not limited to the above-mentioned at least one processing unit 2010, the above-mentioned at least one storage unit 2020, and a bus 2030 connecting different system components including the storage unit 2020 and the processing unit 2010.

[0251] The storage unit stores program code, which can be executed by the processing unit 2010, so that the processing unit 2010 performs the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present specification. For example, the processing unit 2010 can perform the following steps of the above method embodiments: determining whether the time domain recovery flag bit is set in the code stream data; in the case where the time domain recovery flag bit is set, determining the number of remaining frames; and performing frame padding based on the number of remaining frames to obtain a first frame padding segment.

[0252] The storage unit 2020 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 20201 and / or a cache memory unit 20202, and can further include a read-only memory (ROM) 20203.

[0253] ​The storage unit 2020 also includes a number of program modules 20205 that are stored in the memory 20204, including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or a combination of which can include implementation of a network environment.

[0254] The bus 2030 can represent one or more of several types of bus structures, including a storage unit bus or bus controller, a peripheral bus, a graphics acceleration port, a processing unit bus, or a local bus using any of a variety of bus architectures.

[0255] The electronic device 2000 can also be coupled via the bus 2030 to one or more external devices 2040, such as one or more keyboards, one or more pointing devices, one or more Bluetooth devices, etc. The electronic device 2000 can also be coupled to one or more devices that enable a user to interact with the electronic device 2000 in a way not directly related to accessing the storage unit, and / or one or more devices that enable the electronic device 2000 to communicate with one or more other computing devices. Such communication can be facilitated by an Input / Output (I / O) interface 2050. The electronic device 2000 can also be coupled to one or more networks (such as a Local Area Network (LAN), a Wide Area Network (WAN), and / or the Internet) via a network adapter 2060. As depicted, the network adapter 2060 is coupled to the other components of the electronic device 2000 via the bus 2030. It should be appreciated that although the network adapter 2060 is depicted as a single component, the network adapter 2060 can comprise two or more components that operate together to facilitate the communication of the electronic device 2000 with one or more other computing devices. It should also be appreciated that not all components of the electronic device 2000 are necessary to practice the subject disclosure, and some components have been simplified or omitted for the sake of clarity.

[0256] Those skilled in the art will readily recognize that the example embodiments described herein can be implemented using software and / or hardware in combination with software. Thus, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or a network, and includes a number of instructions to make a computing device (such as a personal computer, a server, a terminal device, or a network device, etc.) execute the methods according to the embodiments of the present disclosure.

[0257] In the example embodiments of the present disclosure, a computer readable storage medium is also provided, which can be a readable signal medium or a readable storage medium. A program product is stored thereon, which can implement the method of the present disclosure. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing terminal equipment to perform the steps described in the above "example method" section according to various example embodiments of the present disclosure when the program product is run on the terminal equipment.

[0258] More specific examples of the computer readable storage medium in the present disclosure can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of the foregoing.

[0259] In the present disclosure, the computer readable storage medium can include a data signal carried in the baseband or as a part of a carrier wave propagating through the program codes, in which the readable program codes are borne. Such a propagating data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal, or any appropriate combination of the foregoing. The readable signal medium can also be any readable medium other than the readable storage medium, which can send, propagate or transmit programs for use by or in connection with an instruction execution system, apparatus or device.

[0260] Optionally, the program codes contained on the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any appropriate combination of the foregoing.

[0261] In specific implementation, the program codes for performing the operations of the present disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" language or similar programming languages. The program codes can be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, through the Internet by connecting to an Internet service provider).

[0262] It should be noted that, although several modules or units of the devices for action execution are mentioned in the above detailed description, the division into such modules or units is not mandatory. Indeed, according to an embodiment of the disclosure, the features and functionalities of two or more of the above-described modules or units can be embodied in one module or unit. Conversely, the features and functionalities of one of the above-described modules or units can be further divided into several modules or units.

[0263] Furthermore, although the various steps of the methods in the disclosure are described in a particular order in the drawings, this is not required or implied as to the order of the steps or that all of the steps shown must be performed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, one step can be broken into multiple steps, etc.

[0264] From the above description of the embodiments, those skilled in the art will readily perceive that the example embodiments described herein can be implemented by software and / or by software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or network, and includes several instructions to make a computing device (which can be a personal computer, server, mobile terminal, or network device, etc.) execute the method according to the embodiments of the disclosure.

[0265] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. The disclosure is intended to cover any variations, uses, or adaptations of the disclosure following the general principles thereof and including such departures from the present disclosure that come within known use or custom in the art to which the disclosure pertains. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the disclosure are indicated by the appended claims.

Claims

1. A data processing method, characterized by, Applied to a decoding end, comprising: Determining whether a time domain recovery flag bit is set in code stream data; In a case where the time domain recovery flag bit is set, determining a number of remaining frames; Based on the number of remaining frames, performing frame filling to obtain a first frame filled segment.

2. The data processing method according to claim 1, characterized in that, The method further comprises: Based on the code stream data, determining a plurality of sampling segments, a number of original frames corresponding to each sampling segment, and a sampling rate corresponding to each sampling segment; Based on the number of original frames corresponding to each sampling segment and the sampling rate corresponding to each sampling segment, performing frame filling to obtain a frame filled segment corresponding to each sampling segment.

3. The data processing method of claim 1, wherein, The code stream data comprises a sequence parameter set and end information; The determining whether the time domain recovery flag bit is set in the code stream data comprises: determining whether the time domain recovery flag bit is set according to the sequence parameter set; The determining the number of remaining frames in the case where the time domain recovery flag bit is set comprises: In the case where the time domain recovery flag bit is set, determining whether target data exists in the end information; In the case where the target data exists, acquiring remaining frame information according to the target data, and determining the number of remaining frames according to the remaining frame information.

4. The data processing method of claim 1, wherein, The code stream data comprises a sequence parameter set and end information; The determining the number of remaining frames in the case where the time domain recovery flag bit is set comprises: Determining whether target data exists in the end information; In the case where the target data exists, determining whether the time domain recovery flag bit is set; In the case where the time domain recovery flag bit is set, acquiring remaining frame information according to the target data, and determining the number of remaining frames according to the remaining frame information.

5. The data processing method of claim 1, wherein, The method further comprises: Acquiring a sampling rate corresponding to a current sampling frame; In the case where the sampling rate corresponding to the current sampling frame is not one, performing frame filling based on the sampling rate, the current sampling frame, and a neighboring sampling frame of the current sampling frame to obtain a second frame filled segment corresponding to the current sampling frame.

6. The data processing method according to claim 5, characterized in that, The method further comprises: Acquiring a sampling rate corresponding to the current sampling frame; In the case where the sampling rate corresponding to the current sampling frame is one, determining a third frame filled segment based on the current sampling frame.

7. The data processing method according to claim 6, characterized in that, The method further comprises: Determining a decoded video based on the first frame filled segment, the second frame filled segment, and the third frame filled segment.

8. The data processing method of claim 1, wherein, The method further comprises: In the case where the time domain recovery flag bit is set, determining whether a time domain sampling rate corresponding to a current sampling frame is switched; In the case where the time domain sampling rate corresponding to the current sampling frame is switched, acquiring a sampling rate corresponding to the current sampling frame; Performing frame filling based on the sampling rate.

9. The data processing method according to claim 8, characterized in that, The method further comprises: In the case where the time domain sampling rate corresponding to the current sampling frame is not switched, acquiring a sampling rate corresponding to a sampling frame before the current sampling frame; Performing frame filling based on the sampling rate corresponding to the sampling frame before the current sampling frame.

10. The data processing method of claim 1, wherein, The method further comprises: In the case where the time domain recovery flag bit is set, determining whether a sampling rate maintaining flag corresponding to a current sampling frame exists; In the case where the sampling rate maintaining flag corresponding to the current sampling frame exists, determining whether a sampling rate maintaining flag bit is set; In the case where the sampling rate maintaining flag bit is set, acquiring a sampling rate maintaining flag corresponding to a sampling frame before the current sampling frame according to the sampling rate maintaining flag bit, and determining a sampling rate corresponding to the sampling frame before the current sampling frame based on the sampling rate maintaining flag corresponding to the sampling frame before the current sampling frame; In the case where the sampling rate maintaining flag bit is not set, acquiring a sampling rate corresponding to a sampling frame before the current sampling frame, and performing frame filling based on the sampling rate corresponding to the sampling frame before the current sampling frame. In a case where the sampling rate keeping flag bit is not set, it is determined whether a sampling frame after the current sampling frame corresponds to a sampling rate keeping flag; In a case where the sampling frame after the current sampling frame corresponds to the sampling rate keeping flag, it is determined whether a sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set; In a case where the sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set, the sampling frame after the current sampling frame is filled based on a sampling rate corresponding to a last sampling frame of the sampling frame after the current sampling frame.

11. The data processing method of claim 1, wherein, The method further comprises: In a case where the time domain recovery flag bit is set, it is determined whether a current sampling frame corresponds to a sampling rate keeping flag; In a case where the current sampling frame corresponds to the sampling rate keeping flag, it is determined whether a sampling rate keeping flag bit is set; In a case where the sampling rate keeping flag bit is not set, it is determined whether a sampling frame after the current sampling frame corresponds to a sampling rate keeping flag; In a case where the sampling frame after the current sampling frame corresponds to the sampling rate keeping flag, it is determined whether a sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set; In a case where the sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is not set, a sampling rate corresponding to the sampling frame after the current sampling frame is determined; The sampling frame after the current sampling frame is filled based on the sampling rate.

12. The data processing method of claim 1, wherein, The method further comprises: In a case where the time domain recovery flag bit is set, it is determined whether a current sampling frame corresponds to a sampling rate keeping flag; In a case where the current sampling frame corresponds to the sampling rate keeping flag, it is determined whether a sampling rate keeping flag bit is set; In a case where the sampling rate keeping flag bit is set, a sampling rate corresponding to the current sampling frame is determined; The sampling frame after the current sampling frame is filled based on the sampling rate.

13. The data processing method of claim 1, wherein, The method further comprises: In a case where the time domain recovery flag bit is set, it is determined whether a current sampling frame corresponds to a sampling rate keeping flag; In a case where the current sampling frame corresponds to the sampling rate keeping flag, it is determined whether a sampling rate keeping flag bit is set; In a case where the sampling rate keeping flag bit is set, it is determined whether a sampling frame after the current sampling frame corresponds to a sampling rate keeping flag; In a case where the sampling frame after the current sampling frame corresponds to the sampling rate keeping flag, it is determined whether a sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set; In a case where the sampling rate keeping flag bit corresponding to the sampling frame after the current sampling frame is set, a sampling rate corresponding to the sampling frame after the current sampling frame is determined; The sampling frame after the current sampling frame is filled based on the sampling rate.

14. The data processing method of claim 1, wherein, The method further comprises: In a case where the time domain recovery flag bit is set, it is determined whether a current sampling frame corresponds to a sampling rate keeping flag; In a case where the current sampling frame corresponds to the sampling rate keeping flag, it is determined whether a sampling rate keeping flag bit is set; In a case where the sampling rate keeping flag bit is set, a sampling rate corresponding to the current sampling frame and a keeping number are acquired; In a case where the time domain recovery flag bit is set, it is determined whether a current sampling frame corresponds to a sampling rate keeping flag; In a case where the current sampling frame corresponds to the sampling rate keeping flag, it is determined whether a sampling rate keeping flag bit is set; In a case where the sampling rate keeping flag bit is set, a sampling rate corresponding to the current sampling frame is determined; In a case where the sampling rate keeping flag bit is set, a sampling rate corresponding to the current sampling frame is determined; The sampling frame after the current sampling frame is filled based on the sampling rate. The number of sample frames including the current frame are padded at the sampling rate.

15. The data processing method of claim 1, wherein, The method further comprises: In the case that the time domain recovery flag is set, determining whether the current sample frame corresponds to a sampling rate keeping flag; In the case that the current sample frame corresponds to a sampling rate keeping flag, determining whether a sampling rate keeping flag bit is set; In the case that the sampling rate keeping flag is set, obtaining the sampling rate corresponding to the current sample frame and the number of keeping; Determining whether the sample frame after the number of keeping corresponds to a sampling rate keeping flag; In the case that the sample frame after the number of keeping corresponds to a sampling rate keeping flag, determining whether a sampling rate keeping flag bit corresponding to the sample frame after the number of keeping is set; In the case that the sampling rate keeping flag bit corresponding to the sample frame after the number of keeping is not set, determining the sampling rate corresponding to the sample frame after the number of keeping; Padding the sample frame after the number of keeping based on the sampling rate corresponding to the sample frame after the number of keeping.

16. The data processing method of claim 1, wherein, The method further comprises: In the case that the time domain recovery flag is set, determining whether the current sample frame corresponds to a sampling rate keeping flag; In the case that the current sample frame corresponds to a sampling rate keeping flag, determining whether a sampling rate keeping flag bit is set; In the case that the sampling rate keeping flag is set, obtaining the sampling rate corresponding to the current sample frame and the number of keeping; Determining whether the sample frame after the number of keeping corresponds to a sampling rate keeping flag; In the case that the sample frame after the number of keeping corresponds to a sampling rate keeping flag, determining whether a sampling rate keeping flag bit corresponding to the sample frame after the number of keeping is set; In the case that the sampling rate keeping flag bit corresponding to the sample frame after the number of keeping is set, padding the sample frame after the number of keeping based on the sampling rate corresponding to the current sample frame.

17. The data processing method according to any one of claims 14-16, characterized by, The number of keeping is determined by the number of sample frames between target frames, wherein the target frames include key frames.

18. The data processing method of claim 1, wherein, The method further comprises: In the case that the time domain recovery flag is set, determining whether the current sample frame is a key frame; In the case that the current sample frame is a key frame, determining whether a sampling rate keeping flag bit corresponding to the current sample frame is set; In the case that the sampling rate keeping flag bit corresponding to the current sample frame is set, determining whether the sample frame after the current sample frame is a key frame; In the case that the sample frame after the current sample frame is a key frame, determining the sampling rate corresponding to the sample frame after the current sample frame; Padding based on the sampling rate.

19. The data processing method of claim 1, wherein, The method further comprises: In the case that the time domain recovery flag is set, determining whether the current sample frame is a key frame; In the case that the current sample frame is a key frame, determining whether a sampling rate keeping flag bit corresponding to the current sample frame is set; In the case that the sampling rate keeping flag bit corresponding to the current sample frame is set, determining whether the sample frame after the current sample frame is a key frame; In a case where the sampling frame after the current sampling frame is not a key frame, frame interpolation is performed based on the sampling rate corresponding to the current sampling frame.

20. The data processing method of claim 1, wherein, The method further comprises: In a case where the time domain recovery flag is set, determining whether the current sampling frame is a key frame; In a case where the current sampling frame is a key frame, determining whether the sampling rate retention flag corresponding to the current sampling frame is set; In a case where the sampling rate retention flag corresponding to the current sampling frame is not set, determining whether the sampling frame after the current sampling frame is a key frame; In a case where the sampling frame after the current sampling frame is not a key frame, determining whether the sampling frame after the current sampling frame corresponds to a sampling rate; In a case where the sampling frame after the current sampling frame corresponds to a sampling rate, performing frame interpolation based on the sampling rate.

21. The data processing method of claim 1, wherein, The method further comprises: In a case where the time domain recovery flag is set, determining whether the current sampling frame is a key frame; In a case where the current sampling frame is a key frame, determining whether the sampling rate retention flag corresponding to the current sampling frame is set; In a case where the sampling rate retention flag corresponding to the current sampling frame is not set, determining whether the sampling frame after the current sampling frame is a key frame; In a case where the sampling frame after the current sampling frame is not a key frame, determining whether the sampling frame after the current sampling frame corresponds to a sampling rate; In a case where the sampling frame after the current sampling frame does not correspond to a sampling rate, performing frame interpolation based on the sampling rate corresponding to the current sampling frame.

22. The data processing method according to any one of claims 10 to 16 and 18 to 21, characterized in that, The image information comprises the sampling rate retention flag and time domain sampling change rate information. The time domain sampling change rate information comprises a time domain sampling change rate index.

23. A data processing method, characterized by, Applied to a decoding end, comprising: Performing frame interpolation according to the sampling rate corresponding to the sampling frame.

24. The data processing method according to claim 23, characterized by, The frame interpolation according to the sampling rate corresponding to the sampling frame comprises: Obtaining the sampling rate corresponding to the current sampling frame; In a case where the sampling rate corresponding to the current sampling frame is not one, performing frame interpolation based on the sampling rate, the current sampling frame and the adjacent sampling frame of the current sampling frame to obtain a second frame interpolation segment corresponding to the current sampling frame.

25. The data processing method according to claim 23, characterized by, The frame interpolation according to the sampling rate corresponding to the sampling frame comprises: Obtaining the sampling rate corresponding to the current sampling frame; In a case where the sampling rate corresponding to the current sampling frame is one, determining a third frame interpolation segment based on the current sampling frame.

26. The data processing method of claim 23, wherein, The method further comprises: Determining the number of remaining frames; Performing frame interpolation based on the number of remaining frames to obtain a first frame interpolation segment.

27. The data processing method of claim 23, wherein, The method further comprises: In a case where the time domain recovery flag is set, determining whether the time domain sampling rate corresponding to the current sampling frame is switched; In a case where the time domain sampling rate corresponding to the current sampling frame is switched, obtaining the sampling rate corresponding to the current sampling frame; Performing frame interpolation based on the sampling rate.

28. The data processing method of claim 27, wherein, The method further comprises: In a case where the time domain sampling rate corresponding to the current sampling frame is not switched, obtaining the sampling rate corresponding to the sampling frame before the current sampling frame; Performing frame interpolation based on the sampling rate corresponding to the sampling frame before the current sampling frame.

29. The data processing method of claim 23, wherein, The method further comprises: In a case where the time domain recovery flag is set, it is determined whether the current sampling frame corresponds to a sampling rate maintaining flag; In a case where the current sampling frame corresponds to the sampling rate maintaining flag, it is determined whether the sampling rate maintaining flag bit is set; In a case where the sampling rate maintaining flag bit is not set, it is determined whether a sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag; In a case where the sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag, it is determined whether a sampling rate maintaining flag bit corresponding to the sampling frame after the current sampling frame is set; In a case where the sampling rate maintaining flag bit corresponding to the sampling frame after the current sampling frame is set, the sampling frame after the current sampling frame is filled based on a sampling rate corresponding to a last sampling frame of the sampling frame after the current sampling frame.

30. The data processing method of claim 23, wherein, The method further comprises: In a case where the time domain recovery flag is set, it is determined whether the current sampling frame corresponds to a sampling rate maintaining flag; In a case where the current sampling frame corresponds to the sampling rate maintaining flag, it is determined whether the sampling rate maintaining flag bit is set; In a case where the sampling rate maintaining flag bit is not set, it is determined whether a sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag; In a case where the sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag, it is determined whether a sampling rate maintaining flag bit corresponding to the sampling frame after the current sampling frame is set; In a case where the sampling rate maintaining flag bit corresponding to the sampling frame after the current sampling frame is not set, a sampling rate corresponding to the sampling frame after the current sampling frame is determined; The sampling frame after the current sampling frame is filled based on the sampling rate.

31. The data processing method of claim 23, wherein, The method further comprises: In a case where the time domain recovery flag is set, it is determined whether the current sampling frame corresponds to a sampling rate maintaining flag; In a case where the current sampling frame corresponds to the sampling rate maintaining flag, it is determined whether the sampling rate maintaining flag bit is set; In a case where the sampling rate maintaining flag bit is set, a sampling rate corresponding to the current sampling frame is determined; The sampling frame after the current sampling frame is filled based on the sampling rate.

32. The data processing method of claim 23, wherein, The method further comprises: In a case where the time domain recovery flag is set, it is determined whether the current sampling frame corresponds to a sampling rate maintaining flag; In a case where the current sampling frame corresponds to the sampling rate maintaining flag, it is determined whether the sampling rate maintaining flag bit is set; In a case where the sampling rate maintaining flag bit is not set, it is determined whether a sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag; In a case where the sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag, it is determined whether a sampling rate maintaining flag bit corresponding to the sampling frame after the current sampling frame is set; In a case where the sampling rate maintaining flag bit corresponding to the sampling frame after the current sampling frame is set, a sampling rate corresponding to the sampling frame after the current sampling frame is determined; The sampling frame after the current sampling frame is filled based on the sampling rate.

33. The data processing method of claim 23, wherein, The method further comprises: In a case where the time domain recovery flag is set, it is determined whether the current sampling frame corresponds to a sampling rate maintaining flag; In a case where the current sampling frame corresponds to the sampling rate keeping flag, it is determined whether the sampling rate keeping flag bit is set; In a case where the sampling rate keeping flag bit is set, the sampling rate corresponding to the current sampling frame and the keeping quantity are obtained; The sampling frames including the current frame are interpolated at the sampling rate by the keeping quantity.

34. The data processing method of claim 23, wherein, The method further comprises: In a case where the time domain recovery flag bit is set, it is determined whether the current sampling frame corresponds to the sampling rate keeping flag; In a case where the current sampling frame corresponds to the sampling rate keeping flag, it is determined whether the sampling rate keeping flag bit is set; In a case where the sampling rate keeping flag bit is set, the sampling rate corresponding to the current sampling frame and the keeping quantity are obtained; It is determined whether the sampling frame after the keeping quantity corresponds to the sampling rate keeping flag; In a case where the sampling frame after the keeping quantity corresponds to the sampling rate keeping flag, it is determined whether the sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is set; In a case where the sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is not set, the sampling rate corresponding to the sampling frame after the keeping quantity is determined; The sampling frame after the keeping quantity is interpolated based on the sampling rate corresponding to the sampling frame after the keeping quantity.

35. The data processing method of claim 23, wherein, The method further comprises: In a case where the time domain recovery flag bit is set, it is determined whether the current sampling frame corresponds to the sampling rate keeping flag; In a case where the current sampling frame corresponds to the sampling rate keeping flag, it is determined whether the sampling rate keeping flag bit is set; In a case where the sampling rate keeping flag bit is set, the sampling rate corresponding to the current sampling frame and the keeping quantity are obtained; It is determined whether the sampling frame after the keeping quantity corresponds to the sampling rate keeping flag; In a case where the sampling frame after the keeping quantity corresponds to the sampling rate keeping flag, it is determined whether the sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is set; In a case where the sampling rate keeping flag bit corresponding to the sampling frame after the keeping quantity is set, the sampling frame after the keeping quantity is interpolated based on the sampling rate corresponding to the current sampling frame.

36. The data processing method according to any one of claims 33-35, wherein, The keeping quantity is determined by the quantity of sampling frames between target frames, wherein the target frames include key frames.

37. The data processing method of claim 23, wherein, The method further comprises: In a case where the time domain recovery flag bit is set, it is determined whether the current sampling frame is a key frame; In a case where the current sampling frame is a key frame, it is determined whether the sampling rate keeping flag bit corresponding to the current sampling frame is set; In a case where the sampling rate keeping flag bit corresponding to the current sampling frame is set, it is determined whether the sampling frame after the current sampling frame is a key frame; In a case where the sampling frame after the current sampling frame is a key frame, the sampling rate corresponding to the sampling frame after the current sampling frame is determined; The interpolation is performed based on the sampling rate.

38. The data processing method of claim 23, wherein, The method further comprises: In a case where the time domain recovery flag bit is set, it is determined whether the current sampling frame is a key frame; In a case where the current sampling frame is a key frame, it is determined whether the sampling rate keeping flag bit corresponding to the current sampling frame is set; In a case where the sampling rate maintaining flag corresponding to the current sampling frame is set, it is determined whether a sampling frame after the current sampling frame is a key frame; In a case where the sampling frame after the current sampling frame is not a key frame, the frame is interpolated based on the sampling rate corresponding to the current sampling frame.

39. The data processing method of claim 23, wherein, The method further comprises: In a case where the time domain recovery flag is set, it is determined whether the current sampling frame is a key frame; In a case where the current sampling frame is a key frame, it is determined whether the sampling rate maintaining flag corresponding to the current sampling frame is set; In a case where the sampling rate maintaining flag corresponding to the current sampling frame is not set, it is determined whether a sampling frame after the current sampling frame is a key frame; In a case where the sampling frame after the current sampling frame is not a key frame, it is determined whether the sampling frame after the current sampling frame corresponds to a sampling rate; In a case where the sampling frame after the current sampling frame corresponds to a sampling rate, the frame is interpolated based on the sampling rate.

40. The data processing method of claim 23, wherein, The method further comprises: In a case where the time domain recovery flag is set, it is determined whether the current sampling frame is a key frame; In a case where the current sampling frame is a key frame, it is determined whether the sampling rate maintaining flag corresponding to the current sampling frame is set; In a case where the sampling rate maintaining flag corresponding to the current sampling frame is not set, it is determined whether a sampling frame after the current sampling frame is a key frame; In a case where the sampling frame after the current sampling frame is not a key frame, it is determined whether the sampling frame after the current sampling frame corresponds to a sampling rate; In a case where the sampling frame after the current sampling frame does not correspond to a sampling rate, the frame is interpolated based on the sampling rate corresponding to the current sampling frame.

41. The data processing method according to any of claims 29-35 and 37-40, characterized by, The image information comprises the sampling rate maintaining flag and time domain sampling change rate information; The time domain sampling change rate information comprises a time domain sampling change rate index.

42. The data processing method of claim 23, wherein, The method further comprises: Based on the code stream data, a plurality of sampling segments, a number of original frames corresponding to each sampling segment and a sampling rate corresponding to each sampling segment are determined; Based on the number of original frames corresponding to each sampling segment and the sampling rate corresponding to each sampling segment, the frame is interpolated to obtain a frame segment corresponding to each sampling segment.

43. The data processing method of claim 23, wherein, The code stream data comprises a sequence parameter set and end information; It is determined whether the time domain recovery flag is set according to the sequence parameter set; In a case where the time domain recovery flag is set, it is determined whether the end information has target data; In a case where there is target data, the remaining frame information is obtained according to the target data, and the number of remaining frames is determined according to the remaining frame information.

44. The data processing method of claim 23, wherein, The code stream data comprises a sequence parameter set and end information; It is determined whether the end information has target data; In a case where there is target data, it is determined whether the time domain recovery flag is set; In a case where the time domain recovery flag is set, the remaining frame information is obtained according to the target data, and the number of remaining frames is determined according to the remaining frame information.

45. A data processing method, comprising: Applied to an encoding end, comprising: The video is encoded to obtain code stream data, so that the decoding end implements any one of the data processing methods in claims 1-44 based on the code stream data.

46. A data processing apparatus, characterized in that, Applied to a decoding end, comprising: The video is encoded to obtain code stream data, so that the decoding end implements any one of the data processing methods in claims 1-44 based on the code stream data. The first determining module is configured to determine whether a time domain recovery flag bit is set in the code stream data. The second determining module is configured to determine a number of remaining frames in a case where the time domain recovery flag bit is set. The first frame supplementing module is configured to perform frame supplementing based on the number of remaining frames to obtain a first frame supplemented segment.

47. A data processing apparatus, characterized in that, The application is applied to a decoding end and includes: The fifteenth frame supplementing module is configured to perform frame supplementing according to a sampling rate corresponding to a sampling frame, wherein the code stream data includes at least two different sampling rates.

48. A data processing apparatus, characterized in that, The application is applied to a coding end and includes: A coding module is configured to code a video to obtain code stream data, so that the decoding end implements any one of the data processing methods in claims 1-44 based on the code stream data.

49. An electronic device, comprising: The application includes: A processor; and A memory configured to store executable instructions of the processor. The processor is configured to execute the executable instructions to implement the data processing method in any one of claims 1-44.

50. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the data processing method in any one of claims 1-44.

Images