Data processing method and device, equipment and storage medium

Abstract

The invention provides a signal data processing method and device, equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: carrying out the frame supplementation of a frame between a current sampling frame and a subsequent sampling frame of the current sampling frame based on a target frame, and obtaining a first frame supplementation segment of the current sampling frame; coding and decoding based on variable up-sampling multiplying power are realized.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority to Chinese Patent Application No. 202410460640.4, filed April 16, 2024, entitled “Data Processing Method, Device, Equipment and Storage Medium,” the entire contents of which are incorporated herein by reference. Figure 1 Figure 2 TECHNICAL FIELD

[0003] The present disclosure relates to the technical field of image processing, and particularly relates to a data processing method and device, equipment and a storage medium. BACKGROUND

[0004] With the development of the times, the rate of information transmission is accelerating. Among the current transmitted information, video information accounts for a large proportion.

[0005] In the process of information transmission, the frame extraction coding technology is a commonly used technology in the field of signal coding technology. The current frame extraction coding technology usually extracts frames from the video according to a fixed frame extraction ratio, and then encodes the extracted video. However, with the development of technology, the single up-sampling ratio coding and decoding method has been difficult to meet the needs of users. How to realize the coding and decoding of variable up-sampling ratio is a problem to be solved in the field. SUMMARY

[0006] The present disclosure provides a data processing method, device, equipment and storage medium, which at least realizes the coding and decoding of variable up-sampling ratio to some extent.

[0007] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0008] According to one aspect of the present disclosure, a data processing method is provided, applied to a decoding end, comprising: based on a target frame, performing frame filling on frames between a current sampling frame and a subsequent sampling frame of the current sampling frame, to obtain a first frame filling segment corresponding to the current sampling frame, the target frame including at least one of the current sampling frame, a previous sampling frame of the current sampling frame, and frames between the current sampling frame and the previous sampling frame of the sampling frame.

[0009] In one embodiment of the present disclosure, the method further comprises: based on the target frame, performing frame filling on frames between the current sampling frame and a subsequent frame of the current sampling frame, comprising:

[0010] Based on the target frame and a sampling ratio corresponding to the current sampling frame, performing frame filling on frames between the current sampling frame and the subsequent sampling frame of the current sampling frame, to obtain a first frame filling segment corresponding to the current sampling frame.

[0011] In one embodiment of the present disclosure, the method further comprises: determining whether an extension flag bit is set.

[0012] In an embodiment of the present disclosure, the method further comprises: in the case that the extension flag is set to 1, performing frame interpolation between the current sampling frame and a subsequent sampling frame of the current sampling frame based on a target frame to obtain a first frame-interpolated segment corresponding to the current sampling frame.

[0013] In an embodiment of the present disclosure, the method further comprises: in the case that the extension flag is set to 1, determining whether a time domain recovery flag is set to 1 in the code stream data; in the case that the time domain recovery flag is set to 1, determining a number of remaining frames; and performing frame interpolation based on the number of remaining frames to obtain a second frame-interpolated segment.

[0014] In an embodiment of the present disclosure, the method further comprises: determining the decoded video based on the first frame-interpolated segment and the second frame-interpolated segment.

[0015] In an embodiment of the present disclosure, the method further comprises: in the case that the extension flag is set to 1, determining whether a sampling rate maintaining flag corresponds to the current sampling frame; in the case that the sampling rate maintaining flag corresponds to the current sampling frame, determining whether a sampling rate maintaining flag bit is set to 1; and in the case that the sampling rate maintaining flag bit is set to 1, performing frame interpolation based on a sampling rate corresponding to the current sampling frame.

[0016] In an embodiment of the present disclosure, the method further comprises: in the case that the extension flag is set to 1, determining whether a sampling rate maintaining flag corresponds to the current sampling frame; in the case that the sampling rate maintaining flag corresponds to the current sampling frame, determining whether a sampling rate maintaining flag bit is set to 1; and in the case that the sampling rate maintaining flag bit is not set to 1, determining a sampling rate corresponding to a sampling frame subsequent to the current sampling frame; and performing frame interpolation based on the sampling rate corresponding to the sampling frame subsequent to the current sampling frame.

[0017] According to another aspect of the present disclosure, a data processing method is provided, which is applied to an encoding end and comprises: encoding a video to obtain code stream data, so that a decoding end implements any one of the data processing methods in the decoding end based on the code stream data.

[0018] According to still another aspect of the present disclosure, a data processing apparatus is provided, which is applied to a decoding end and comprises: a first frame-interpolating module configured to perform frame interpolation between a current sampling frame and a subsequent sampling frame of the current sampling frame based on a target frame to obtain a first frame-interpolated segment corresponding to the current sampling frame, the target frame comprising at least one of the current sampling frame, a previous sampling frame of the current sampling frame, and a frame between the current sampling frame and the previous sampling frame of the current sampling frame.

[0019] In one embodiment of the present disclosure, the first frame filling module further comprises a first frame filling unit configured to fill frames between the current sampling frame and a subsequent sampling frame of the current sampling frame based on the target frame and a sampling rate corresponding to the current sampling frame, to obtain a first frame filling segment corresponding to the current sampling frame.

[0020] In one embodiment of the present disclosure, the apparatus further comprises a first determination module configured to determine whether an extension flag is set.

[0021] In one embodiment of the present disclosure, the apparatus further comprises a second frame filling module configured to, in a case where the extension flag is set, fill frames between the current sampling frame and a subsequent sampling frame of the current sampling frame based on the target frame, to obtain a first frame filling segment corresponding to the current sampling frame.

[0022] In one embodiment of the present disclosure, the apparatus further comprises a second determination module configured to, in a case where the extension flag is set, determine whether a time domain recovery flag is set in the code stream data; a third determination module configured to, in a case where the time domain recovery flag is set, determine a number of remaining frames; and a third frame filling module configured to fill frames based on the number of remaining frames, to obtain a second frame filling segment.

[0023] In one embodiment of the present disclosure, the apparatus further comprises a third determination module configured to determine a decoded video based on the first frame filling segment and the second frame filling segment.

[0024] In one embodiment of the present disclosure, the apparatus further comprises a fourth determination module configured to, in a case where the extension flag is set, determine whether a current sampling frame corresponds to a sampling rate maintaining flag; a fifth determination module configured to, in a case where the current sampling frame corresponds to the sampling rate maintaining flag, determine whether a sampling rate maintaining flag is set; and a third frame filling module configured to, in a case where the sampling rate maintaining flag is set, fill frames based on a sampling rate corresponding to the current sampling frame.

[0025] In one embodiment of the present disclosure, the apparatus further comprises a sixth determination module configured to, in a case where the extension flag is set, determine whether a current sampling frame corresponds to a sampling rate maintaining flag; a seventh determination module configured to, in a case where the current sampling frame corresponds to the sampling rate maintaining flag, determine whether a sampling rate maintaining flag is set; an eighth determination module configured to, in a case where the sampling rate maintaining flag is not set, determine a sampling rate corresponding to a sampling frame subsequent to the current sampling frame; and a fourth frame filling module configured to fill frames based on the sampling rate corresponding to the sampling frame subsequent to the current sampling frame.

[0026] According to still another aspect of the present disclosure, there is provided a data processing apparatus applied to an encoding end, comprising an encoding module configured to encode a video to obtain code stream data, so that a decoding end implements any one of the data processing methods in the decoding end based on the code stream data.

[0027] According to yet another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-mentioned data processing method via execution of the executable instructions.

[0028] According to yet another aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon a computer program, which, when executed by a processor, implements the above-mentioned data processing method.

[0029] The data processing method, device, equipment and storage medium provided by the embodiments of the present disclosure perform frame filling between the current sampling frame and the subsequent sampling frame of the current sampling frame based on the target frame, obtain the first frame filling segment of the current sampling frame, and decode according to the already decoded frame. The variable up-sampling ratio based coding and decoding is realized.

[0030] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0031] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0032] Figure 3 A flow chart of a data processing method in an embodiment of the present disclosure is shown;

[0033] Figure 4 A flow chart of another data processing method in an embodiment of the present disclosure is shown;

[0034] Figure 5 A flow chart of another data processing method in an embodiment of the present disclosure is shown;

[0035] Figure 6 A flow chart of another data processing method in an embodiment of the present disclosure is shown;

[0036] Figure 7 A flow chart of another data processing method in an embodiment of the present disclosure is shown;

[0037] Figure 8 A flow chart of another data processing method in an embodiment of the present disclosure is shown;

[0038] Figure 9 A schematic diagram of a data processing device in an embodiment of the present disclosure is shown;

[0039] Figure 1 Fig. 2 shows a schematic diagram of another data processing apparatus in the embodiments of the present disclosure;

[0040] Figure 1 Fig. 3 shows a structural block diagram of an electronic device in the embodiments of the present disclosure. DETAILED DESCRIPTION

[0041] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations can be implemented in any

[0042] In addition, the accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and serve to explain the principles of the present disclosure. The same reference numbers in different drawings represent the same or similar elements.

[0043] It should be understood that each of the steps of the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0044] It should be noted that the terms "first", "second", and the like in the present disclosure are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0045] It should be noted that the terms "one", "multiple" in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that "one" or "multiple" should be understood as "one or more" unless otherwise explicitly indicated in the context.

[0046] To solve the above problems, the embodiments of the present disclosure provide a data processing method, apparatus, device and storage medium.

[0047] In order to describe the present disclosure in detail, Figure 2 Fig. 1 shows a flowchart of a data processing method in the embodiments of the present disclosure. It should be noted that the data processing method in the embodiments of the present disclosure is applied to an encoding end.

[0048] As Figure 2 shown, the data processing method can include:

[0049] S110, based on the target frame, the frames between the current sampling frame and the subsequent sampling frame of the current sampling frame are filled in, and the first filling frame segment corresponding to the current sampling frame is obtained, the target frame includes at least one of the current sampling frame, the previous sampling frame of the current sampling frame and the frame between the current sampling frame and the previous sampling frame of the sampling frame.

[0050] In some embodiments, the encoding mode can be determined in advance.

[0051] In some embodiments, in the random access mode, the frames between the current sampling frame and the subsequent sampling frame of the current sampling frame can also be filled in based on the current sampling frame and the subsequent sampling frame of the current sampling frame.

[0052] In some embodiments, in the random access mode, the frames between the current sampling frame and the subsequent sampling frame of the current sampling frame can also be filled in based on the target frame, and the first filling frame segment corresponding to the current sampling frame is obtained, the target frame includes at least one of the current sampling frame, the previous sampling frame of the current sampling frame and the frame between the current sampling frame and the previous sampling frame of the sampling frame.

[0053] In some embodiments, in the low delay mode, the frames between the current sampling frame and the subsequent sampling frame of the current sampling frame can be filled in based on the target frame, and the first filling frame segment corresponding to the current sampling frame is obtained, the target frame includes at least one of the current sampling frame, the previous sampling frame of the current sampling frame and the frame between the current sampling frame and the previous sampling frame of the sampling frame.

[0054] In some embodiments, the sampling rate corresponding to the current sampling frame can be determined.

[0055] In some embodiments, the number of frames between the current sampling frame and the subsequent sampling frame of the current sampling frame can be determined based on the sampling rate corresponding to the current sampling frame. After determining the number of frames between the current sampling frame and the subsequent sampling frame of the current sampling frame, the above-mentioned number of frames can be filled in based on the target frame.

[0056] In some embodiments, after determining the sampling ratio, the number of frames between the current sampling frame and the subsequent sampling frame of the current sampling frame is determined based on the sampling ratio. For example, the number of frames between the current sampling frame and the subsequent sampling frame of the current sampling frame can be represented by TemporaryResamplingRatio-1. For example, TemporaryResamplingRatio = 2^(temporal_resampling_ratio_idx)-1. In some embodiments, temporal_resampling_ratio_idx can be the sampling ratio between the current sampling frame and the subsequent sampling frame of the current sampling frame. In some embodiments, temporal_resampling_ratio_idx can be the temporal sampling ratio index between the current sampling frame and the subsequent sampling frame of the current sampling frame, and the sampling ratio between the current sampling frame and the subsequent sampling frame of the current sampling frame can be determined based on the temporal sampling ratio index between the current sampling frame and the subsequent sampling frame of the current sampling frame.

[0057] In some embodiments, the frame filling can include repeating the target frame or inputting the target frame into a trained prediction model.

[0058] In some embodiments, the frame filling can further include determining a sliding window, and filling the frames based on the sliding window and the target frame.

[0059] In some embodiments, the frame filling can further include filling the frames based on a frame filling algorithm such as linear interpolation and the target frame.

[0060] Figure 3 A flowchart of another data processing method in the embodiments of the present disclosure is shown. It should be noted that the data processing method in the embodiments of the present disclosure is applied to the decoding end.

[0061] As shown in Figure 3 , the data processing method can include:

[0062] S210, determining whether an extension flag is set;

[0063] S220, in the case where the extension flag is set, filling the frames between the current sampling frame and the subsequent sampling frame of the current sampling frame based on the target frame to obtain a first frame filling segment corresponding to the current sampling frame, the target frame including at least one of the current sampling frame, the previous sampling frame of the current sampling frame, and the frames between the current sampling frame and the previous sampling frame of the sampling frame.

[0064] In some embodiments, the extension flag position bit can be set to Sps_vcm_extension_flag = 1. The extension flag bit is not set to Sps_vcm_extension_flag = 0.

[0065] In some embodiments, the extension flag bit can be used to indicate whether the VCM standard syntax signaling is enabled. The VCM standard syntax signaling is enabled when the flag bit is set, and the VCM standard syntax signaling is disabled when the flag bit is not set.

[0066] Exemplary:

[0067]

[0068] It should be noted that ph_temporal_resampling_ratio_changed_flag = 1 indicates that the temporal resampling ratio of the temporal domain reconstruction changes. ph_temporal_resampling_ratio_changed_flag = 0 indicates that the temporal domain reconstruction uses the historical temporal resampling ratio. The historical temporal resampling ratio is a temporary variable in the decoder, which is updated by the changed temporal resampling ratio.

[0069] ph_temporal_resampling_ratio_idx indicates the temporal resampling ratio when the temporal domain reconstruction function is enabled and ph_temporal_resampling_ratio_changed_flag is activated. The range of ph_temporal_resampling_ratio_idx is 0 to 3. The variable TemporalResamplingRatio is derived as follows:

[0070] TemporalResamplingRatio = 2^(ph_temporal_resampling_ratio_idx).

[0071] The data processing method provided by the embodiments of the present disclosure is based on a target frame to perform frame interpolation between a current sampling frame and a subsequent sampling frame of the current sampling frame, to obtain a first interpolation frame segment of the current sampling frame, and to perform decoding according to the already decoded frame. The variable upsampling ratio based coding and decoding is realized.

[0072] Figure 4 Another flowchart of a data processing method in the embodiments of the present disclosure is shown. It should be noted that the data processing method in the embodiments of the present disclosure is applied to the decoding end.

[0073] As shown in Figure 4 the data processing method can comprise:

[0074] S310, determine whether the extension flag is set.

[0075] S320, in the case where the extension flag is set, perform frame interpolation between the current sample frame and a subsequent sample frame of the current sample frame based on the target frame to obtain a first frame-interpolated segment corresponding to the current sample frame.

[0076] Exemplary:

[0077]

[0078] It should be noted that Eob_num_temporal_remai indicates the number of frames that need to be recovered after the last sample frame.

[0079] The variable TotalNumberOfFrames is derived as follows:

[0080]

[0081] TemporalResamplingRatio[i] specifies the temporal sampling rate between the i-th and (i+1)-th input pictures.

[0082] The data processing method provided by the embodiment of the present disclosure performs frame interpolation between the current sample frame and a subsequent sample frame of the current sample frame based on the target frame to obtain a first frame-interpolated segment of the current sample frame, and decodes based on the already decoded frame. The variable upsampling rate based coding and decoding is realized.

[0083] Figure 5 Another flowchart of a data processing method in the embodiment of the present disclosure is shown. It should be noted that the data processing method in the embodiment of the present disclosure is applied to the decoding end.

[0084] As Figure 5 shown, the data processing method can include:

[0085] S410, determine whether the extension flag is set.

[0086] S420, in the case where the extension flag is set, determine whether the temporal recovery flag is set in the code stream data.

[0087] S430, in the case where the temporal recovery flag is set, determine the number of remaining frames.

[0088] S440, perform frame interpolation based on the number of remaining frames to obtain a second frame-interpolated segment.

[0089] In some embodiments, the time domain restoration flag can be temproal_restoration_flag. In some embodiments, the time domain flag restoration position bit can be set as temproal_restoration_flag = 1, and the time domain flag restoration position bit can be unset as temproal_restoration_flag = 0. In some embodiments, after obtaining the bitstream data, the bitstream data can be parsed first, and then it is determined whether the time domain restoration flag is set in the bitstream data.

[0090] In some embodiments, the remaining frames can be the frames in the original video that are not sampled after the last sampled frame. For example, the number of the remaining frames can be num_temporal_remain. For example, if the original video contains 1, 2, 3, 4, and 5 frames in total, and the 1st and 3rd frames are sampled, then the 4th and 5th frames are the remaining frames, and the number of the remaining frames is 2.

[0091] In some embodiments, after determining the number of the remaining frames, the last sampled frame repetition method can be used for frame interpolation. In some embodiments, after determining the number of the remaining frames, the method of inputting at least one sampled frame into an intelligent model can also be used for frame interpolation. In some embodiments, the second frame interpolation segment can include a segment composed of the remaining frames.

[0092] In some embodiments, the length of the decoded video frame can be

[0093]

[0094] wherein num_temporal_remain is the number of the remaining frames, N is the number of the sampled frames, i is the sampled frame, and TemporalResamplingratio is the sampling ratio between the i-th sampled frame and the i+1-th sampled frame. In some embodiments, the first frame interpolation segment and the second frame interpolation segment can both be empty segments, i.e., the frame interpolation segment does not exist.

[0095] For example,

[0096]

[0097] It should be noted that Sps_vcm_extension_flag = 1 indicates that there is a VCM standard syntax element. Sps_vcm_extension_flag = 0 indicates that there is no VCM standard syntax element.

[0098] Sps_temporal_restoration_flag = 1 indicates that the temporal restoration function is enabled. Sps_temporal_restoration_flag = 0 indicates that the temporal restoration function is disabled.

[0099] sps_temporal_restoration_ratio_index specifies the sequence-level temporal sampling ratio index, and the initial value of the temporary variable history temporal sampling ratio index is sps_temporal_restoration_ratio_index.

[0100] Figure 6 Another flowchart of a data processing method in the embodiments of the present disclosure is shown. It should be noted that the data processing method in the embodiments of the present disclosure is applied to a decoding end.

[0101] As shown in Figure 6 , the data processing method can include:

[0102] S510, determining whether an extension flag bit is set.

[0103] S520, in the case where the extension flag bit is set, determining whether a current sampling frame corresponds to a sampling ratio maintenance flag.

[0104] In some embodiments, the current sampling frame can be an I frame. The I frame, also known as an intra picture or key frame, is a complete picture and can be decoded without reference to other frames. In video compression encoding, the I frame is usually the first frame of each group of pictures (GOP), and after moderate compression, it can be used as a reference point for random access. Due to its independence, the I frame can ensure the integrity and accuracy of the video data during transmission.

[0105] In some embodiments, before determining the maintenance flag bit, it can also be determined that the encoding mode is a random access mode or a low delay mode.

[0106] In some embodiments, the ratio consistent flag can be set or not set. For example, when the ratio consistent flag is set, it can be represented as ratio_consisent_flag = 1 or ratio_changed_flag = 0. When the ratio consistent flag is not set, it can be represented as ratio_consisent_flag = 0 or ratio_changed_flag = 1. For example, ratio_consisent_flag = 1 or ratio_changed_flag = 0 can also indicate that the sampling rate switching flag is not set; ratio_consisent_flag = 0 or ratio_changed_flag = 1 can also indicate that the sampling rate switching flag is set. In some embodiments, when the sampling rate switching flag is set, i.e., temporal resampling ratio changed flag equal to 1, the subsequent sampling rate uses the changed sampling rate. When the sampling rate switching flag is not set, i.e., temporal resampling ratio changed flag equal to 0, the subsequent sampling rate uses the sampling rate corresponding to the previous sampling frame.

[0107] S530, in the case of the sampling rate consistent flag corresponding to the current sampling frame, determine whether the sampling rate consistent flag is set.

[0108] S540, in the case of the sampling rate consistent flag being set, perform frame interpolation based on the sampling rate corresponding to the current sampling frame.

[0109] For example:

[0110]

[0111]

[0112] It should be noted that ph_temporal_resampling_ratio_changed_flag = 1 indicates that the temporal sampling rate of the temporal reconstruction has changed. ph_temporal_resampling_ratio_changed_flag = 0 indicates that the temporal reconstruction uses the historical temporal sampling rate. The historical temporal sampling rate is a temporary variable in the decoder, which is updated by the changed temporal sampling rate.

[0113] For example:

[0114]

[0115] It should be noted that ph_temporal_resampling_ratio_idx represents the time domain resampling ratio when the time domain resampling function is enabled and ph_temporal_resampling_ratio_changed_flag is activated. The range of ph_temporal_resampling_ratio_idx is 0 to 3. The variable TemporalResamplingRatio is derived as follows:

[0116] TemporalResamplingRatio = 2^(ph_temporal_resampling_ratio_idx).

[0117] In some embodiments, the sampling ratio can be ratio_idx. In some embodiments, the time domain sampling ratio index can be ratio_idx. In some embodiments, the last sampling frame of the sampling frame after can be a sampling frame continuous with the sampling frame after. In some embodiments, in the case that the sampling ratio maintenance flag bit corresponding to the current sampling frame is not set, it can be determined whether the sampling ratio maintenance flag bit corresponding to the sampling frame after the current sampling frame is set. In the case that the sampling ratio maintenance flag bit corresponding to the sampling frame after the current sampling frame is not set, the sampling ratio corresponding to the sampling frame after the current sampling frame is determined; and the sampling frame after the current sampling frame is filled based on the sampling ratio.

[0118] In some embodiments, in the case that the sampling ratio maintenance flag bit corresponding to the current sampling frame is set, it can be determined whether the sampling ratio maintenance flag bit corresponding to the sampling frame after the current sampling frame is set; in the case that the sampling ratio maintenance flag bit corresponding to the sampling frame after the current sampling frame is set, the sampling ratio corresponding to the sampling frame after the current sampling frame is determined; and the sampling frame after the current sampling frame is filled based on the sampling ratio. In some embodiments, in the case that the sampling ratio maintenance flag bit corresponding to the current sampling frame is set, it can be determined whether the sampling ratio maintenance flag bit corresponding to the sampling frame after the current sampling frame is set; in the case that the sampling ratio maintenance flag bit corresponding to the sampling frame after the current sampling frame is not set, the sampling ratio corresponding to the sampling frame after the current sampling frame is determined; and the sampling frame after the current sampling frame is filled based on the sampling ratio.

[0119] In some embodiments, when the sampling rate keeping flag is set, the sampling rate corresponding to the current sampling frame and the keeping number can also be obtained. The sampling frames including the current frame are padded at the sampling rate for the keeping number. Whether the sampling frame after the keeping number corresponds to the sampling rate keeping flag is determined. When the sampling frame after the keeping number corresponds to the sampling rate keeping flag, whether the sampling rate keeping flag bit corresponding to the sampling frame after the keeping number is set is determined. When the sampling rate keeping flag bit corresponding to the sampling frame after the keeping number is not set, the sampling rate corresponding to the sampling frame after the keeping number is determined. The sampling frame after the keeping number is padded based on the sampling rate corresponding to the sampling frame after the keeping number.

[0120] In some embodiments, the keeping number is determined by the number of sampling frames between target frames. The target frames can be key frames. In some embodiments, the target frames can also include frames marked by an encoder.

[0121] In some embodiments, whether the sampling rate keeping flag bit is set, the time domain sampling information such as the time domain sampling rate index, is not limited to the high-level syntax structure such as SEI / AU / PPS. For example, a picture parameter set (PPS) with a new ID can also be added. The PPS with the new ID contains the temporal_ratio_idx. The I frame and the frame with the ratio_changed_flag of 1 adopt the PPS structure with the new ID, and the remaining frames adopt the existing PPS structure. The frame adopting the existing PPS structure is associated with the temporal_ratio_idx in the PPS corresponding to the forward adjacent I frame or the frame with the ratio_changed_flag of 1. Therefore, the judgment of the ratio_consistent_flag or the ratio_changed_flag is hidden in the judgment of the ID of the PPS. Similar operations can also be added to the NAL header with a new ID or the SEI with a new ID.

[0122] The NAL unit is the basic syntax structure of NAL, which contains one byte of header information and a series of byte streams from VCL called raw byte sequence payload (RBSP). The NAL header is one byte, in which the 7th bit is a forbidden bit, which is 0 by default in coding, and can be set to 1 when the network identifies that there is a bit error in the unit, so that the recipient discards the unit. The 6th to 5th bits are the priority, with a value range of 0 to 3, and the higher the value, the more important the current NAL, which needs to be protected first. The 4th to 0th bits represent the type of NAL, such as slice in non-IDR image and slice not partitioned, slice in IDR image, supplemental enhancement information SEI, sequence parameter set SPS, picture parameter set PPS, etc.

[0123] Among them, SEI (Supplemental Enhancement Information) is an important component in the H.264 / AVC video coding standard. SEI is a kind of additional enhancement information, which can contain a series of parameters and data related to video coding and transmission.

[0124] In some embodiments, it can be determined whether the current sampling frame is a key frame based on the codec.

[0125] In some embodiments, in the case that the current sampling frame is a key frame, it is determined whether the sampling rate maintaining flag corresponding to the current sampling frame is set, in the case that the sampling rate maintaining flag is set, it is determined whether the sampling frame after the current sampling frame is a key frame, in the case that the sampling frame after the current sampling frame is a key frame, the sampling rate corresponding to the sampling frame after the current sampling frame is determined. The frame is filled based on the sampling rate.

[0126] In some embodiments, in the case that the sampling frame after the current sampling frame is not a key frame, the frame is filled based on the sampling rate corresponding to the current sampling frame. In some embodiments, in the case that the sampling frame after the current sampling frame is not a key frame, it can also be determined whether the sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag. If it corresponds to the sampling rate maintaining flag and the sampling rate maintaining flag is set, the frame is filled based on the sampling rate of the previous sampling frame of the sampling frame after the current sampling frame. In some embodiments, in the case that the sampling frame after the current sampling frame is not a key frame, it can also be determined whether the sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag. If it corresponds to the sampling rate maintaining flag and the sampling rate maintaining flag is not set, it is determined whether the sampling frame after the current sampling frame corresponds to the sampling rate; in the case that the sampling frame after the current sampling frame corresponds to the sampling rate, the frame is filled based on the sampling rate.

[0127] Figure 7 Another flowchart of a data processing method in the embodiments of the present disclosure is shown. It should be noted that the data processing method in the embodiments of the present disclosure is applied to the decoding end.

[0128] As Figure 7 shown, the method can include:

[0129] S610, in the case that the extension flag is set, it is determined whether the current sampling frame corresponds to the sampling rate maintaining flag;

[0130] S620, in the case that the current sampling frame corresponds to the sampling rate maintaining flag, it is determined whether the sampling rate maintaining flag is set;

[0131] S630, determining the sampling rate corresponding to the sampling frame after the current sampling frame in the case that the sampling rate maintaining flag is not set;

[0132] S640, performing frame interpolation based on the sampling rate corresponding to the sampling frame after the current sampling frame.

[0133] In some embodiments, the method further comprises: in the case that the sampling frame after the current sampling frame does not correspond to the sampling rate, performing frame interpolation based on the sampling rate corresponding to the current sampling frame. In some embodiments, before determining whether the sampling frame after the current sampling frame corresponds to the time domain sampling rate, it is further required to determine whether the sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag. If it corresponds to the sampling rate maintaining flag and the sampling rate maintaining flag is set, frame interpolation is performed based on the sampling rate of the previous sampling frame of the sampling frame after the current sampling frame.

[0134] In some embodiments, the method further comprises: in the case that the sampling rate maintaining flag corresponding to the current sampling frame is not set, determining whether the sampling frame after the current sampling frame is a key frame, and in the case that the sampling frame after the current sampling frame is not a key frame, determining whether the sampling frame after the current sampling frame corresponds to the sampling rate. In some embodiments, in the case that the sampling frame after the current sampling frame is not a key frame, it is further required to determine whether the sampling frame after the current sampling frame corresponds to the sampling rate maintaining flag. If it corresponds to the sampling rate maintaining flag and the sampling rate maintaining flag is not set, it is determined whether the sampling frame after the current sampling frame corresponds to the sampling rate, and frame interpolation is performed based on the sampling rate in the case that the sampling frame after the current sampling frame corresponds to the sampling rate.

[0135] In some embodiments, the method further comprises: determining the plurality of sampling segments, the original frame number corresponding to each sampling segment and the sampling rate corresponding to each sampling segment based on the code stream data;

[0136] Performing frame interpolation based on the original frame number corresponding to each sampling segment and the sampling rate corresponding to each sampling segment to obtain the frame-interpolated segment corresponding to each sampling segment.

[0137] In some embodiments, the method further comprises: the code stream data comprises a sequence parameter set and end information; and determining whether the time domain recovery flag is set in the code stream data comprises: determining whether the time domain recovery flag is set according to the sequence parameter set; and in the case that the time domain recovery flag is set, determining the number of remaining frames comprises: in the case that the time domain recovery flag is set, determining whether the end information has target data; and in the case that the end information has target data, obtaining the remaining frame information according to the target data and determining the number of remaining frames according to the remaining frame information.

[0138] In some embodiments, the method further comprises: the bitstream data comprises a sequence parameter set and end information; in a case where the temporal restoration flag is set, determining the number of remaining frames comprises: determining whether the end information has target data; in a case where the target data exists, determining whether the temporal restoration flag is set; in a case where the temporal restoration flag is set, acquiring the remaining frame information according to the target data, and determining the number of remaining frames according to the remaining frame information.

[0139] In some embodiments, the sequence parameter set comprises a SPS, the target data comprises rbsp data, and the end information comprises an EOS (End of Sequence) or an EOB (End of Bitstream).

[0140] In some embodiments, When temporal_restoration_flag is equal to 1, if TemporalResamplingRatio[i] is not 1, put a pair of i-th and (i+1)-th temporal input pictures, and TemporalResamplingRatio[i] into the process specified by subclause 8.2.3. The (TemporalResamplingRatio[i]-1) interpolated pictures that are obtained as output of the process are included in output order in VcmOutputPicList between i-th and (i+1)-th temporal input pictures given as input to the process, which i shall be in the range of 0 to (num_temporal_input_pics-2). At last, the last temporal input picture will be duplicated num_temporal_remain times at the end of VcmOutputPicList, to restore the sequence to the original length.

[0141] It is noted that when temporal_restoration_flag = 1, if TemporalResamplingRatio[i] is not 1, then the pair of i-th and (i+1)-th temporal input pictures and TemporalResamplingRatio[i] are passed into the process specified in clause 8.2.3. The process outputs (TemporalResamplingRatio[i] - 1) pictures that are inserted in VcmOutputPicList between the i-th and (i+1)-th pictures in the output order, where i is in the range of 0 to (num_temporal_input_pics - 2). Finally, the last num_temporal_remain pictures in VcmOutputPicList are copied from the last temporal input picture to restore the original length of the sequence.

[0142] In some embodiments, when there is no target data in the end information, the remaining frame number is represented as 0, and the last frame sample frame is the last frame of the original sequence.

[0143] In some embodiments, certain frames belong to the sampling rate change flag bit which is always set to 1, such as I frames.

[0144] In some embodiments, the picture information includes a sampling rate maintenance flag and temporal sampling change rate information; the temporal sampling change rate information includes a temporal sampling change rate index.

[0145] In some embodiments, the sequence level parameter includes a temporal restoration flag and a sequence level temporal sampling rate, when the temporal restoration flag is 1, the sequence level temporal sampling rate is obtained, and if the PH does not update the temporal sampling information subsequently, the sequence level temporal sampling rate is used.

[0146] The sequence level parameter includes a temporal restoration flag and a sequence level temporal sampling rate, when the temporal restoration flag is 1, the sequence level temporal sampling rate is obtained, and if the PH does not update the temporal sampling information subsequently, the sequence level temporal sampling rate is used.

[0147] The flag bit is set to 1

[0148] In some embodiments, the picture information includes a picture header (PH).

[0149] In some embodiments, the target data includes original byte sequence payload (rbsp) data.

[0150] Based on the same inventive concept, the data processing method in the embodiments of the present disclosure is also provided, as follows. Since the principle of solving the problem of the method embodiments is similar to the above-mentioned method embodiments, the implementation of the method embodiments can be referred to the implementation of the above-mentioned method embodiments, and the repeated parts will not be described here.

[0151] The method comprises: performing frame interpolation according to a sampling rate corresponding to the sampling frame.

[0152] In some embodiments, performing frame interpolation according to the sampling rate corresponding to the sampling frame comprises:

[0153] obtaining the sampling rate corresponding to the current sampling frame;

[0154] In the case that the sampling rate corresponding to the current sampling frame is not one, performing frame interpolation based on the sampling rate, the current sampling frame and the adjacent sampling frame of the current sampling frame to obtain a second frame interpolation segment corresponding to the current sampling frame.

[0155] In some embodiments, performing frame interpolation according to the sampling rate corresponding to the sampling frame comprises:

[0156] obtaining the sampling rate corresponding to the current sampling frame;

[0157] In the case that the sampling rate corresponding to the current sampling frame is one, determining a third frame interpolation segment based on the current sampling frame.

[0158] In some embodiments, the method further comprises:

[0159] determining the number of remaining frames;

[0160] performing frame interpolation based on the number of remaining frames to obtain a first frame interpolation segment.

[0161] In some embodiments, the sequence-level parameters comprise a time domain recovery flag and a time domain maintenance flag, and in the case that both flags are set, a sequence-level sampling rate is obtained, and the sampling rate is used for frame interpolation of subsequent sequences except the remaining frames. In some embodiments, the above-mentioned setting of both flags can include that both flags are 1.

[0162] Based on the same inventive concept, the data processing method in the embodiments of the present disclosure is also provided, as follows. Since the principle of solving the problem of the method embodiments is similar to the above-mentioned method embodiments, the implementation of the method embodiments can be referred to the implementation of the above-mentioned method embodiments, and the repeated parts will not be described here.

[0163] It should be noted that the data processing method in the embodiments of the present disclosure is applied to the encoding end to encode the video to obtain the code stream data, so that the decoding end can implement any one of the data processing methods in the above-mentioned embodiments based on the code stream data.

[0164] Based on the same inventive concept, the present disclosure also provides a data processing apparatus, as follows. Since the principle of solving problems of the apparatus embodiment is similar to the above-mentioned method embodiment, the implementation of the apparatus embodiment can be referred to the implementation of the above-mentioned method embodiment, and the repeated parts will not be described here.

[0165] Figure 8 A schematic diagram of another data processing apparatus in the embodiments of the present disclosure is shown. It should be noted that the data processing apparatus in the embodiments of the present disclosure is applied to a decoding end.

[0166] As shown in Figure 8 , the data processing apparatus 700 can include a first frame padding module 710 configured to pad frames between a current sample frame and a subsequent sample frame of the current sample frame based on a target frame to obtain a first padded frame segment corresponding to the current sample frame, the target frame including at least one of the current sample frame, a previous sample frame of the current sample frame, and frames between the current sample frame and the previous sample frame of the sample frame.

[0167] In an embodiment of the present disclosure, the first frame padding module further includes a first frame padding unit configured to pad the frames between the current sample frame and the subsequent sample frame of the current sample frame based on the target frame and a sampling rate corresponding to the current sample frame to obtain the first padded frame segment corresponding to the current sample frame.

[0168] In an embodiment of the present disclosure, the apparatus further includes a first determination module configured to determine whether an extension flag bit is set.

[0169] In an embodiment of the present disclosure, the apparatus further includes a second frame padding module configured to, in a case where the extension flag bit is set, pad the frames between the current sample frame and the subsequent sample frame of the current sample frame based on the target frame to obtain the first padded frame segment corresponding to the current sample frame.

[0170] In an embodiment of the present disclosure, the apparatus further includes a second determination module configured to, in a case where the extension flag bit is set, determine whether a time domain recovery flag bit is set in the code stream data; a third determination module configured to, in a case where the time domain recovery flag bit is set, determine a number of remaining frames; and a third frame padding module configured to pad based on the number of remaining frames to obtain a second padded frame segment.

[0171] In an embodiment of the present disclosure, the apparatus further includes a third determination module configured to determine a decoded video based on the first padded frame segment and the second padded frame segment.

[0172] In one embodiment of the present disclosure, the apparatus further comprises a fourth determination module configured to determine whether the current sampling frame corresponds to a sampling rate maintaining flag in a case where the extension flag is set.

[0173] In one embodiment of the present disclosure, the apparatus further comprises a sixth determination module configured to determine whether the current sampling frame corresponds to a sampling rate maintaining flag in a case where the extension flag is set.

[0174] a seventh determination module configured to determine whether a sampling rate maintaining flag bit is set in a case where the current sampling frame corresponds to the sampling rate maintaining flag; an eighth determination module configured to determine a sampling rate corresponding to a sampling frame after the current sampling frame in a case where the sampling rate maintaining flag bit is not set; and a fourth frame supplement module configured to perform frame supplement based on the sampling rate corresponding to the sampling frame after the current sampling frame.

[0175] The data processing apparatus provided by the embodiments of the present disclosure determines whether a time domain recovery flag bit is set in the code stream data, determines the number of remaining frames in a case where the time domain recovery flag bit is set, performs frame supplement based on the number of remaining frames to obtain a first frame supplement segment. The variable up-sampling rate based coding is realized.

[0176] Based on the same inventive concept, the present disclosure further provides a data processing apparatus in the embodiments, as follows. Since the principle of the apparatus embodiments to solve the problem is similar to the above-mentioned method embodiments, the implementation of the apparatus embodiments can be referred to the implementation of the above-mentioned method embodiments, and the repeated parts will not be described here.

[0177] Figure 9 A schematic diagram of another data processing apparatus in the embodiments of the present disclosure is shown. It should be noted that the data processing apparatus in the embodiments of the present disclosure is applied to the encoding end. As shown in Figure 9 The data processing apparatus 800 can comprise an encoding module 810 configured to encode a video to obtain code stream data, so that the decoding end implements any one of the data processing methods in the decoding end based on the code stream data.

[0178] The data processing apparatus provided by the embodiments of the present disclosure can be used to execute the positioning method provided by each of the above-mentioned method embodiments, and the implementation principle and technical effects are similar. For the sake of simplicity, they will not be described here.

[0179] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method or a program product. Therefore, various aspects of the present disclosure can be embodied as a whole hardware implementation, a whole software implementation (including firmware, microcode, etc.), or an implementation of hardware and software aspects combined, which can be collectively referred to as "circuitry", "module" or "system" here.

[0180] The electronic device 900 according to this embodiment of the present disclosure will be described below with reference to Figure 9 ​ The display electronic device 900 is merely an example and should not impose any limitation on the function and use range of the embodiments of the present disclosure.

[0181] As shown in ​ The electronic device 900 is in the form of a general computing device. The components of the electronic device 900 can include, but are not limited to, the at least one processing unit 910 described above, the at least one storage unit 902 described above, and a bus 930 connecting different system components, including the storage unit 902 and the processing unit 910.

[0182] The storage unit stores program code that can be executed by the processing unit 910, so that the processing unit 910 performs the steps described in the "Exemplary Method" section above according to various exemplary embodiments of the present disclosure. For example, the processing unit 910 can perform the following steps of the above method embodiments: determining whether a time domain recovery flag is set in the bitstream data; determining the number of remaining frames in the case where the time domain recovery flag is set; and performing frame padding based on the number of remaining frames to obtain a first padded segment.

[0183] The storage unit 902 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 9021 and / or a cache memory unit 9022, and can further include a read-only memory (ROM) 9023.

[0184] The storage unit 902 can also include a program / utility 9024 having a set of program modules 9025, including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof can include implementation of a network environment.

[0185] The bus 930 can represent one or more of several types of bus structures, including a storage unit bus or storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit bus, or a local bus using any of a variety of bus architectures.

[0186] ​The electronic device 900 can also be in data communication with one or more external devices 940 (e.g., a keyboard, a pointing device, a Bluetooth device, etc.) or one or more devices that enable a user to interact with the electronic device 900 (e.g., a

[0187] Those skilled in the art will readily understand that the example embodiments described herein can be implemented by software and / or by hardware coupled with software, as described above. As such, the techniques of the present disclosure can be embodied in a software product, which can be stored in a non-volatile storage medium (e.g., a CD-ROM, a USB flash drive, a flash memory, etc.) or transmitted over a network, including one or more instructions for causing a computing device (e.g., a personal computer, a server, a network appliance, etc.) to perform the steps described in the above embodiments of the present disclosure.

[0188] In the example embodiments of the present disclosure, a computer readable storage medium can also be provided, which can be a readable signal medium or a readable storage medium. The program product stored thereon can implement the above method of the present disclosure. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing an end device to perform the steps described in the above "Example Methods" section according to various example embodiments of the present disclosure when the program product is run on the end device.

[0189] More specific examples of the computer readable storage medium in the present disclosure can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0190] In this disclosure, a computer readable storage medium can include a data signal transported over a carrier wave and can take the form of any appropriate medium including but not limited to phase or frequency modulation techniques, as well as other techniques. Still yet, a computer readable medium can be any medium that can be read by a computer including magnetic storage media (e.g., floppy disks); optical storage media (e.g., CD-ROMs); and tangible storage media that is impregnated with a substance such as gold and / or silver; and others as will occur to those skilled in the art. A computer readable medium can also be any medium that can be used to store or transfer the desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0191] Optionally, program code embodied on a computer readable medium can be transmitted by any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0192] In one or more embodiments, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over a computer-readable medium, such as a non-transitory computer-readable medium, as described above. The term "computer-readable medium" includes both computer storage media and communication media, including but not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store or transfer the desired information or instructions, and which can be accessed by a computer.

[0193] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, such a division is not mandatory. Indeed, features and functionalities of two or more modules or units described above can be embodied in one module or unit according to embodiments of the present disclosure. Conversely, features and functionalities of one module or unit described above can be further divided into multiple modules or units.

[0194] Moreover, although the various steps of the methods in the present disclosure are described in a particular order in the figures, this is not required or implied as to the order of execution of the steps, nor is it required that all of the steps be executed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, one step can be broken into multiple steps, etc.

[0195] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the methods according to the embodiments of the present disclosure.

[0196] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure following the general principles thereof and including such departures from the present disclosure that come within known use or custom of the art to which the present disclosure pertains. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A data processing method, characterized by, Applied to a decoding end, comprising: Based on the target frame, the current sampling frame and the frame between the subsequent sampling frame of the current sampling frame are supplemented, and the first supplementary frame segment corresponding to the current sampling frame is obtained.

2. The data processing method of claim 1, wherein, The method further comprises: Determine whether the extension flag bit is set.

3. The data processing method of claim 1, wherein, The method further comprises: In the case where the extension flag bit is set, based on the target frame, the current sampling frame and the frame between the subsequent sampling frame of the current sampling frame are supplemented, and the first supplementary frame segment corresponding to the current sampling frame is obtained.

4. The data processing method according to claim 3, characterized in that, The method further comprises: In the case where the extension flag bit is set, determine whether the time domain recovery flag bit is set in the code stream data; 5. The data processing method according to claim 4, characterized in that, In the case where the time domain recovery flag bit is set, determine the number of remaining frames; Based on the number of remaining frames, the frame is supplemented to obtain the second supplementary frame segment. The method further comprises: Based on the first supplementary frame segment and the second supplementary frame segment, the decoded video is determined.

6. The data processing method according to claim 5, characterized in that, The method further comprises: In the case where the extension flag bit is set, determine whether the current sampling frame corresponds to the sampling rate retention flag; 7. The data processing method according to claim 3, characterized in that, In the case where the current sampling frame corresponds to the sampling rate retention flag, determine whether the sampling rate retention flag bit is set; In the case where the sampling rate retention flag bit is set, the frame is supplemented based on the sampling rate corresponding to the current sampling frame. The method further comprises: In the case where the extension flag bit is set, determine whether the current sampling frame corresponds to the sampling rate retention flag; 8. The data processing method according to claim 3, characterized in that, In the case where the current sampling frame corresponds to the sampling rate retention flag, determine whether the sampling rate retention flag bit is set; In the case where the sampling rate retention flag bit is not set, determine the sampling rate corresponding to the sampling frame after the current sampling frame; Based on the sampling rate corresponding to the sampling frame after the current sampling frame, the frame is supplemented. Applied to an encoding end, comprising: Encoding the video to obtain code stream data, so that the decoding end implements any one of the data processing methods in claims 1-8 based on the code stream data.

9. A data processing method, characterized by, Applied to a decoding end, comprising: A first supplementary frame module is configured to supplement the frame between the current sampling frame and the subsequent sampling frame of the current sampling frame based on the target frame, and obtain the first supplementary frame segment corresponding to the current sampling frame.

10. A data processing apparatus, characterized by, Applied to an encoding end, comprising: An encoding module is configured to encode the video to obtain code stream data, so that the decoding end implements any one of the data processing methods in claims 1-9 based on the code stream data.

11. A data processing apparatus, characterized by Comprising: A processor; 12. An electronic device, comprising: And A memory for storing executable instructions of the processor; ​ ​ The processor is configured to execute the data processing method of any one of claims 1-9 by executing the executable instructions.

13. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the data processing method of any one of claims 1-9.

Images