Video processing method, device and storage medium

Abstract

An embodiment of the present invention provides a video processing method, device and storage medium, the method comprising: determining a first channel indicator for transmitting a coded video; determining the number of network layers of a coding network based on the first channel indicator; and using the coding network to encode a target frame of the video to be encoded. The method is applied to the field of video processing technology, determining the number of network layers of the coding network based on the determined first channel indicator for transmitting the coded video, using the coding network to encode the target frame of the video to be encoded, and dynamically adjusting the network layer of the coding network based on the channel quality status, thereby adjusting the number of frames encoded and decoded during the video encoding process, thereby realizing video encoding and decoding that is dynamically adjusted according to the channel status.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a video processing method, device and storage medium. Background Art

[0002] Traditional algorithms divide frames into key (I) frames and reference (P or B) frames in the temporal dimension. Reference frames compress data by recording relative changes, maintaining a strict hierarchical relationship. Spatially, frames are then divided into blocks for post-processing, using Shannon's source coding theory as a framework for compression based on statistical characteristics. After image frames are processed by the encoding network, entropy coding is used to further compress redundant information. However, traditional video compression involves multiple steps—encoding, quantization, decoding, and entropy coding—designed separately and independently. This coding approach is not only labor-intensive but also relies on carefully designed and independent steps, making it impossible to jointly optimize the encoder and decoder. Summary of the Invention

[0003] Embodiments of the present invention provide a video processing method, device, and storage medium.

[0004] The technical solution of the embodiment of the present invention is achieved as follows:

[0005] An embodiment of the present invention provides a video processing method, the method comprising:

[0006] Determining a first channel indicator for the transmitted encoded video;

[0007] determining the number of network layers of the coding network based on the first channel indicator;

[0008] The encoding network is used to encode a target frame of the video to be encoded.

[0009] In the above solution, determining the first channel indicator of the transmission-encoded video includes:

[0010] Determine a fluctuation mean of a channel quality measurement value within a first preset time period; wherein the channel quality measurement value includes: received signal power RSRP, received signal quality RSRQ and / or signal-to-noise ratio SNR;

[0011] A first channel indicator is determined according to the fluctuation mean.

[0012] In the above solution, encoding the target frame of the video to be encoded using the encoding network includes:

[0013] Determining, according to the length of the GOP of the video to be encoded and the nth network layer, an nth target frame and a previous frame and a next frame of the nth target frame to be encoded by the nth network layer; wherein n is a positive integer less than N; and N is the number of network layers;

[0014] encoding the nth target frame based on a previous frame and a following frame of the nth target frame to obtain encoding information of the nth target frame;

[0015] The coding information of the nth target frame is input to the (n+1)th network layer.

[0016] In the above solution, encoding the nth target frame based on the previous frame and the next frame of the nth target frame to obtain the encoding information code of the nth target frame includes:

[0017] Performing motion estimation on a previous frame of the n-th target frame and the n-th target frame to obtain a first motion feature;

[0018] Performing motion estimation on a subsequent frame of the n-th target frame and the n-th target frame to obtain a second motion feature;

[0019] The n-th target frame is encoded based on the first motion feature and the second motion feature to obtain encoding information of the n-th target frame.

[0020] In the above solution, the method further includes:

[0021] Extracting a first image feature of a previous frame of an nth target frame of the to-be-encoded video, and extracting a second image feature of a subsequent frame of the nth target frame;

[0022] The nth target frame is encoded according to the first image feature, the first motion feature, the second image feature, and the second motion feature to obtain encoding information of the nth target frame.

[0023] In the above solution, encoding the target frame of the video to be encoded using the encoding network further includes:

[0024] Determining the channel quality measurement value and network type of the nth network layer within a second preset time period;

[0025] Determining an nth second channel indicator for transmitting the encoded video according to the channel quality measurement value and the network type;

[0026] The nth second channel indicator is input to the n+1th network layer, wherein the second channel indicator is used by the n+1th network layer to encode the n+1th target frame.

[0027] In the above solution, determining the nth second channel indicator for transmitting the encoded video according to the channel quality measurement value and the network type includes:

[0028] Establishing a channel state node based on the network type and the channel quality measurement value within a second preset time period;

[0029] Establishing an nth adjacency matrix according to the channel state node;

[0030] The nth second channel index is determined according to the n-1th adjacency matrix and the nth adjacency matrix.

[0031] In the above solution, the method further includes:

[0032] decoding the encoded information output by the N network layers based on the second channel indicator to obtain decoded information;

[0033] Verifying whether the decoded information is correct;

[0034] If correct, it is determined that the encoding of the video to be encoded is successful.

[0035] An embodiment of the present invention further provides a video processing device, the device comprising: a first determining module, a second determining module and an encoding module;

[0036] The first determining module is used to determine a first channel indicator for transmitting the encoded video;

[0037] The second determining module is configured to determine the number of network layers of the coding network based on the first channel indicator;

[0038] The encoding module is configured to encode a target frame of the video to be encoded using the encoding network.

[0039] In the above solution, the first determining module is further configured to:

[0040] Determine a fluctuation mean of a channel quality measurement value within a first preset time period; wherein the channel quality measurement value includes: received signal power RSRP, received signal quality RSRQ and / or signal-to-noise ratio SNR;

[0041] A first channel indicator is determined according to the fluctuation mean.

[0042] In the above solution, the encoding module is further used to:

[0043] Determining, according to the length of the GOP of the video to be encoded and the nth network layer, an nth target frame and a previous frame and a next frame of the nth target frame to be encoded by the nth network layer; wherein n is a positive integer less than N; and N is the number of network layers;

[0044] encoding the nth target frame based on a previous frame and a following frame of the nth target frame to obtain encoding information of the nth target frame;

[0045] The coding information of the nth target frame is input to the (n+1)th network layer.

[0046] In the above solution, the encoding module is further used to:

[0047] Performing motion estimation on a previous frame of the n-th target frame and the n-th target frame to obtain a first motion feature;

[0048] Performing motion estimation on a subsequent frame of the n-th target frame and the n-th target frame to obtain a second motion feature;

[0049] The n-th target frame is encoded based on the first motion feature and the second motion feature to obtain encoding information of the n-th target frame.

[0050] In the above solution, the encoding module is further used to:

[0051] Extracting a first image feature of a previous frame of an nth target frame of the to-be-encoded video, and extracting a second image feature of a subsequent frame of the nth target frame;

[0052] The nth target frame is encoded according to the first image feature, the first motion feature, the second image feature, and the second motion feature to obtain encoding information of the nth target frame.

[0053] In the above solution, the device further includes: a third determination module, a fourth determination module and an input module;

[0054] The third determining module is configured to determine the channel quality measurement value and network type of the nth network layer within a second preset time period;

[0055] The fourth determining module is configured to determine an nth second channel indicator for transmitting the encoded video according to the channel quality measurement value and the network type;

[0056] The input module is used to input the nth second channel indicator to the n+1th network layer, wherein the second channel indicator is used for the n+1th network layer to encode the n+1th target frame.

[0057] In the above solution, the fourth determining module is further configured to:

[0058] Establishing a channel state node based on the network type and the channel quality measurement value within a second preset time period;

[0059] Establishing an nth adjacency matrix according to the channel state node;

[0060] The nth second channel index is determined according to the n-1th adjacency matrix and the nth adjacency matrix.

[0061] In the above solution, the device further comprises: a decoding module and a verification module;

[0062] The decoding module is configured to decode the encoded information output by the N network layers based on the second channel indicator to obtain decoded information;

[0063] The verification module is used to verify whether the decoding information is correct; if correct, it is determined that the encoding of the video to be encoded is successful.

[0064] An embodiment of the present invention further provides a video processing device, comprising: a processor and a memory for storing a computer program that can be run on the processor; wherein the processor is configured to execute the steps of any one of the above-mentioned video processing methods when running the computer program.

[0065] An embodiment of the present invention further provides a computer storage medium, characterized in that the computer storage medium stores computer-executable instructions; after the computer-executable instructions are executed by a processor, the steps of the above-mentioned video processing method can be implemented.

[0066] In an embodiment of the present invention, the number of network layers of a coding network is determined based on a determined first channel indicator of a video after transmission encoding, the coding network is used to encode a target frame of a video to be encoded, and the network layer of the coding network is dynamically adjusted based on the channel quality status, thereby adjusting the number of frames encoded and decoded during the video encoding process, thereby achieving video encoding and decoding that is dynamically adjusted according to the channel status. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0068] Figure 1 A schematic flow chart of the video processing method provided by the present invention;

[0069] Figure 2 A schematic flow chart of the video processing method provided by the present invention;

[0070] Figure 3A schematic diagram of the information interaction flow of the encoding network in the video processing method provided by the present invention;

[0071] Figure 4 A schematic diagram of the information interaction flow of the decoding network in the video processing method provided by the present invention;

[0072] Figure 5 A schematic diagram of the structure of a video processing device provided by an embodiment of the present invention;

[0073] Figure 6 A schematic structural diagram of another video processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0074] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0075] Unless otherwise specified, the embodiments and features of the embodiments in this application may be combined with each other in any manner. The steps shown in the flowcharts of the accompanying drawings may be executed in a computer system, such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown.

[0076] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are for the purpose of describing embodiments of the present invention only and are not intended to limit the present invention.

[0077] Video content is naturally temporally correlated, with only minor changes between frames. Therefore, existing mainstream video codecs (such as H.263, H.264, or HEVC (H.265)) take temporal correlation into account. These algorithms consist of frame type analysis, intra-frame / inter-frame prediction, transformation, quantization, filtering, and entropy coding. These deep neural network-based image interpolation and extrapolation techniques attempt to compress video in the temporal dimension.

[0078] Image interpolation attempts to compute an unseen frame between two reference frames. Most image insertion networks build on the structure of an encoder and decoder, which involves shifting pixels in the temporal dimension, estimating spatial variance, convolution kernels, and generating a flow field. These methods are combined with forward and backward prediction to form the final output. Image extrapolation algorithms attempt to infer future frames from existing frames, which is a more challenging approach. Both image interpolation and extrapolation work well for small time steps and are commonly used to create slow-motion video frames or infer the content of frames one second in the future.

[0079] The channel state indicator involved in this application reflects the channel properties of the communication link, which describes the state of the channel through which the signal is transmitted. In the 5G system, link adaptation in both the uplink and downlink directions is controlled by the base station. The 5G base station gNB selects the most appropriate modulation and coding scheme under the current channel conditions based on the Channel Quality Indicator (CQI) reported by the terminal to meet the spectrum efficiency under a specific bit error rate and packet error rate, ensuring maximum data rate.

[0080] 5G's adaptive schemes based on channel metrics primarily include adaptive modulation and coding, power control, hybrid automatic repeat request (HART), and channel selective scheduling. Adaptive modulation and coding adjusts the coding method and coding rate based on channel variations. When channel quality is good, the modulation level and coding rate are increased to increase system throughput; when channel quality is poor, the modulation level and coding rate are reduced to increase transmission reliability. Power control reduces transmit power when channel conditions are good and increases it when conditions are poor. Hybrid automatic repeat request (HART) adjusts data redundancy to achieve retransmission / combining gain at the receiving end. Channel selective scheduling selects time-frequency resources with better channel conditions for data transmission based on wireless channel measurement results.

[0081] An embodiment of the present invention provides a video processing method. Figure 1 Schematic diagram of the flow of the video processing method provided by the present invention; Figure 1 As shown, the method includes:

[0082] Step S101: determining a first channel index for transmitting the encoded video;

[0083] Step S102: determining the number of network layers of the coding network based on the first channel indicator;

[0084] Step S103: using the encoding network to encode the target frame of the video to be encoded.

[0085] In the above-mentioned step S101, determining the first channel indicator of the transmission-encoded video includes: determining a fluctuation mean of a channel quality measurement value within a first preset time period; wherein the channel quality measurement value includes: received signal power RSRP, received signal quality RSRQ and / or signal-to-noise ratio SNR; and determining the first channel indicator based on the fluctuation mean.

[0086] Specifically, the first preset time period may be a time period set by the coding network or a time period defined by the user, which is not specifically limited here;

[0087] Determine the fluctuation means of received signal power RSRP, received signal quality RSRQ, and signal-to-noise ratio SNR within a first preset time period respectively; sum the fluctuation means of RSRP, the fluctuation means of RSRQ, and the fluctuation means of SNR to determine a first channel indicator.

[0088] Specifically, all RSRP values, RSRQ values, and SNR values ​​measured within a first preset time period are determined; based on all the measured RSRP values, a fluctuation mean of RSRP is calculated; based on all the measured RSRQ values, a fluctuation mean of RSRQ is calculated; based on all the measured SNR values, a fluctuation mean of SNR is calculated;

[0089] Summing the fluctuation mean RSRP', the fluctuation mean RSRQ', and the fluctuation mean SNR' to determine a first channel indicator α;

[0090] α=sum(RSRP', RSRQ', SNR').

[0091] In the above step S102, according to a preset mapping relationship between the first channel indicator and the number of network layers, the number of network layers of the coding network is determined based on the first channel indicator;

[0092] Specifically, according to the first channel indicator value, the first channel indicator value range corresponding to the number of network layers in the mapping relationship is compared; the first channel indicator value range to which the first channel indicator value belongs is determined, and the number of network layers corresponding to the first channel indicator value is determined.

[0093] It should be noted that the first channel index is proportional to the number of network layers; the larger the first channel index is, the more network layers there are; and the smaller the first channel index is, the fewer network layers there are.

[0094] In this embodiment, a first channel indicator of a channel for transmitting encoded video is determined, and based on the determined first channel indicator, the number of network layers of a coding network is determined, and the coding network is used to encode target frames of the video to be encoded. In this way, the network layer of the coding network is dynamically adjusted according to the channel quality status, thereby adjusting the number of frames to be encoded and decoded during the video encoding process, thereby achieving adaptive video coding. The network structure of the video coding network is dynamically adjusted according to the channel status. When the channel quality is good, the modulation level and coding rate are increased to increase the system throughput; when the channel quality is poor, the modulation level and coding rate are reduced to increase transmission reliability.

[0095] Furthermore, the encoding of the target frame of the video to be encoded using the encoding network includes: determining the nth target frame and the previous frame and the next frame of the nth target frame encoded by the nth network layer according to the length of the GOP of the video to be encoded and the nth network layer; wherein, n is a positive integer less than N; N is the number of network layers; based on the previous frame and the next frame of the nth target frame, encoding the nth target frame to obtain the encoding information of the nth target frame; and inputting the encoding information of the nth target frame into the (n+1)th network layer.

[0096] Specifically, the GOP strategy affects encoding quality. A GOP is a group of consecutive pictures. The first picture in a GOP is a key frame (I frame). The I frame ensures that the GOP can be decoded independently without referencing other pictures.

[0097] Specifically, N is the determined number of network layers. In some embodiments, when N is equal to 1, n is equal to N is equal to 1; in other embodiments, when N is a positive integer greater than 1, the nth network layer is any one of the 1st to Nth network layers; here, n is a positive integer less than N; N can be any positive integer value, such as 2, 3, 4 or 5, etc.

[0098] To be more specific, assume that:

[0099] The length of the GOP of the video to be encoded is recorded as M, where M can be 12; the number of network layers N can be 3;

[0100] When n is equal to 1, the previous frame of the first target frame encoded by the first network layer is: the first frame in the current GOP of the video to be encoded; the following frame of the first target frame encoded by the first network layer is: the first frame in the next GOP of the video to be encoded, that is, the 13th frame; the first target frame is: the 7th frame in the current GOP;

[0101] The first target frame is the 7th frame; based on the 1st frame, which is the previous frame, and the 13th frame, which is the next frame, of the 1st target frame, the 7th frame is encoded to obtain the encoding information of the 1st target frame; the encoding information of the 1st target frame is input into the 2nd network layer.

[0102] When n is equal to 2, the previous frame of the second target frame encoded by the second network layer is: the first frame in the current GOP of the video to be encoded; the following frame of the second target frame is: the first target frame, that is, the seventh frame in the current GOP; the second target frame is determined to be: the fourth frame in the current GOP;

[0103] The previous frame of the second target frame encoded by the second network layer is the 7th frame in the current GOP of the video to be encoded; the following frame of the second target frame is the 13th frame, which is the first frame in the next GOP of the video to be encoded; and the second target frame is determined to be the 10th frame in the current GOP.

[0104] The second target frames are the 4th and 10th frames; based on the upper context frames 1st and 7th frames of the second target frame, and the lower context frames 7th and 13th frames of the second target frame, the second target frames, i.e., the 4th and 10th frames, are encoded to obtain the encoding information of the second target frame; the encoding information of the second target frame is input into the 3rd network layer.

[0105] When n is equal to 3, the previous frame of the third target frame encoded by the third network layer is: the first frame in the current GOP of the video to be encoded; the following frame of the third target frame is: the fourth frame in the second target frame; the third target frame is determined to be: the second and third frames in the current GOP;

[0106] The previous frame of the third target frame encoded by the third network layer is the fourth frame in the second target frame; the following frame of the third target frame is the seventh frame in the current GOP; the second target frame is determined to be the fifth and sixth frames in the current GOP;

[0107] The previous frame of the third target frame encoded by the third network layer is the seventh frame in the current GOP; the following frame of the third target frame is the tenth frame in the second target frame; the second target frame is determined to be the eighth and ninth frames in the current GOP;

[0108] The previous frame of the third target frame encoded by the third network layer is the 10th frame in the second target frame; the following frame of the third target frame is the 13th frame, which is the first frame in the next GOP of the video to be encoded; the third target frame is determined to be the 11th and 12th frames in the current GOP;

[0109] The third target frame is frame 2, frame 3, frame 5, frame 6, frame 8, frame 9, frame 11 and frame 12; based on the previous frame and the following frame of the third target frame: frame 1, frame 4, frame 7, frame 10 and frame 13, the third target frame, i.e. frame 2, frame 3, frame 5, frame 6, frame 8, frame 9, frame 11 and frame 12, is encoded to obtain the encoding information of the third target frame; the encoding information of the third target frame is input into the third network layer.

[0110] Furthermore, encoding the nth target frame based on the previous frame and the following frame of the nth target frame to obtain the coding information code of the nth target frame includes: performing motion estimation on the previous frame of the nth target frame and the nth target frame to obtain a first motion feature; performing motion estimation on the following frame of the nth target frame and the nth target frame to obtain a second motion feature; encoding the nth target frame based on the first motion feature and the second motion feature to obtain the coding information of the nth target frame.

[0111] Specifically, the motion features are obtained through the optical flow estimation method, and the motion features of the video image are determined by estimating the movement amount of the same object in the video image to the next frame; optical flow estimation is generally divided into sparse optical flow estimation and dense optical flow estimation; sparse optical flow estimation refers to selecting some feature points in the image for optical flow estimation and tracking, while dense optical flow estimation is to describe the optical flow of each pixel in the image.

[0112] Specifically, in some embodiments, the nth target frame is divided into multiple non-overlapping macroblocks, and the displacement of all pixels in the macroblock is the same. Then, based on each macroblock, the macroblock most similar to the current macroblock of the nth target frame is queried from a specific search range in the previous frame and the following frame of the nth target frame according to a certain matching criterion; the macroblock in the previous frame and the following frame of the nth target frame that is most similar to the current macroblock is called a matching block; the relative displacement between the pixels in the matching block and the pixels of the current macroblock in the nth target frame is called a motion vector.

[0113] Determining a macroblock similar to a macroblock in the nth target frame from a frame preceding the nth target frame as a matching block; determining a relative displacement between pixels in the matching block in the frame preceding the nth target frame and pixels in the macroblock in the nth target frame to obtain a first motion feature;

[0114] A macroblock similar to a macroblock in the nth target frame is determined from a subsequent frame of the nth target frame and is determined as a matching block; a relative displacement between pixels in the matching block in the subsequent frame of the nth target frame and pixels in the macroblock in the nth target frame is determined to obtain a second motion feature; and the nth target frame is encoded based on the first motion feature and the second motion feature to obtain encoding information of the nth target frame.

[0115] In other embodiments, a specific macroblock or specific pixel point in the nth target frame is determined, and based on the position of the specific macroblock or specific pixel point in the previous frame and the following frame of the nth target frame, the relative displacement, i.e., the motion vector, of the specific macroblock or specific pixel point in the previous frame and the following frame of the nth target frame and the specific macroblock or specific pixel point in the nth target frame is determined.

[0116] Determine a specific macroblock or specific pixel in an nth target frame, and determine a position of the specific macroblock or specific pixel from a frame preceding the nth target frame; determine a relative displacement of the specific macroblock or specific pixel between the frame preceding the nth target frame and the nth target frame to obtain a first motion feature;

[0117] The method comprises the steps of: determining the position of the specific macroblock or specific pixel point from the following frame of the nth target frame; determining the relative displacement of the specific macroblock or specific pixel point between the following frame of the nth target frame and the nth target frame to obtain a second motion feature; encoding the nth target frame according to the first motion feature and the second motion feature to obtain encoding information of the nth target frame.

[0118] Furthermore, the first image feature of the previous frame of the nth target frame of the video to be encoded is extracted, and the second image feature of the following frame of the nth target frame is extracted; the nth target frame is encoded according to the first image feature, the first motion feature, the second image feature and the second motion feature to obtain the encoding information of the nth target frame.

[0119] Specifically, the first image feature includes at least: semantic information of a previous frame of the nth target frame and pixel information of a frame adjacent to the previous frame of the nth target frame; the second image feature includes at least: semantic information of a subsequent frame of the nth target frame and pixel information of a frame adjacent to the subsequent frame of the nth target frame;

[0120] Specifically, the image features of the upper and lower frames of the nth target frame of the video to be encoded are respectively extracted through the Unet network to generate a first image feature and a second image feature; wherein the first image feature includes: the semantic information of the upper frame and the pixel information of the adjacent frame adjacent to the upper frame; the second image feature includes: the semantic information of the lower frame and the pixel information of the adjacent frame adjacent to the lower frame; through the encoding network, the corresponding nth target frame is encoded according to the first image feature, the first motion feature, the second image feature and the second motion feature to obtain the encoding information of the nth target frame.

[0121] Furthermore, the use of the coding network to encode the target frame of the video to be encoded also includes: determining the channel quality measurement value and network type of the nth network layer within a second preset time period; determining the nth second channel indicator of the video after transmission encoding based on the channel quality measurement value and network type; inputting the nth second channel indicator to the n+1th network layer, wherein the second channel indicator is used for the n+1th network layer to encode the n+1th target frame.

[0122] Specifically, the second preset time period can be a time period set by the coding network or a time period customized by the user, which is not specifically limited here; the network type includes: 3G network, 4G network or 5G network, etc.; the channel quality measurement value includes: received signal power RSRP, received signal quality RSRQ and / or signal-to-noise ratio SNR.

[0123] Furthermore, determining the nth second channel indicator of the transmitted encoded video based on the channel quality measurement value and the network type includes: establishing a channel status node based on the network type and the channel quality measurement value within a second preset time period; establishing an nth adjacency matrix based on the channel status node; determining the nth second channel indicator based on the n-1th adjacency matrix and the nth adjacency matrix; transmitting the nth second channel indicator to the encoding network, performing a convolution operation on the nth target frame to encode the nth target frame, and obtaining encoding information of the nth target frame.

[0124] Specifically, each channel quality measurement value determined within the second preset time period is determined as a channel state node; the adjacency matrix is ​​a collection of channel state nodes of the same network type within a continuous time period; it should be noted that when the network type is switched once, an adjacency matrix is ​​generated.

[0125] Specifically, each channel quality measurement value of the nth network layer within a second preset time period is determined as a channel state node; the channel state nodes of the same network type within consecutive time periods are stored in a one-dimensional matrix P, P = [H1 H2 ... H U-1 H U ], where U identifies the number of channel state nodes with the same network type in the continuous time period; based on the correlation between the channel state nodes at each moment and their adjacent channel state nodes, an nth adjacency matrix is ​​generated according to the channel state nodes with the same network type in the continuous time period, and the nth adjacency matrix is ​​represented by A. h 1,U For the one-dimensional matrix H1 and H U The connection relationship (edge ​​or arc) data between two channel state nodes. The channel state nodes with a connection relationship are represented as 1, and the others are represented as 0.

[0126] Performing feature aggregation on the channel state nodes with a connection relationship in the nth adjacency matrix to generate a feature aggregation vector of the target channel state node; performing feature aggregation on the channel state nodes with a connection relationship in the n-1th adjacency matrix to generate a feature aggregation vector of the neighbor channel state nodes; and aggregating the feature aggregation vectors of the neighbor channel state nodes and the feature aggregation vector of the target channel state node to generate the nth second channel index;

[0127] The feature aggregation vector of the neighbor channel state node generated based on the n-1th adjacency matrix is ​​marked as Wherein, N(v) is the set of neighbor nodes in the n-1th adjacency matrix; the feature aggregation vector of the target channel state node generated based on the nth adjacency matrix is ​​marked as: The nth second channel index is marked as Z (n) , Where A is the nth adjacency matrix, Z is the second channel index, D is the normalized matrix of A, and W is the weight to be optimized;

[0128] The nth second channel indicator is transmitted to the deep convolutional neural network (Inception) v4 network in the encoding network, and a convolution operation is performed on the nth target frame to encode the nth target frame to obtain the encoding information of the nth target frame. The Inception v4 network here adds the ResNet method to the original Inception method to improve the training speed of the neural network and enhance the performance.

[0129] It should be noted that in this embodiment, different adjacency matrices are constructed according to the network type and integrated into the coding network, so that the coding network has the ability to distinguish network types and classify and build maps when the network is unstable or the network type switches. In addition, based on the determined channel quality and network type, a graph structure is constructed with the time dimension as the connection, so as to determine the second channel indicator to describe the channel state of the short time slice, and realize the combination of the channel state and the coding network to adjust the brightness, color richness and other factors that directly affect the coding efficiency of a single frame of the picture during the encoding process.

[0130] Furthermore, encoding information output by N network layers is decoded based on the second channel indicator to obtain decoding information; and it is verified whether the decoding information is correct; if correct, it is determined that the encoding of the video to be encoded is successful.

[0131] Specifically, the encoding network outputs encoding information output by N network layers and the second channel indicator in the N network layers; wherein the encoding information includes information such as the first image feature, the first motion feature, the second image feature, and the second motion feature;

[0132] Specifically, the decoding network decodes the coded information output by the corresponding network layer according to the second channel indicators in the N network layers to generate N decoded information.

[0133] In some embodiments, the similarity between the video restored based on N decoding information and the video to be encoded is determined; if the similarity is greater than a preset threshold, it is determined that the decoding information is correct and the encoding of the video to be encoded is successful; if the similarity is less than the preset threshold, it is determined that the decoding information is incorrect and the encoding of the video to be encoded fails.

[0134] If the encoding is successful, the successfully encoded target video is sent to the receiving device.

[0135] If encoding fails, you need to re-encode.

[0136] In other embodiments, the encoded information output by the corresponding network layer is decoded according to the second channel indicators in the N network layers. If decoding information is generated, it proves that the encoding of the video to be encoded is successful; if decoding information cannot be generated, it proves that the encoding of the video to be encoded has failed.

[0137] In this embodiment, the network level of the coding network is dynamically adjusted based on the channel quality status, thereby adjusting the number of frames encoded and decoded during the video encoding process, achieving adaptive video coding. The network structure of the video coding network is dynamically adjusted based on the channel status. When the channel quality is good, the modulation level and coding rate are increased to increase the system throughput; when the channel quality is poor, the modulation level and coding rate are reduced to increase transmission reliability. In addition, different adjacency matrices are constructed based on the network type and integrated into the coding network. This enables the coding network to distinguish network types and classify and construct maps in the event of network instability or network type switching. Based on the determined channel quality and network type, a graph structure is constructed with the time dimension as the connection, thereby determining a second channel indicator to describe the channel status of a short time slice. This realizes the combination of the channel status and the coding network to adjust the brightness and color richness of the image during the encoding process, which directly affect the coding efficiency of a single frame.

[0138] the following, Figure 2 is a flow chart of the video processing method, Figure 3 is a schematic diagram of the information interaction flow of the encoding network in the video processing method, Figure 4 FIG1 is a schematic diagram of the information interaction flow of the decoding network in the video processing method; and FIG1 is a specific example to illustrate the communication method according to the embodiment of the present invention.

[0139] Step 1: Calculate the first channel index;

[0140] Determine a fluctuation mean of channel quality measurement values ​​within a step size of [t-3, t+3] at a current time t; wherein the channel quality measurement values ​​include received signal power RSRP, received signal quality RSRQ, and signal-to-noise ratio SNR; determine a first channel indicator α based on the fluctuation mean of the channel quality measurement values; wherein the first channel indicator determines the tier of the currently used codec and is calculated as follows:

[0141]

[0142] The level in the above formula corresponds to the coding network level. As the level increases, the channel quality improves and more frame transmissions can be supported; otherwise, the channel quality decreases.

[0143] Step 2: Determine the target frame at each level and the context frame corresponding to the target frame;

[0144] Assume that the length of the GOP of the video to be encoded is recorded as M, where M can be 12; the number of network layers N can be 3;

[0145] When n is equal to 1, the first target frame is: the 7th frame in the current GOP;

[0146] The previous frame of the first target frame encoded by the first network layer is the first frame in the current GOP of the video to be encoded; the following frame of the first target frame encoded by the first network layer is the first frame in the next GOP of the video to be encoded, that is, the 13th frame; the first target frame is the 7th frame in the current GOP;

[0147] When n is equal to 2, the second target frame is the 4th frame and the 10th frame;

[0148] The previous frame of the second target frame encoded by the second network layer is: the first frame in the current GOP of the video to be encoded; the following frame of the second target frame is: the first target frame, that is, the seventh frame in the current GOP; the second target frame is determined to be: the fourth frame in the current GOP;

[0149] The previous frame of the second target frame encoded by the second network layer is the 7th frame in the current GOP of the video to be encoded; the following frame of the second target frame is the 13th frame, which is the first frame in the next GOP of the video to be encoded; and the second target frame is determined to be the 10th frame in the current GOP.

[0150] When n is equal to 3, the third target frame is the 2nd frame, the 3rd frame, the 5th frame, the 6th frame, the 8th frame, the 9th frame, the 11th frame and the 12th frame;

[0151] The previous frame of the third target frame encoded by the third network layer is: the first frame in the current GOP of the video to be encoded; the following frame of the third target frame is: the fourth frame in the second target frame; the third target frame is determined to be: the second and third frames in the current GOP;

[0152] The previous frame of the third target frame encoded by the third network layer is the fourth frame in the second target frame; the following frame of the third target frame is the seventh frame in the current GOP; the second target frame is determined to be the fifth and sixth frames in the current GOP;

[0153] The previous frame of the third target frame encoded by the third network layer is the seventh frame in the current GOP; the following frame of the third target frame is the tenth frame in the second target frame; the second target frame is determined to be the eighth and ninth frames in the current GOP;

[0154] The previous frame of the third target frame encoded by the third network layer is: the 10th frame in the second target frame; the following frame of the third target frame is: the 1st frame in the next GOP of the video to be encoded, that is, the 13th frame; the third target frame is determined to be: the 11th frame and the 12th frame in the current GOP.

[0155] Step 3: Perform motion estimation on the context frame and target frame in the three levels, and input the motion estimation results into the encoding network in the form of gradients.

[0156] Determining a macroblock similar to a macroblock in the nth target frame from a frame preceding the nth target frame as a matching block; determining a relative displacement between pixels in the matching block in the frame preceding the nth target frame and pixels in the macroblock in the nth target frame to obtain a first motion feature;

[0157] determining, from a subsequent frame of the nth target frame, a macroblock similar to the macroblock in the nth target frame as a matching block; determining a relative displacement between pixels in the matching block in the subsequent frame of the nth target frame and pixels in the macroblock in the nth target frame to obtain a second motion feature;

[0158] The first motion feature and the second motion feature are input into a coding network, and the n-th target frame is encoded to obtain coding information of the n-th target frame.

[0159] The calculation process is as follows. Assume that at time t, the object at pixel (x, y) is located at position (x+u, y+v) at time t+Δt. I(x, y, t) = I(x+u, y+v, t+Δt). After Taylor expansion, the partial derivatives of the image brightness in the x-axis and y-axis directions at the pixel point are expressed in matrix form, that is, the gradient I' in the x-direction of the image x , the gradient in the y direction I' y , the formula is as follows:

[0160] Step 4: Use the Unet network to extract single-frame image features from the previous frame features and the following frame features of the nth target frame of the video to be encoded, and generate the first image features and the second image features;

[0161] Extracting image features of a previous frame and a next frame of the nth target frame of the video to be encoded respectively through a Unet network to generate a first image feature and a second image feature; wherein the first image feature includes: semantic information of the previous frame and pixel information of adjacent frames adjacent to the previous frame; and the second image feature includes: semantic information of the next frame and pixel information of adjacent frames adjacent to the next frame;

[0162] Step 5: Calculate the second channel index.

[0163] Step 5.1: Differentiate by network type and establish a graph structure that distinguishes network types in non-Euclidean space using temporal proximity as edges.

[0164] The graph structure contains two types of connection relationships: temporally adjacent connections and network-type connection sets. Temporally adjacent connections exist only between data of the same network type and temporally adjacent to each other, describing the temporal correlation of the data. Network-type connections are the collection of temporally adjacent connections of the same network type within consecutive time periods. When the network type changes s times, there are s+1 collections.

[0165] Determine each channel quality measurement value and network type within a time [t-10, t+10]; the channel quality measurement value includes: received signal power RSRP, received signal quality RSRQ and / or signal-to-noise ratio SNR.

[0166] Determine each channel quality measurement value of the nth network layer within the time period [t-10, t+10] as a channel state node; store the channel state nodes of the same network type within the time period [t-10, t+10] in a one-dimensional matrix P, P = [H1 H2 ... H 19 H 20 ], where the number U of channel state nodes with the same network type in the time period [t-10, t+10] is 20; based on the correlation between the channel state nodes at each moment and their adjacent channel state nodes, the nth adjacency matrix is ​​generated according to the channel state nodes with the same network type in consecutive time periods. The nth adjacency matrix is ​​represented by A. h 1,U For the one-dimensional matrix H1 and H U The connection relationship (edge ​​or arc) data between two channel state nodes, the position with a connection relationship is 1, and the others are 0. U is a positive integer greater than 0 and less than or equal to 20.

[0167] Step 5.2: Calculate the feature aggregation vectors of the neighboring channel state nodes related to the target channel state node, and aggregate the feature aggregation vectors of the neighboring channel state nodes with the feature aggregation vector of the target channel state node to generate the nth second channel index. The calculation formula is expressed as:

[0168] The feature aggregation vector of the neighbor channel state node generated based on the n-1th adjacency matrix is ​​marked as Wherein, N(v) is the set of neighbor nodes in the n-1th adjacency matrix;

[0169] The feature aggregation vector of the target channel state node generated based on the nth adjacency matrix is ​​marked as:

[0170] The nth second channel index is marked as Z (n) , Where A is the nth adjacency matrix, Z is the second channel index, D is the normalized matrix of A, and W is the weight to be optimized;

[0171] Here, the nth second channel indicator is transmitted to the Inception V4 network in the encoding network, and a convolution operation is performed on the nth target frame to encode the nth target frame to obtain the encoding information of the nth target frame.

[0172] Step 6: The encoder takes the results of steps 3, 4, and 5 as input. The main branch uses a convolutional network combined with LSTM. Its specific parameters and structure are as follows: Figure 3 As shown in Figure 3 The figure is a schematic diagram of the information interaction flow of the encoding network in the video processing method.

[0173] Step 7: After convolution, the output of step 5 is nonlinearized using the tanh function and then quantized using the sign function. The quantized result is input into the Pixel-CNN for entropy coding.

[0174] Step 8: The decoder takes the encoder output and secondary channel indicators as input. The module structure corresponds to the encoder. It deconvolves the input data and restores the encoded low-resolution image to a high-resolution image based on the pixel-shuffle algorithm. Its specific parameters and structure are as follows: Figure 4 As shown, Figure 4 The figure is a schematic diagram of the information interaction flow of the decoding network in the video processing method.

[0175] Step 9: Combine steps 1-8 to build a complete end-to-end video encoding and decoding network, and use the norm of the generated frame and the original frame as the loss function to optimize the network weights.

[0176] like Figure 5 , which is a schematic structural diagram of a video processing device provided by an embodiment of the present invention, the device includes: a first determining module 501, a second determining module 502 and an encoding module 503;

[0177] The first determining module 501 is configured to determine a first channel indicator for transmitting the encoded video; the second determining module 502 is configured to determine the number of network layers of the encoding network based on the first channel indicator;

[0178] The encoding module 503 is configured to use the encoding network to encode the target frame of the video to be encoded.

[0179] Specifically, the first determination module 501 is further used to: determine a fluctuation mean of a channel quality measurement value within a first preset time period; wherein the channel quality measurement value includes: received signal power RSRP, received signal quality RSRQ and / or signal-to-noise ratio SNR; and determine a first channel indicator based on the fluctuation mean.

[0180] Specifically, the encoding module 503 is also used to: determine the nth target frame and the previous frame and the following frame of the nth target frame encoded by the nth network layer according to the length of the GOP of the video to be encoded and the nth network layer; wherein, n is a positive integer less than N; N is the number of network layers; based on the previous frame and the following frame of the nth target frame, encode the nth target frame to obtain the encoding information of the nth target frame; and input the encoding information of the nth target frame into the n+1th network layer.

[0181] Specifically, the encoding module 503 is also used to: perform motion estimation on the previous frame of the nth target frame and the nth target frame to obtain a first motion feature; perform motion estimation on the following frame of the nth target frame and the nth target frame to obtain a second motion feature; and encode the nth target frame based on the first motion feature and the second motion feature to obtain encoding information of the nth target frame.

[0182] Specifically, the encoding module 503 is also used to: extract the first image feature of the previous frame of the nth target frame of the video to be encoded, and extract the second image feature of the following frame of the nth target frame; encode the nth target frame according to the first image feature, the first motion feature, the second image feature and the second motion feature to obtain the encoding information of the nth target frame.

[0183] Specifically, the apparatus further includes: a third determination module 504, a fourth determination module 505 and an input module 506;

[0184] The third determining module 504 is configured to determine the channel quality measurement value and network type of the nth network layer within a second preset time period;

[0185] The fourth determining module 505 is configured to determine an nth second channel indicator for transmitting the encoded video according to the channel quality measurement value and the network type;

[0186] The input module 506 is configured to input the nth second channel indicator to the n+1th network layer, wherein the second channel indicator is used by the n+1th network layer to encode the n+1th target frame.

[0187] Specifically, the fourth determination module 505 is also used to: establish a channel status node based on the network type and the channel quality measurement value within the second preset time period; establish an nth adjacency matrix according to the channel status node; and determine the nth second channel indicator according to the n-1th adjacency matrix and the nth adjacency matrix.

[0188] Specifically, the device further includes: a decoding module 507 and a verification module 508;

[0189] The decoding module 507 is configured to decode the coded information output by the N network layers based on the second channel indicator to obtain decoded information;

[0190] The verification module 508 is used to verify whether the decoding information is correct; if correct, it is determined that the encoding of the video to be encoded is successful.

[0191] To implement the method of the embodiment of the present invention, the embodiment of the present invention provides a video processing device, specifically, as Figure 6 As shown, the apparatus 60 includes a processor 601 and a memory 602 for storing a computer program that can be run on the processor;

[0192] In which, when the processor 601 is used to run the computer program, it executes: determining a first channel indicator for the transmitted encoded video; determining the number of network layers of the encoding network based on the first channel indicator; and using the encoding network to encode the target frame of the video to be encoded.

[0193] In one embodiment, the processor 601 is further configured to, when running the computer program, execute: determining a fluctuation mean of a channel quality measurement value within a first preset time period; wherein the channel quality measurement value includes: received signal power RSRP, received signal quality RSRQ and / or signal-to-noise ratio SNR; and determining a first channel indicator based on the fluctuation mean.

[0194] In one embodiment, the processor 601 is further used to execute the following when running the computer program: determining the nth target frame and the previous frame and the next frame of the nth target frame encoded by the nth network layer according to the length of the GOP of the video to be encoded and the nth network layer; wherein n is a positive integer less than N; N is the number of network layers; based on the previous frame and the next frame of the nth target frame, encoding the nth target frame to obtain the encoding information of the nth target frame; and inputting the encoding information of the nth target frame into the n+1th network layer.

[0195] In one embodiment, the processor 601 is further used to execute, when running the computer program: performing motion estimation on the previous frame of the nth target frame and the nth target frame to obtain a first motion feature; performing motion estimation on the following frame of the nth target frame and the nth target frame to obtain a second motion feature; and encoding the nth target frame based on the first motion feature and the second motion feature to obtain encoding information of the nth target frame.

[0196] In one embodiment, the processor 601 is also used to execute the following steps when running the computer program: extracting the first image feature of the previous frame of the nth target frame of the video to be encoded, and extracting the second image feature of the following frame of the nth target frame; encoding the nth target frame according to the first image feature, the first motion feature, the second image feature and the second motion feature to obtain the encoding information of the nth target frame.

[0197] In one embodiment, the processor 601 is further used to execute the following when running the computer program: determining the channel quality measurement value and network type of the nth network layer within a second preset time period; determining the nth second channel indicator of the video after transmission encoding based on the channel quality measurement value and network type; and inputting the nth second channel indicator to the n+1th network layer, wherein the second channel indicator is used for the n+1th network layer to encode the n+1th target frame.

[0198] In one embodiment, the processor 601 is further configured to, when running the computer program, execute: establishing a channel state node based on the network type and the channel quality measurement value within the second preset time period; establishing an nth adjacency matrix based on the channel state node; and determining the nth second channel indicator based on the n-1th adjacency matrix and the nth adjacency matrix.

[0199] In one embodiment, the processor 601 is further used to execute, when running the computer program: decoding the encoding information output by N network layers based on the second channel indicator to obtain decoding information; verifying whether the decoding information is correct; if correct, determining that the encoding of the video to be encoded is successful.

[0200] It should be noted that the video processing device and the video processing method provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0201] Of course, in actual application, Figure 6 As shown, the apparatus 60 may further include: at least one network interface 603. The various components in the video processing apparatus 60 are coupled together via a bus system 604. It is understood that the bus system 604 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 604 is not shown in FIG. Figure 6 In the figure, various buses are labeled as bus system 604. There can be at least one processor 601. The network interface 603 is used for wired or wireless communication between the video processing device 60 and other devices.

[0202] The memory 602 in this embodiment of the present invention is used to store various types of data to support the operation of the communication device 60 .

[0203] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 601. Processor 601 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 601 or by software instructions. Processor 601 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. Processor 601 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium located in memory 602. Processor 601 reads information from memory 602 and, in conjunction with its hardware, completes the steps of the above method.

[0204] In an exemplary embodiment, the video processing device 60 can be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to execute the aforementioned method.

[0205] In an exemplary embodiment, the present invention further provides a computer-readable storage medium, such as a memory 602 including a computer program. The computer program can be executed by the processor 601 of the video processing device 60 to complete the steps of the aforementioned method.

[0206] Specifically, an embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program performs the following operations: determining a first channel indicator of a video after transmission encoding; determining the number of network layers of a coding network based on the first channel indicator; and using the coding network to encode a target frame of the video to be encoded.

[0207] In one embodiment, when the computer program is executed by a processor, the computer program performs the following steps: determining a fluctuation mean of a channel quality measurement value within a first preset time period; wherein the channel quality measurement value includes: received signal power RSRP, received signal quality RSRQ and / or signal-to-noise ratio SNR; and determining a first channel indicator based on the fluctuation mean.

[0208] In one embodiment, when the computer program is executed by a processor, the following is performed: determining the nth target frame and the previous frame and the next frame of the nth target frame encoded by the nth network layer according to the length of the GOP of the video to be encoded and the nth network layer; wherein n is a positive integer less than N; N is the number of network layers; encoding the nth target frame based on the previous frame and the next frame of the nth target frame to obtain the encoding information of the nth target frame; and inputting the encoding information of the nth target frame into the (n+1)th network layer.

[0209] In one embodiment, when the computer program is executed by a processor, the following is performed: performing motion estimation on the previous frame of the nth target frame and the nth target frame to obtain a first motion feature; performing motion estimation on the following frame of the nth target frame and the nth target frame to obtain a second motion feature; and encoding the nth target frame based on the first motion feature and the second motion feature to obtain encoding information of the nth target frame.

[0210] In one embodiment, when the computer program is executed by a processor, the following steps are performed: extracting a first image feature of a previous frame of the nth target frame of the video to be encoded, and extracting a second image feature of a following frame of the nth target frame; encoding the nth target frame according to the first image feature, the first motion feature, the second image feature, and the second motion feature to obtain encoding information of the nth target frame.

[0211] In one embodiment, when the computer program is executed by a processor, the computer program performs the following steps: determining the channel quality measurement value and network type of the nth network layer within a second preset time period; determining the nth second channel indicator of the transmitted encoded video based on the channel quality measurement value and the network type; and inputting the nth second channel indicator to the n+1th network layer, wherein the second channel indicator is used for the n+1th network layer to encode the n+1th target frame.

[0212] In one embodiment, when the computer program is executed by a processor, the computer program performs the following operations: establishing a channel state node based on the network type and the channel quality measurement value within the second preset time period; establishing an nth adjacency matrix based on the channel state node; and determining the nth second channel indicator based on the n-1th adjacency matrix and the nth adjacency matrix.

[0213] In one embodiment, when the computer program is executed by the processor, the computer program performs the following steps: decoding the encoded information output by N network layers based on the second channel indicator to obtain decoded information; verifying whether the decoded information is correct; and if correct, determining that the encoding of the video to be encoded is successful.

[0214] It should be noted that the computer-readable storage medium provided in the embodiment of the present invention can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM; or it can be various devices including one or any combination of the above memories.

[0215] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present invention are included in the scope of protection of the present invention.

Claims

1. A video processing method, characterized in that: The method comprises: Determining a first channel indicator for the transmitted encoded video; determining the number of network layers of the coding network based on the first channel indicator; Using the encoding network to encode a target frame of the video to be encoded; wherein, using the encoding network to encode the target frame of the video to be encoded includes: Determining, according to the length of the GOP of the video to be encoded and the nth network layer, an nth target frame and a previous frame and a next frame of the nth target frame to be encoded by the nth network layer; wherein n is a positive integer less than N; and N is the number of the network layers; encoding the nth target frame based on a previous frame and a following frame of the nth target frame to obtain encoding information of the nth target frame; Inputting the coding information of the nth target frame into the n+1th network layer; determining a channel quality measurement value and a network type of an nth network layer within a second preset time period; Determining an nth second channel indicator for transmitting the encoded video according to the channel quality measurement value and the network type; The nth second channel indicator is input to the n+1th network layer, wherein the second channel indicator is used by the n+1th network layer to encode the n+1th target frame.

2. The video processing method according to claim 1, wherein: The determining of the first channel indicator of the transmission-encoded video includes: Determine a fluctuation mean of a channel quality measurement value within a first preset time period; wherein the channel quality measurement value includes: received signal power RSRP, received signal quality RSRQ and / or signal-to-noise ratio SNR; A first channel indicator is determined according to the fluctuation mean.

3. The video processing method according to claim 1, wherein: The encoding of the nth target frame based on the previous frame and the next frame of the nth target frame to obtain the encoding information of the nth target frame includes: Performing motion estimation on a previous frame of the n-th target frame and the n-th target frame to obtain a first motion feature; Performing motion estimation on a subsequent frame of the n-th target frame and the n-th target frame to obtain a second motion feature; The n-th target frame is encoded based on the first motion feature and the second motion feature to obtain encoding information of the n-th target frame.

4. The video processing method according to claim 3, wherein: The method further comprises: Extracting a first image feature of a previous frame of an nth target frame of the to-be-encoded video, and extracting a second image feature of a subsequent frame of the nth target frame; The nth target frame is encoded according to the first image feature, the first motion feature, the second image feature, and the second motion feature to obtain encoding information of the nth target frame.

5. The video processing method according to claim 1, wherein: The determining, according to the channel quality measurement value and the network type, an nth second channel indicator for transmitting the encoded video includes: Establishing a channel state node based on the network type and the channel quality measurement value within a second preset time period; Establishing an nth adjacency matrix according to the channel state node; The nth second channel index is determined according to the n-1th adjacency matrix and the nth adjacency matrix.

6. The video processing method according to claim 5, characterized in that: The method further comprises: decoding the encoded information output by the N network layers based on the second channel indicator to obtain decoded information; Verifying whether the decoded information is correct; If correct, it is determined that the encoding of the video to be encoded is successful.

7. A video processing device, characterized in that: The device includes: a first determination module, a second determination module, an encoding module, a third determination module, a fourth determination module and an input module; The first determining module is used to determine a first channel indicator for transmitting the encoded video; The second determining module is configured to determine the number of network layers of the coding network based on the first channel indicator; The encoding module is configured to encode a target frame of the video to be encoded using the encoding network; The encoding module is further configured to determine, based on the length of the GOP of the video to be encoded and the nth network layer, an nth target frame and a previous frame and a next frame of the nth target frame to be encoded by the nth network layer; wherein n is a positive integer less than N; and N is the number of network layers; based on the previous frame and the next frame of the nth target frame, encode the nth target frame to obtain encoding information of the nth target frame; and input the encoding information of the nth target frame into the (n+1)th network layer; The third determining module is used to determine the channel quality measurement value and network type of the nth network layer within a second preset time period; The fourth determining module is configured to determine an nth second channel indicator for transmitting the encoded video according to the channel quality measurement value and the network type; The input module is used to input the nth second channel indicator to the n+1th network layer, wherein the second channel indicator is used for the n+1th network layer to encode the n+1th target frame.

8. The video processing device according to claim 7, wherein: The first determining module is further configured to: Determine a fluctuation mean of a channel quality measurement value within a first preset time period; wherein the channel quality measurement value includes: received signal power RSRP, received signal quality RSRQ and / or signal-to-noise ratio SNR; A first channel indicator is determined according to the fluctuation mean.

9. The video processing device according to claim 7, wherein: The encoding module is also used for: Performing motion estimation on a previous frame of the n-th target frame and the n-th target frame to obtain a first motion feature; Performing motion estimation on a subsequent frame of the n-th target frame and the n-th target frame to obtain a second motion feature; The n-th target frame is encoded based on the first motion feature and the second motion feature to obtain encoding information of the n-th target frame.

10. The video processing device according to claim 9, wherein: The encoding module is also used for: Extracting a first image feature of a previous frame of an nth target frame of the to-be-encoded video, and extracting a second image feature of a subsequent frame of the nth target frame; The nth target frame is encoded according to the first image feature, the first motion feature, the second image feature, and the second motion feature to obtain encoding information of the nth target frame.

11. The video processing device according to claim 7, wherein: The fourth determining module is further configured to: Establishing a channel state node based on the network type and the channel quality measurement value within a second preset time period; Establishing an nth adjacency matrix according to the channel state node; The nth second channel index is determined according to the n-1th adjacency matrix and the nth adjacency matrix.

12. The video processing device according to claim 11, wherein: The device further comprises: a decoding module and a verification module; The decoding module is configured to decode the encoded information output by the N network layers based on the second channel indicator to obtain decoded information; The verification module is used to verify whether the decoding information is correct; if correct, it is determined that the encoding of the video to be encoded is successful.

13. A video processing device, characterized in that: The apparatus comprises: a processor and a memory for storing a computer program capable of running on the processor; Wherein, when the processor is used to run the computer program, it executes the steps of any one of the video processing methods of claims 1 to 6.

14. A computer storage medium, characterized in that The computer storage medium stores computer-executable instructions; after the computer-executable instructions are executed by the processor, the video processing method according to any one of claims 1 to 6 can be implemented.

Images