Video compression method and device based on tensor decomposition and trainable mask

Abstract

The embodiment of the invention provides a video compression method and device based on tensor decomposition and a trainable mask, the method can be applied to a sending end, and the method comprises the steps: obtaining video data, and generating a video tensor according to the video data, the video tensor comprising coordinate data and pixel features of pixel points of a video frame of the video data; generating three tensor planes and corresponding plane vectors according to the coordinate data, and determining plane parameters of the tensor planes and vector parameters of the plane vectors corresponding to the tensor planes according to the pixel features to form decomposition data; determining a mask plane corresponding to the tensor plane according to the decomposition data, and compressing plane parameters of the tensor plane according to the mask plane to obtain a compressed plane; determining transmission data according to the compression plane and the plane vector, and sending the transmission data to a receiving end; according to the scheme, the video can be compressed and then transmitted, so that the occupation of resources is reduced.

Description

Technical Field

[0001] The present application relates to the field of computer information technology, and in particular to a video compression method and device based on tensor decomposition and trainable mask. Background Art

[0002] With the rapid development of computer hardware, people have higher requirements for the clarity and resolution of images. Video data consists of multiple video frames (video frame sequences), and a video frame contains multiple pixels, each of which contains a corresponding color (pixel information).

[0003] Some solutions convert pixel information of video frames of video data into bit streams for transmission, which occupies a high amount of resources such as bandwidth. Summary of the invention

[0004] The embodiments of the present application provide a video compression method and device based on tensor decomposition and trainable masks. The method and device can extract the tensor of video data, decompose the tensor to form tensor planes and plane vectors, compress the tensor planes through a trainable mask, and transmit the compressed tensor planes and plane vectors, thereby reducing resource usage.

[0005] The technical solution is as follows:

[0006] In a first aspect, the present application provides a video compression method based on tensor decomposition and trainable mask, which is applied to a sending end, and the method includes: obtaining video data, and generating a video tensor based on the video data, the video tensor including coordinate data and pixel features of pixel points of a video frame of the video data; based on the coordinate data, generating three tensor planes and corresponding plane vectors, and based on the pixel features, determining the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane to form decomposed data; based on the decomposed data, determining the mask plane corresponding to the tensor plane, and compressing the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane; based on the compressed plane and the plane vector, determining the transmission data, and sending it to the receiving end.

[0007] Furthermore, the step of determining pixel features includes: determining pixel information of pixel points of the video frame based on the video data, and generating a multi-layer perceptron to synchronize to the receiving end, so that the pixel features are restored to pixel information at the receiving end based on the multi-layer perceptron; processing the target pixel information of the target pixel point based on the multi-layer perceptron to obtain the target pixel feature of the target pixel point.

[0008] Furthermore, the method also includes: acquiring adjacent pixel points of the target pixel point, and acquiring adjacent pixel features of the adjacent pixel points; and supplementing the adjacent pixel features into the target pixel features of the target pixel point.

[0009] Furthermore, the step of determining coordinate data includes: determining a first coordinate value and a second coordinate value based on the position of the pixel point in the video frame; determining a third coordinate value based on the time of the video frame corresponding to the pixel point in the video data; and determining coordinate data based on the first coordinate value, the second coordinate value and the third coordinate value.

[0010] Furthermore, the method of generating three tensor planes and corresponding plane vectors based on the coordinate data includes: generating a first tensor plane, a second tensor plane and a third tensor plane based on the coordinate data, and generating corresponding plane vectors; wherein the first tensor plane is a plane corresponding to the first coordinate value and the second coordinate value, and the plane vector corresponding to the first tensor plane corresponds to the third coordinate value; the second tensor plane is a plane corresponding to the first coordinate value and the third coordinate value, and the plane vector corresponding to the second tensor plane corresponds to the second coordinate value; the third tensor plane is a plane corresponding to the second coordinate value and the third coordinate value, and the plane vector corresponding to the third tensor plane corresponds to the first coordinate value.

[0011] Furthermore, the method of determining the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane based on the pixel features includes: generating the plane intermediate parameters of the tensor plane and the vector intermediate parameters of the plane vector corresponding to the tensor plane based on the pixel features; determining the reconstructed video based on the plane intermediate parameters of the tensor plane and the vector intermediate parameters of the plane vector, and comparing the difference between the reconstructed video and the video data to obtain parameter adjustment information; iteratively adjusting the plane intermediate parameters and the vector intermediate parameters based on the parameter adjustment information to determine the plane parameters of the tensor plane and the vector parameters of the plane vector corresponding to the tensor plane.

[0012] Furthermore, the method of determining the mask plane corresponding to the tensor plane based on the decomposed data includes: generating the mask plane, setting the mask value on the mask plane to an intermediate value, and determining a mask threshold, wherein the mask value on the mask plane that is greater than the mask threshold participates in feature calculation; determining an adjustment amount of the intermediate value on the mask plane based on the decomposed data and the intermediate value on the mask plane, and adjusting the intermediate value until the final value of each mask value of the mask plane is determined.

[0013] In the second aspect, the present application provides a video processing method based on tensor decomposition and trainable masks, which is applied at the receiving end, and the method includes: obtaining transmission data, the transmission data including three compression planes corresponding to the video data and plane vectors corresponding to each compression plane; determining first feature information, second feature information and third feature information of pixel points of a video frame of the video data based on the three compression planes and the plane vectors corresponding to each compression plane; obtaining coordinate data and pixel features of pixel points of a video frame of the video data based on the first feature information, the second feature information and the third feature information; performing video restoration based on the coordinate data and pixel features to obtain a restored video.

[0014] In the third aspect, the present application provides a video compression device based on tensor decomposition and trainable mask, which is used at the sending end, and the device includes: a video tensor acquisition module, which is used to acquire video data and generate a video tensor based on the video data, and the video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data; a video tensor decomposition module, which is used to generate three tensor planes and corresponding plane vectors based on the coordinate data, and determine the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane based on the pixel features to form decomposed data; a tensor plane compression module, which is used to determine the mask plane corresponding to the tensor plane based on the decomposition data, and compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane; a transmission data determination module, which is used to determine the transmission data based on the compression plane and the plane vector, and send it to the receiving end.

[0015] In a fourth aspect, the present application provides a network device, comprising: a memory, a transceiver, and a processor; wherein the memory is used to store a computer program; the transceiver is used to send and receive data under the control of the processor; and the processor is used to read the computer program in the memory and execute the method described in the first aspect and the second aspect.

[0016] In a fifth aspect, the present application provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in the first aspect and the second aspect is implemented.

[0017] The beneficial effects of the technical solution provided by this application are:

[0018] The scheme of the present application can be applied in a video compression scenario, and may involve a transmitter and a receiver. The transmitter compresses the video data and sends the compressed data to the receiver. The receiver restores the video based on the compressed data to play the video. The transmitter can obtain the video data, extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the scheme also uses a trainable mask plane to update the mask value corresponding to the mask plane by iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, so as to transmit the compressed plane and the plane vector to the receiver. After the receiver receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The scheme can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Specifically, the scheme can obtain video data and generate a video tensor based on the video data. The video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data. The coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data. The pixel features are determined according to the pixel information of the pixel point; based on the coordinate data, three tensor planes and corresponding plane vectors are generated, and based on the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data; based on the decomposition data, the mask plane corresponding to the tensor plane is determined, and the plane parameters of the tensor plane are compressed according to the mask plane to obtain a compressed plane; based on the compressed plane and the plane vector, the transmission data is determined and sent to the receiving end. After the receiving end receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0020] Figure 1 It is a schematic diagram of the steps of a video compression method based on tensor decomposition and trainable mask according to an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of tensor decomposition of an embodiment of the present application;

[0022] Figure 3 It is a flowchart of a video compression method based on tensor decomposition and trainable mask according to an embodiment of the present application;

[0023] Figure 4 is a flowchart of a video processing method based on tensor decomposition and trainable mask according to an embodiment of the present application;

[0024] Figure 5 It is a structural schematic diagram of a video compression device based on tensor decomposition and trainable mask according to an embodiment of the present application;

[0025] Figure 6 is a structural schematic diagram of a video processing device based on tensor decomposition and trainable mask according to an embodiment of the present application;

[0026] Figure 7 It is a structural block diagram of a network device according to an embodiment of the present application;

[0027] Figure 8 This is a structural block diagram of a user device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout identify the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0029] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms, and "multiple" refers to two or more, and other quantifiers are similar. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may be intermediate elements. In addition, the "connection" or "coupling" used here may include wireless connection or wireless coupling. The term "and / or" used here describes the association relationship of associated objects, indicating that there may be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0030] The scheme of the present application can be applied in a video compression scenario, and may involve a transmitter and a receiver. The transmitter compresses the video data and sends the compressed data to the receiver. The receiver restores the video based on the compressed data to play the video. The transmitter can obtain the video data, extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the scheme also uses a trainable mask plane to update the mask value corresponding to the mask plane by iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, so as to transmit the compressed plane and the plane vector to the receiver. After the receiver receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The scheme can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Among them, the coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data, and the pixel feature is determined based on the pixel information of the pixel point; the three tensor planes each correspond to two of the three dimensions of the coordinate data, and the plane vector corresponding to the tensor plane corresponds to one of the three dimensions of the coordinate data, and is different from the two dimensions of the tensor plane.

[0031] This scheme proposes a video compression method based on tensor decomposition and trainable mask. Consider using tensor decomposition for video to achieve dimensionality reduction of video spatial complexity. Decomposing the entire video as a whole can make full use of the spatiotemporal information of the video, thereby improving the expression ability of the model. Through trainable masks, the sparsity of the plane can be significantly improved while maintaining a high reconstruction quality of the video, thereby achieving compression of the video volume.

[0032] In some schemes, the encoding part of the video compression method has a lot of manual design, and the potential of video compression has not been fully utilized. The neural network video compression method currently considers frame-by-frame compression, ignoring the temporal order between frames and insufficient information utilization. At present, many methods using the neural network video compression method have restrictions on the input video resolution ratio, and the video needs to be cropped before compression, which makes video compression difficult. To solve the above problems, this scheme takes the following measures: Video tensor decomposition, decomposing the video into tensor planes and vectors, reducing the dimension of the tensor and reducing the spatial complexity. Tensor decomposition decomposes the entire video from the whole, and uses the spatiotemporal information of the entire video to improve the model performance. The decomposed tensor plane is sparsely distributed through a trainable mask to reduce the number of parameters of the tensor plane. The tensor plane and vector obtained by decomposition can be used to restore and reconstruct the video. As a video compression method based on tensor decomposition and trainable masks, this scheme can be applied to videos of different resolutions without cropping.

[0033] This solution can be used with different videos without restrictions on video type and resolution. Use tensor decomposition to decompose the video into tensor planes and vectors to achieve dimensionality reduction of spatial complexity. Aiming at the optimization goal of video compression, use trainable masks to continuously sparse the tensor planes during training, reduce the number of parameters required to express the video, and form a joint training method of tensor decomposition and mask. The main contents include: performing tensor decomposition on the video to obtain tensor planes and vectors. Set the corresponding trainable mask plane for each tensor plane. Set the corresponding mask threshold, and train the tensor decomposition and mask jointly.

[0034] like Figure 1 As shown in the figure, this scheme compares a pixel point in a video frame sequence to a three-dimensional point in space, which carries the color feature of the corresponding pixel. A feature grid is used to represent a video. This method can be used to model a video. The video can be modeled as a feature grid. The feature grid can be regarded as a 4D tensor. Each item in the tensor represents a pixel. The pixel contains the coordinates of the video pixel point and multi-dimensional color feature information. Subsequently, the feature decomposition method based on pixel points is used to decompose the features of each pixel point of the video along the tensor plane and vector. In this way, the video content can be stored in plane coefficients and vectors without the limitation of video resolution ratio. In addition, the video is represented by learnable tensor planes and vectors, so only the plane coefficients and vector coefficients need to be sampled to reconstruct the video, and no additional decoder is required for decoding. A trainable mask plane designed for video feature grids is proposed for the decomposed tensor plane to achieve higher sparsity. The goal is to set most of the coefficients to zero without significantly reducing the rendering quality, thereby reducing the number of parameters and improving the efficiency of parameter expression.

[0035] A pixel point xi in a video frame sequence is analogous to a three-dimensional point in space. The coordinates of xi are (hi, wi, ti). The color feature fc(xi) of the corresponding pixel is carried at point xi. The conversion of color feature to color c can be achieved by a small multi-layer perceptron (MLP). In this way, the video can be modeled as a feature grid, which can be regarded as a 4D tensor. For each pixel point in the video sequence, random sampling is usually used in the sampling process of the pixel point during the training process. This scheme proposes neighboring point supplementation based on random sampling. Neighboring point supplementation means that considering the certain similarity of regional pixel values, the prediction of the color of each pixel point should consider the influence of other pixels around it. In the sampling process, the sampling point is taken as the center, and the neighboring points around the sampling point are sampled, and the features of the neighboring points are extracted to supplement the sampling point.

[0036] Combination Figure 2 In this scheme, the 4D tensor grid decomposition of video modeling is used for subsequent video reconstruction. The 4D tensor is decomposed along the tensor plane M and the vector v using tensor decomposition, and the video content is stored in the plane coefficients and vectors. There is no restriction on the video resolution ratio. The video is represented by the learnable tensor plane M and the vector v. Subsequently, only the plane coefficients and the vector coefficients need to be sampled to reconstruct the video.

[0037] A trainable mask plane designed for video feature grids is proposed for the decomposed tensor plane M to achieve higher sparsity and higher compression efficiency. The feature grid is decomposed into multiple matrix planes through tensor decomposition. In order to reduce the number of parameters in the plane and increase the proportion of zero elements, a learnable mask plane is used. Each matrix plane is initialized with a mask plane of all 1s. During the training process, the value of the mask plane will be continuously iteratively updated. Only the matrix plane parameters corresponding to the mask values ​​greater than the mask threshold (λmask) on the mask plane will participate in the calculation of the features, and the mask values ​​lower than the mask threshold will not participate in the calculation of the features. It is expected that most of the coefficients can be reset to zero without significantly reducing the rendering quality, reducing the number of parameters and improving the efficiency of parameter expression.

[0038] Specifically, this scheme includes the following steps: a pixel point xi in the video frame sequence is analogized to a three-dimensional point in space, and the video is modeled as a feature grid, which can be regarded as a 4D tensor. During the sampling process of the pixel points during the training process, the influence of other pixels around it is considered. During the sampling process, the neighboring points around the sampling point are sampled with the sampling point as the center, and the features of the neighboring points are extracted to supplement the sampling point. The 4D tensor grid of video modeling is decomposed into a tensor plane M and a vector v for subsequent video reconstruction. A trainable mask plane designed for video feature grids is proposed for the decomposed tensor plane M to achieve higher compression efficiency.

[0039] The step of modeling the video as a feature grid includes: a pixel point xi in the video frame sequence is analogized to a three-dimensional point in space, the coordinates of xi are (hi, wi, ti), and the video is modeled as a feature grid fc. The color feature fc(xi) of the corresponding pixel is carried at the point xi, and the conversion of the color feature to the color c can be achieved by a small MLP. The video can be reconstructed through the feature grid and the small MLP.

[0040] The step of decomposing the 4D tensor grid of video modeling into a tensor plane M and a vector v includes: decomposing the features of the 4D tensor grid of video modeling along the tensor plane M and the vector v; sampling and rendering the plane coefficients and the vector coefficients to reconstruct the video, and calculating the mean square error of the pixel values ​​between the reconstructed video and the original video as a loss function loss; using the loss function for training the plane coefficients and the vector coefficients to optimize the reconstructed video representation;

[0041] The steps of using a trainable mask plane (learnable mask) to compress a tensor plane include: the feature grid is decomposed by VM to obtain multiple matrix planes. In order to reduce the number of parameters in the plane and increase the proportion of zero elements, a learnable mask plane is used. Each mask plane is initialized with a mask value of all 1s, and a mask threshold λmask is set. Only the matrix plane parameters corresponding to the mask values ​​greater than λmask on the mask plane will participate in the calculation of the features. The sum of the plane mask values ​​is calculated as the mask loss function lossmask; the mask loss function lossmask is used for iterative update of the mask value of the mask plane. Afterwards, the compressed tensor plane and plane vector can be sent to the receiving end as a model code stream to perform video restoration at the receiving end.

[0042] This scheme models the video as a feature grid. Compared with many current neural network video compression methods, this method is more robust in input and can handle different video types and different video resolutions without additional cropping. The use of tensor decomposition and mask plane joint training can make full use of the spatiotemporal information of the video, thereby improving the efficiency of video compression.

[0043] Based on the above embodiments, the present application also provides a video compression method based on tensor decomposition and trainable mask, such as Figure 3 As shown, applied at the sending end, the method includes:

[0044] Step 102: Obtain video data, and generate a video tensor based on the video data, where the video tensor includes coordinate data and pixel features of pixel points of a video frame of the video data.

[0045] Step 104: Generate three tensor planes and corresponding plane vectors based on the coordinate data, and determine the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane based on the pixel features to form decomposition data.

[0046] Step 106: Determine the mask plane corresponding to the tensor plane based on the decomposed data, and compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane.

[0047] Step 108: Determine the transmission data based on the compression plane and the plane vector, and send it to the receiving end.

[0048] The scheme of the present application can be applied in a video compression scenario, and may involve a transmitter and a receiver. The transmitter compresses the video data and sends the compressed data to the receiver. The receiver restores the video based on the compressed data to play the video. The transmitter can obtain the video data, extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the scheme also uses a trainable mask plane to update the mask value corresponding to the mask plane by iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, so as to transmit the compressed plane and the plane vector to the receiver. After the receiver receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The scheme can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Specifically, the scheme can obtain video data and generate a video tensor based on the video data. The video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data. The coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data. The pixel features are determined according to the pixel information of the pixel point; based on the coordinate data, three tensor planes and corresponding plane vectors are generated, and based on the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data; based on the decomposition data, the mask plane corresponding to the tensor plane is determined, and the plane parameters of the tensor plane are compressed according to the mask plane to obtain the compressed plane; based on the compressed plane and the plane vector, the transmission data is determined and sent to the receiving end. After the receiving end receives the transmission data and extracts the compressed plane and the plane vector corresponding to the pixel point, the first feature information, the second feature information and the third feature information corresponding to the pixel point can be determined, and then fused to obtain the coordinate data and pixel features of the pixel point to restore it to the pixel information to form the restored video.

[0049] The process of restoring the compressed information to the video at the receiving end is the reverse process of the video compression at the sending end. The video processing process of the receiving end corresponds to that of the sending end. The processing flow of the receiving end can refer to the processing flow of the sending end. Among them, for each video data, the sending end can generate a corresponding multi-layer perceptron, which is used to determine the relationship between pixel information (such as pixel value) and pixel features, and synchronize it to the receiving end. The receiving end uses the multi-layer perceptron to restore the pixel features to pixel information to restore it to the video. Specifically, as an optional embodiment, the step of determining the pixel features includes: determining the pixel information of the pixel points of the video frame based on the video data, and generating a multi-layer perceptron to synchronize it to the receiving end, so that the pixel features are restored to pixel information at the receiving end based on the multi-layer perceptron; processing the target pixel information of the target pixel point based on the multi-layer perceptron to obtain the target pixel feature of the target pixel point.

[0050] Considering that there is a certain similarity in the pixel values ​​of the region, the prediction of the color of each pixel point should consider the influence of other pixels around it. In the sampling process, the sampling point is taken as the center, and the neighboring points around the sampling point are sampled, and the features of the neighboring points are extracted to supplement the sampling points. In addition to extracting the pixel features of the target pixel point, this scheme can also fuse the features of the adjacent pixel points into the target pixel point. Specifically, as an optional embodiment, the method also includes: obtaining the adjacent pixel points of the target pixel point, and obtaining the adjacent pixel features of the adjacent pixel points; supplementing the adjacent pixel features to the target pixel features of the target pixel point. Adjacent pixel points may include one or more adjacent pixel points in the same video frame, and may also include adjacent pixel points of adjacent frames, so as to enhance the content contained in the pixel features and enhance the subsequent rendering effect.

[0051] This solution can use time as a dimension and combine the two dimensions of the video frame to form coordinate data. Specifically, as an optional embodiment, the step of determining the coordinate data includes: determining the first coordinate value and the second coordinate value based on the position of the pixel point in the video frame; determining the third coordinate value based on the time of the video frame corresponding to the pixel point in the video data; determining the coordinate data based on the first coordinate value, the second coordinate value and the third coordinate value. A segment of video data can correspond to multiple tensor planes, and there can be multiple tensor planes of the same type. The three tensor planes each correspond to two of the three dimensions of the coordinate data, and the plane vector corresponding to the tensor plane corresponds to one of the three dimensions of the coordinate data, and is different from the two dimensions of the tensor plane. Specifically, as an optional embodiment, the three tensor planes and corresponding plane vectors are generated based on the coordinate data, including: based on the coordinate data, a first tensor plane, a second tensor plane and a third tensor plane are generated, and corresponding plane vectors are generated; wherein the first tensor plane is a plane corresponding to the first coordinate value and the second coordinate value, and the plane vector corresponding to the first tensor plane corresponds to the third coordinate value; the second tensor plane is a plane corresponding to the first coordinate value and the third coordinate value, and the plane vector corresponding to the second tensor plane corresponds to the second coordinate value; the third tensor plane is a plane corresponding to the second coordinate value and the third coordinate value, and the plane vector corresponding to the third tensor plane corresponds to the first coordinate value.

[0052] The present solution can update the parameters of the tensor plane and the plane vector in an iterative manner until the parameters converge (such as convergence when the difference between the reconstructed video and the video data is small). Specifically, as an optional embodiment, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined based on pixel features, including: generating plane intermediate parameters of the tensor plane and vector intermediate parameters of the plane vector corresponding to the tensor plane based on pixel features; determining the reconstructed video based on the plane intermediate parameters of the tensor plane and the vector intermediate parameters of the plane vector, and comparing the difference between the reconstructed video and the video data to obtain parameter adjustment information; iteratively adjusting the plane intermediate parameters and the vector intermediate parameters based on the parameter adjustment information to determine the plane parameters of the tensor plane and the vector parameters of the plane vector corresponding to the tensor plane.

[0053] In order to further compress the tensor plane, after decomposing the video tensor into the tensor plane and the vector, a mask plane can be generated to further compress the tensor plane, and the initial value of the mask plane can be generated, and the mask value of the mask plane can be updated in an iterative manner until the final value is determined to better compress the tensor plane. The mask threshold corresponds to the quality of the restored video corresponding to the compressed parameter, and the video of the corresponding quality can be determined by setting different thresholds. This scheme can reconstruct the video according to the compressed plane after the intermediate value processing, and compare the difference between the reconstructed video and the original video to determine the adjustment amount of the intermediate value. Specifically, as an optional embodiment, the mask plane corresponding to the tensor plane is determined according to the decomposed data, including: generating a mask plane, setting the mask value on the mask plane to the intermediate value, and determining the mask threshold, wherein the mask value on the mask plane greater than the mask threshold participates in feature calculation; determining the adjustment amount of the intermediate value on the mask plane according to the decomposed data and the intermediate value on the mask plane, and adjusting the intermediate value until the final value of each mask value of the mask plane is determined.

[0054] Based on the above embodiments, the present application also provides a video processing method based on tensor decomposition and trainable mask, which is applied at the receiving end, such as Figure 4 As shown, the method includes:

[0055] Step 202: Acquire transmission data, where the transmission data includes three compression planes corresponding to the video data and plane vectors corresponding to each compression plane.

[0056] Step 204: Determine first feature information, second feature information, and third feature information of pixel points of a video frame of the video data based on the three compression planes and the plane vectors corresponding to the compression planes.

[0057] Step 206: Obtain coordinate data and pixel features of pixel points of a video frame of the video data based on the first feature information, the second feature information, and the third feature information.

[0058] Step 208: Perform video restoration based on the coordinate data and pixel features to obtain a restored video.

[0059] The implementation of the embodiments of the present application is similar to that of the above embodiments. The specific implementation can refer to the specific implementation of the above embodiments, which will not be repeated here.

[0060] The scheme of the present application can be applied in a video compression scenario, and may involve a transmitter and a receiver. The transmitter compresses the video data and sends the compressed data to the receiver. The receiver restores the video based on the compressed data to play the video. The transmitter can obtain the video data, extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the scheme also uses a trainable mask plane to update the mask value corresponding to the mask plane by iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, so as to transmit the compressed plane and the plane vector to the receiver. After the receiver receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The scheme can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Specifically, the scheme can obtain video data and generate a video tensor based on the video data. The video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data. The coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data. The pixel features are determined according to the pixel information of the pixel point; based on the coordinate data, three tensor planes and corresponding plane vectors are generated, and based on the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data; based on the decomposition data, the mask plane corresponding to the tensor plane is determined, and the plane parameters of the tensor plane are compressed according to the mask plane to obtain the compressed plane; based on the compressed plane and the plane vector, the transmission data is determined and sent to the receiving end. After the receiving end receives the transmission data and extracts the compressed plane and the plane vector corresponding to the pixel point, the first feature information, the second feature information and the third feature information corresponding to the pixel point can be determined, and then fused to obtain the coordinate data and pixel features of the pixel point to restore it to the pixel information to form the restored video. For each video data, the transmitter can generate a corresponding multi-layer perceptron, which is used to determine the relationship between pixel information (such as pixel value) and pixel features, and synchronize it to the receiver. The receiver uses the multi-layer perceptron to restore the pixel features to pixel information to restore it to video. The process of restoring the compressed information to video at the receiver is the inverse process of video compression at the transmitter, and the video processing process at the receiver corresponds to that at the transmitter.

[0061] Based on the above embodiments, the present application also provides a video compression device based on tensor decomposition and trainable mask, which is applied to the sending end, such as Figure 5 As shown, the device comprises:

[0062] The video tensor acquisition module 302 is used to acquire video data and generate a video tensor based on the video data. The video tensor includes coordinate data and pixel features of pixel points of a video frame of the video data.

[0063] The video tensor decomposition module 304 is used to generate three tensor planes and corresponding plane vectors based on the coordinate data, and determine the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane based on the pixel features to form decomposition data.

[0064] The tensor plane compression module 306 is used to determine the mask plane corresponding to the tensor plane according to the decomposed data, and compress the plane parameters of the tensor plane according to the mask plane to obtain a compressed plane.

[0065] The transmission data determination module 308 is used to determine the transmission data according to the compression plane and the plane vector, and send the data to the receiving end.

[0066] The implementation of the embodiments of the present application is similar to that of the above embodiments. The specific implementation can refer to the specific implementation of the above embodiments, which will not be repeated here.

[0067] The scheme of the present application can be applied in a video compression scenario, and may involve a transmitter and a receiver. The transmitter compresses the video data and sends the compressed data to the receiver. The receiver restores the video based on the compressed data to play the video. The transmitter can obtain the video data, extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the scheme also uses a trainable mask plane to update the mask value corresponding to the mask plane by iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, so as to transmit the compressed plane and the plane vector to the receiver. After the receiver receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The scheme can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Specifically, the scheme can obtain video data and generate a video tensor based on the video data. The video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data. The coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data. The pixel features are determined according to the pixel information of the pixel point; based on the coordinate data, three tensor planes and corresponding plane vectors are generated, and based on the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data; based on the decomposition data, the mask plane corresponding to the tensor plane is determined, and the plane parameters of the tensor plane are compressed according to the mask plane to obtain the compressed plane; based on the compressed plane and the plane vector, the transmission data is determined and sent to the receiving end. After the receiving end receives the transmission data and extracts the compressed plane and the plane vector corresponding to the pixel point, the first feature information, the second feature information and the third feature information corresponding to the pixel point can be determined, and then fused to obtain the coordinate data and pixel features of the pixel point to restore it to the pixel information to form the restored video. For each video data, the transmitter can generate a corresponding multi-layer perceptron, which is used to determine the relationship between pixel information (such as pixel value) and pixel features, and synchronize it to the receiver. The receiver uses the multi-layer perceptron to restore the pixel features to pixel information to restore it to video. The process of restoring the compressed information to video at the receiver is the inverse process of video compression at the transmitter, and the video processing process at the receiver corresponds to that at the transmitter.

[0068] Based on the above embodiments, the present application also provides a video processing device based on tensor decomposition and trainable mask, which is applied to a receiving end, such as Figure 6 As shown, the device comprises:

[0069] The transmission data acquisition module 402 is used to acquire transmission data, where the transmission data includes three compression planes corresponding to the video data and plane vectors corresponding to each compression plane.

[0070] The pixel feature extraction module 404 is used to determine the first feature information, the second feature information and the third feature information of the pixel points of the video data video frame according to the three compression planes and the plane vectors corresponding to the compression planes.

[0071] The pixel feature acquisition module 406 is used to obtain the coordinate data and pixel features of the pixel points of the video frame of the video data according to the first feature information, the second feature information and the third feature information.

[0072] The restored video generation module 408 is used to restore the video according to the coordinate data and pixel features to obtain the restored video.

[0073] The implementation of the embodiments of the present application is similar to that of the above embodiments. The specific implementation can refer to the specific implementation of the above embodiments, which will not be repeated here.

[0074] The scheme of the present application can be applied in a video compression scenario, and may involve a transmitter and a receiver. The transmitter compresses the video data and sends the compressed data to the receiver. The receiver restores the video based on the compressed data to play the video. The transmitter can obtain the video data, extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the scheme also uses a trainable mask plane to update the mask value corresponding to the mask plane by iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, so as to transmit the compressed plane and the plane vector to the receiver. After the receiver receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The scheme can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Specifically, the scheme can obtain video data and generate a video tensor based on the video data. The video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data. The coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data. The pixel features are determined according to the pixel information of the pixel point; based on the coordinate data, three tensor planes and corresponding plane vectors are generated, and based on the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data; based on the decomposition data, the mask plane corresponding to the tensor plane is determined, and the plane parameters of the tensor plane are compressed according to the mask plane to obtain the compressed plane; based on the compressed plane and the plane vector, the transmission data is determined and sent to the receiving end. After the receiving end receives the transmission data and extracts the compressed plane and the plane vector corresponding to the pixel point, the first feature information, the second feature information and the third feature information corresponding to the pixel point can be determined, and then fused to obtain the coordinate data and pixel features of the pixel point to restore it to the pixel information to form the restored video. For each video data, the transmitter can generate a corresponding multi-layer perceptron, which is used to determine the relationship between pixel information (such as pixel value) and pixel features, and synchronize it to the receiver. The receiver uses the multi-layer perceptron to restore the pixel features to pixel information to restore it to video. The process of restoring the compressed information to video at the receiver is the inverse process of video compression at the transmitter, and the video processing process at the receiver corresponds to that at the transmitter.

[0075] It should be noted that the division of units and / or modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit and / or module in each embodiment of the present application may be integrated into a processing unit and / or module, or each unit and / or module may exist physically separately, or two or more units and / or modules may be integrated into one unit and / or module. The above-mentioned integrated units and / or modules may be implemented in the form of hardware or in the form of software functional units and / or modules.

[0076] If the integrated units and / or modules are implemented in the form of software functional units and / or modules and sold or used as independent products, they can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0077] In addition, the data transmission device and the data transmission method provided in the above embodiments are based on the same application concept. Since the principles of solving problems by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0078] Figure 7 A structural block diagram of a network device is shown according to an exemplary embodiment.

[0079] like Figure 7 As shown, the network device 1100 includes at least: a processor 1110 , a memory 1120 , and a transceiver 1130 .

[0080] The transceiver 1130 is used to receive and send data under the control of the processor 1110 .

[0081] exist Figure 7In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 1110 and memory represented by memory 1120. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 1130 may be a plurality of components, namely, a transmitter and a receiver, providing units and / or modules for communicating with various other devices on a transmission medium, which may include a wireless channel, a wired channel, an optical cable, and other transmission media.

[0082] The processor 1110 is responsible for managing the bus architecture and general processing, and the memory 1120 can store data used by the processor 1110 when performing operations.

[0083] Optionally, the processor 1110 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD), and the processor 1110 may also adopt a multi-core architecture. The processor 1110 and the memory 1120 may also be physically arranged separately.

[0084] The processor 1110 calls the computer program stored in the memory 1120 to execute any one of the methods for allocating a cell radio network temporary identifier provided in the above embodiments of the present application according to the obtained executable instructions.

[0085] Figure 8 A structural block diagram of a user equipment is shown according to an exemplary embodiment.

[0086] like Figure 8 As shown, the user equipment 1300 includes at least: a processor 1310 , a memory 1320 , and a transceiver 1330 .

[0087] The transceiver 1330 is used to receive and send data under the control of the processor 1310 .

[0088] exist Figure 8In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 1310 and various circuits of memory represented by memory 1320 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 1330 may be a plurality of components, namely, a transmitter and a receiver, providing units and / or modules for communicating with various other devices on a transmission medium, including transmission media such as wireless channels, wired channels, and optical cables. For different user devices, the user interface 1340 may also be an interface capable of externally and internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0089] The processor 1310 is responsible for managing the bus architecture and general processing, and the memory 1320 can store data used by the processor 1310 when performing operations.

[0090] Optionally, the processor 1310 may be a CPU (central processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or a CPLD (Complex Programmable Logic Device), and the processor 1310 may also adopt a multi-core architecture. The processor 1310 and the memory 1320 may also be physically arranged separately.

[0091] The processor 1310 calls the computer program stored in the memory 1320 to execute any one of the methods for allocating a cell radio network temporary identifier provided in the above embodiments of the present application according to the obtained executable instructions.

[0092] It should be noted here that the above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0093] In addition, a storage medium is provided in an embodiment of the present application, on which a computer program is stored, and when the computer program is executed by a processor, the data transmission method in each of the above embodiments is implemented. The storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid state drive (SSD)), etc.

[0094] In an embodiment of the present application, a program product is provided, for example, the program product is an FPGA chip or a DSP chip, and the program product includes executable instructions, and the executable instructions are stored in a storage medium. The processor reads the executable instructions from the storage medium, so that when the executable instructions are executed by the processor, the data transmission method in each of the above embodiments is implemented.

[0095] The scheme of the present application can be applied in a video compression scenario, and may involve a transmitter and a receiver. The transmitter compresses the video data and sends the compressed data to the receiver. The receiver restores the video based on the compressed data to play the video. The transmitter can obtain the video data, extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the scheme also uses a trainable mask plane to update the mask value corresponding to the mask plane by iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, so as to transmit the compressed plane and the plane vector to the receiver. After the receiver receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The scheme can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Specifically, the scheme can obtain video data and generate a video tensor based on the video data. The video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data. The coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data. The pixel features are determined according to the pixel information of the pixel point; based on the coordinate data, three tensor planes and corresponding plane vectors are generated, and based on the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data; based on the decomposition data, the mask plane corresponding to the tensor plane is determined, and the plane parameters of the tensor plane are compressed according to the mask plane to obtain the compressed plane; based on the compressed plane and the plane vector, the transmission data is determined and sent to the receiving end. After the receiving end receives the transmission data and extracts the compressed plane and the plane vector corresponding to the pixel point, the first feature information, the second feature information and the third feature information corresponding to the pixel point can be determined, and then fused to obtain the coordinate data and pixel features of the pixel point to restore it to the pixel information to form the restored video. For each video data, the transmitter can generate a corresponding multi-layer perceptron, which is used to determine the relationship between pixel information (such as pixel value) and pixel features, and synchronize it to the receiver. The receiver uses the multi-layer perceptron to restore the pixel features to pixel information to restore it to video. The process of restoring the compressed information to video at the receiver is the inverse process of video compression at the transmitter, and the video processing process at the receiver corresponds to that at the transmitter.

[0096] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0097] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0099] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0100] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0101] The above is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be included in the scope of protection of the present application.

Claims

1. A video compression method based on tensor decomposition and trainable mask, characterized in that: Applied at the sending end, the method includes: Acquire video data, and generate a video tensor based on the video data, wherein the video tensor includes coordinate data and pixel features of pixel points of a video frame of the video data; According to the coordinate data, three tensor planes and corresponding plane vectors are generated, and according to the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposed data; According to the decomposed data, a mask plane corresponding to the tensor plane is determined, and plane parameters of the tensor plane are compressed according to the mask plane to obtain a compressed plane; According to the compression plane and the plane vector, the transmission data is determined and sent to the receiving end.

2. The method according to claim 1, characterized in that The steps of determining pixel features include: Determine pixel information of pixel points of a video frame according to the video data, and generate a multi-layer perceptron to synchronize to a receiving end, so that the pixel features are restored to pixel information according to the multi-layer perceptron at the receiving end; The target pixel information of the target pixel is processed according to the multi-layer perceptron to obtain the target pixel feature of the target pixel.

3. The method according to claim 2, characterized in that The method further comprises: Obtaining adjacent pixels of the target pixel and obtaining adjacent pixel features of the adjacent pixels; The adjacent pixel features are added to the target pixel features of the target pixel.

4. The method according to claim 1, characterized in that The steps of determining coordinate data include: Determine a first coordinate value and a second coordinate value according to a position of the pixel point in the video frame; Determine a third coordinate value according to the time of the video frame corresponding to the pixel point in the video data; Coordinate data is determined according to the first coordinate value, the second coordinate value and the third coordinate value.

5. The method according to claim 4, characterized in that The three tensor planes and corresponding plane vectors are generated according to the coordinate data, including: Based on the coordinate data, a first tensor plane, a second tensor plane and a third tensor plane are generated, and corresponding plane vectors are generated; wherein, the first tensor plane is a plane corresponding to the first coordinate value and the second coordinate value, and the plane vector corresponding to the first tensor plane corresponds to the third coordinate value; the second tensor plane is a plane corresponding to the first coordinate value and the third coordinate value, and the plane vector corresponding to the second tensor plane corresponds to the second coordinate value; the third tensor plane is a plane corresponding to the second coordinate value and the third coordinate value, and the plane vector corresponding to the third tensor plane corresponds to the first coordinate value.

6. The method according to claim 1, characterized in that Determining the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane according to the pixel features includes: Generate a plane intermediate parameter of the tensor plane and a vector intermediate parameter of the plane vector corresponding to the tensor plane according to the pixel features; Determine the reconstructed video according to the plane intermediate parameters of the tensor plane and the vector intermediate parameters of the plane vector, and compare the difference between the reconstructed video and the video data to obtain parameter adjustment information; The plane intermediate parameters and the vector intermediate parameters are iteratively adjusted according to the parameter adjustment information to determine the plane parameters of the tensor plane and the vector parameters of the plane vector corresponding to the tensor plane.

7. The method according to claim 1, characterized in that The step of determining the mask plane corresponding to the tensor plane according to the decomposed data includes: Generate a mask plane, set the mask value on the mask plane to an intermediate value, and determine a mask threshold, wherein the mask value on the mask plane that is greater than the mask threshold participates in feature calculation; According to the decomposed data and the intermediate values ​​on the mask plane, the adjustment amount of the intermediate values ​​on the mask plane is determined, and the intermediate values ​​are adjusted until the final values ​​of the mask values ​​of the mask plane are determined.

8. A video processing method based on tensor decomposition and trainable mask, characterized in that: Applied at the receiving end, the method includes: Acquire transmission data, where the transmission data includes three compression planes corresponding to the video data and plane vectors corresponding to each compression plane; Determine first feature information, second feature information and third feature information of a pixel point of a video frame of the video data according to three compression planes and plane vectors corresponding to each compression plane; Obtaining coordinate data and pixel features of pixel points of a video frame of the video data according to the first feature information, the second feature information and the third feature information; The video is restored based on the coordinate data and pixel features to obtain the restored video.

9. A video compression device based on tensor decomposition and trainable mask, characterized in that: Applied at the transmitting end, the device comprises: A video tensor acquisition module is used to acquire video data and generate a video tensor based on the video data, wherein the video tensor includes coordinate data and pixel features of pixel points of a video frame of the video data; The video tensor decomposition module is used to generate three tensor planes and corresponding plane vectors according to the coordinate data, and determine the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane according to the pixel characteristics to form decomposition data; A tensor plane compression module, used for determining a mask plane corresponding to the tensor plane according to the decomposed data, and compressing the plane parameters of the tensor plane according to the mask plane to obtain a compressed plane; The transmission data determination module is used to determine the transmission data according to the compression plane and the plane vector, and send it to the receiving end.

10. A network device, characterized in that: include: A memory, a transceiver, and a processor; wherein the memory is used to store a computer program; the transceiver is used to send and receive data under the control of the processor; The processor is used to read the computer program in the memory and execute the method according to claims 1-8.

Images