Video compression method and device based on tensor decomposition and trainable mask

Abstract

An embodiment of the present application provides a video compression method and device based on tensor decomposition and trainable mask, which can be applied at the sending end. The method includes: obtaining video data and generating a video tensor based on the video data, where the video tensor includes coordinate data and pixel features of pixel points of a video frame of the video data; generating three tensor planes and corresponding plane vectors based on the coordinate data, and determining the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane based on the pixel features to form decomposed data; determining the mask plane corresponding to the tensor plane based on the decomposition data, and compressing the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane; determining the transmission data based on the compressed plane and the plane vector, and sending it to the receiving end; this scheme can compress the video before transmission to reduce resource occupation.

Description

Technical Field

[0001] The present application relates to the field of computer information technology, and in particular to a video compression method and device based on tensor decomposition and trainable masks. Background Art

[0002] With the rapid development of computer hardware, people have higher requirements for image clarity and resolution. Video data consists of multiple video frames (video frame sequences), each of which contains multiple pixels, and each pixel contains a corresponding color (pixel information).

[0003] Some solutions convert pixel information of video frames of video data into code streams for transmission, which occupies a high amount of resources such as bandwidth. Summary of the Invention

[0004] The embodiments of the present application provide a video compression method and device based on tensor decomposition and trainable masks. The method and device can extract the tensor of video data, decompose the tensor to form a tensor plane and plane vectors, compress the tensor plane through a trainable mask, and transmit the compressed tensor plane and plane vectors, thereby reducing resource usage.

[0005] The technical solution is as follows:

[0006] In the first aspect, the present application provides a video compression method based on tensor decomposition and trainable mask, which is applied to the sending end, and the method includes: obtaining video data, and generating a video tensor based on the video data, the video tensor including the coordinate data and pixel features of the pixel points of the video frame of the video data; based on the coordinate data, generating three tensor planes and corresponding plane vectors, and based on the pixel features, determining the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane to form decomposition data; based on the decomposition data, determining the mask plane corresponding to the tensor plane, and compressing the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane; based on the compressed plane and the plane vector, determining the transmission data, and sending it to the receiving end.

[0007] Furthermore, the step of determining pixel features includes: determining pixel information of pixel points of the video frame based on the video data, and generating a multi-layer perceptron to synchronize to the receiving end, so that the pixel features are restored to pixel information based on the multi-layer perceptron at the receiving end; processing the target pixel information of the target pixel point based on the multi-layer perceptron to obtain the target pixel feature of the target pixel point.

[0008] Furthermore, the method further includes: obtaining adjacent pixel points of the target pixel point, and obtaining adjacent pixel features of the adjacent pixel points; and supplementing the adjacent pixel features to the target pixel features of the target pixel point.

[0009] Furthermore, the step of determining coordinate data includes: determining a first coordinate value and a second coordinate value based on the position of the pixel point in the video frame; determining a third coordinate value based on the time of the video frame corresponding to the pixel point in the video data; and determining coordinate data based on the first coordinate value, the second coordinate value and the third coordinate value.

[0010] Furthermore, the method of generating three tensor planes and corresponding plane vectors based on the coordinate data includes: generating a first tensor plane, a second tensor plane and a third tensor plane based on the coordinate data, and generating corresponding plane vectors; wherein, the first tensor plane is a plane corresponding to the first coordinate value and the second coordinate value, and the plane vector corresponding to the first tensor plane corresponds to the third coordinate value; the second tensor plane is a plane corresponding to the first coordinate value and the third coordinate value, and the plane vector corresponding to the second tensor plane corresponds to the second coordinate value; the third tensor plane is a plane corresponding to the second coordinate value and the third coordinate value, and the plane vector corresponding to the third tensor plane corresponds to the first coordinate value.

[0011] Furthermore, the method of determining the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane based on the pixel features includes: generating the plane intermediate parameters of the tensor plane and the vector intermediate parameters of the plane vector corresponding to the tensor plane based on the pixel features; determining the reconstructed video based on the plane intermediate parameters of the tensor plane and the vector intermediate parameters of the plane vector, and comparing the difference between the reconstructed video and the video data to obtain parameter adjustment information; iteratively adjusting the plane intermediate parameters and the vector intermediate parameters based on the parameter adjustment information to determine the plane parameters of the tensor plane and the vector parameters of the plane vector corresponding to the tensor plane.

[0012] Furthermore, the method of determining the mask plane corresponding to the tensor plane based on the decomposed data includes: generating the mask plane, setting the mask value on the mask plane to the intermediate value, and determining the mask threshold, wherein the mask value on the mask plane that is greater than the mask threshold participates in feature calculation; determining the adjustment amount of the intermediate value on the mask plane based on the decomposed data and the intermediate value on the mask plane, and adjusting the intermediate value until the final value of each mask value of the mask plane is determined.

[0013] In the second aspect, the present application provides a video processing method based on tensor decomposition and trainable masks, which is applied at the receiving end. The method includes: obtaining transmission data, the transmission data including three compression planes corresponding to the video data and plane vectors corresponding to each compression plane; determining the first feature information, second feature information and third feature information of the pixel points of the video frame of the video data based on the three compression planes and the plane vectors corresponding to each compression plane; obtaining the coordinate data and pixel features of the pixel points of the video frame of the video data based on the first feature information, the second feature information and the third feature information; performing video restoration based on the coordinate data and pixel features to obtain a restored video.

[0014] In the third aspect, the present application provides a video compression device based on tensor decomposition and trainable mask, which is used at the sending end, and the device includes: a video tensor acquisition module, which is used to acquire video data and generate a video tensor based on the video data, and the video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data; a video tensor decomposition module, which is used to generate three tensor planes and corresponding plane vectors based on the coordinate data, and determine the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane based on the pixel features to form decomposition data; a tensor plane compression module, which is used to determine the mask plane corresponding to the tensor plane based on the decomposition data, and compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane; a transmission data determination module, which is used to determine the transmission data based on the compressed plane and the plane vector, and send it to the receiving end.

[0015] In a fourth aspect, the present application provides a network device comprising: a memory, a transceiver, and a processor; wherein the memory is used to store a computer program; the transceiver is used to send and receive data under the control of the processor; and the processor is used to read the computer program in the memory and execute the methods described in the first and second aspects.

[0016] In a fifth aspect, the present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described in the first and second aspects is implemented.

[0017] The beneficial effects of the technical solution provided by this application are:

[0018] The solution of the present application can be applied in video compression scenarios, and may involve a sending end and a receiving end. The sending end compresses the video data and sends the compressed data to the receiving end. The receiving end restores the video based on the compressed data to play the video. The sending end can obtain the video data and extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the present solution also uses a trainable mask plane to update the mask value corresponding to the mask plane through iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, and transmit the compressed plane and the plane vector to the receiving end. After the receiving end receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The present solution can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Specifically, this solution can obtain video data and generate a video tensor based on the video data. The video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data. The coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data. The pixel features are determined based on the pixel information of the pixel point. Based on the coordinate data, three tensor planes and corresponding plane vectors are generated. Based on the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data. Based on the decomposition data, the mask plane corresponding to the tensor plane is determined, and the plane parameters of the tensor plane are compressed based on the mask plane to obtain a compressed plane. Based on the compressed plane and the plane vector, the transmission data is determined and sent to the receiving end. After receiving the compressed plane and the plane vector, the receiving end can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0020] Figure 1 1 is a schematic diagram of the steps of a video compression method based on tensor decomposition and trainable mask according to an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of tensor decomposition according to an embodiment of the present application;

[0022] Figure 3 1 is a flowchart of a video compression method based on tensor decomposition and trainable mask according to an embodiment of the present application;

[0023] Figure 4 1 is a flowchart of a video processing method based on tensor decomposition and trainable mask according to an embodiment of the present application;

[0024] Figure 5 1 is a structural diagram of a video compression device based on tensor decomposition and trainable mask according to an embodiment of the present application;

[0025] Figure 6 1 is a structural diagram of a video processing device based on tensor decomposition and trainable mask according to an embodiment of the present application;

[0026] Figure 7 This is a structural block diagram of a network device according to an embodiment of the present application;

[0027] Figure 8 This is a structural block diagram of a user device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout identify the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present application and are not to be construed as limiting the present application.

[0029] Those skilled in the art will understand that, unless otherwise specified, the singular forms "a," "an," "the," and "the" used herein may also include the plural forms, and "a plurality" refers to two or more, and other quantifiers are similar. It should be further understood that the term "comprising" used in the specification of this application refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connection or wireless coupling. The term "and / or" used herein describes the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0030] The solution of the present application can be applied in video compression scenarios, and may involve a sending end and a receiving end. The sending end compresses the video data and sends the compressed data to the receiving end. The receiving end restores the video based on the compressed data to play the video. The sending end can obtain the video data and extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the present solution also uses a trainable mask plane to update the mask value corresponding to the mask plane through iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, and transmit the compressed plane and the plane vector to the receiving end. After the receiving end receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The present solution can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Among them, the coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data, and the pixel features are determined based on the pixel information of the pixel point; the three tensor planes each correspond to two of the three dimensions of the coordinate data, and the plane vector corresponding to the tensor plane corresponds to one of the three dimensions of the coordinate data, and is different from the two dimensions of the tensor plane.

[0031] This proposal proposes a video compression method based on tensor decomposition and trainable masks. Tensor decomposition is considered for video, achieving dimensionality reduction of spatial complexity. Decomposing the entire video as a whole fully utilizes the video's temporal and spatial information, thereby improving the model's expressive power. Trainable masks significantly improve planar sparsity while maintaining high reconstruction quality, thereby achieving video volume compression.

[0032] Some schemes involve extensive manual design in the encoding portion of video compression methods, which underutilizes the full potential of video compression. Current neural network video compression methods primarily consider frame-by-frame compression, ignoring the temporal order between frames and underutilizing information. Many current neural network video compression methods impose limitations on the input video resolution ratio and require pre-compression video cropping, making video compression difficult. To address these issues, this scheme employs the following measures: Tensor decomposition of the video into tensor planes and vectors reduces the tensor's dimensionality and spatial complexity. Tensor decomposition decomposes the entire video holistically, leveraging the video's spatiotemporal information to improve model performance. The resulting tensor planes are sparsified using trainable masks, reducing the number of parameters. The resulting tensor planes and vectors can then be used to restore and reconstruct the video. As a video compression method based on tensor decomposition and trainable masks, this scheme is applicable to videos of varying resolutions without the need for cropping.

[0033] This solution can be used with different videos without restrictions on video type and resolution. Use tensor decomposition to decompose the video into tensor planes and vectors to achieve dimensionality reduction of spatial complexity. Aiming at the optimization goal of video compression, a trainable mask is used to continuously sparsify the tensor plane during training, reducing the number of parameters required to express the video, forming a joint training method of tensor decomposition and masking. The main contents include: performing tensor decomposition on the video to obtain tensor planes and vectors. Setting the corresponding trainable mask plane for each tensor plane. Setting the corresponding mask threshold, and training the tensor decomposition and masking jointly.

[0034] like Figure 1 As shown, this scheme compares a pixel in a video frame sequence to a three-dimensional point in space, carrying the color features of the corresponding pixel. A video is represented using a feature grid. This method allows the video to be modeled as a feature grid, which can be viewed as a 4D tensor. Each item in the tensor represents a pixel, which contains both the coordinates of the video pixel and multidimensional color features. Subsequently, pixel-based feature decomposition is used to decompose the features of each video pixel along tensor planes and vectors. This allows the video content to be stored in plane coefficients and vectors, without the limitations of video resolution scale. Furthermore, since the video is represented using learnable tensor planes and vectors, video reconstruction only requires sampling the plane coefficients and vector coefficients, eliminating the need for an additional decoder. A trainable mask plane designed specifically for the video feature grid is proposed for the decomposed tensor plane to achieve higher sparsity. The goal is to set most coefficients to zero without significantly reducing rendering quality, thereby reducing the number of parameters and improving parameter representation efficiency.

[0035] A pixel point xi in a video frame sequence is analogous to a three-dimensional point in space, with the coordinates of xi being (hi, wi, ti). At point xi, the color feature fc(xi) of the corresponding pixel is carried. The conversion of color features to color c can be achieved using a small multi-layer perceptron (MLP). This allows the video to be modeled as a feature grid, which can be viewed as a 4D tensor. For each pixel in a video sequence, random sampling is typically used during training. This scheme proposes neighboring point supplementation based on random sampling. Neighboring point supplementation means that the color prediction of each pixel should take into account the influence of other surrounding pixels, considering the certain similarity of regional pixel values. During the sampling process, the neighboring points around the sampling point are sampled with the sampling point as the center, and the neighboring point features are extracted to supplement the sampling point.

[0036] Combine Figure 2 This scheme decomposes the 4D tensor grid of video modeling for subsequent video reconstruction. It uses tensor decomposition to decompose the features of the 4D tensor along the tensor plane M and vector v, and stores the video content in plane coefficients and vectors. There is no limit on the video resolution ratio. The video is represented by the learnable tensor plane M and vector v. Subsequently, only the plane coefficients and vector coefficients need to be sampled to reconstruct the video.

[0037] A trainable mask plane designed specifically for the video feature grid is proposed for the decomposed tensor plane M to achieve higher sparsity and higher compression efficiency. The feature grid is decomposed into multiple matrix planes through tensor decomposition. To reduce the number of parameters in the planes and increase the proportion of zero elements, a learnable mask plane is used. Each matrix plane is initialized with a mask plane of all 1s. During training, the values ​​of the mask plane are continuously updated iteratively. Only the matrix plane parameters corresponding to mask values ​​greater than the mask threshold (λmask) on the mask plane participate in feature calculations, while mask values ​​below the mask threshold do not participate in feature calculations. The goal is to reduce the number of parameters and improve parameter expression efficiency by zeroing out most coefficients without significantly reducing rendering quality.

[0038] Specifically, this solution includes the following steps: a pixel point xi in a video frame sequence is analogized to a three-dimensional point in space, and the video is modeled as a feature grid, which can be regarded as a 4D tensor. During the sampling process of the pixel points during training, the influence of other surrounding pixels is considered. During the sampling process, the neighboring points around the sampling point are sampled with the sampling point as the center, and the features of the neighboring points are extracted to supplement the sampling point. The 4D tensor grid of the video model is decomposed into a tensor plane M and a vector v for subsequent video reconstruction. A trainable mask plane designed for the video feature grid is proposed for the decomposed tensor plane M to achieve higher compression efficiency.

[0039] The video is modeled as a feature grid. Each pixel point xi in the video frame sequence is analogized to a three-dimensional point in space, with the coordinates of xi being (hi, wi, ti). The video is then modeled as a feature grid fc. Point xi carries the color feature fc(xi) of the corresponding pixel. The conversion of color features to color c can be accomplished using a small MLP. The feature grid and the small MLP enable video reconstruction.

[0040] The step of decomposing a 4D tensor grid of video modeling into a tensor plane M and a vector v includes: decomposing features of the 4D tensor grid of video modeling along the tensor plane M and the vector v; sampling and rendering plane coefficients and vector coefficients to reconstruct a video, calculating a mean square error of pixel values ​​between the reconstructed video and the original video as a loss function; and applying the loss function to training the plane coefficients and the vector coefficients to optimize the reconstructed video representation.

[0041] The steps of using a trainable mask plane (learnable mask) to compress the tensor plane include: decomposing the feature grid through VM to obtain multiple matrix planes. In order to reduce the number of parameters in the plane and increase the proportion of zero elements, a learnable mask plane is used. Each mask plane is initialized with a mask value of all 1s, and a mask threshold λmask is set. Only the matrix plane parameters corresponding to the mask values ​​greater than λmask on the mask plane will participate in the feature calculation. The sum of the plane mask values ​​is calculated as the mask loss function lossmask; the mask loss function lossmask is used for iterative updates of the mask values ​​of the mask plane. Afterwards, the compressed tensor plane and plane vector can be sent to the receiving end as a model code stream for video restoration at the receiving end.

[0042] This approach models the video as a feature grid. Compared to many existing neural network video compression methods, this approach is more robust to input and can handle different video types and resolutions without requiring additional cropping. By jointly training with tensor decomposition and mask planes, it fully exploits the spatiotemporal information in the video, thereby improving video compression efficiency.

[0043] On the basis of the above embodiments, the embodiment of the present application further provides a video compression method based on tensor decomposition and trainable mask, such as Figure 3 As shown, applied at the sending end, the method includes:

[0044] Step 102: Obtain video data and generate a video tensor based on the video data. The video tensor includes coordinate data and pixel features of pixel points of a video frame of the video data.

[0045] Step 104: Generate three tensor planes and corresponding plane vectors based on the coordinate data, and determine the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane based on the pixel features to form decomposition data.

[0046] Step 106: Determine the mask plane corresponding to the tensor plane based on the decomposed data, and compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane.

[0047] Step 108: Determine the transmission data based on the compression plane and the plane vector, and send it to the receiving end.

[0048] The solution of the present application can be applied in video compression scenarios, and may involve a sending end and a receiving end. The sending end compresses the video data and sends the compressed data to the receiving end. The receiving end restores the video based on the compressed data to play the video. The sending end can obtain the video data and extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the present solution also uses a trainable mask plane to update the mask value corresponding to the mask plane through iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, and transmit the compressed plane and the plane vector to the receiving end. After the receiving end receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The present solution can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Specifically, this solution can obtain video data and generate a video tensor based on the video data. The video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data. The coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data. The pixel features are determined based on the pixel information of the pixel point. Based on the coordinate data, three tensor planes and corresponding plane vectors are generated. Based on the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data. Based on the decomposition data, the mask plane corresponding to the tensor plane is determined, and the plane parameters of the tensor plane are compressed based on the mask plane to obtain a compressed plane. Based on the compressed plane and the plane vector, the transmission data is determined and sent to the receiving end. After the receiving end receives the transmission data and extracts the compressed plane and plane vector corresponding to the pixel point, the first feature information, second feature information, and third feature information corresponding to the pixel point can be determined, and then fused to obtain the coordinate data and pixel features of the pixel point to restore it to pixel information to form a restored video.

[0049] The process of restoring the compressed information to a video at the receiving end is the inverse process of the video compression at the sending end. The video processing process at the receiving end corresponds to that at the sending end, and the processing flow at the receiving end can refer to the processing flow at the sending end. Among them, for each video data, the sending end can generate a corresponding multi-layer perceptron, which is used to determine the relationship between pixel information (such as pixel value) and pixel features, and synchronize it to the receiving end. The receiving end uses the multi-layer perceptron to restore the pixel features to pixel information to restore it to a video. Specifically, as an optional embodiment, the step of determining pixel features includes: determining pixel information of pixel points of a video frame based on video data, and generating a multi-layer perceptron to synchronize it to the receiving end, so that the pixel features are restored to pixel information at the receiving end based on the multi-layer perceptron; processing the target pixel information of the target pixel point based on the multi-layer perceptron to obtain the target pixel feature of the target pixel point.

[0050] Taking into account the existence of certain similarities in regional pixel values, the prediction of the color of each pixel point should take into account the influence of other pixels around it. During the sampling process, the sampling point is taken as the center, and the neighboring points around the sampling point are sampled, and the features of the neighboring points are extracted to supplement the sampling points. In addition to extracting the pixel features of the target pixel point, this solution can also fuse the features of the adjacent pixel points into the target pixel point. Specifically, as an optional embodiment, the method also includes: obtaining the adjacent pixel points of the target pixel point, and obtaining the adjacent pixel features of the adjacent pixel points; supplementing the adjacent pixel features to the target pixel features of the target pixel point. Adjacent pixel points can include one or more adjacent pixel points in the same video frame, and can also include adjacent pixel points of adjacent frames, so as to enhance the content contained in the pixel features and enhance the subsequent rendering effect.

[0051] This solution can use time as a dimension and combine it with the two dimensions of the video frame to form coordinate data. Specifically, as an optional embodiment, the step of determining the coordinate data includes: determining the first coordinate value and the second coordinate value based on the position of the pixel point in the video frame; determining the third coordinate value based on the time of the video frame corresponding to the pixel point in the video data; determining the coordinate data based on the first coordinate value, the second coordinate value and the third coordinate value. A segment of video data can correspond to multiple tensor planes, and there can be multiple tensor planes of the same type. The three tensor planes each correspond to two of the three dimensions of the coordinate data, and the plane vector corresponding to the tensor plane corresponds to one of the three dimensions of the coordinate data, and is different from the two dimensions of the tensor plane. Specifically, as an optional embodiment, the three tensor planes and corresponding plane vectors are generated based on the coordinate data, including: generating a first tensor plane, a second tensor plane and a third tensor plane based on the coordinate data, and generating corresponding plane vectors; wherein, the first tensor plane is a plane corresponding to the first coordinate value and the second coordinate value, and the plane vector corresponding to the first tensor plane corresponds to the third coordinate value; the second tensor plane is a plane corresponding to the first coordinate value and the third coordinate value, and the plane vector corresponding to the second tensor plane corresponds to the second coordinate value; the third tensor plane is a plane corresponding to the second coordinate value and the third coordinate value, and the plane vector corresponding to the third tensor plane corresponds to the first coordinate value.

[0052] This solution can update the parameters of the tensor plane and the plane vector in an iterative manner until the parameters converge (such as convergence when the difference between the reconstructed video and the video data is small). Specifically, as an optional embodiment, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined based on pixel features, including: generating plane intermediate parameters of the tensor plane and vector intermediate parameters of the plane vector corresponding to the tensor plane based on pixel features; determining the reconstructed video based on the plane intermediate parameters of the tensor plane and the vector intermediate parameters of the plane vector, and comparing the difference between the reconstructed video and the video data to obtain parameter adjustment information; iteratively adjusting the plane intermediate parameters and the vector intermediate parameters based on the parameter adjustment information to determine the plane parameters of the tensor plane and the vector parameters of the plane vector corresponding to the tensor plane.

[0053] To further compress the tensor plane, after decomposing the video tensor into tensor planes and vectors, a mask plane can be generated to further compress the tensor plane. Initial values ​​for the mask plane can be generated, and the mask values ​​of the mask plane can be updated iteratively until a final value is determined, thereby further compressing the tensor plane. The mask threshold corresponds to the quality of the restored video corresponding to the compressed parameters, and the corresponding video quality can be determined by setting different thresholds. This solution can reconstruct the video based on the compressed plane after processing the intermediate value, and compare the difference between the reconstructed video and the original video to determine the adjustment amount for the intermediate value. Specifically, as an optional embodiment, determining the mask plane corresponding to the tensor plane based on the decomposed data includes: generating the mask plane, setting the mask value on the mask plane to the intermediate value, and determining the mask threshold, wherein mask values ​​on the mask plane greater than the mask threshold participate in feature calculation; determining the adjustment amount for the intermediate value on the mask plane based on the decomposed data and the intermediate value on the mask plane, and adjusting the intermediate value until the final value of each mask value on the mask plane is determined.

[0054] On the basis of the above embodiments, the embodiment of the present application further provides a video processing method based on tensor decomposition and trainable mask, which is applied at the receiving end, such as Figure 4 As shown, the method includes:

[0055] Step 202: Acquire transmission data, where the transmission data includes three compression planes corresponding to the video data and plane vectors corresponding to each compression plane.

[0056] Step 204 : Determine first feature information, second feature information, and third feature information of the pixel points of the video frame of the video data based on the three compression planes and the plane vectors corresponding to the compression planes.

[0057] Step 206: Obtain coordinate data and pixel features of the pixel points of the video frame of the video data based on the first feature information, the second feature information, and the third feature information.

[0058] Step 208: Perform video restoration based on the coordinate data and pixel features to obtain a restored video.

[0059] The implementation of the embodiment of the present application is similar to that of the above embodiment. The specific implementation can refer to the specific implementation of the above embodiment and will not be repeated here.

[0060] The solution of the present application can be applied in video compression scenarios, and may involve a sending end and a receiving end. The sending end compresses the video data and sends the compressed data to the receiving end. The receiving end restores the video based on the compressed data to play the video. The sending end can obtain the video data and extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the present solution also uses a trainable mask plane to update the mask value corresponding to the mask plane through iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, and transmit the compressed plane and the plane vector to the receiving end. After the receiving end receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The present solution can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Specifically, this solution can obtain video data and generate a video tensor based on the video data. The video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data. The coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data. The pixel features are determined based on the pixel information of the pixel point. Based on the coordinate data, three tensor planes and corresponding plane vectors are generated. Based on the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data. Based on the decomposition data, the mask plane corresponding to the tensor plane is determined, and the plane parameters of the tensor plane are compressed based on the mask plane to obtain a compressed plane. Based on the compressed plane and the plane vector, the transmission data is determined and sent to the receiving end. After the receiving end receives the transmission data and extracts the compressed plane and plane vector corresponding to the pixel point, the first feature information, second feature information, and third feature information corresponding to the pixel point can be determined, and then fused to obtain the coordinate data and pixel features of the pixel point to restore it to pixel information to form a restored video. For each piece of video data, the transmitter generates a corresponding multi-layer perceptron (MLP) to determine the relationship between pixel information (such as pixel values) and pixel features, and synchronizes this information to the receiver. The receiver then uses the MLP to convert the pixel features back into pixel information, which is then converted back into video. The process of converting compressed information back into video at the receiver is the inverse of the video compression process at the transmitter, and the video processing processes at the receiver and transmitter are identical.

[0061] On the basis of the above embodiments, the embodiment of the present application further provides a video compression device based on tensor decomposition and trainable mask, which is applied at the sending end, such as Figure 5 As shown, the device includes:

[0062] The video tensor acquisition module 302 is used to acquire video data and generate a video tensor based on the video data. The video tensor includes coordinate data and pixel features of pixel points of a video frame of the video data.

[0063] The video tensor decomposition module 304 is used to generate three tensor planes and corresponding plane vectors based on the coordinate data, and determine the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane based on the pixel features to form decomposition data.

[0064] The tensor plane compression module 306 is used to determine the mask plane corresponding to the tensor plane according to the decomposed data, and compress the plane parameters of the tensor plane according to the mask plane to obtain a compressed plane.

[0065] The transmission data determination module 308 is used to determine the transmission data according to the compression plane and the plane vector, and send the data to the receiving end.

[0066] The implementation of the embodiment of the present application is similar to that of the above embodiment. The specific implementation can refer to the specific implementation of the above embodiment and will not be repeated here.

[0067] The solution of the present application can be applied in video compression scenarios, and may involve a sending end and a receiving end. The sending end compresses the video data and sends the compressed data to the receiving end. The receiving end restores the video based on the compressed data to play the video. The sending end can obtain the video data and extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the present solution also uses a trainable mask plane to update the mask value corresponding to the mask plane through iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, and transmit the compressed plane and the plane vector to the receiving end. After the receiving end receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The present solution can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Specifically, this solution can obtain video data and generate a video tensor based on the video data. The video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data. The coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data. The pixel features are determined based on the pixel information of the pixel point. Based on the coordinate data, three tensor planes and corresponding plane vectors are generated. Based on the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data. Based on the decomposition data, the mask plane corresponding to the tensor plane is determined, and the plane parameters of the tensor plane are compressed based on the mask plane to obtain a compressed plane. Based on the compressed plane and the plane vector, the transmission data is determined and sent to the receiving end. After the receiving end receives the transmission data and extracts the compressed plane and plane vector corresponding to the pixel point, the first feature information, second feature information, and third feature information corresponding to the pixel point can be determined, and then fused to obtain the coordinate data and pixel features of the pixel point to restore it to pixel information to form a restored video. For each piece of video data, the transmitter generates a corresponding multi-layer perceptron (MLP) to determine the relationship between pixel information (such as pixel values) and pixel features, and synchronizes this information to the receiver. The receiver then uses the MLP to convert the pixel features back into pixel information, which is then converted back into video. The process of converting compressed information back into video at the receiver is the inverse of the video compression process at the transmitter, and the video processing processes at the receiver and transmitter are identical.

[0068] On the basis of the above embodiments, the embodiment of the present application further provides a video processing device based on tensor decomposition and trainable mask, which is applied at the receiving end, such as Figure 6 As shown, the device includes:

[0069] The transmission data acquisition module 402 is configured to acquire transmission data, where the transmission data includes three compression planes corresponding to the video data and plane vectors corresponding to each compression plane.

[0070] The pixel feature extraction module 404 is used to determine first feature information, second feature information and third feature information of pixel points of the video data video frame based on the three compression planes and the plane vectors corresponding to the compression planes.

[0071] The pixel feature acquisition module 406 is configured to obtain coordinate data and pixel features of pixel points of a video frame of the video data based on the first feature information, the second feature information, and the third feature information.

[0072] The restored video generation module 408 is used to restore the video according to the coordinate data and pixel features to obtain the restored video.

[0073] The implementation of the embodiment of the present application is similar to that of the above embodiment. The specific implementation can refer to the specific implementation of the above embodiment and will not be repeated here.

[0074] The solution of the present application can be applied in video compression scenarios, and may involve a sending end and a receiving end. The sending end compresses the video data and sends the compressed data to the receiving end. The receiving end restores the video based on the compressed data to play the video. The sending end can obtain the video data and extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the present solution also uses a trainable mask plane to update the mask value corresponding to the mask plane through iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, and transmit the compressed plane and the plane vector to the receiving end. After the receiving end receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The present solution can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Specifically, this solution can obtain video data and generate a video tensor based on the video data. The video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data. The coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data. The pixel features are determined based on the pixel information of the pixel point. Based on the coordinate data, three tensor planes and corresponding plane vectors are generated. Based on the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data. Based on the decomposition data, the mask plane corresponding to the tensor plane is determined, and the plane parameters of the tensor plane are compressed based on the mask plane to obtain a compressed plane. Based on the compressed plane and the plane vector, the transmission data is determined and sent to the receiving end. After the receiving end receives the transmission data and extracts the compressed plane and plane vector corresponding to the pixel point, the first feature information, second feature information, and third feature information corresponding to the pixel point can be determined, and then fused to obtain the coordinate data and pixel features of the pixel point to restore it to pixel information to form a restored video. For each piece of video data, the transmitter generates a corresponding multi-layer perceptron (MLP) to determine the relationship between pixel information (such as pixel values) and pixel features, and synchronizes this information to the receiver. The receiver then uses the MLP to convert the pixel features back into pixel information, which is then converted back into video. The process of converting compressed information back into video at the receiver is the inverse of the video compression process at the transmitter, and the video processing processes at the receiver and transmitter are identical.

[0075] It should be noted that the division of units and / or modules in the embodiments of the present application is schematic and is merely a logical functional division. In actual implementation, there may be other division methods. In addition, the functional units and / or modules in the various embodiments of the present application may be integrated into one processing unit and / or module, or each unit and / or module may exist physically alone, or two or more units and / or modules may be integrated into one unit and / or module. The above-mentioned integrated units and / or modules may be implemented in the form of hardware or in the form of software functional units and / or modules.

[0076] If the integrated units and / or modules are implemented in the form of software functional units and / or modules and sold or used as independent products, they can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0077] In addition, the data transmission device and data transmission method provided in the above embodiments are based on the same application concept. Since the principles of solving problems by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0078] Figure 7 A structural block diagram of a network device is shown according to an exemplary embodiment.

[0079] like Figure 7 As shown, the network device 1100 includes at least: a processor 1110 , a memory 1120 and a transceiver 1130 .

[0080] The transceiver 1130 is used to receive and send data under the control of the processor 1110 .

[0081] exist Figure 7In the embodiment of the present invention, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 1110 and memory represented by memory 1120. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be further described herein. The bus interface provides an interface. The transceiver 1130 may be a plurality of components, i.e., a transmitter and a receiver, providing units and / or modules for communicating with various other devices over a transmission medium, such as a wireless channel, a wired channel, an optical cable, or the like.

[0082] The processor 1110 is responsible for managing the bus architecture and general processing, and the memory 1120 can store data used by the processor 1110 when performing operations.

[0083] Optionally, the processor 1110 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a complex programmable logic device (CPLD). The processor 1110 may also employ a multi-core architecture. The processor 1110 and the memory 1120 may also be physically separated.

[0084] The processor 1110 calls the computer program stored in the memory 1120 to execute any one of the methods for allocating a cell radio network temporary identifier provided in the above embodiments of the present application according to the obtained executable instructions.

[0085] Figure 8 A structural block diagram of a user equipment is shown according to an exemplary embodiment.

[0086] like Figure 8 As shown, the user equipment 1300 includes at least: a processor 1310 , a memory 1320 and a transceiver 1330 .

[0087] The transceiver 1330 is used to receive and send data under the control of the processor 1310.

[0088] exist Figure 8In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 1310 and memory represented by memory 1320. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 1330 may be a plurality of components, i.e., a transmitter and a receiver, providing units and / or modules for communicating with various other devices on a transmission medium, such as wireless channels, wired channels, optical cables, and other transmission media. For different user devices, the user interface 1340 may also be an interface capable of connecting external or internal devices as required, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0089] The processor 1310 is responsible for managing the bus architecture and general processing, and the memory 1320 can store data used by the processor 1310 when performing operations.

[0090] Optionally, the processor 1310 may be a CPU (central processing unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a CPLD (Complex Programmable Logic Device). The processor 1310 may also employ a multi-core architecture. The processor 1310 and the memory 1320 may also be physically separated.

[0091] The processor 1310 calls the computer program stored in the memory 1320 to execute any one of the methods for allocating a cell radio network temporary identifier provided in the above embodiments of the present application according to the obtained executable instructions.

[0092] It should be noted here that the above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0093] In addition, an embodiment of the present application provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the data transmission method of each of the above embodiments. The storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as a floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0094] In an embodiment of the present application, a program product is provided. For example, the program product is an FPGA chip or a DSP chip. The program product includes executable instructions stored in a storage medium. A processor reads the executable instructions from the storage medium, so that when the executable instructions are executed by the processor, the data transmission method described in each of the above embodiments is implemented.

[0095] The solution of the present application can be applied in video compression scenarios, and may involve a sending end and a receiving end. The sending end compresses the video data and sends the compressed data to the receiving end. The receiving end restores the video based on the compressed data to play the video. The sending end can obtain the video data and extract the video tensor of the video data, decompose the video tensor into three tensor planes and corresponding plane vectors, and determine the parameters of the tensor plane and the plane vector based on the pixel features; the present solution also uses a trainable mask plane to update the mask value corresponding to the mask plane through iterative updating, so as to compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane, and transmit the compressed plane and the plane vector to the receiving end. After the receiving end receives the compressed plane and the plane vector, it can restore the pixel features corresponding to each pixel point and restore it to pixel information to form a restored video. The present solution can convert the video data into a video tensor, decompose it into low-dimensional tensor planes and plane vectors, and can also compress the tensor plane, thereby reducing the occupation of resources such as bandwidth during transmission. Specifically, this solution can obtain video data and generate a video tensor based on the video data. The video tensor includes the coordinate data and pixel features of the pixel points of the video frame of the video data. The coordinate data includes the position of the pixel point in the video frame and the time of the video frame corresponding to the pixel point in the video data. The pixel features are determined based on the pixel information of the pixel point. Based on the coordinate data, three tensor planes and corresponding plane vectors are generated. Based on the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data. Based on the decomposition data, the mask plane corresponding to the tensor plane is determined, and the plane parameters of the tensor plane are compressed based on the mask plane to obtain a compressed plane. Based on the compressed plane and the plane vector, the transmission data is determined and sent to the receiving end. After the receiving end receives the transmission data and extracts the compressed plane and plane vector corresponding to the pixel point, the first feature information, second feature information, and third feature information corresponding to the pixel point can be determined, and then fused to obtain the coordinate data and pixel features of the pixel point to restore it to pixel information to form a restored video. For each piece of video data, the transmitter generates a corresponding multi-layer perceptron (MLP) to determine the relationship between pixel information (such as pixel values) and pixel features, and synchronizes this information to the receiver. The receiver then uses the MLP to convert the pixel features back into pixel information, which is then converted back into video. The process of converting compressed information back into video at the receiver is the inverse of the video compression process at the transmitter, and the video processing processes at the receiver and transmitter are identical.

[0096] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) that contain computer-usable program code.

[0097] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor-readable memory produce an article of manufacture comprising an instruction device that implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These processor-executable instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0100] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0101] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be included in the scope of protection of the present application.

Claims

1. A video compression method based on tensor decomposition and trainable mask, characterized in that Applied at the sending end, the method includes: Acquire video data and generate a video tensor based on the video data, where the video tensor includes coordinate data and pixel features of pixel points of a video frame of the video data; According to the coordinate data, three tensor planes and corresponding plane vectors are generated, and according to the pixel features, the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane are determined to form decomposition data; Determine the mask plane corresponding to the tensor plane according to the decomposed data, and compress the plane parameters of the tensor plane according to the mask plane to obtain a compressed plane; According to the compression plane and plane vector, the transmission data is determined and sent to the receiving end.

2. The method according to claim 1, wherein The steps of determining pixel features include: Determine pixel information of pixel points of the video frame based on the video data, and generate a multi-layer perceptron to synchronize to the receiving end, so that the pixel features can be restored to pixel information based on the multi-layer perceptron at the receiving end; The target pixel information of the target pixel is processed according to the multi-layer perceptron to obtain the target pixel feature of the target pixel.

3. The method according to claim 2, wherein The method further comprises: Obtaining adjacent pixels of the target pixel and obtaining adjacent pixel features of the adjacent pixels; The adjacent pixel features are added to the target pixel features of the target pixel.

4. The method according to claim 1, wherein The steps for determining coordinate data include: Determine a first coordinate value and a second coordinate value according to a position of the pixel point in the video frame; Determine a third coordinate value based on the time of the video frame corresponding to the pixel point in the video data; Coordinate data is determined according to the first coordinate value, the second coordinate value, and the third coordinate value.

5. The method according to claim 4, wherein The three tensor planes and corresponding plane vectors are generated based on the coordinate data, including: Based on the coordinate data, a first tensor plane, a second tensor plane and a third tensor plane are generated, and corresponding plane vectors are generated; wherein, the first tensor plane is a plane corresponding to the first coordinate value and the second coordinate value, and the plane vector corresponding to the first tensor plane corresponds to the third coordinate value; the second tensor plane is a plane corresponding to the first coordinate value and the third coordinate value, and the plane vector corresponding to the second tensor plane corresponds to the second coordinate value; the third tensor plane is a plane corresponding to the second coordinate value and the third coordinate value, and the plane vector corresponding to the third tensor plane corresponds to the first coordinate value.

6. The method according to claim 1, wherein Determining the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane based on the pixel features includes: Generate plane intermediate parameters of the tensor plane and vector intermediate parameters of the plane vector corresponding to the tensor plane according to the pixel features; Determine a reconstructed video according to the plane intermediate parameters of the tensor plane and the vector intermediate parameters of the plane vector, and compare the difference between the reconstructed video and the video data to obtain parameter adjustment information; The plane intermediate parameters and the vector intermediate parameters are iteratively adjusted according to the parameter adjustment information to determine the plane parameters of the tensor plane and the vector parameters of the plane vector corresponding to the tensor plane.

7. The method according to claim 1, wherein The step of determining a mask plane corresponding to the tensor plane based on the decomposed data includes: Generate a mask plane, set the mask value on the mask plane to an intermediate value, and determine a mask threshold, wherein the mask value on the mask plane that is greater than the mask threshold participates in feature calculation; According to the decomposed data and the intermediate values ​​on the mask plane, the adjustment amount of the intermediate values ​​on the mask plane is determined, and the intermediate values ​​are adjusted until the final values ​​of the mask values ​​of the mask plane are determined.

8. A video processing method based on tensor decomposition and trainable mask, characterized in that Applied at the receiving end, the method includes: Acquire transmission data, where the transmission data includes three compression planes corresponding to the video data and plane vectors corresponding to each compression plane; Determining first feature information, second feature information, and third feature information of a pixel point of a video frame of the video data based on the three compression planes and the plane vectors corresponding to the compression planes; Obtaining coordinate data and pixel features of pixel points of a video frame of the video data based on the first feature information, the second feature information, and the third feature information; The video is restored based on the coordinate data and pixel features to obtain the restored video.

9. A video compression device based on tensor decomposition and trainable mask, characterized in that Applied at the transmitting end, the device includes: A video tensor acquisition module is used to acquire video data and generate a video tensor based on the video data. The video tensor includes coordinate data and pixel features of pixel points of a video frame of the video data; The video tensor decomposition module is used to generate three tensor planes and corresponding plane vectors based on the coordinate data, and determine the plane parameters of each tensor plane and the vector parameters of the plane vector corresponding to the tensor plane based on the pixel features to form decomposition data; A tensor plane compression module is used to determine a mask plane corresponding to the tensor plane based on the decomposed data, and compress the plane parameters of the tensor plane based on the mask plane to obtain a compressed plane; The transmission data determination module is used to determine the transmission data based on the compression plane and the plane vector, and send it to the receiving end.

10. A network device, characterized in that: include: A memory, a transceiver, and a processor; wherein the memory is used to store computer programs; the transceiver is used to send and receive data under the control of the processor; The processor is configured to read the computer program in the memory and execute the method according to claims 1-8.

Images