An image data processing method, apparatus, device and medium

Abstract

The application discloses an image data processing method and device, equipment and medium, and relates to the technical field of computers. In the image data processing process, the first tensor of the parameters of the deep learning model is replaced by the approximation tensor of the second tensor, the approximation tensor of the second tensor is processed by the element scale of coarse sampling and fine sampling of the second tensor, the sub-tensors under each element scale are obtained, and then the tensor decomposition and combination of the sub-tensors corresponding to each element scale are obtained. The coarse sampling and fine sampling change the number of elements of the tensor dimension of the sub-tensors, reduce the calculation complexity during the tensor decomposition, and also reduce the calculation complexity of the data processing in the image processing process. The approximation tensor of the second tensor is obtained by the tensor decomposition and combination of each sub-tensor, has a low storage complexity, reduces the memory space to a certain extent in the image data processing process, and improves the image data processing efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an image data processing method and device, equipment and medium. BACKGROUND

[0002] With the development of computer vision and natural language processing, deep neural networks play an increasingly important role in these artificial intelligence fields.

[0003] Because the deep neural network obtained by using the model compression method in the image data processing process has a large number of layers and a large amount of parameters, which brings a large storage space, it is of great significance to reduce the occupied storage space. In the current parameter compression process of deep neural network parameters using tensor decomposition, the original high-dimensional tensor is directly decomposed, which leads to a high computational complexity of data processing in the tensor decomposition process. At the same time, the storage amount corresponding to the original high-dimensional tensor is large, so that the storage space corresponding to the high-dimensional tensor after decomposition is also large, which occupies a large space for data in the image data processing process.

[0004] Therefore, how to reduce the storage space and reduce the computational complexity of data processing in the tensor decomposition process in the image data processing process is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0005] The purpose of the present application is to provide an image data processing method, device, equipment and medium to solve the technical problems of high computational complexity and large storage space in the current image data processing process.

[0006] To solve the above technical problems, the present application provides an image data processing method, comprising:

[0007] obtaining initial image data;

[0008] calling a deep learning model to input the initial image data, and obtaining the output result of the deep learning model;

[0009] taking the output result as the processing result of the initial image data;

[0010] wherein the parameters of the deep learning model include a first tensor; the first tensor is obtained by replacing the second tensor with an approximation tensor; the approximation tensor of the second tensor is obtained by tensor decomposition and combination of each element scale corresponding sub-tensor; each element scale corresponding sub-tensor is obtained by coarse sampling and fine sampling processing of the second tensor; the processing mode of coarse sampling and fine sampling processing is that the tensor dimension of the second tensor is unchanged, and the number of elements corresponding to the tensor dimension of the second tensor is changed to obtain the sub-tensor.

[0011] In one aspect, the sub-tensors corresponding to each element size include initial sub-tensors and overlapping sub-tensors, and the sub-tensors corresponding to each element size are obtained through coarse sampling and fine sampling of the second tensor, including:

[0012] The elements corresponding to the first dimension to the last dimension of the second tensor are respectively subjected to fine sampling to obtain dimension elements corresponding to target dimensions of the initial sub-tensors corresponding to each element size, wherein the number of elements subjected to fine sampling is the same as the number of elements of the dimensions of the second tensor;

[0013] The sampling ratio of the coarse sampling is obtained;

[0014] The elements of the remaining dimensions except the target dimensions in the initial sub-tensors corresponding to each element size are respectively subjected to coarse sampling according to the sampling ratio to obtain dimension elements after sampling, wherein the number of elements subjected to coarse sampling is different from the number of elements of the dimensions of the second tensor;

[0015] The dimension elements after sampling and the dimension elements of the target dimensions are combined to form the initial sub-tensors, and the dimension elements after sampling are processed according to the number of dimensions of the second tensor to obtain the overlapping sub-tensors;

[0016] The initial sub-tensors and the overlapping sub-tensors are taken as the sub-tensors corresponding to each element size.

[0017] On the other hand, when the sampling ratio of the coarse sampling is the same, the elements of the remaining dimensions except the target dimensions in the initial sub-tensors corresponding to each element size are respectively subjected to coarse sampling according to the sampling ratio to obtain dimension elements after sampling, including:

[0018] Taking the target dimension as a fixed dimension, the number of elements corresponding to the dimension positions of the second tensor in the remaining dimensions is subjected to coarse sampling according to the sampling ratio to obtain dimension elements after sampling.

[0019] On the other hand, the number of initial sub-tensors is the same as the number of dimensions of the second tensor.

[0020] On the other hand, the number of overlapping sub-tensors reserved is one, and the corresponding relationship of the second tensor, the initial sub-tensors and the overlapping sub-tensors is as follows:

[0021] The number of dimensions of the second tensor is reduced by 1 to obtain a repetition coefficient corresponding to the overlapping sub-tensors;

[0022] The repetition coefficient is multiplied with the overlapping sub-tensors to obtain a first sub-tensor;

[0023] The initial sub-tensors are added to obtain a second sub-tensor;

[0024] The second sub-tensor is subtracted from the first sub-tensor to obtain a second tensor.

[0025] In another aspect, when the sampling ratio of the coarse sampling processing is multiple and different, the elements of the remaining dimensions in the initial sub-tensor corresponding to each element size are respectively subjected to coarse sampling processing according to the sampling ratio to obtain dimension elements after sampling, comprising:

[0026] The number of elements corresponding to the dimension positions of the second tensor of the remaining dimensions is respectively subjected to coarse sampling processing according to the sampling ratio to obtain dimension elements after sampling, wherein the number of the sampling ratio is less than or equal to the number of dimensions of the remaining dimensions.

[0027] In another aspect, the determination process of the sampling ratio of the coarse sampling processing comprises:

[0028] Obtaining the dimension element number rule and the integer rule of the coarse sampling processing;

[0029] Determining the sampling ratio according to the dimension element number rule and the integer rule;

[0030] The dimension element number rule is that the number of dimension elements after sampling of the initial sub-tensor is less than the number of dimension elements of the second tensor.

[0031] The integer rule is that there is a positive proportional relationship between the number of dimension elements after sampling of the initial sub-tensor and the number of dimension elements of the second tensor, and the coefficient is an integer.

[0032] In another aspect, the sampling ratio is the Nth power of the number of dimension elements of the second tensor, wherein N is greater than 0 and less than 1.

[0033] In another aspect, the approximation tensor of the second tensor is obtained by tensor decomposition and combination of the sub-tensors corresponding to each element size, comprising:

[0034] The initial sub-tensor and the overlapping sub-tensor in each element size corresponding sub-tensor are respectively subjected to tensor decomposition to obtain corresponding low-rank tensors.

[0035] The low-rank tensors are combined to obtain the approximation tensor of the second tensor.

[0036] In another aspect, the initial sub-tensor and the overlapping sub-tensor in each element size corresponding sub-tensor are respectively subjected to tensor decomposition to obtain corresponding low-rank tensors, comprising:

[0037] Selecting the current sub-tensor in the initial sub-tensor and the overlapping sub-tensor in each element size corresponding sub-tensor;

[0038] determining the tensor rank corresponding to each dimension according to each dimension of the current sub-tensor;

[0039] determining the tensor core corresponding to the current sub-tensor according to the tensor rank corresponding to each dimension and the element corresponding to each dimension;

[0040] taking the tensor core corresponding to the current sub-tensor as the low-rank tensor after tensor decomposition of the current sub-tensor.

[0041] In another aspect, the tensor core corresponding to the first dimension and the tensor core corresponding to the last dimension in each tensor core are two-dimensional matrices.

[0042] In another aspect, combining the low-rank tensors to obtain the approximation tensor of the second tensor comprises:

[0043] adding the low-rank tensors corresponding to each initial sub-tensor to obtain a first combined tensor;

[0044] multiplying the low-rank tensors corresponding to the overlapping sub-tensors with the corresponding repetition coefficients to obtain a second combined tensor;

[0045] subtracting the first combined tensor from the second combined tensor to obtain the approximation tensor of the second tensor.

[0046] In another aspect, after the second tensor is coarsely sampled and finely sampled to obtain sub-tensors corresponding to each element scale, before the sub-tensors corresponding to each element scale are tensor decomposed and combined to obtain the approximation tensor of the second tensor, the method further comprises:

[0047] obtaining the storage complexity corresponding to the second tensor and the storage complexity corresponding to each sub-tensor corresponding to each element scale;

[0048] determining whether the storage complexity corresponding to the second tensor is greater than the storage complexity corresponding to each sub-tensor corresponding to each element scale;

[0049] if yes, determining that each sub-tensor corresponding to each element scale is verified successfully.

[0050] In another aspect, when the number of sampling ratios is one, the determination process of the storage complexity corresponding to each sub-tensor corresponding to each element scale comprises:

[0051] the measure of the storage complexity of the initial sub-tensor is obtained from the sampling ratio;

[0052] the measure of the storage complexity of the overlapping sub-tensor is obtained from the relationship between the sampling ratio and the repetition coefficient corresponding to the overlapping sub-tensor.

[0053] On the other hand, after the second tensor is coarsely sampled and finely sampled to obtain the sub-tensors corresponding to the element scales, before the sub-tensors corresponding to the element scales are decomposed and combined to obtain the approximation tensor of the second tensor, the method further comprises:

[0054] Obtaining the number of elements in each dimension corresponding to each sub-tensor and the number of elements in each dimension corresponding to the second tensor;

[0055] Determining the compression ratio of each dimension corresponding to each sub-tensor based on the number of elements in each dimension corresponding to each sub-tensor and the number of elements in each dimension corresponding to the second tensor;

[0056] Determining whether the compression ratio of each dimension corresponding to each sub-tensor is the same as the sampling ratio;

[0057] If yes, it is determined that the verification of each sub-tensor corresponding to the element scale is successful.

[0058] On the other hand, after the sub-tensors corresponding to the element scales are decomposed and combined to obtain the approximation tensor of the second tensor, the method further comprises:

[0059] Obtaining the first storage complexity of the low-rank tensor approximation result corresponding to the second tensor;

[0060] Determining the low-rank tensor approximation result corresponding to each sub-tensor according to the low-rank tensor corresponding to each sub-tensor, and performing space complexity processing on the low-rank tensor approximation result corresponding to each sub-tensor to obtain the second storage complexity corresponding to each sub-tensor;

[0061] Determining whether the first storage complexity is greater than each second storage complexity;

[0062] If yes, it is determined that the verification of each sub-tensor corresponding to the element scale is successful.

[0063] On the other hand, if the storage complexity corresponding to the second tensor is less than or equal to the storage complexity corresponding to each sub-tensor corresponding to the element scale, the method further comprises:

[0064] Obtaining the actual sub-tensor corresponding to the storage complexity of the second tensor which is less than or equal to the storage complexity corresponding to each sub-tensor corresponding to the element scale;

[0065] Adjusting the sampling ratio according to the relationship between the number of actual sub-tensors and the number of sub-tensors corresponding to the element scale to obtain an adjusted sampling ratio, and returning to the step of coarsely sampling the elements of the remaining dimensions except the target dimension in each initial sub-tensor corresponding to the element scale to obtain the dimension elements after sampling according to the adjusted sampling ratio.

[0066] To solve the above technical problems, the application further provides an image data processing device, comprising:

[0067] An acquisition module is configured to acquire initial image data.

[0068] A calling module is configured to call a deep learning model to input the initial image data and acquire an output result of the deep learning model.

[0069] A determination module is configured to determine the output result as a processing result of the initial image data.

[0070] The parameters of the deep learning model comprise a first tensor; the first tensor is obtained by replacing a second tensor with an approximation tensor of the second tensor; the approximation tensor of the second tensor is obtained by tensor decomposition and combination of a sub-tensor corresponding to each element scale; the sub-tensor corresponding to each element scale is obtained by coarse sampling and fine sampling of the second tensor; the processing mode of the coarse sampling and the fine sampling is that the tensor dimension of the second tensor is unchanged, and the number of elements corresponding to the tensor dimension of the second tensor is changed to obtain the sub-tensor.

[0071] To solve the above technical problems, the application further provides an image data processing device, comprising:

[0072] A memory is configured to store a computer program.

[0073] A processor is configured to execute the computer program to realize the steps of the image data processing method.

[0074] To solve the above technical problems, the application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the image data processing method.

[0075] The application provides an image data processing method, comprising the following steps: acquiring initial image data; calling a deep learning model to input the initial image data and acquiring an output result of the deep learning model as a processing result of the initial image data; wherein the parameters of the deep learning model comprise a first tensor; the first tensor is obtained by replacing a second tensor with an approximation tensor of the second tensor; the approximation tensor of the second tensor is obtained by tensor decomposition and combination of a sub-tensor corresponding to each element scale; the sub-tensor corresponding to each element scale is obtained by coarse sampling and fine sampling of the second tensor; the processing mode of the coarse sampling and the fine sampling is that the tensor dimension of the second tensor is unchanged, and the number of elements corresponding to the tensor dimension of the second tensor is changed to obtain the sub-tensor.

[0076] The application has the beneficial effect that in the image data processing process, the parameters of the deep learning model include a first tensor, the first tensor is replaced by an approximation tensor of a second tensor, the approximation tensor of the second tensor is processed by element scale through coarse sampling and fine sampling of the second tensor, sub-tensors under each element scale are obtained, and tensor decomposition and combination are performed on the sub-tensors corresponding to each element scale to obtain the corresponding coarse sampling and fine sampling, which changes the number of elements of the tensor dimension of the sub-tensor, and compared with the high order and large number of elements of the second tensor (the original high-dimensional tensor directly decomposed by traditional tensor decomposition), the calculation complexity of tensor decomposition is reduced when the sub-tensors are decomposed, and the calculation complexity of data processing in the image processing process is also reduced. Compared with the second tensor, the approximation tensor of the second tensor is obtained by tensor decomposition and combination of each sub-tensor, which has a lower storage complexity, reduces the memory space to a certain extent in the image data processing process, and improves the image data processing efficiency.

[0077] The coarse sampling and fine sampling processes provided in this embodiment reduce the number of elements of the sub-tensors of each element scale compared to the second tensor, so as to reduce the computational complexity of tensor decomposition when performing tensor decomposition on each sub-tensor, and also reduce the computational complexity of data processing during image processing; when the sampling ratio is one, the coarse sampling process and the corresponding correspondence between the initial sub-tensor, overlapping sub-tensor and the second tensor eliminate duplicate overlapping sub-tensors, so that the accuracy of the sub-tensors obtained at each element scale is improved; when the sampling ratio is multiple, the coarse sampling process and the corresponding process for determining the number of initial sub-tensors make the sub-tensors at each element scale ordered and accurate, which is convenient for subsequent tensor decomposition; low-rank tensors are obtained through coarse sampling, fine sampling and tensor decomposition, and the low-rank tensors are combined to obtain an approximation to the second tensor. Relative to the second tensor, each sub-tensor is obtained by performing tensor decomposition and combination on each sub-tensor. The approximate tensor has a lower storage complexity, which reduces the memory space in the image data processing process to a certain extent, thereby improving the image data processing efficiency. Before the sub-tensor is decomposed, the storage complexity of each sub-tensor is compared with the second tensor to verify that the storage complexity of the sub-tensor is lower than that of the second tensor. If the verification is successful, the subsequent tensor decomposition of the sub-tensor is performed to improve the data storage efficiency and avoid the problem of large computational complexity and storage space occupied by data processing caused by tensor decomposition when the verification is unsuccessful. The storage complexity of the low-rank approximation result is compared and verified, and the storage complexity of the tensor decomposition after the coarse sampling of the sub-tensor is double-verified. Compared with the method of directly decomposing the original high-order tensor (the second tensor), the former has lower storage complexity. If the verification fails, the sampling ratio is adjusted, and the sub-tensor is determined by coarse sampling and fine sampling based on different sampling ratios to facilitate subsequent tensor decomposition. The sampling ratio is adjusted in time to improve the accuracy of image data processing.

[0078] In addition, the present invention also provides an image data processing device, equipment and medium, which have the same beneficial effects as the above-mentioned image data processing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0080] Figure 1 A flowchart of an image data processing method provided by an embodiment of the present invention;

[0081] Figure 2 A determination schematic diagram of an initial sub-tensor and an overlapping sub-tensor corresponding to a sampling ratio of one is provided for an embodiment of the present application;

[0082] Figure 3 A tensor decomposition schematic diagram of a current sub-tensor is provided for an embodiment of the present application;

[0083] Figure 4 A structure diagram of an image data processing apparatus is provided for an embodiment of the present application;

[0084] Figure 5 A structure diagram of an image data processing device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0085] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0086] The core of the present application is to provide an image data processing method, apparatus, device and medium, so as to solve the technical problems of high computational complexity and large storage space in the corresponding tensor decomposition process in the current image data processing process.

[0087] In order to enable those skilled in the art to better understand the present application scheme, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0088] In image data processing, the corresponding model compression method can adopt matrix decomposition compression and tensor decomposition compression method. The matrix decomposition compression has a relatively low data processing efficiency. The tensor decomposition compression is to decompose the tensors such as weights and offsets of each layer of the model into low-order tensors by high-order singular value decomposition. However, the high-order singular value decomposition is based on original high-dimensional tensor decomposition, and the corresponding computational complexity is high when the high-order tensor is decomposed, which increases the amount of calculation in the image data processing process. Similarly, the corresponding storage space of the high-order tensor decomposition is also large. Therefore, the image data processing method provided by the present application can solve the above technical problems.

[0089] Figure 1 A flowchart of an image data processing method provided for an embodiment of the present application is shown in Figure 1 The method comprises the following steps:

[0090] S11: acquiring initial image data;

[0091] S12: call the deep learning model to input the initial image data, and obtain an output result of the deep learning model;

[0092] The parameters of the deep learning model include a first tensor; the first tensor is obtained by replacing a second tensor by an approximation tensor of the second tensor; the approximation tensor of the second tensor is obtained by tensor decomposition and combination of a sub-tensor corresponding to each element scale; the sub-tensor corresponding to each element scale is obtained by coarse sampling and fine sampling of the second tensor; the processing mode of the coarse sampling and the fine sampling is that the tensor dimension of the second tensor is unchanged, and the number of elements corresponding to the tensor dimension of the second tensor is changed to obtain the sub-tensor;

[0093] S13: taking the output result as a processing result of the initial image data.

[0094] Specifically, the initial image data in the embodiment can be image processing, or can be processing of corresponding images in a video processing process, and the application scenario of the data processing is not specifically limited, and can be set according to actual conditions.

[0095] In the embodiment, the initial image data is input into the deep learning model, and subsequent feature extraction processing is performed to obtain an output result corresponding to the deep learning model, and the specific feature processing is performed based on the deep learning model.

[0096] The deep learning model can be a neural network model, a deep neural network, a convolutional neural network, a recurrent neural network, a long short-term memory network, and a graph neural network, etc. The embodiment is not limited to a specific model, and each deep learning model includes an input layer, a hidden layer, and an output layer. Each layer is divided into multiple layers based on different deep learning models, such as each layer of neurons and links between the input layer and the output layer.

[0097] The parameters of the deep learning model include a first tensor, which is in the form of an input parameter before each layer operation in the deep learning model, and the first tensor is a tensor composed of different array dimensions. Taking the input layer, the hidden layer and the output layer in the deep learning model as examples, for the input layer, the parameter data based on the initial image data is converted to obtain a sub-tensor, the sub-tensor is decomposed to obtain an approximation tensor of a second tensor, the second tensor is replaced by the approximation tensor of the second tensor to obtain a first tensor, which is a low-rank tensor, as an input parameter of the input layer for tensor operation in the input layer. Similarly, the input parameters of the hidden layer and the output layer also include the first tensor. The first tensor is replaced by the approximation tensor of the second tensor, which is obtained based on the tensor decomposition and combination of the sub-tensor corresponding to each element size, and further, the second tensor is subjected to coarse sampling processing and fine sampling processing to obtain the sub-tensor corresponding to each element size. The second tensor of the application is converted based on the parameters corresponding to the initial image data, such as input parameters, weight parameters or offset parameters.

[0098] Tensor decomposition is a tool for accelerating the initial deep learning model, which decomposes one layer into multiple smaller layers, and the total number of floating point operations and weights will be smaller. Tensor decomposition to the weights of the layer, assuming that the network layer is parameterized, the weights are represented by a matrix or a low-rank tensor, and the information or main component hidden in the initial image data is extracted by tensor decomposition.

[0099] The second tensor is converted based on the parameters corresponding to the initial image data, which can be converted by using a conversion function or by using data processing software. Usually, the reshape function is used to convert the parameter data into a matrix with a specific dimension. The conversion processing in this embodiment is the same as the conventional conversion into a high-order tensor, or it can be different, which is not limited here.

[0100] Different from the traditional high-order tensor direct tensor decomposition mode, in the embodiment, the second tensor is processed to obtain the corresponding sub-tensor under each element scale based on sampling processing. That is, the second tensor is first processed into a plurality of low element scale sub-tensors. The low element scale sub-tensor has the same tensor dimension as the second tensor. For example, if the tensor dimension of the second tensor is 3D, the tensor dimension of each element scale sub-tensor after processing is also 3D. The number of elements in the tensor dimension of each element scale sub-tensor is different from the number of elements in the corresponding tensor dimension of the second tensor. Not all the number of elements in the tensor dimension of each sub-tensor is different from the number of elements in the corresponding tensor dimension of the second tensor. Based on the processing mode of coarse sampling processing and fine sampling processing, it is considered that part of each sub-tensor is consistent with the second tensor. If the number of elements in each tensor dimension of each sub-tensor is different from the number of elements in the corresponding tensor dimension of the second tensor, the second tensor cannot be restored after each sub-tensor is combined, so that the processed sub-tensor is meaningless.

[0101] In the coarse sampling processing in the embodiment, the number of elements in the same tensor dimension corresponding to the second tensor is reduced. The coarse sampling is based on a sampling ratio. The ratio value of the sampling ratio is not limited here, as long as the number of elements in the same tensor dimension corresponding to the second tensor is reduced by integer division.

[0102] In the fine sampling processing in the embodiment, the number of elements in the tensor dimension corresponding to the sub-tensor after fine sampling processing is the same as the number of elements in the same tensor dimension of the second tensor. The fine sampling processing is to keep the number of elements in the same tensor dimension of the sub-tensor consistent with the number of elements in the same tensor dimension of the second tensor, so as to restore the second tensor by combining the sub-tensors subsequently. The number of elements in the tensor dimension corresponding to the sub-tensor after fine sampling processing is the same as the number of elements in the same tensor dimension of the second tensor. The corresponding tensor dimension can be one or multiple. In the case of multiple tensor dimensions with the same number of elements, the storage space and the number of repeated sub-tensors are also large. Therefore, considering the storage space and the less repetitive sub-tensors subsequently, only one tensor dimension corresponding to the fine sampling processing is reserved to ensure that part of each sub-tensor is consistent with the second tensor.

[0103] The element scale represents the number of elements of a tensor, and is calculated as the product of the number of elements in each dimension. Since the sub-tensor is divided into multiple parts, the sub-tensors have different element scales. The sub-tensors corresponding to each element scale are decomposed and combined to obtain the approximation tensor of the second tensor. The tensor decomposition here can be the same as or different from the current tensor decomposition algorithm, and is not limited here. The object of the tensor decomposition here is the sub-tensor of each element scale, which has a lower computational complexity than the object of the traditional tensor decomposition (the second tensor).

[0104] After the low-rank tensors of the sub-tensors corresponding to each element scale are obtained by tensor decomposition, the low-rank tensors are combined to obtain the approximation tensor of the second tensor. The purpose is to combine the low-rank approximations of different sub-tensors to approximate the original high-order tensor (the second tensor). The combination method of the present embodiment is to add the low-rank tensors after tensor decomposition, but the addition here is not the addition of the low-rank tensors corresponding to the sub-tensors, but the addition according to the specific type of the sub-tensor and the corresponding coefficient relationship to approximate the second tensor.

[0105] After obtaining the approximation tensor of the second tensor, the parameters in the deep learning model, i.e., the second tensor, are replaced to obtain the first tensor. In addition, the deep learning model is called to input the initial image data to obtain the output result of the initial image data, so that the computational complexity of the entire image data processing process is reduced, and at the same time, since the computational complexity is reduced, the combination of the low-rank tensors after sub-tensor decomposition to approximate the second tensor has a lower storage complexity.

[0106] The embodiment of the present application provides an image data processing method, initial image data is acquired; a deep learning model is called to input the initial image data, and an output result of the deep learning model is acquired to serve as a processing result of the initial image data; wherein parameters of the deep learning model comprise a first tensor; the first tensor is obtained by replacing a second tensor; the second tensor is obtained by tensor decomposition and combination of each element scale corresponding sub-tensor; each element scale corresponding sub-tensor is obtained by coarse sampling processing and fine sampling processing of the second tensor; the processing mode of the coarse sampling processing and the fine sampling processing is that the tensor dimension of the second tensor is unchanged, and the number of elements corresponding to the tensor dimension of the second tensor is changed to obtain the sub-tensor. In the image data processing process, since the parameters of the deep learning model comprise the first tensor, the first tensor is obtained by replacing the second tensor, the second tensor is obtained by coarse sampling and fine sampling of the element scale of the second tensor, the sub-tensor under each element scale is obtained, and the second tensor is obtained by tensor decomposition and combination of each element scale corresponding sub-tensor, the coarse sampling and the fine sampling change the number of elements of the tensor dimension of the sub-tensor, and the high order and the large number of elements of the second tensor (the original high-dimensional tensor directly decomposed by the traditional tensor decomposition) are reduced, the calculation complexity of the tensor decomposition is reduced when the sub-tensor is decomposed, and the calculation complexity of the data processing in the image processing process is also reduced. Compared with the second tensor, the second tensor is obtained by tensor decomposition and combination of each sub-tensor, the storage complexity is low, the memory space is reduced to a certain extent in the image data processing process, and the image data processing efficiency is improved.

[0107] On the basis of the above-mentioned embodiment, in some embodiments, the corresponding sub-tensor in the coarse sampling processing and the fine sampling processing will be repeatedly sampled, therefore, the sub-tensor corresponding to each element scale after the sampling processing comprises an initial sub-tensor and an overlapping sub-tensor. The initial sub-tensor is the sub-tensor combined after the tensor dimension after the coarse sampling processing and the fine sampling processing, and the overlapping sub-tensor is the sub-tensor based on the repeated sampling in the coarse sampling processing. The sub-tensor corresponding to each element scale is obtained by the coarse sampling processing and the fine sampling processing of the second tensor, comprising:

[0108] The elements corresponding to the first dimension to the last dimension of the second tensor are respectively subjected to fine sampling processing to serve as the dimension elements of the target dimension corresponding to each element scale corresponding initial sub-tensor, wherein the number of elements of the fine sampling processing is the same as the number of elements of the dimension corresponding to the second tensor;

[0109] The sampling ratio of the coarse sampling processing is acquired;

[0110] The elements of the initial sub-tensor corresponding to each element size are respectively subjected to coarse sampling processing on the dimensions other than the target dimension to obtain dimension elements after sampling, wherein the number of elements subjected to coarse sampling processing is different from the number of elements of the dimension corresponding to the second tensor;

[0111] The dimension elements after sampling are combined with the dimension elements of the target dimension to form an initial sub-tensor, and the dimension elements after sampling are processed according to the number of dimensions of the second tensor to obtain an overlapping sub-tensor;

[0112] The initial sub-tensor and the overlapping sub-tensor are used as sub-tensors corresponding to each element size.

[0113] Specifically, after ensuring that a part of each sub-tensor is consistent with the second tensor to facilitate subsequent approximation of the second tensor, fine sampling processing can be performed on several dimensions of a sub-tensor and the corresponding several dimensions of the second tensor, or only one dimension of a sub-tensor and the corresponding one dimension of the second tensor, without limitation. In this embodiment, in order to ensure that a part of the sub-tensor is the same as the second tensor while reducing the calculation complexity and storage complexity, one dimension of a sub-tensor is kept the same as the element number of the corresponding dimension at the same position of the second tensor, for example, the second dimension of the sub-tensor is the same as the element number of the second dimension of the second tensor.

[0114] Therefore, the elements corresponding to the first dimension to the last dimension of the second tensor are respectively subjected to fine sampling processing as the dimension elements of the target dimension corresponding to each element size of the initial sub-tensor. Here, the target dimension is a fixed dimension under each initial sub-tensor, and the element number here is the same as the element number of the dimension position of the second tensor. For example, the first dimension of the first sub-tensor is the same as the element number of the first dimension of the second tensor, the second dimension of the second sub-tensor is the same as the element number of the second dimension of the second tensor, and so on, and the last dimension of the last sub-tensor is the same as the element number of the last dimension of the second tensor. That is, the target dimension of each initial sub-tensor is one, and fine sampling processing can obtain the number of dimension elements of the target dimension of each initial sub-tensor.

[0115] In the above embodiment, coarse sampling processing is based on a sampling ratio. In order to facilitate subsequent combination and subsequent data calculation, there can be a direct proportional relationship between the number of elements of a certain dimension of the second tensor and the number of elements of the initial sub-tensor, or there can be a cubic, square exponential relationship.

[0116] It should be noted that the sampling ratio in the embodiment can be one or more, and the corresponding sampling ratio can be the same or different each time the coarse sampling processing is performed. In different cases, that is, there are multiple corresponding sampling ratios.

[0117] The coarse sampling processing is based on the sampling ratio to sample the elements of the remaining dimensions in each initial sub-tensor except the target dimension to obtain the sampled dimension elements. The number of elements of the corresponding dimension position in the coarse sampling processing of the embodiment is different from the number of elements of the same dimension position in the second tensor, and the number of elements of the dimension position of the initial sub-tensor subjected to the coarse sampling processing is in a sampling ratio relationship with the number of elements of the same dimension position in the second tensor.

[0118] The sampled dimension elements and the dimension elements of the target dimension are combined to form an initial sub-tensor. The combination method here is a splicing process. For example, the dimension of the sub-tensor A1' is 3, the number of elements of the target dimension (the first dimension) is 1000, and the number of elements of the remaining dimensions is 10. Therefore, the element size of the corresponding sub-tensor A1' is 1000*10*10.

[0119] The sampled dimension elements are processed according to the number of dimensions of the second tensor to obtain an overlapping sub-tensor. That is, the number of dimensions of the second tensor is the same as the number of dimensions of the overlapping sub-tensor, that is, the same dimension number. The sampled dimension elements are repeated sampling points. According to the dimension number and the sampled dimension elements, the overlapping sub-tensor is combined. Taking the sub-tensor A1' in the above paragraph as an example, the sampling ratio of the corresponding sub-tensor is 1:100, the number of elements of the remaining dimensions is 10, that is, the number of sampled dimension elements. The corresponding overlapping sub-tensor is one, and the element size of the overlapping sub-tensor is 10*10*10.

[0120] The coarse sampling and fine sampling processing processes provided in the embodiment reduce the element size of each sub-tensor compared to the number of elements of the second tensor, so as to reduce the calculation complexity of tensor decomposition and the calculation complexity of data processing in the image processing process.

[0121] On the basis of the above embodiment, the corresponding sampling ratio of the coarse sampling processing can be the same or different. In the same case, that is, the sampling ratio of the coarse sampling processing is one. In some embodiments, the elements of the remaining dimensions in each initial sub-tensor except the target dimension are respectively subjected to coarse sampling processing according to the sampling ratio to obtain the sampled dimension elements, comprising:

[0122] Taking the target dimension as a fixed dimension, the number of elements corresponding to the dimension position of the second tensor of the remaining dimensions is subjected to coarse sampling processing according to the sampling ratio to obtain the sampled dimension elements.

[0123] Specifically, taking the target dimension as the fixed dimension, the number of elements of the remaining dimensions corresponding to the dimension position of the second tensor is coarsely sampled according to the sampling ratio to obtain the sampled dimension elements, such as the second tensor A is I1xI2x...xI d ; I represents the number of elements corresponding to each dimension, and d represents the dimension of the second tensor, that is, d dimensions. Taking the target dimension as the fixed dimension, the target dimension is respectively taken as a fixed dimension of each sub-tensor from the first dimension to the d-th dimension. The first dimension I1 is finely sampled, and the other elements are coarsely sampled to obtain the tensor form of the sub-tensor A1 as I1xJ2x...xJ d , wherein J2x...xJ d is the sampled dimension element of each remaining dimension obtained by the same sampling ratio, and J represents the number of elements of each remaining dimension of the sub-tensor. By analogy, the tensor form of the d-th sub-tensor A d is J1xJ2x...xJ d-1 xI d .

[0124] On the basis of the above embodiment, since one sampling ratio is used, the number of initial sub-tensors is the same as the number of dimensions of the second tensor, that is, the number of elements of each dimension of the second tensor is taken as the number of elements of the target dimension corresponding to the same dimension position of the initial sub-tensor. The number of dimensions of the second tensor is also the number of dimensions that are finely sampled and taken as the target dimension of the initial sub-tensor. The determination process of the initial sub-tensor is simple and clear, so as to reduce the calculation complexity in subsequent tensor decomposition. The number of elements corresponding to the remaining dimensions of the initial sub-tensor is determined based on the sampling ratio.

[0125] For example, the element size of the second tensor B is 1000x1000x1000, the sampling ratio is 1:100, the initial sub-tensor obtained by coarse sampling and fine sampling is 3, which is the same as the number of dimensions (3) of the second tensor. The element size of the initial sub-tensor B1 is 1000x10x10, the element size of the initial sub-tensor B2 is 10x1000x10, and the element size of the initial sub-tensor B3 is 10x10x1000.

[0126] In the case of a sampling ratio of 1, the number of retained overlapping sub-tensors is one. In some embodiments, the corresponding relationship of the second tensor, the initial sub-tensor and the overlapping sub-tensor is as follows:

[0127] The number of dimensions of the second tensor is reduced by 1 to obtain the repetition coefficient corresponding to the overlapping sub-tensor;

[0128] The repetition coefficient is multiplied by the overlapping sub-tensor to obtain the first sub-tensor;

[0129] The initial sub-tensors are added to obtain a second sub-tensor;

[0130] The second sub-tensor is subtracted from the first sub-tensor to obtain a second tensor.

[0131] Figure 2 A determination diagram of initial sub-tensors and overlapping sub-tensors corresponding to a sampling ratio of one is provided for an embodiment of the present application, as shown in the figure. Figure 2 Taking a 3-dimensional tensor of a second tensor 11x11x11 as an example, the corresponding initial sub-tensor 1 is 11x3x3, the initial sub-tensor 2 is 3x11x3, the initial sub-tensor 3 is 3x3x11, and the overlapping sub-tensor is 3x3x3. At this time, the overlapping sub-tensor is repeatedly sampled three times, and one overlapping sub-tensor is retained. Therefore, (3-1) overlapping sub-tensors need to be subtracted, that is, one overlapping sub-tensor is retained. Here, 3 represents the dimension, and it should be noted that the number of elements in each dimension in the embodiment is represented by the intersection of the boxes in the matrix, and the number of elements of the overlapping sub-tensor is represented by a black solid point. Figure 2

[0132] The relationship between the second tensor, the initial sub-tensor and the overlapping sub-tensor is that since the overlapping sub-tensor corresponds to 3 overlaps, the embodiment only needs to retain one overlapping sub-tensor, and the other 2 overlaps are deleted. Therefore, the corresponding relationship is:

[0133] A=A1+……+A d -(d-1)A0;

[0134] Wherein, A is the second tensor, A1 to A d are initial sub-tensors, A0 is the overlapping sub-tensor, d is the dimension of the second tensor, d-1 is the repetition coefficient, (d-1)A0 is the first sub-tensor, and A1+……+A d is the second sub-tensor.

[0135] The coarse sampling processing process and the corresponding relationship between the initial sub-tensor, the overlapping sub-tensor and the second tensor in the case of a sampling ratio of one are provided in the embodiment, which eliminates the repeated overlapping sub-tensor and improves the accuracy of the sub-tensor under each element scale.

[0136] In other embodiments, in the case of a sampling ratio of multiple in coarse sampling processing, that is, in the case of a sampling ratio of multiple in coarse sampling processing and at different times, the elements of the remaining dimensions except the target dimension in the initial sub-tensor corresponding to each element scale are respectively subjected to coarse sampling processing to obtain the dimension elements after sampling, including:

[0137] ​The element number corresponding to the dimension position of the second tensor of the remaining dimensions is respectively coarsely sampled according to each sampling ratio to obtain a dimension element after sampling, wherein the number of sampling ratios is less than or equal to the number of dimensions of the remaining dimensions.

[0138] Specifically, the element number corresponding to the dimension position of the second tensor of the remaining dimensions is respectively coarsely sampled according to each sampling ratio to obtain a dimension element after sampling. Taking the second tensor C'(1000x1000x1000) as an example, there are multiple sampling ratios, including 1:100 and 1:20, and the corresponding coarse sampling needs to be processed based on each sampling ratio. The element size of the initial sub-tensor C1' is 1000x10x50; the element size of the initial sub-tensor C2' is 1000x50x10; the element size of the initial sub-tensor C3' is 50x1000x10; the element size of the initial sub-tensor C4' is 50x10x1000; the element size of the initial sub-tensor C5' is 10x50x1000; and the element size of the initial sub-tensor C6' is 10x1000x50. As for the initial sub-tensor C1' and the initial sub-tensor C2', the element number of the first dimension is the same as the first dimension of the second tensor, so the target dimension is the first dimension, and each sampling ratio is sampled respectively. The second dimension and the third dimension of the initial sub-tensor C1' are sampled according to the respective sampling ratios, and the sampling ratio of the second dimension of the initial sub-tensor C2' is different from that of the initial sub-tensor C1'.

[0139] It should be noted that the number of sampling ratios can be the same as or different from the number of dimensions of the remaining dimensions, which can be set according to actual conditions. If the number of sampling ratios is less than the number of dimensions of the remaining dimensions, the element number under each remaining dimension is processed in turn based on each sampling ratio. If the number of sampling ratios is greater than the number of dimensions of the remaining dimensions, appropriate sampling ratios need to be selected to be less than or equal to the number of dimensions of the remaining dimensions. Of course, the more sub-tensors, the better, although the computational complexity and storage complexity of the tensor decomposition of the sub-tensor are relatively reduced compared with the tensor decomposition of the second tensor, but the total number of sub-tensors is increasing, so that the processing time occupied by the tensor decomposition of each sub-tensor is also increasing. Therefore, the number of sampling ratios can be set according to actual conditions.

[0140] The coarse sampling processing process and the number of initial sub-tensors determination process provided in the embodiment make the sub-tensors of each element size ordered and accurate, which is convenient for subsequent tensor decomposition.

[0141] On the basis of the above-mentioned embodiments, in some embodiments, the determination process of the sampling ratio of the coarse sampling processing includes:

[0142] obtaining a dimension element number rule and an integer rule of the coarse sampling processing;

[0143] determining a sampling ratio according to the dimension element number rule and the integer rule;

[0144] The dimension element number rule is that the dimension element number of the initial sub-tensor after sampling is less than the dimension element number of the second tensor.

[0145] The integer rule is that there is a positive proportional relationship with an integer coefficient between the dimension element number of the initial sub-tensor after sampling and the dimension element number of the second tensor.

[0146] Specifically, the selection of the sampling ratio mainly considers the size relationship between the dimension element number of the initial sub-tensor after sampling and the dimension element number of the second tensor and whether it is an integer division. The dimension element number of the initial sub-tensor after sampling needs to be less than the dimension element number of the second tensor, and only the dimension element number of the second tensor is reduced. In addition, the corresponding sampling ratio needs to realize the integer division processing of the dimension number of the second tensor, so that the subsequent division of the remaining sub-tensors is more convenient and fast.

[0147] Therefore, the sampling ratio is selected based on the dimension element number rule and the integer rule.

[0148] In some embodiments, the sampling ratio is the Nth power of the dimension element number of the second tensor, where N is greater than 0 and less than 1.

[0149] Specifically, the sampling ratio is based on the Nth power of the dimension element number of the second tensor, N is greater than 0 and less than 1, which is based on the case of taking the square root to reduce the dimension element number of the second tensor.

[0150] The determination process of the specific value of the sampling ratio provided in this embodiment makes the selection of the sampling ratio diversified and flexible, and makes the sub-tensors obtained by the coarse sampling processing more diversified, which is convenient for the approximation tensor obtained after the subsequent tensor decomposition.

[0151] In some embodiments, the approximation tensor of the second tensor is obtained by tensor decomposition and combination of each element scale corresponding sub-tensor, including:

[0152] tensor decomposing the initial sub-tensor and the overlapping sub-tensor in each element scale corresponding sub-tensor to obtain corresponding low-rank tensors;

[0153] combining each low-rank tensor to obtain the approximation tensor of the second tensor.

[0154] Specifically, since the sub-tensors are divided into two categories, initial sub-tensors and overlapping sub-tensors, in the present embodiment, the tensor decomposition is performed on all sub-tensors respectively during the tensor decomposition process. The tensor decomposition method here can be the same as the traditional tensor decomposition method, or it can be different, which can be set according to the actual situation. For the tensor decomposition method different from the traditional tensor decomposition method, a new tensor decomposition method can be used for sub-tensor decomposition. In order to ensure the integrity of the low-rank tensor combination process and the consistency of the low-rank tensor, the corresponding tensor decomposition method is the same during the tensor decomposition process of each sub-tensor.

[0155] The current tensor decomposition method can be canonical decomposition (Candecomp / Parafac Decomposition, CP), high-order singular value decomposition, and quantized tensor column decomposition, which is not limited here. A variety of decomposition methods can be used to verify the corresponding more appropriate decomposition method in the present embodiment, so that the low-rank tensor is more approximate to the second tensor after combination.

[0156] In some embodiments, the sampled tensor decomposition of the present embodiment is used to find the low-rank approximation of high-order tensors, so the second tensor is represented as a series of small-scale tensor cores. The initial sub-tensors and overlapping sub-tensors in each element scale corresponding sub-tensor are respectively decomposed to obtain the corresponding low-rank tensor, including:

[0157] Selecting the current sub-tensor in the initial sub-tensor and the overlapping sub-tensor in the element scale corresponding sub-tensor;

[0158] Determining the tensor rank corresponding to each dimension according to each dimension of the current sub-tensor;

[0159] Determining the tensor core corresponding to the current sub-tensor according to the tensor rank corresponding to each dimension and the corresponding element;

[0160] Taking the tensor core corresponding to the current sub-tensor as the low-rank tensor after tensor decomposition of the current sub-tensor.

[0161] Specifically, each sub-tensor is respectively decomposed, the corresponding tensor rank is determined based on the dimension of the current sub-tensor, and the corresponding tensor core is determined according to the tensor rank corresponding to each dimension and the corresponding element. Taking the first initial sub-tensor A1 as an example, the specific formula is as follows:

[0162]

[0163] Where, i1, j2, …, j d are the element indexes of the 1st, 2nd, …, dth dimensions of the current sub-tensor (the first initial sub-tensor) A1; G k is the tensor r decomposed under the kth dimension of the current sub-tensor (the first initial sub-tensor) A1k-1 ×P k ×r k , k=1,2,……,d, it should be noted that, when k=1, P1=I1; when k=2,……,d, P2,……,d=J2,……,J d ;I1,J2,……,J d The number of elements in the 1st, 2nd, ..., dth dimensions of the current sub-tensor respectively; r k is the rank of the current sub-tensor, which is a relatively small integer after the tensor decomposition is completed. d is the tensor rank index, corresponding to the tensor rank r0, r1, ..., r d ; d is the dimension of the second tensor, which is also the dimension of the sub-tensor; after the entire tensor is decomposed, it becomes d small-scale tensor cores G k , k=1, 2,……, d.

[0164] Based on the above formula, the right side of the formula is defined as (low-rank tensor), where each element satisfies Combined with the above formula, it means approximately equal, that is, the tensor core is taken as the low-rank tensor after decomposition of the current sub-tensor, which corresponds to the low-rank tensor approximation tensor of the current sub-tensor.

[0165] Figure 3 A schematic diagram of tensor decomposition of a current sub-tensor provided by an embodiment of the present invention is shown as follows: Figure 3 As shown, each box represents a tensor, the surrounding line segments represent indices, and the number of indices represents the dimension of the tensor. The i1, j2, ..., j in the brackets of the current sub-tensor A1 on the left side of the figure are d are the element indices of the 1st, 2nd, ..., dth dimensions of the current sub-tensor (the first initial sub-tensor) A1, and the tensor cores (G 1 (i1)G 2 (j2)……G d (j d ));G k It is the tensor r after the k-th dimension decomposition of the current sub-tensor (the first initial sub-tensor) A1 k-1 ×P k ×r k , k=1,2,……,d, it should be noted that, when k=1, P1=I1; when k=2,……,d, P2,……,d=J2,……,J d ;I1,J2,……,J d The number of elements in the 1st, 2nd, ..., dth dimensions of the current sub-tensor respectively; r kis the rank of the tensor of the current sub-tensor, which is a small integer after the tensor decomposition is completed. a0, a1, …, a d is the rank of the tensor of the current sub-tensor, which is a small integer after the tensor decomposition is completed. a0, a1, …, a d ; d is the dimension of the second tensor, and also the dimension of the sub-tensor; after the entire tensor decomposition is completed, it is d small-scale tensor cores G k , k = 1, 2, …, d.

[0166] In some embodiments, in combination with the above formula, the tensor core corresponding to the first dimension and the tensor core corresponding to the last dimension in each tensor core are two-dimensional matrices.

[0167] Specifically, the tensor core corresponding to the first dimension and the tensor core corresponding to the last dimension of the tensor core of the current sub-tensor are two-dimensional matrices, that is, G 1 and G d , so r0= r d = 1 needs to be satisfied.

[0168] In combination with the above formula, the index i1, j2, …, j d of the sub-tensor A1 in the bracket on the left side limits a scalar, that is, the element size is 1x1, and the element size of the first index and the last index on the right side of the formula is also 1, so the first tensor core and the last tensor core become two-dimensional matrices.

[0169] In the combination process in this embodiment, considering that the number of overlapping sub-tensors is different due to the number of sampling ratios, in the case of one sampling ratio, the corresponding overlapping sub-tensor can be set based on the repetition coefficient. Mainly considering that the second tensor is approximated in the combination process, so on the basis of the above embodiment, the same relationship exists between the low-rank tensors corresponding to each sub-tensor in the relationship between the corresponding second tensor, initial sub-tensor and overlapping sub-tensor.

[0170] In some embodiments, the low-rank tensors are combined to obtain an approximated tensor of the second tensor, including:

[0171] Adding the low-rank tensors corresponding to the initial sub-tensors to obtain a first combined tensor;

[0172] Multiplying the low-rank tensors corresponding to the overlapping sub-tensors by the corresponding repetition coefficients to obtain a second combined tensor;

[0173] Subtracting the first combined tensor from the second combined tensor to obtain the approximated tensor of the second tensor.

[0174] Specifically, the formula is as follows:

[0175]

[0176] wherein A represents a second tensor, represents a low-rank tensor corresponding to an initial sub-tensor, represents a low-rank tensor corresponding to an overlapping sub-tensor, d represents a dimension of the second tensor, and also a dimension of the sub-tensor, and d-1 represents a repetition coefficient. represents a first combined tensor, represents a second combined tensor.

[0177] The low-rank tensors are obtained through coarse sampling processing and fine sampling processing and tensor decomposition, and the approximation of the second tensor is obtained by combining the low-rank tensors. The approximation tensor of the second tensor is obtained through tensor decomposition and combination of each sub-tensor, and the storage complexity is relatively low. To some extent, the memory space is reduced in the image data processing process, and the image data processing efficiency is improved.

[0178] In some embodiments, in order to verify the storage complexity of the sub-tensor, after the second tensor is subjected to coarse sampling processing and fine sampling processing to obtain the sub-tensor corresponding to each element scale, before the sub-tensor corresponding to each element scale is subjected to tensor decomposition and combination to obtain the approximation tensor of the second tensor, the method further comprises:

[0179] obtaining the storage complexity corresponding to the second tensor and the storage complexity corresponding to the sub-tensor corresponding to each element scale;

[0180] determining whether the storage complexity corresponding to the second tensor is greater than the storage complexity corresponding to the sub-tensor corresponding to each element scale;

[0181] If yes, it is determined that the sub-tensor corresponding to each element scale is verified successfully.

[0182] Specifically, based on the complexity, the embodiment verifies by storage complexity, and obtains the storage complexity of the second tensor and the sub-tensor corresponding to each element scale. The storage complexity can be the same as the traditional storage complexity processing mode, which is not limited here.

[0183] determining whether the storage complexity corresponding to the second tensor is greater than the storage complexity corresponding to the sub-tensor corresponding to each element scale, if yes, it is determined that the storage complexity of the sub-tensor corresponding to each element scale is lower than the storage complexity of the second tensor, which further indicates that the storage complexity of the sub-tensor in the embodiment is relatively low. After successful verification, the step of obtaining the approximation tensor of the second tensor by tensor decomposition and combination of the sub-tensor corresponding to each element scale is entered for tensor decomposition and combination.

[0184] In some embodiments, the determination of the storage complexity corresponding to each sub-tensor is also related to the sampling ratio. When the number of sampling ratios is one, the process of determining the storage complexity corresponding to the sub-tensor corresponding to each element size includes:

[0185] The corresponding metric for the storage complexity of the initial subtensor is obtained by the sampling ratio;

[0186] The corresponding metric for the storage complexity of overlapping sub-tensors is obtained by the relationship between the sampling ratio and the repetition coefficient corresponding to the overlapping sub-tensors.

[0187] Specifically, the second tensor A satisfies I1=I2=……=I d =n, the corresponding storage complexity is O(n d ), the initial sub-tensor A of each element size in this embodiment k , k=1,……,d satisfies When , the storage complexity is Since the sampling ratio is m=n 1 / 2 , m represents the number of elements in each dimension of the sampling ratio of the coarsely sampled object (initial sub-tensor), n represents the number of elements in each dimension of the second tensor, then the storage complexity of the initial sub-tensor corresponds to the metric Obtained from the sampling ratio.

[0188] The corresponding metric for the storage complexity of overlapping sub-tensors is based on the relationship between the sampling ratio and the repetition coefficient (d-1) corresponding to the overlapping sub-tensor, I1 = n, The storage complexity of the overlapping sub-tensors is Therefore, both the initial sub-tensor and the overlapping sub-tensor have less complexity than the second tensor.

[0189] This embodiment provides a method for comparing the storage complexity of each sub-tensor with the second tensor before performing tensor decomposition of the sub-tensors to verify that the storage complexity of the sub-tensor is lower than the storage complexity of the second tensor. If the verification is successful, subsequent tensor decomposition of the sub-tensors is performed to improve data storage efficiency and avoid the problem of large computational complexity and storage space occupied by data processing caused by tensor decomposition due to unsuccessful verification.

[0190] In some embodiments, after performing coarse sampling and fine sampling on the second tensor to obtain sub-tensors corresponding to the scale of each element, and before performing tensor decomposition and combination on the sub-tensors corresponding to the scale of each element to obtain an approximate tensor of the second tensor, the method further includes:

[0191] Get the number of elements in each dimension of the sub-tensor corresponding to each element scale and the number of elements in each dimension of the second tensor;

[0192] determining the compression ratio of each dimension corresponding to each sub-tensor based on the element number of each dimension corresponding to the sub-tensor of each element scale and the element number of each dimension corresponding to the second tensor;

[0193] determining whether the compression ratio of each dimension corresponding to each sub-tensor is the same as the sampling ratio;

[0194] If yes, it is determined that the sub-tensor of each element scale is verified successfully.

[0195] Specifically, the verification by storage complexity in the above embodiment is verified by compression ratio in this embodiment. The element number of each dimension of the sub-tensor and the element number of each dimension of the second tensor are obtained. The compression ratio of each dimension is determined for different element numbers. Then, whether the compression ratio is the same as the preset sampling ratio is determined. If yes, it means that the verification is successful, and the coarse sampling processing and the fine sampling processing of the sub-tensor are strictly performed according to the sampling ratio. In the case of successful verification, the subsequent tensor decomposition and combination of the sub-tensor are performed, that is, the sub-tensor of each element scale is decomposed and combined to obtain the approximation tensor of the second tensor.

[0196] Before the sub-tensor is decomposed, the compression ratio of each sub-tensor is compared with the preset sampling ratio to verify the correctness of the preset sampling ratio of the sub-tensor in the coarse sampling and fine sampling processing. In the case of successful verification, the subsequent tensor decomposition is performed, the data storage efficiency is improved, and the problem of large amount of calculation and storage space occupied by data processing caused by unsuccessful verification is avoided.

[0197] On the basis of the above embodiment, the verification before the decomposition of the sub-tensor is performed to ensure that the complexity of each sub-tensor is lower than that of the second tensor. In some embodiments, after the sub-tensor of each element scale is decomposed and combined to obtain the approximation tensor of the second tensor, the following steps are further included:

[0198] obtaining a first storage complexity of a low-rank tensor approximation result corresponding to the second tensor;

[0199] determining a corresponding low-rank tensor approximation result according to the low-rank tensor corresponding to each sub-tensor of each element scale, and performing spatial complexity processing on the low-rank tensor approximation result corresponding to each sub-tensor to obtain a corresponding second storage complexity;

[0200] determining whether the first storage complexity is greater than each second storage complexity;

[0201] If yes, it is determined that the sub-tensor of each element scale is verified successfully.

[0202] Specifically, based on the pre-decomposition verification of the above embodiment, the re-verification in this embodiment further determines that the tensor decomposition of the sub-tensor in this embodiment has a lower complexity through the double verification. The first storage complexity of the low-rank tensor approximation result of the second tensor is obtained, and the low-rank tensor approximation result of the low-rank tensor of each element scale corresponding sub-tensor is also obtained. The spatial complexity is processed to obtain the corresponding second storage complexity.

[0203] If the first storage complexity is greater than each second storage complexity, it indicates that the storage complexity of the low-rank tensor approximation result of each sub-tensor is lower than the storage complexity of the low-rank tensor approximation result of the second tensor.

[0204] It should be noted that the low-rank tensor approximation result of the sub-tensor and the low-rank tensor approximation result of the second tensor have the same tensor decomposition method. Different tensor decomposition methods result in different low-rank tensors. Therefore, in order to compare the sub-tensor and the second tensor, the verification error caused by the tensor decomposition method is further excluded to improve the accuracy of the verification.

[0205] Correspondingly, the storage complexity of the low-rank tensor approximation result of the second tensor is O(dR 2 n), where R is the rank of the tensor of the second tensor, and the storage complexity of the low-rank tensor approximation result of the sub-tensor is O(dr 2 n), where r is the rank of the tensor of the sub-tensor, and r << R. Therefore, the storage complexity of the low-rank approximation result obtained by the tensor decomposition of the sub-tensor is lower, so as to realize the efficient compression of the parameters of the deep learning model.

[0206] The comparison and verification of the storage complexity of the low-rank approximation result provided in this embodiment double-verify the storage complexity of the tensor decomposition after the coarse sampling of the sub-tensor. Compared with the method of directly performing tensor decomposition on the original high-order tensor (the second tensor), the former has a lower storage complexity.

[0207] In some embodiments, if the storage complexity of the second tensor is less than or equal to the storage complexity of each element scale corresponding sub-tensor, it further includes:

[0208] Obtaining the actual sub-tensor corresponding to the storage complexity of the second tensor which is less than or equal to the storage complexity of each element scale corresponding sub-tensor;

[0209] According to the relationship between the number of actual sub-tensors and the number of sub-tensors corresponding to the element size, the sampling ratio is adjusted to obtain an adjusted sampling ratio, and returned to the step of performing coarse sampling processing on the elements of the remaining dimensions except the target dimension in the initial sub-tensor corresponding to the element size according to the sampling ratio to obtain the dimension elements after sampling.

[0210] Specifically, if there is a second tensor corresponding to a storage complexity less than or equal to the storage complexity of the sub-tensor, the actual sub-tensor less than or equal to the storage complexity of the sub-tensor needs to be obtained, and the sampling ratio is further adjusted based on the relationship between the number of actual sub-tensors and the total number of sub-tensors to obtain an adjusted sampling ratio. For example, there are only a small number of actual sub-tensors, and the corresponding sampling ratio can be reduced compared to the original sampling ratio. If there are more actual sub-tensors, a larger step size needs to be adjusted to obtain the adjusted sampling ratio. After obtaining the new sampling ratio, coarse sampling and fine sampling processing are performed again to obtain a new initial sub-tensor.

[0211] In the case of verification failure provided by the embodiment, the sampling ratio is adjusted, coarse sampling processing and fine sampling processing are performed based on different sampling ratios to determine the sub-tensor, which facilitates subsequent tensor decomposition. The sampling ratio is adjusted in time to improve the accuracy of image data processing.

[0212] The above detailed description of the image data processing method corresponds to each embodiment, and on this basis, the application also discloses an image data processing device corresponding to the above method, Figure 4 A structure diagram of an image data processing device provided by an embodiment of the application is shown in FIG. 1. Figure 4 As shown in the figure, the image data processing device comprises:

[0213] The acquisition module 11 is configured to acquire initial image data.

[0214] The calling module 12 is configured to call a deep learning model to input the initial image data, and acquire an output result of the deep learning model.

[0215] The determination module 13 is configured to determine the output result as a processing result of the initial image data.

[0216] The parameters of the deep learning model include a first tensor; the first tensor is obtained by replacing an approximation tensor of a second tensor; the approximation tensor of the second tensor is obtained by tensor decomposition and combination of sub-tensors corresponding to element sizes; the sub-tensors corresponding to the element sizes are obtained by coarse sampling processing and fine sampling processing of the second tensor; the processing mode of the coarse sampling processing and the fine sampling processing is a mode of changing the number of elements corresponding to the tensor dimensions of the second tensor to obtain the sub-tensors without changing the tensor dimensions of the second tensor.

[0217] In an aspect, the sub-tensors corresponding to each element scale include an initial sub-tensor and an overlap sub-tensor, and the sub-tensors corresponding to each element scale are obtained by performing coarse sampling processing and fine sampling processing on the second tensor, including:

[0218] The first processing submodule is configured to perform fine sampling processing on elements corresponding to a first dimension to a last dimension of the second tensor respectively to obtain dimension elements of a target dimension corresponding to each initial sub-tensor, wherein the number of elements subjected to the fine sampling processing is the same as the number of elements of the dimension corresponding to the second tensor;

[0219] The first obtaining submodule is configured to obtain a sampling ratio of the coarse sampling processing;

[0220] The second processing submodule is configured to perform coarse sampling processing on elements of dimensions other than the target dimension in each initial sub-tensor according to the sampling ratio to obtain dimension elements after sampling, wherein the number of elements subjected to the coarse sampling processing is different from the number of elements of the dimension corresponding to the second tensor;

[0221] The third processing submodule is configured to combine the dimension elements after sampling and the dimension elements of the target dimension to form the initial sub-tensor, and to process the dimension elements after sampling according to the number of dimensions of the second tensor to obtain the overlap sub-tensor;

[0222] The first sub-submodule is configured to take the initial sub-tensor and the overlap sub-tensor as the sub-tensors corresponding to each element scale.

[0223] In another aspect, when the sampling ratio of the coarse sampling processing is the same, the second processing submodule includes:

[0224] The fourth processing submodule is configured to take the target dimension as a fixed dimension, and perform coarse sampling processing on the number of elements corresponding to the dimension positions of the second tensor according to the sampling ratio to obtain the dimension elements after sampling.

[0225] In another aspect, the number of initial sub-tensors is the same as the number of dimensions of the second tensor.

[0226] In another aspect, the number of overlap sub-tensors reserved is one, and the corresponding relationship among the second tensor, the initial sub-tensor and the overlap sub-tensor is as follows:

[0227] The first obtaining submodule is configured to obtain a repetition coefficient corresponding to the overlap sub-tensor by subtracting 1 from the number of dimensions of the second tensor;

[0228] The second obtaining submodule is configured to multiply the repetition coefficient and the overlap sub-tensor to obtain a first sub-tensor;

[0229] The fifth processing submodule is configured to add each initial sub-tensor to obtain a second sub-tensor;

[0230] The third obtaining sub-module is configured to subtract the second sub-tensor from the first sub-tensor to obtain a second tensor.

[0231] In another aspect, when the sampling ratio of the coarse sampling processing is multiple and different, the second processing sub-module comprises:

[0232] The sixth processing sub-module is configured to perform coarse sampling processing on the element numbers corresponding to the dimension positions of the second tensor in the remaining dimensions with the target dimension as a fixed dimension and according to the sampling ratios respectively, to obtain dimension elements after sampling, wherein the number of the sampling ratios is less than or equal to the number of the remaining dimensions.

[0233] In another aspect, the first obtaining sub-module comprises:

[0234] The second obtaining sub-module is configured to obtain a dimension element number rule and an integer rule of the coarse sampling processing.

[0235] The first determining sub-module is configured to determine the sampling ratios according to the dimension element number rule and the integer rule.

[0236] The dimension element number rule is that the number of the dimension elements after sampling of the initial sub-tensor is less than the number of the dimension elements of the second tensor.

[0237] The integer rule is that there is a positive proportional relationship between the number of the dimension elements after sampling of the initial sub-tensor and the number of the dimension elements of the second tensor, and the coefficient is an integer.

[0238] In another aspect, the sampling ratio is Nth power of the number of the dimension elements of the second tensor, wherein N is greater than 0 and less than 1.

[0239] In another aspect, the approximation tensor of the second tensor is obtained by tensor decomposition and combination of the sub-tensors corresponding to the element scales, comprising:

[0240] The first tensor decomposition sub-module is configured to perform tensor decomposition on the initial sub-tensors and the overlapping sub-tensors in the sub-tensors corresponding to the element scales respectively to obtain corresponding low-rank tensors.

[0241] The first combination sub-module is configured to combine the low-rank tensors to obtain the approximation tensor of the second tensor.

[0242] In another aspect, the first tensor decomposition sub-module comprises:

[0243] The first selecting sub-module is configured to select a current sub-tensor from the initial sub-tensors and the overlapping sub-tensors in the sub-tensors corresponding to the element scales.

[0244] The second determining sub-module is configured to determine the tensor ranks corresponding to the dimensions according to the dimensions of the current sub-tensor.

[0245] The third determining submodule is configured to determine the tensor core corresponding to the current sub-tensor according to the tensor rank corresponding to each dimension and the element corresponding to each dimension;

[0246] The second determining submodule is configured to determine the low-rank tensor after tensor decomposition of the current sub-tensor as the tensor core corresponding to the current sub-tensor.

[0247] In another aspect, the tensor core corresponding to the first dimension and the tensor core corresponding to the last dimension in each tensor core are two-dimensional matrices.

[0248] In another aspect, the first combining submodule includes:

[0249] The seventh processing submodule is configured to add the low-rank tensors corresponding to the initial sub-tensors to obtain a first combined tensor;

[0250] The eighth processing submodule is configured to multiply the low-rank tensors corresponding to the overlapping sub-tensors by the corresponding repetition coefficients to obtain a second combined tensor;

[0251] The ninth processing submodule is configured to subtract the first combined tensor from the second combined tensor to obtain the approximation tensor of the second tensor.

[0252] In another aspect, after the second tensor is subjected to coarse sampling processing and fine sampling processing to obtain the sub-tensors corresponding to each element scale, before the sub-tensors corresponding to each element scale are subjected to tensor decomposition and combination to obtain the approximation tensor of the second tensor, the method further includes:

[0253] The third obtaining submodule is configured to obtain the storage complexity corresponding to the second tensor and the storage complexity corresponding to the sub-tensors corresponding to each element scale;

[0254] The first judging submodule is configured to determine whether the storage complexity corresponding to the second tensor is greater than the storage complexity corresponding to the sub-tensors corresponding to each element scale; if yes, the first verifying submodule is triggered;

[0255] The first verifying submodule is configured to determine that the verification of the sub-tensors corresponding to each element scale is successful.

[0256] In another aspect, when the number of sampling ratios is one, the determination process of the storage complexity corresponding to the sub-tensors corresponding to each element scale includes:

[0257] The fourth obtaining submodule is configured to obtain the metric corresponding to the storage complexity of the initial sub-tensor from the sampling ratio;

[0258] The fifth obtaining submodule is configured to obtain the metric corresponding to the storage complexity of the overlapping sub-tensor from the relationship between the sampling ratio and the repetition coefficient corresponding to the overlapping sub-tensor.

[0259] On the other hand, after the second tensor is coarsely sampled and finely sampled to obtain the sub-tensors corresponding to the element scales, before the sub-tensors corresponding to the element scales are tensor decomposed and combined to obtain the approximation tensor of the second tensor, the method further comprises:

[0260] The fourth obtaining sub-module is configured to obtain the number of elements in each dimension corresponding to each sub-tensor and the number of elements in each dimension corresponding to the second tensor;

[0261] The fourth determining sub-module is configured to determine the compression ratio of each dimension corresponding to each sub-tensor based on the number of elements in each dimension corresponding to each sub-tensor and the number of elements in each dimension corresponding to the second tensor;

[0262] The second judging sub-module is configured to judge whether the compression ratio of each dimension corresponding to each sub-tensor is the same as the sampling ratio; if yes, the second verifying sub-module is triggered;

[0263] The second verifying sub-module is configured to determine that the verification of each sub-tensor corresponding to each element scale is successful.

[0264] On the other hand, after the sub-tensors corresponding to the element scales are tensor decomposed and combined to obtain the approximation tensor of the second tensor, the method further comprises:

[0265] The fifth obtaining sub-module is configured to obtain the first storage complexity of the low-rank tensor approximation result corresponding to the second tensor;

[0266] The fifth determining sub-module is configured to determine the low-rank tensor approximation result corresponding to each sub-tensor according to the low-rank tensor corresponding to each sub-tensor, and perform spatial complexity processing on the low-rank tensor approximation result corresponding to each sub-tensor to obtain the corresponding second storage complexity;

[0267] The third judging sub-module is configured to judge whether the first storage complexity is greater than each second storage complexity; if yes, the third verifying sub-module is triggered;

[0268] The third verifying sub-module is configured to determine that the verification of each sub-tensor corresponding to each element scale is successful.

[0269] On the other hand, if the storage complexity corresponding to the second tensor is less than or equal to the storage complexity corresponding to each sub-tensor corresponding to each element scale, the method further comprises:

[0270] The sixth obtaining sub-module is configured to obtain the actual sub-tensor to which the storage complexity corresponding to the second tensor is less than or equal to the storage complexity corresponding to each sub-tensor corresponding to each element scale;

[0271] The first adjusting sub-module is configured to adjust the sampling ratio according to the relationship between the actual number of sub-tensors and the number of sub-tensors corresponding to the element scale, to obtain an adjusted sampling ratio, and trigger the second processing sub-module.

[0272] Since the embodiments of the device part correspond to the embodiments described above, the embodiments of the device part are described with reference to the embodiments of the method part described above, and will not be described here.

[0273] For the image data processing device provided in the embodiments of the present application, please refer to the method embodiments described above, and the present application will not be described here, which has the same beneficial effects as the image data processing method described above.

[0274] Figure 5 The structural diagram of the image data processing device provided in the embodiments of the present application is shown in FIG. 1, which includes: Figure 5

[0275] The memory 21 is configured to store a computer program.

[0276] The processor 22 is configured to execute the computer program to realize the steps of the image data processing method.

[0277] The image data processing device provided in the embodiments can include but is not limited to a smart phone, a tablet computer, a notebook computer, or a desktop computer, etc.

[0278] The processor 22 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 22 can be implemented in at least one of a hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA). The processor 22 can also include a main processor and a coprocessor. The main processor is a processor for processing data in a wake-up state, also known as a central processing unit (CPU). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 22 can be integrated with a graphics processor (GPU) for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 22 can also include an artificial intelligence (AI) processor for processing machine learning-related computing operations.

[0279] ​The memory 21 can include one or more computer-readable storage media. The computer-readable storage media can be non-transitory. The memory 21 can also include high-speed random access memory and can include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices. In this embodiment, the memory 21 is used to store at least the following computer program 211, wherein the computer program is loaded and executed by the processor 22, and can implement the related steps of the image data processing method disclosed in any of the preceding embodiments. In addition, the resources stored in the memory 21 can also include an operating system 212 and data 213, etc., and the storage mode can be temporary storage or permanent storage. The operating system 212 can include Windows, Unix, Linux, etc. The data 213 can include but is not limited to data related to the image data processing method, etc.

[0280] In some embodiments, the image data processing device can further include a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27.

[0281] Those skilled in the art can understand that the structure shown in the above embodiments does not constitute a limitation on the image data processing device, and can include more or fewer components than those shown. Figure 5

[0282] The processor 22 implements the image data processing method provided in any of the preceding embodiments by invoking the instructions stored in the memory 21.

[0283] For the image data processing device provided by the present application, please refer to the above method embodiments, and the present application will not be repeated here, which has the same beneficial effects as the above image data processing method.

[0284] Further, the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and the computer program is executed by the processor 22 to implement the steps of the above image data processing method.

[0285] ​It can be understood that if the method in the above embodiment is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the present application or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and performs all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0286] For the computer readable storage medium provided by the present application, please refer to the above method embodiment, and the present application will not be repeated here, which has the same beneficial effects as the above image data processing method.

[0287] The above provides a detailed description of the image data processing method, device, equipment and medium provided by the present application. The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the related parts are described in the method part. It should be pointed out that for ordinary skilled in the art, without departing from the principle of the present application, the present application can be improved and modified, and these improvements and modifications also fall within the protection scope of the present application.

[0288] It should be noted that in the present specification, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the term "include" "contain" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitation, the element defined by the sentence "including a…" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

Claims

1. A method for processing image data, characterized in that: include: Obtaining initial image data; Calling a deep learning model to input the initial image data and obtaining an output result of the deep learning model; outputting a result as a result of processing the initial image data; Among them, the parameters of the deep learning model include a first tensor; the first tensor is obtained by replacing the approximation tensor of the second tensor; the approximation tensor of the second tensor is obtained by tensor decomposition and combination of sub-tensors corresponding to the scale of each element; the sub-tensors corresponding to the scale of each element are obtained by coarse sampling and fine sampling of the second tensor; the processing method of coarse sampling and fine sampling is that the tensor dimension of the second tensor remains unchanged, and the number of elements corresponding to the tensor dimension of the second tensor is changed to obtain the sub-tensor.

2. The image data processing method according to claim 1, wherein: The sub-tensors corresponding to the scale of each element include an initial sub-tensor and an overlapping sub-tensor. The sub-tensors corresponding to the scale of each element are obtained by coarse sampling and fine sampling of the second tensor, including: Fine-sample the elements corresponding to the first dimension to the last dimension of the second tensor to use them as the dimensional elements of the target dimension corresponding to the initial sub-tensor corresponding to the scale of each element, wherein the number of elements fine-sampled is the same as the number of elements in the corresponding dimension of the second tensor; Get the sampling ratio of coarse sampling processing; performing coarse sampling processing on the elements of the remaining dimensions except the target dimension in the initial sub-tensor corresponding to each element scale according to the sampling ratio to obtain sampled dimensional elements, wherein the number of elements processed by the coarse sampling is different from the number of elements of the dimension corresponding to the second tensor; Combining the sampled dimensional elements with the dimensional elements of the target dimension to form an initial sub-tensor; and processing the sampled dimensional elements according to the number of dimensions of the second tensor to obtain overlapping sub-tensors; The initial sub-tensor and the overlapping sub-tensor are used as sub-tensors corresponding to the scale of each element.

3. The image data processing method according to claim 2, wherein: When the sampling ratios of the coarse sampling process are the same, coarse sampling processes are performed on the elements of the remaining dimensions except the target dimension in the initial sub-tensor corresponding to each element scale according to the sampling ratio to obtain sampled dimensional elements, including: The target dimension is used as a fixed dimension, and the number of elements corresponding to the dimensional positions of the second tensor corresponding to the remaining dimensions is coarsely sampled according to the sampling ratio to obtain the sampled dimensional elements.

4. The image data processing method according to claim 3, wherein: The number of initial sub-tensors is the same as the number of dimensions of the second tensor.

5. The image data processing method according to claim 4, characterized in that: The number of retained overlapping sub-tensors is one, and the corresponding relationship between the second tensor, the initial sub-tensor, and the overlapping sub-tensor is as follows: Subtract 1 from the dimension of the second tensor to get the repetition coefficient corresponding to the overlapping sub-tensor; Multiplying the repetition coefficient and the overlapping sub-tensor to obtain a first sub-tensor; Adding the initial sub-tensors together to obtain the second sub-tensor; The second sub-tensor is subtracted from the first sub-tensor to obtain a second tensor.

6. The image data processing method according to claim 2, wherein: When the sampling ratios of the coarse sampling process are multiple and different, the elements of the remaining dimensions except the target dimension in the initial sub-tensor corresponding to each element scale are subjected to coarse sampling according to the sampling ratios to obtain the sampled dimensional elements, including: Taking the target dimension as a fixed dimension, the number of elements corresponding to the dimensional positions of the second tensor corresponding to the remaining dimensions are coarsely sampled according to each of the sampling ratios to obtain the sampled dimensional elements, wherein the number of the sampling ratios is less than or equal to the number of dimensions of the remaining dimensions.

7. The image data processing method according to any one of claims 2 to 6, characterized in that: The process of determining the sampling ratio in the coarse sampling process includes: Obtain the number of dimension elements and integer rules for coarse sampling processing; Determining the sampling ratio according to the dimensional element number rule and the integer rule; The rule for the number of dimensional elements is that the number of dimensional elements of the initial sub-tensor after sampling is less than the number of dimensional elements of the second tensor; The integer rule is that there is a positive proportional relationship between the number of dimensional elements of the sampled initial sub-tensor and the number of dimensional elements of the second tensor, with a coefficient being an integer.

8. The image data processing method according to any one of claims 2 to 6, characterized in that: The sampling ratio is N to the power of the number of dimensional elements of the second tensor, where N is greater than 0 and less than 1.

9. The image data processing method according to claim 7, wherein: The approximation tensor of the second tensor is obtained by tensor decomposition and combination of sub-tensors corresponding to the scale of each element, including: Perform tensor decomposition on the initial sub-tensor and overlapping sub-tensor in the sub-tensor corresponding to each element scale to obtain the corresponding low-rank tensor; The low-rank tensors are combined to obtain an approximation tensor of the second tensor.

10. The image data processing method according to claim 9, wherein: The initial sub-tensor and overlapping sub-tensor in the sub-tensor corresponding to each element scale are decomposed into corresponding low-rank tensors, including: Select the current sub-tensor from the initial sub-tensor and the overlapping sub-tensor within the sub-tensor corresponding to the size of each element; Determine the tensor rank corresponding to each dimension according to the dimensions of the current sub-tensor; The tensor rank and corresponding elements of each dimension are used to determine the tensor core corresponding to the current sub-tensor; The tensor core corresponding to the current sub-tensor is used as the low-rank tensor after the tensor decomposition of the current sub-tensor.

11. The image data processing method according to claim 10, wherein: In each tensor core, the tensor core corresponding to the first dimension and the tensor core corresponding to the last dimension are two-dimensional matrices.

12. The image data processing method according to claim 10, wherein: Combining the low-rank tensors to obtain an approximation tensor of the second tensor includes: Adding the low-rank tensors corresponding to the initial sub-tensors to obtain a first combined tensor; Multiplying the low-rank tensor corresponding to the overlapping sub-tensor with the corresponding repetition coefficient to obtain a second combined tensor; The first combined tensor and the second combined tensor are subtracted to obtain an approximation tensor of the second tensor.

13. The image data processing method according to claim 12, wherein: After the second tensor is subjected to coarse sampling and fine sampling to obtain sub-tensors corresponding to the scale of each element, and before the sub-tensors corresponding to the scale of each element are subjected to tensor decomposition and combination to obtain an approximation tensor of the second tensor, the method further includes: Get the storage complexity corresponding to the second tensor and the storage complexity corresponding to the sub-tensor corresponding to the size of each element; Determine whether the storage complexity corresponding to the second tensor is greater than the storage complexity corresponding to the sub-tensor corresponding to the size of each element; If so, it is determined that the sub-tensor corresponding to the size of each element is successfully verified.

14. The image data processing method according to claim 13, wherein: When the number of the sampling ratios is one, the process of determining the storage complexity of the sub-tensor corresponding to each element size includes: The corresponding metric of the storage complexity of the initial sub-tensor is obtained by the sampling ratio; The metric corresponding to the storage complexity of the overlapping sub-tensors is obtained by the relationship between the sampling ratio and the repetition coefficient corresponding to the overlapping sub-tensors.

15. The image data processing method according to claim 12, wherein: After the second tensor is subjected to coarse sampling and fine sampling to obtain sub-tensors corresponding to the scale of each element, and before the sub-tensors corresponding to the scale of each element are subjected to tensor decomposition and combination to obtain an approximation tensor of the second tensor, the method further includes: Get the number of elements in each dimension of the sub-tensor corresponding to each element scale and the number of elements in each dimension of the second tensor; Determine the compression ratio of each dimension corresponding to each sub-tensor based on the number of elements in each dimension corresponding to the sub-tensor corresponding to each element scale and the number of elements in each dimension corresponding to the second tensor; Determine whether the compression ratio of each dimension corresponding to each sub-tensor is the same as the sampling ratio; If they are the same, the sub-tensor corresponding to the scale of each element is verified successfully.

16. The image data processing method according to claim 13, wherein: After performing tensor decomposition and combination of the sub-tensors corresponding to the scale of each element to obtain the approximate tensor of the second tensor, it also includes: Get the first storage complexity of the low-rank tensor approximation result corresponding to the second tensor; Determine a corresponding low-rank tensor approximation result according to a low-rank tensor corresponding to a sub-tensor corresponding to each element scale, and perform space complexity processing on the low-rank tensor approximation result corresponding to each sub-tensor to obtain a corresponding second storage complexity; Determine whether the first storage complexity is greater than each of the second storage complexities; If so, it is determined that the sub-tensor corresponding to the size of each element is successfully verified.

17. The image data processing method according to claim 13, wherein: If the storage complexity of the second tensor is less than or equal to the storage complexity of the sub-tensor corresponding to the size of each element, it also includes: Get the actual sub-tensor whose storage complexity corresponding to the second tensor is less than or equal to the storage complexity corresponding to the sub-tensor corresponding to the size of each element; According to the relationship between the number of actual sub-tensors and the number of sub-tensors corresponding to the scale of each element, the sampling ratio is adjusted to obtain the adjusted sampling ratio, and the step of performing coarse sampling processing on the elements of the remaining dimensions except the target dimension in the initial sub-tensor corresponding to each element scale according to the sampling ratio to obtain the sampled dimensional elements is returned.

18. An image data processing device, characterized in that: include: An acquisition module, used for acquiring initial image data; A calling module, configured to call a deep learning model to input the initial image data and obtain an output result of the deep learning model; a determination module, configured to use an output result as a processing result of the initial image data; Among them, the parameters of the deep learning model include a first tensor; the first tensor is obtained by replacing the approximation tensor of the second tensor; the approximation tensor of the second tensor is obtained by tensor decomposition and combination of sub-tensors corresponding to the scale of each element; the sub-tensors corresponding to the scale of each element are obtained by coarse sampling and fine sampling of the second tensor; the processing method of coarse sampling and fine sampling is that the tensor dimension of the second tensor remains unchanged, and the number of elements corresponding to the tensor dimension of the second tensor is changed to obtain the sub-tensor.

19. An image data processing device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the image data processing method according to any one of claims 1 to 17 when executing the computer program.

20. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image data processing method according to any one of claims 1 to 17 are implemented.

Images