End-to-end 360-degree image compression method based on multiple scales

Abstract

The invention relates to the technical field of image coding, in particular to a multi-scale-based end-to-end 360-degree image compression method, which comprises the following steps that: an image compression model is constructed and trained, the image compression model comprises three layers of paths, and each layer of path comprises a coding module, a self-adaptive scale block coding module and a decoding module; end-to-end 360-degree image compression is realized through the trained image compression model; according to the invention, the compression efficiency can be improved, and the bit rate redundancy caused by excessive stretching is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image coding technology, and in particular to a multi-scale end-to-end 360-degree image compression method. Background Art

[0002] 360° images, also known as panoramic images, are a new type of media that can record the full range of vision of the scene being shot, and allow users to explore the scene from any perspective. Nowadays, with the rapid development of virtual reality (VR) devices and the popularity of mobile phones that can shoot panoramic images, the market size of panoramic images has also been greatly expanded. Compared with flat images, panoramic images can cover more content, so they require more storage space and transmission bandwidth, which also poses a great challenge to the growing storage and transmission requirements of panoramic images. In order to adapt to the development of VR devices in the future, a novel image compression method that adapts to panoramic images must be proposed.

[0003] Currently, 360° images need to be converted into planar images with specific projections through projection methods such as equirectangular projection (ERP), and then saved using existing codecs. However, there are some problems when using the ERP method to project a sphere onto a rectangle to obtain an ERP image. First, the object will be deformed, which makes the deformation on the ERP image inconsistent with what people see on VR devices or 360° devices; second, ERP images also have the problem of unbalanced sampling. Currently, in ERP images, regions of different latitudes are represented by the same number of pixels, that is, the width of the ERP image, so objects in high latitudes will be oversampled. Different techniques have been studied to overcome these problems, for example, introducing regional downsampling and reprojection methods to reduce the bits on the oversampled area. However, these methods either decompose the ERP image into tiles, downsample them, and then merge the tiles into a new image; or map the sphere to a polyhedron and merge all surfaces into a new planar image. In both cases, the new image will destroy the local smoothness of the natural image and create artificial sharp edges between different tiles or surfaces.

[0004] With the booming development of end-to-end image generation and entropy coding, people are increasingly looking to abandon pixel coding in traditional image compression methods and use feature coding based on deep neural networks. Deep neural networks have proven to have more reliable global smoothness and higher compression rates than traditional models in the field of image coding. Some scholars use autoencoder-like architectures to learn nonlinear transformations and combine them with quantization to optimize the entire process in an end-to-end manner, thereby achieving higher compression efficiency than traditional manual methods. Other scholars have shown that combining autoregression and hierarchical priors can better exploit the probabilistic structure of potential representations. Although these end-to-end models perform well for 2D images, they are less effective for 360° images because they cannot adequately address the overstretching problem in ERP images. Recent studies have explored deep neural network models for 360° image compression. Some scholars introduced an adaptive 360° image tile representation that optimizes the size of each tile to address the overstretching problem. Other scholars have proposed the first end-to-end optimized 360° image compression model, which uses an adaptive latitude map to mask features, thereby reducing rate loss. Although their work achieved state-of-the-art performance in end-to-end 360° image compression, they did not consider that different regions of a 360° image should be processed separately.

[0005] In general, there are two problems with current 360° image compression: (1) There is a lack of effective optimization methods for intermediate features of 360° images. (2) There is no deep neural network model that is very effective for the overstretching characteristics of the two poles of 360° images. Summary of the invention

[0006] To solve the above problems, the present invention provides a multi-scale end-to-end 360-degree image compression method, characterized in that an image compression model is constructed and trained, the image compression model includes a three-layer path, each layer path includes an encoding module, an adaptive scale block encoding module and a decoding module; end-to-end 360-degree image compression is achieved through the trained image compression model, including the following steps:

[0007] S1. Perform ERP projection on the acquired 360-degree image to form a 2D projection image, and divide the 2D projection image into three groups of images;

[0008] S2. Input the i=1, 2, 3 groups of images into the encoding module in the i-th layer path to obtain a set of encoding features;

[0009] S3. input the i-th group of coding features into the adaptive scaled block coding module in the i-th layer path to obtain a group of optimal block pattern maps, each group of optimal block pattern maps includes multiple blocks; wherein the size of all blocks in the first group of optimal block pattern maps is 1×8, the size of all blocks in the second group of optimal block pattern maps is 2×4, and the size of all blocks in the third group of optimal block pattern maps is 4×4;

[0010] S4. The i-th group of images and the i-th group of coding features are subjected to an average pooling layer to obtain a group of similar merged features, and each group of similar merged features is quantized and upsampled to obtain a group of quantized features;

[0011] S5. Input the i-th group of quantized features into the decoding module in the i-th layer path, and merge the outputs of the decoding modules of the three-layer paths to obtain a 360-degree image compression result.

[0012] Beneficial effects of the present invention:

[0013] ERP projection of 360-degree images will lead to overstretching, which increases storage requirements without corresponding visual effect returns. In order to improve compression efficiency, it is important to reduce the bit rate redundancy caused by such overstretching. To this end, the present invention proposes a multi-scale end-to-end 360-degree image compression method.

[0014] First, in order to adapt to the over-stretching rate of different regions, the present invention introduces multi-scale rectangular convolution kernels to capture the rectangular receptive field and reduce overfitting according to the over-stretching rate. Secondly, an adaptive scale block method is proposed, which simultaneously considers the over-stretching rate and texture complexity of 360° images, effectively reducing the bitrate redundancy of over-stretching and less important areas. At the same time, a new end-to-end compression framework specifically for 360° images is proposed. Experimental results show that compared with the latest learned 360° image compression model, our model outperforms all other image compression methods and reduces the bitrate by 15.76%. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of the method of the present invention;

[0016] Figure 2 This is a schematic diagram of overstretching of the ERP image of the present invention;

[0017] Figure 3 It is a schematic diagram of the structure of the image compression model of the present invention;

[0018] Figure 4 It is a detailed structural diagram of each module of the model of the present invention;

[0019] Figure 5 is the rate-distortion curve of various convolution kernel sizes in the first group of the present invention;

[0020] Figure 6 It is a specific implementation of the adaptive scale block method of the present invention;

[0021] Figure 7 It is the simulation result diagram of the present invention;

[0022] Figure 8 Schematic diagram of model training according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] At present, although some scholars have proposed the first 360° image compression model based on end-to-end optimization, their model is still designed based on the characteristics of traditional planar images. In their specific scheme, the intermediate features need to be reduced based on mask adaptation to reduce the bit rate of the intermediate features, which can be understood as reducing the intermediate features. However, in the projected image generated by the projection of the 360-degree image, the severity of overstretching between different regions is inconsistent. They did not consider the independence between different regions but processed the entire image uniformly. This cannot efficiently take into account high-latitude and low-latitude regions. At the same time, reducing features will cause vacancies in feature information. Although this method uses latitude-adaptive reduction, the oversampling of the 360° image is caused by projection stretching, that is, a small amount of information is stretched to fill the image. Therefore, we believe that simple feature reduction is not the optimal solution, and the equatorial region of the 360° image is not stretched and should not be reduced.

[0025] In addition, currently, the networks for 360° image compression all use the network structure designed for flat images. If a model equipped with a special deep neural network is not proposed, it will be difficult to fundamentally solve the oversampling problem of 360° images.

[0026] Based on the above foundation, the present invention provides a multi-scale end-to-end 360-degree image compression method, including constructing and training an image compression model, wherein the image compression model includes a three-layer path, each layer path includes an encoding module, an adaptive scale block encoding module and a decoding module; end-to-end 360-degree image compression is achieved through the trained image compression model, such as Figure 1 As shown, the following steps are included:

[0027] S1. Perform ERP projection on the acquired 306-degree image to form a 2D projection image, and divide the 2D projection image into three groups of images.

[0028] Specifically, a 360° image is usually represented by a sphere with longitude and latitude as indexes. In order to facilitate compression, the sphere needs to be projected onto a 2D image. Figure 2 As shown, regions A and B on the sphere have different sizes, but after projection mapping, they appear as regions of equal size in the ERP image. This leads to an overstretching of horizontal pixels between the sphere and the ERP image content, a problem known as horizontal overstretching. Specifically, the closer a region is to a pole, the smaller its area on the sphere and the more noticeable the stretching in the corresponding ERP image. Pixel stretching varies with latitude. To quantify the extent of this distortion, we introduce an overstretching factor.

[0029] Dividing too many regions will make the model too complicated, while dividing too few regions will reduce the efficiency of the model. According to the stretch distribution of the ERP image, the present invention divides the ERP image into 16 regions according to the height of the ERP image. Encoding each region separately will complicate the process; therefore, we calculate the overstretch factor value of each region, merge the regions with similar average overstretch ratios, and form three groups of images. Based on this, we divide our model into three layers according to the divided regions. For each layer, the model uses convolution kernels and pooling blocks of different scales for processing respectively.

[0030] Table 1 Classification criteria

[0031]

[0032]

[0033] Specifically, as shown in Table 1, the first column represents the number of rows of the regional image in the 2D projection image to which it belongs, the second column represents the overstretching factor value of the regional image, the third column is the average overstretching ratio of the regional image, and the fourth column is the group to which the regional image belongs. Step S1 divides the 2D projection image into three groups of images according to the overstretching factor, including:

[0034] The 2D projection image is evenly divided into 16 regional images according to the height; each regional image uses its own central pixel to calculate the overstretching factor value through the following formula:

[0035]

[0036] Among them, Lat(i) represents the overstretching factor value of the i-th region image, η represents the scale parameter weight, and θ irepresents the latitude value of the i-th regional image, h represents the height of the 2D projection image, and N represents the number of divided regional images; the average overstretching ratio of each regional image is calculated by the overstretching factor value;

[0037] The first region image, the second region image, the fifteenth region image, and the sixteenth region image constitute a first group of images;

[0038] The 3rd region image, the 4th region image, the 13th region image and the 14th region image constitute the second group of images;

[0039] The 5th region image, the 6th region image, the 7th region image, the 8th region image, the 9th region image, the 10th region image, the 11th region image and the 12th region image constitute the third group of images.

[0040] S2. Input the i=1, 2, 3 groups of images into the encoding module in the i-th layer path to obtain a set of encoding features.

[0041] S3. Input the i-th group of coding features into the adaptive scale block coding module in the i-th layer path to obtain a group of optimal block pattern maps, each group of optimal block pattern maps includes multiple blocks; wherein the size of all blocks in the first group of coding features is 1×8, the size of all blocks in the second group of coding features is 2×4, and the size of all blocks in the third group of coding features is 4×4.

[0042] S4. The i-th group of coding features and the i-th group of best block pattern images are subjected to an average pooling layer to obtain a group of similar merged features, and each group of similar merged features is quantized and upsampled to obtain a group of quantized features.

[0043] S5. Input the i-th group of quantized features into the decoding module in the i-th layer path, and merge the outputs of the decoding modules of the three-layer paths to obtain a 360-degree image compression result.

[0044] In one embodiment, if Figure 3 As shown, the image compression model includes three-layer paths, each of which includes an encoding module, an adaptive scale block encoding module and a decoding module.

[0045] Preferably, the encoding module adopts the design of the current mainstream end-to-end image compression model, uses convolutional layers for feature extraction, and enhances the performance of the network through residual blocks (RBs) and attention modules. Since the ERP image (i.e., the 2D projection image) has an overstretching problem, it is difficult for the square convolution kernel to effectively extract features from the overstretched area; at the same time, the square convolution kernel is prone to overfitting. Therefore, the present invention proposes to use a rectangular convolution kernel for effective substitution.

[0046] like Figure 4As shown in the figure, the encoding modules of the three-layer path have the same structure, including 4 rectangular convolutional layers, 3 residual blocks and 2 attention modules; among them, a residual block is connected between every two adjacent rectangular convolutional layers, and an attention module is connected after the second residual block and the fourth rectangular convolutional layer respectively.

[0047] Specifically, most end-to-end networks use 5×5 convolution kernels to balance computational complexity and receptive field size, however, the present invention proposes to use convolution kernels with larger horizontal size relative to height. Since the average overstretching ratios between the three groups of images are very different, it is necessary to design different rectangular convolution kernels for each group of images.

[0048] In this embodiment, five rectangular convolution kernels are specifically designed for the first group of images, balancing the trade-off between overstretching and time efficiency. In order to evaluate the distortion, the present invention uses WS-PSNR as an indicator. Figure 5 The rate-distortion curves of five different rectangular convolution kernels are shown. It can be observed that the 5×7 rectangular convolution kernel performs the worst in the first-layer path compared to other rectangular convolution kernels. In contrast, the 5×9, 5×11, and 5×13 rectangular convolution kernels show similar performance. However, larger rectangular convolution kernels lead to longer training and inference times. In particular, the 5×11 and 5×13 rectangular convolution kernels require more time to process but provide performance comparable to the 5×9 rectangular convolution kernel. Considering the time consumption, we selected the 5×9 rectangular convolution kernel for the first group of images, which can also be said to be the first-layer path. The same testing method is applied to the other groups. Finally, the image compression model uses rectangular convolution kernels of three scales to match the three groups of images. The first group of images has the most serious over-stretching, so in the first layer path used to process the first group of images, a horizontally wider 5×9 rectangular convolution kernel is used; for the second group of images with a moderate degree of over-stretching, a 5×7 rectangular convolution kernel is used to match them in the second layer path used to process the second group of images; because the average over-stretching rate of the third group of images is close to 1, a 5×5 rectangular convolution kernel is used for processing in the third layer path used to process the third group of images.

[0049] Preferably, if Figure 3 As shown in FIG. 1 , the structures of the adaptive scale block coding modules of the three-layer paths are the same, and both include two adaptive scale block units; each adaptive scale block unit includes a first convolution layer, a residual block, a rectangular convolution layer, a second convolution layer, a BN layer, and a Gumbel softmax layer that are cascaded in sequence.

[0050] Specifically, due to the overstretching problem, overstretched pixels are usually very similar, so the bit rate cost is reduced by merging. As shown in Table 1, the average overstretching ratio of the first group of images is about 1:7.2, the average overstretching ratio of the second group of images is about 1:2, and the average overstretching ratio of the third group of images is about 1:1.3. In order to match these average overstretching ratios, the present invention designs three sizes of blocks. Figure 6 As shown in (a), for the first layer with an average overstretching ratio of 1:7.2, the present invention designs a 1×8 horizontal block to approximately correspond to the ratio; for the second layer, a 2×4 block is used to correspond; for the third layer, a 4×4 square block is used to match the ratio; the purpose of this is to adaptively compress the code rate of unimportant areas. The size of the block determines the size of each group of encoded features that can be pooled.

[0051] Not only overstretching, texture complexity is also related to the design of the block. In order to adapt to different texture complexities, the present invention divides different scales for each size of block, such as Figure 6 (a). Specifically, Figure 6 Block 1 in layer 1 in (a) is suitable for features with complex textures to retain detailed information, block 2 or block 3 is more suitable for simpler or overstretched textures, and the remaining blocks are designed for medium texture complexity. After a set of coded features of each group of images passes through the corresponding adaptive scale block coding module, the neural network selects the optimal pooling mode for each coded feature in each set of coded features, that is, the optimal pooling size and pooling scale are selected for each code to form a block with the best size mode, thereby obtaining a set of optimal block pattern maps.

[0052] exist Figure 6 (b) takes one of the coding features in the third group as an example. First, an adaptive scale block coding module is used to generate a block of the best size mode for the coding feature. Then, the coding feature and its blocks of the best size mode are passed through the average pooling layer to merge similar values. Then, a quantization operation is performed to obtain a quantized value, which is upsampled to generate the final quantized feature and transmitted to the decoding module. We observe that similar values ​​in the upper right corner of the coding feature are merged into one value, while different values ​​in the lower right corner are retained to achieve adaptive compression bit rate.

[0053] Preferably, the decoding modules of the three-layer path have the same structure, all including 4 transposed rectangular convolution layers, 3 residual blocks and 2 attention modules; wherein, a residual block is connected between every two adjacent transposed rectangular convolution layers, and an attention module is respectively connected before the second residual block and the first transposed rectangular convolution layer; in the first-layer path, the rectangular convolution kernel size used by all transposed rectangular convolution layers is 5×9; in the second-layer path, the rectangular convolution kernel size used by all transposed rectangular convolution layers is 5×7; in the third-layer path, the rectangular convolution kernel size used by all transposed rectangular convolution layers is 5×5.

[0054] Preferably, rate distortion is a widely used indicator for evaluating model coding performance. During the training process of the image compression model, the training is performed with the goal of minimizing the rate distortion loss. The rate distortion loss is calculated as follows:

[0055]

[0056] Among them, Loss represents the rate distortion loss, R i Represents the quantized features corresponding to the i-th group of images The number of digits, D i represents the distortion value of the i-th group of images, λ represents the hyperparameter, pooling(·) represents the average pooling operation, and P(·) represents the Gaussian mixture distribution model; X i represents the i-th group of images, represents the output of the i-th group of images after the image compression model, Represents the operation of calculating the mean square error. i It represents the pixel group weight of the i-th group of images. The pixel group weight is obtained by adding the position weights of all pixels in the group of images. The position weight of each pixel is calculated as follows:

[0057]

[0058] ω(k,l) represents the position weight of the pixel point (k,l) in the 2D projection image, and h represents the height of the 2D projection image.

[0059] Specifically, the present invention uses a training dataset designed for end-to-end 360° image compression in the training phase. The training dataset consists of 18,160 high-quality ERP images, each with a resolution of 512×1024 pixels. The images in the training dataset include panoramic natural images and panoramic indoor images, which have rich textures and natural structures and can closely reflect the photographic environment of the real world.

[0060] At the same time, the present invention selects 200 high-quality ERP images from the training data set to form a test data set. In order to evaluate the objective quality of 360° images in the observation space, the present invention uses quality perception methods weighted to spherical uniformity, including WS-PSNR and WS-SSIM. These methods are intended to provide an accurate assessment of image quality.

[0061] The present invention trains the image compression model in two stages. During the model training process, an entropy estimation network is additionally added. Figure 8 As shown. In the first stage, the model was trained for 24 times using a learning rate of ξ = 0.001; in the second stage, the learning rate was reduced to ξ = 0.0001 and trained for another 12 times until the model stabilized. The encoding module, decoding module, and entropy estimation network were all trained together using the same learning rate. During training, the batch size was set to 4. The Adam optimizer was used, which is based on PyTorch and supports CUDA. WS MSE was used as the distortion loss for WS-PSNR. In order to fine-tune the model in the second stage, WS-MSE and WS-SSIM were used as the distortion losses and trained for another 12 times.

[0062] In one embodiment, in order to evaluate the effectiveness of the method proposed in the present invention, the method is compared with the most advanced learning-based models, including Li360, Balle18, Minnen18 and traditional compression standards such as VVC intra, BPG, and JPEG.

[0063] like Figure 7 As shown, our method significantly outperforms all other learning-based methods in terms of both WS PSNR and WS-SSIM. It is worth noting that Li360 represents the latest achievements in the field of 360° image compression. Our method outperforms Li360 by 0.5dB at a WS PSNR of 0.16bpp, and we achieve an average bitrate reduction of 15.76% compared to Li360 on the same dataset. In addition, VVC Intra is a well-known video compression standard that incorporates various hand-crafted modules. The results show that our model performs better than VVC Intra.

[0064] In the present invention, unless otherwise clearly stipulated and limited, the terms such as "installation", "setting", "connection", "fixation" and "rotation" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, ordinary technicians in this field can understand the specific meanings of the above terms in the present invention according to the specific circumstances.

[0065] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-scale end-to-end 360-degree image compression method, characterized in that: Constructing and training an image compression model, wherein the image compression model includes a three-layer path, each layer path includes an encoding module, an adaptive scale block encoding module, and a decoding module; implementing end-to-end 360-degree image compression through the trained image compression model, including the following steps: S1. Perform ERP projection on the acquired 360-degree image to form a 2D projection image, and divide the 2D projection image into three groups of images; S2. Input the i=1, 2, 3 groups of images into the encoding module in the i-th layer path to obtain a set of encoding features; S3. input the i-th group of coding features into the adaptive scaled block coding module in the i-th layer path to obtain a group of optimal block pattern maps, each group of optimal block pattern maps includes multiple blocks; wherein the size of all blocks in the first group of optimal block pattern maps is 1×8, the size of all blocks in the second group of optimal block pattern maps is 2×4, and the size of all blocks in the third group of optimal block pattern maps is 4×4; S4. The i-th group of coding features and the i-th group of best block pattern images are subjected to an average pooling layer to obtain a group of similar merged features, and each group of similar merged features is quantized and upsampled to obtain a group of quantized features; S5. Input the i-th group of quantized features into the decoding module in the i-th layer path, and merge the outputs of the decoding modules of the three-layer paths to obtain a 360-degree image compression result.

2. The multi-scale end-to-end 360-degree image compression method according to claim 1, characterized in that: Step S1 divides the 2D projection image into three groups of images according to the overstretching factor, including: The 2D projection image is evenly divided into 16 regional images according to the height; The first region image, the second region image, the fifteenth region image, and the sixteenth region image constitute a first group of images; The 3rd region image, the 4th region image, the 13th region image and the 14th region image constitute the second group of images; The 5th region image, the 6th region image, the 7th region image, the 8th region image, the 9th region image, the 10th region image, the 11th region image and the 12th region image constitute the third group of images.

3. The multi-scale end-to-end 360-degree image compression method according to claim 1, characterized in that: The encoding modules of the three-layer paths have the same structure, including 4 rectangular convolutional layers, 3 residual blocks and 2 attention modules; among them, a residual block is connected between every two adjacent rectangular convolutional layers, and an attention module is connected after the second residual block and the fourth rectangular convolutional layer respectively.

4. The multi-scale end-to-end 360-degree image compression method according to claim 1, characterized in that: The structures of the adaptive scale block coding modules of the three-layer paths are the same, and both include two adaptive scale block units; each adaptive scale block unit includes a first convolution layer, a residual block, a rectangular convolution layer, a second convolution layer, a BN layer, and a Gumbel softmax layer that are cascaded in sequence.

5. The multi-scale end-to-end 360-degree image compression method according to any one of claims 3 or 4, characterized in that: In the first path, all rectangular convolution layers use a rectangular convolution kernel size of 5×9; in the second path, all rectangular convolution layers use a rectangular convolution kernel size of 5×7; in the third path, all rectangular convolution layers use a rectangular convolution kernel size of 5×5.

6. The multi-scale end-to-end 360-degree image compression method according to claim 1, characterized in that: The decoding modules of the three-layer paths have the same structure, including 4 transposed rectangular convolutional layers, 3 residual blocks and 2 attention modules; among them, a residual block is connected between every two adjacent transposed rectangular convolutional layers, and an attention module is connected before the second residual block and the first transposed rectangular convolutional layer respectively; in the first-layer path, the rectangular convolution kernel size used by all transposed rectangular convolutional layers is 5×9; in the second-layer path, the rectangular convolution kernel size used by all transposed rectangular convolutional layers is 5×7; in the third-layer path, the rectangular convolution kernel size used by all transposed rectangular convolutional layers is 5×5.

7. The multi-scale end-to-end 360-degree image compression method according to claim 1, characterized in that: In the training process of the image compression model, the training is performed with the goal of minimizing the rate-distortion loss; the calculation formula of the rate-distortion loss is: Among them, Loss represents the rate distortion loss, R i Represents the quantized features corresponding to the i-th group of images The number of digits, D i represents the distortion value of the i-th group of images, λ represents the hyperparameter, pooling(·) represents the average pooling operation, P(·) represents the Gaussian mixture distribution model, ω i represents the pixel group weight of the i-th group of images, X i represents the i-th group of images, represents the output of the i-th group of images after the image compression model, Represents an operation that computes the mean squared error.

Images