Method and data processing system for lossy image or video encoding, transmission and decoding

Abstract

A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving first and second input images; estimating optical flow information indicative of a difference between representations of the first and second input images; receiving first and second output images, wherein the first and second output images comprise approximations of the first and second input images encoded and decoded by one or more neural networks; estimating optical flow information indicative of a difference between representations of the first output image and the second output image; evaluating a function based on a difference between the estimated optical flow information associated with the first and second input images and the estimated optical flow information associated with the first and second output images.

Description

[0001] Method and data processing system for lossy image or video encoding, transmission and decoding BACKGROUND This invention relates to a method and system for lossy image or video encoding, transmission and decoding, a method, apparatus, computer program and computer readable storage medium for lossy image or video encoding and transmission, and a method, apparatus, computer program and computer readable storage medium for lossy image or video receipt and decoding. There is increasing demand from users of communications networks for images and video content. Demand is increasing not just for the number of images viewed, and for the playing time of video; demand is also increasing for higher resolution content. This places increasing demand on communications networks and increases their energy use because of the larger amount of data being transmitted. To reduce the impact of these issues, image and video content is compressed for transmission across the network. The compression of image and video content can be lossless or lossy compression. In lossless compression, the image or video is compressed such that all of the original information in the content can be recovered on decompression. However, when using lossless compression there is a limit to the reduction in data quantity that can be achieved. In lossy compression, some information is lost from the image or video during the compression process. Known compression techniques attempt to minimise the apparent loss of information by the removal of information that results in changes to the decompressed image or video that is not particularly noticeable to the human visual system. JPEG, JPEG2000, AVC, HEVC and AVI are examples of compression processes for image and / or video files. In general terms, known lossy image compression techniques use the spatial correlations between pixels in images to remove redundant information during compression. For example, in an image of a blue sky, if a given pixel is blue, there is a high likelihood that the neighbouring pixels, and their neighbouring pixels, and so on, are also blue. There is accordingly no need to retain all the raw pixel data. Instead, we can retain only a subset of the pixels which take up fewer bits and infer the pixel values of the other pixels using information derived from spatial correlations. A similar approach is applied in known lossy video compression techniques. That is, spatial correlations between pixels allow the removal of redundant information during compression. However, in video compression, there is further information redundancy in the form of temporal correlations. For example, in a video of an aircraft flying across a blue-sky background, most of the pixels of the blue sky do not change at all between frames of the video. The most of the blue sky pixel data for the frame at position t = 0 in the video is identical to that at position t = 10. Storing this identical, temporally correlated, information is inefficient. Instead, only the blue sky pixel data for a subset of the frames is stored and the rest are inferred from information derived from temporal correlations. In the realm of lossy video compression in particular, the removal of redundant temporally correlated information in a video sequence is known inter-frame redundancy. One technique using inter-frame redundancy that is widely used in standard video compression algorithms involves the categorization of video frames into three types: I-frames, P-frames, and B-frames. Each frame type carries distinct properties concerning their encoding and decoding process, playing different roles in achieving high compression ratios while maintaining acceptable visual quality. I-frames, or intra-coded frames, serve as the foundation of the video sequence. These frames are self-contained, each one encoding a complete image without reference to any other frame. In terms of compression, I-frames are least compressed among all frame types, thus carrying the most data. However, their independence provides several benefits, including being the starting point for decompression and enabling random access, crucial for functionalities like fast-forwarding or rewinding the video. P-frames, or predictive frames, utilize temporal redundancy in video sequences to achieve greater compression. Instead of encoding an entire image like an I-frame, a P-frame represents the difference between itself and the closest preceding I- or P-frame. The process, known as motion compensation, identifies and encodes only the changes that have occurred, thereby significantly reducing the amount of data transmitted. Nonetheless, P-frames are dependent on previous frames for decoding. Consequently, any error during the encoding or transmission process may propagate to subsequent frames, impacting the overall video quality. B-frames, or bidirectionally predictive frames, represent the highest level of compression. Unlike P-frames, B-frames use both the preceding and following frames as references in their encoding process. By predicting motion both forwards and backwards in time, B-frames encode only the differences that cannot be accurately anticipated from the previous and next frames, leading to substantial data reduction. Although this bidirectional prediction makes B-frames more complex to generate and decode, it does not propagate decoding errors since they are not used as references for other frames. Artificial intelligence (AI) based compression techniques achieve compression and decom- pression of images and videos through the use of trained neural networks in the compression and decompression process. Typically, during training of the neutral networks, the difference between the original image and video and the compressed and decompressed image and video is analyzed and the parameters of the neural networks are modified to reduce this difference while minimizing the data required to transmit the content. However, AI based compression methods may achieve poor compression results in terms of the appearance of the compressed image or video or the amount of information required to be transmitted. An example of an AI based image compression process comprising a hyper-network is described in Ballé, Johannes, et al. “Variational image compression with a scale hyperprior.” arXiv preprint arXiv:1802.01436 (2018), which is hereby incorporated by reference. An example of an AI based video compression approach is shown in Agustsson, E., Minnen, D., Johnston, N., Balle, J., Hwang, S. J., and Toderici, G. (2020), Scale-space flow for end-to-end optimized video compression. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (pp. 8503-8512), which is hereby incorporated by reference. A further example of an AI based video compression approach is shown in Mentzer, F., Agustsson, E., Ballé, J., Minnen, D., Johnston, N., and Toderici, G. (2022, November). Neural video compression using gans for detail synthesis and propagation. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXVI (pp. 562-578), which is hereby incorporated by reference. Figure 3 of this reference shows an architecture that calculates optical flow with a flow model, UFlow, and encodes the calculated optical flow with a flow encoder, Eflow. SUMMARY According to the present invention there is provided a method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving first and second input images; estimating optical flow information indicative of a difference between representations of the first and second input images; receiving first and second output images, wherein the first and second output images comprise approximations of the first and second input images encoded and decoded by one or more neural networks; estimating optical flow information indicative of a difference between representations of the first output image and the second output image; evaluating a function based on a difference between the estimated optical flow information associated with the first and second input images and the estimated optical flow information associated with the first and second output images; updating the parameters of the one or more neural networks based on the evaluated function; and repeating the above steps using a first set of input images to produce one or more trained neural network. Optionally, estimating optical flow information between representations of the first and second input images comprises estimating a pixel difference between a pixel in the representation of the first input image and one or more pixels in the representation of the second input image. Optionally, estimating optical flow information between representations of the first and second output images comprises estimating a pixel difference between a pixel in the representation of the first output image and one or more pixels in the representation of the second output image. Optionally, said pixel difference comprises a representation of one or more cost volumes. Optionally, evaluating the function comprises estimating a difference between (i) the represen- tation of the one or more cost volumes associated with the first and second input images, and (ii) the representation of one or more cost volumes associated with the first and second output images. Optionally, the difference between a pixel in the representation of the first input image and one or more pixels in the representation of the second input image, and / or the difference between a pixel in the representation of the first output image and one or more pixels in the representation of the second output image, is estimated for pixels in a predetermined pixel radius. Optionally, the method comprises performing a mean absolute difference calculation to estimate said pixel difference. Optionally, the method comprises performing a dot product calculation to estimate said pixel difference. Optionally, the method comprises producing the representations of the first and second input images for a plurality of spatial resolutions with said one or more neural networks. Optionally, the method comprises estimating a difference between (i) the representation of the one or more cost volumes associated with the first and second input images at each of said plurality of different spatial resolutions, and (ii) the representation of one or more cost volumes associated with the first and second output images at each said plurality of different spatial resolutions. Optionally, the method comprises: (i) encoding the first input image to produce a latent representation using a first neural network of the one or more neural networks; (ii) decoding the latent representation to produce the first output image using a second neural network of the one or more neural networks. Optionally, the method comprises: (iii) encoding the estimated optical flow information indicative of a difference between representations of the first and second input images to produce a latent representation using a third neural network of the one or more neural networks; (iv) decoding the latent representation to produce the estimated optical flow information using a fourth neural network of the one or more neural networks; and (v) using the optical flow information and the first output image to produce the second output image. Optionally, the method comprises: entropy encoding the estimated optical flow information and / or the latent representation of the optical flow information into a bitstream, and entropy decoding the bitstream to produce the estimated optical flow information and / or latent representation of the optical flow information. Optionally, the function further comprises a term based on a size of the bitstream. Optionally, the estimated optical flow information indicative of a difference between a representation of the first and second input images comprises a difference between the latent representation of the first input image and a latent representation of the second input image. Optionally, the estimated optical flow information indicative of a difference between a representation of the first and second output images comprises a difference between a latent representation of the first output image and a latent representation of the second output image. According to a further aspect, there is provided a data processing system configured to perform the above method. According to a further aspect, there is provided a computer-readable storagemedium comprising instructions which, when executed by a computer, cause the computer to carry out the above method. According to a further aspect, there is provided a method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; estimating optical flow information distributed according to a first distribution from the first image image and the second image using a first neural network, the optical flow information being indicative of a difference between a representation of the first image and the second image; transmitting the optical flow information to a second computer system; decoding the optical flow information distributed according to the first distribution using a second neural network; and using the second image and the decoded optical flow information to produce an output image, wherein the output image is an approximation of the first image. Optionally, the second image comprises a previously decoded image. Optionally, the transmitting of the optical flow information is performed without using a dedicated optical flow encoder. Optionally, the transmitting of the optical flow information is performed without encoding the optical flow information into a latent representation. Optionally, the method comprises: entropy encoding the optical flow information distributed according to the first distribution into a bitstream, and transmitting the bitstream to the second computer system. Optionally, the method comprises: entropy decoding the bitstream at the second computer system to produce the optical flow information distributed according to the first distribution. Optionally, estimating optical flow information comprises using the first neural network to produce the representation of the previously decoded image and the representation of the input image. Optionally, estimating optical flow information comprises estimating a pixel difference between a pixel in the representation of the previously decoded image and one or more pixels in the representation of the input image. Optionally, said pixel difference comprises a representation of one or more cost volumes. Optionally, the one or more cost volumes are associated with representations the previously decoded image and the input image at a plurality of spatial resolutions. Optionally, the method comprises estimating the optical flow information from the representa- tion of the one or more cost volumes. Optionally, estimating the optical flow information from the representation of the one or more cost volumes is performed with the first neural network. Optionally, the method comprises producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation; transmitting the latent representation to the first computer system; and decoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information distributed according to a first distribution from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and the representation of the input image; and transmitting the optical flow information to a second computer system. Optionally, the transmitting of the optical flow information is performed without encoding the optical flow information into a latent representation. Optionally, the method comprises producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation; and decoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image. According to a further aspect, there is provided a method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image and a previously decoded image; estimating optical flow information distributed according to a first distribution from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and a representation of the input image; decoding the optical flow information distributed according to the first distribution using a second neural network; and using the previously decoded image and the decoded optical flow information to produce an output image, wherein the output image is an approximation of the input image. evaluating a loss function; updating the parameters of the first and / or second neural networks based on the evaluated loss function; and repeating the above steps using a set of input images to produce first and / or second trained neural networks. Optionally, decoding of the optical flow information is performed without encoding the optical flow information into a latent representation. Optionally, the method comprises producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation; decoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image; and updating the parameters of the third and / or fourth neural networks based on the evaluated loss function. According to a further aspect, there is provided a method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and a representation of the input image; transmitting the optical flow information to a second computer system; decoding the optical flow information using a second neural network;and using the previously decoded image and the decoded optical flow information to produce an output image, wherein the output image is an approximation of the input image; wherein estimating the optical flow information comprises estimating differences between representations of the previously decoded image and representations of the input image in pixel search blocks of a plurality of sizes. Optionally, themethod comprises estimating said differences for a plurality of spatial resolutions of the representations of the input image and the previously decoded image, wherein a pixel search block of a first size is used for a first spatial resolution, and a pixel search block of a second size is used for a second spatial resolution. Optionally, the first size is larger than the second size, and the first spatial resolution is a lower spatial resolution than the second spatial resolution. Optionally, the first and second sizes are defined by a pixel search radius around a center pixel coordinate of the pixel search block. Optionally, the first size comprises a pixel search radius of 3. Optionally, the second size comprises a pixel search radius of 1. Optionally, the differences between the representations of the previously decoded image and representations of the input image comprise mean absolute differences. Optionally, the difference between the representations of the previously decoded image and representations of the input image comprise cost volumes. Optionally, the method comprises producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation; and decoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image. Optionally, themethod comprises estimating said differences for a plurality of spatial resolutions of representations of the input image and the previously decoded image by sequentially performing the steps of: (i) estimating a difference associated with a first spatial resolution; (ii) upsampling the estimated difference to a second spatial resolution; (iii) estimating a difference associated with the second spatial resolution; and (iv) combining the upsampled difference estimation with the estimated difference. Optionally, themethod comprises estimating said differences for a plurality of spatial resolutions of representations of the input image and the previously decoded image by performing the steps of: (i) estimating a difference associated with a first spatial resolution; (ii) estimating a difference associated with a second spatial resolution; (iii) upsampling, in parallel, the estimated differences associated with the first and second spatial resolutions; and (iv) combining the upsampled differences. Optionally, estimating said differences for a plurality of spatial resolutions of representations of the input image and the previously decoded image comprises performing a warpless difference estimation. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information distributed according to a first distribution from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and the representation of the input image; and transmitting the optical flow information to a second computer system; wherein estimating the optical flow information comprises estimating differences between representations of the previously decoded image and representations of the input image in pixel search blocks of a plurality of sizes. Optionally, themethod comprises estimating said differences for a plurality of spatial resolutions of the input image and the previously decoded image, wherein a pixel search block of a first size is used for a first spatial resolution, and a pixel search block of a second size is used for a second spatial resolution. Optionally, the method comprises producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation; and decoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image. According to a further aspect, there is provided a method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and a representation of the input image, and said estimating the optical flow information comprising performing a warpless difference estimation; transmitting the optical flow information to a second computer system; decoding the optical flow information using a second neural network;and using the previously decoded image and the decoded optical flow information to produce an output image, wherein the output image is an approximation of the input image. According to a further aspect, there is provided a method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and a representation of the input image; transmitting the optical flow information to a second computer system; decoding the optical flow information using a second neural network;and using the previously decoded image and the decoded optical flow information to produce an output image, wherein the output image is an approximation of the input image; wherein estimating the optical flow information comprises estimating differences between representations of the previously decoded image and representations of the input image in at least one shaped pixel search block. Optionally, estimating the optical flow information comprises estimating differences between representations of the previously decoded image and representations of the input image in a plurality of pixel search blocks having different shapes. Optionally, at least one of said shapes comprises a diamond shape. Optionally, themethod comprises estimating said differences for a plurality of spatial resolutions of the input image and the previously decoded image, wherein a pixel search block of a first shape is used for a first spatial resolution, and a pixel search block of a second shape is used for a second spatial resolution. Optionally, the shape of the pixel search block used for a highest spatial resolution comprises a diamond shape. Optionally, the diamond shaped pixel search block used for a highest spatial resolution define a pixel search radius of 1. Optionally, one or more shaped pixel search blocks other than the pixel search block used for the highest spatial resolution defines a pixel search radius of 3. Optionally, the differences between the representations of the previously decoded image and representations of the input image comprise mean absolute differences. Optionally, the difference between the representations of the previously decoded image and representations of the input image comprise cost volumes. Optionally, the method comprises producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation; and decoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information distributed according to a first distribution from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and the representation of the input image; and transmitting the optical flow information to a second computer system; wherein estimating the optical flow information comprises estimating differences between representations of the previously decoded image and representations of the input image in at least one shaped pixel search block. Optionally, estimating the optical flow information comprises estimating differences between representations of the previously decoded image and representations of the input image in a plurality of pixel search blocks having different shapes. Optionally, at least one of said shapes comprises a diamond shape. Optionally, themethod comprises estimating said differences for a plurality of spatial resolutions of the input image and the previously decoded image, wherein a pixel search block of a first shape is used for a first spatial resolution, and a pixel search block of a second shape is used for a second spatial resolution. Optionally, the shape of the pixel search block used for a highest spatial resolution comprises a diamond shape. Optionally, the method comprises producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation; and decoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image. According to a further aspect, there is provided a data processing system configured to perform any of the above methods. According to a further aspect, there is provided a computer-readable storagemedium comprising instructions which, when executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image at a first computer system; with a first neural network, producing first optical flow information using the first image and the second image, each represented at a first resolution, the first optical flow information being indicative of a difference between the first image and the second image; with a second neural network, producing second optical flow information using the first image and the second image, each represented at a second resolution, and using the first optical flow information, the second optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the second optical flow information to a second computer system; with a third neural network, decoding the information representing the second optical flow information to produce an approximation of the second optical flow information; and with a fourth neural network, producing an output image using the second optical flow information, wherein the output image is an approximation of the first image. Optionally, the method comprises downsampling the first image and the second image to the first resolution. Optionally, the second resolution is greater than the first resolution. Optionally, the first optical flow information is stored as an integer data type having a dynamic range corresponding to a coarse motion range, and wherein the second optical flow information is stored as an integer data type having a dynamic range corresponding to a fine motion range. Optionally, the coarse information range is greater than the fine motion range. Optionally, the integer data type is an int8 data type. Optionally, using the first optical flow information comprises warping the second image with the first optical flow information, and wherein producing the second optical flow information comprises using the warped second image, the second optical flow information being indicative of a difference between the first image and the warped second image. Optionally, the method comprises estimating a difference between the first image and the warped second image, and using the difference to produce the second optical flow information. Optionally, the difference comprises a brightness difference. Optionally, using the difference comprises convolving a representation of the difference with one or more outputs of one or more layers of the second neural network. Optionally, the method comprises downsampling the first image and the second image to the second resolution. Optionally, the method comprises feeding one or more outputs of one or more layers of the first neural network forward to the second neural network, and wherein producing the second optical flow information comprises using the one or more outputs of one or more layers of the first neural network. Optionally, using the one or more outputs of one or more layers of the first neural network comprises convolving the one or more outputs with one or more outputs of one or more layers of the second neural network. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, the method comprising the steps of: receiving a first image and a second image at a first computer system; with a first neural network, producing first optical flow information using the first image and the second image, each represented at a first resolution, the first optical flow information being indicative of a difference between the first image and the second image; with a second neural network, producing second optical flow information using the first image and the second image, each represented at a second resolution, and using the first optical flow information, the second optical flow information being indicative of a difference between the first image and the second image; and transmitting information representing the second optical flow information to a second computer system. According to a further aspect, there is provided a method for lossy image or video receipt and decoding, the method comprising the steps of: receiving information representing second optical flow information at a second computer system, the second optical flow information produced with a second neural network using a first and a second image, each represented at a second resolution, and using first optical flow information, the second optical flow information being indicative of a difference between the first image and the second image; with a third neural network, decoding the information representing the second optical flow information to produce an approximation of the second optical flow information; and with a fourth neural network, producing an output image using the second optical flow information, wherein the output image is an approximation of the first image. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image at a first computer system; with a first neural network, producing first optical flow information using the first image and the second image, each represented at a first resolution, the first optical flow information being indicative of a difference between the first image and the second image; with a second neural network, producing second optical flow information using the first image and the second image, each represented at a second resolution, and using the first optical flow information, the second optical flow information being indicative of a difference between the first image and the second image; with a fifth neural network, producing third optical flow information using the first image and the second image, each represented at a third resolution, and using the second optical flow information, the third optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the third optical flow information to a second computer system; transmitting information representing the third optical flow information to a second computer system; with a third neural network, decoding the information representing the third optical flow information to produce an approximation of the third optical flow information; and with a fourth neural network, producing an output image using the third optical flow information, wherein the output image is an approximation of the first image. Optionally, the third resolution is greater than the first resolution and second resolution. Optionally, the first optical flow information is stored as an integer data type having a dynamic range corresponding to a first coarseness of motion range, wherein the second optical flow information is stored as an integer data type having a dynamic range corresponding to second coarseness of motion range, and wherein the third optical flow information is stored as an integer data type having a dynamic range corresponding to a third coarseness of motion range. Optionally, the integer data type is an int8 data type. Optionally, the first coarseness of motion range is greater than the second coarseness of motion range, and wherein the second coarseness of motion range is greater than the third coarseness of motion range. According to a further aspect, there is provided a data processing apparatus configured to perform any of the above methods. According to a further aspect, there is provided computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing first optical flow information using the first image and the second image, the first optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the first optical flow information to a second computer system; with a second neural network and using second optical flow information indicative of difference between the second image and a third image of the sequence, decoding the information representing the first optical flow information to produce an approximation of the first optical flow information; and with a third neural network, producing an output image using the first optical flow information, wherein the output image is an approximation of the first image. Optionally, using the second optical flow information comprises producing first entropy parameters indicative of a distribution of the information representing the first optical flow information. Optionally, the method comprises entropy encoding and entropy decoding the information representing the first optical flow information using the first entropy parameters Optionally, the method comprises, with a fourth neural network, producing a hyper latent representation of the first optical flow information using the first optical flow information and using the second optical flow information. Optionally, the method comprises with a fifth neural network producing the first entropy parameters by decoding the hyper latent representation. Optionally, the method comprises: with a sixth neural network producing a hyper hyper latent representation of the first optical flow information using the hyper latent representation of the first optical flow information and using the second optical flow information. Optionally, using the second optical flow information comprises producing second entropy parameters indicative of a distribution of the hyper latent representation. Optionally, the method comprises entropy encoding and entropy decoding the hyper latent representation using the second entropy parameters. Optionally, using the second optical flow information comprises conditioning the hyper latent representation produced by the fourth neural network on the second optical flow information. Optionally, said conditioning comprises combining the second optical flow information and the information representing the first optical flow information using one or more convolution operations and / or one ormore concatenation operations, and inputting the combined information into the fourth neural network. Optionally, using the second optical flow information comprises conditioning the hyper hyper latent representation produced by the sixth neural network on the second optical flow information. Optionally, said conditioning comprises combining the second optical flow information and the hyper latent representation using one or more convolution operations and / or one or more concatenation operations, and inputting the combined information into the sixth neural network. Optionally, the method comprises producing the second entropy parameters using the second optical flow information without using a hyper hyper latent representation. Optionally, the second optical flow information comprises one or more outputs of one or more intermediate layers of the first neural network. Optionally, the method comprises identifying the first image and the second image as a chronologically first frame pair of the image sequence, and using synthetic second optical flow information as the second optical flow information. Optionally, the method comprises identifying the first image and the second image as a chronologically first frame pair of the image sequence, and encoding, transmitting and decoding the first image without using second optical flow information. According to a further aspect, there is provided a method for lossy image or video receipt and decoding, the method comprising the steps of: receiving information representing first optical flow information at a second computer system, the first optical flow information produced using a first neural network and being indicative of a difference between a first image and a second image of an image sequence; with a second neural network and using second optical flow information indicative of difference between the second image and a third image of the sequence, decoding the information representing the first optical flow information to produce an approximation of the first optical flow information; and with a third neural network, producing an output image using the first optical flow information, wherein the output image is an approximation of the first image. According to a further aspect, there is provided a method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence; with a first neural network, producing first optical flow information using the first image and the second image, the first optical flow information being indicative of a difference between the first image and the second image; with a second neural network and using second optical flow information indicative of difference between the second image and a third image of the sequence, decoding the information representing the first optical flow information to produce an approximation of the first optical flow information; with a third neural network, producing an output image using the first optical flow information, wherein the output image is an approximation of the first image; evaluating a function based on a difference between the estimated first image and the output image; updating the parameters of the first, second and / or third networks based on the evaluated function; and repeating the above steps using a first set of image sequences to produce trained first, second and third neural networks. Optionally, the first set of image sequences comprises at least one image sequence with a group of picture (GOP) size of at least four frames. According to a further aspect, there is provided a data processing apparatus configured to perform any of the above methods. According to a further aspect, there is provided a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing actual first optical flow information using the first image, the second image, and using predicted optical flow information, the actual first optical flow information being indicative of a difference between the first image and the second image, and the predicted optical flow information being based on second optical flow information indicative of a difference between the second image and a third image of the sequence; transmitting information representing the actual first optical flow information to a second computer system; with a second neural network, decoding the information representing the actual first optical flow information to produce an approximation of the actual first optical flow information; and with a third neural network, producing an output image using the actual first optical flow information, wherein the output image is an approximation of the first image. Optionally, the method comprises producing the predicted optical flow information by performing a forward warping operation on the second optical flow information. Optionally, the forward warping operation comprises a soft splat operation. Optionally, the second optical flow information is further indicative of a difference between the third image of the sequence and a fourth image of the sequence. Optionally, themethod comprises conditioning the first actual optical flow information produced by the first neural network on the predicted optical flow information. Optionally, said conditioning comprises combining the predicted optical flow information with the first image and / or the second image by performing a convolution operation and / or a concatenation operation before performing a forward pass of the first neural network. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing actual first optical flow information using the first image, the second image, and using predicted optical flow information, the actual first optical flow information being indicative of a difference between the first image and the second image, and the predicted optical flow information being based on second optical flow information indicative of a difference between the second image and a third image of the sequence; transmitting information representing the actual first optical flow information to a second computer system. According to a further aspect, there is provided a method for lossy image or video receipt and decoding, the method comprising the steps of: receiving information representing actual first optical flow information at a second computer system, the actual first optical flow information produced using a first image, a second image, and using predicted optical flow information, the actual first optical flow information being indicative of a difference between the first image and the second image, and the predicted optical flow information being based on second optical flow information indicative of a difference between the second image and a third image of the sequence; with a second neural network, decoding the information representing the actual first optical flow information to produce an approximation of the actual first optical flow information; and with a third neural network, producing an output image using the actual first optical flow information, wherein the output image is an approximation of the first image. According to a further aspect, there is provided a method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence; with a first neural network, producing actual first optical flow information using the first image, the second image, and predicted optical flow information, the actual first optical flow information being indicative of a difference between the first image and the second image, and the predicted optical flow information being based on second optical flow information indicative of a difference between the second image and a third image of the sequence; with a second neural network, decoding the information representing the actual first optical flow information to produce an approximation of the actual first optical flow information; and with a third neural network, producing an output image using the actual first optical flow information, wherein the output image is an approximation of the first image. evaluating a function based on a difference between the estimated first image and the output image; updating the parameters of the first, second and / or third networks based on the evaluated function; and repeating the above steps using a first set of image sequences to produce trained first, second and third neural networks. Optionally, the first set of image sequences comprises at least one image sequence with a group of picture (GOP) size of at least four frames. According to a further aspect, there is provided a data processing apparatus configured to perform any of the above methods. According to a further aspect, there is provided a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the optical flow information to a second computer system; with a second neural network decoding the information representing the optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image, wherein said step of producing optical flow information comprises performing a pooling operation on a representation of said optical flow information. Optionally, the pooling operation converts the representation of said optical flow information into a motion descriptor indicative of global motion between the first image and the second image. Optionally, the method comprises performing the pooling operation on a hyper latent represen- tation of the optical flow information. Optionally, the method comprises combining the hyper latent representation of the optical flow information with a representation of pixel location information before performing said pooling operation. Optionally, the method comprises performing an unpooling operation on a reconstruction of the hyper latent representation of the optical flow information. Optionally, the method comprises combining the reconstruction of the hyper latent representa- tion of the optical flow information with a representation of pixel location information after performing the unpooling operation. Optionally, the representation of pixel position information comprises a positional embedding. Optionally, the method comprises producing the positional embedding by embedding a tensor representing pixel coordinates associated with the first image or second image with a fourth neural network. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the optical flow information to a second computer system; wherein said step of producing optical flow information comprises performing a pooling operation on a representation of said optical flow information. According to a further aspect, there is provided a method for lossy image or video receipt and decoding, the method comprising the steps of: receiving at a second computer system information representing optical flow information indicative of a difference between a first image and a second image of an image sequence, the representation of optical flow information produced with a first neural network using the first and the second image; with a second neural network decoding the information representing the optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image, wherein said step of decoding comprises performing an unpooling operation on a representation of said optical flow information. According to a further aspect, there is provided a data processing apparatus configured to perform any of the above methods. According to a further aspect, there is provided a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the optical flow information to a second computer system; with a second neural network decoding the information representing the optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image, wherein said step of producing optical flow information is based on a representation of pixel position information. Optionally, the representation of pixel position information comprises a positional embedding. Optionally, the method comprises producing the positional embedding by embedding a tensor representing pixel coordinates with a fourth neural network. Optionally, the method comprises producing said optical flow information from representations of the first image and the second image downsampled to a plurality of resolutions. Optionally, the method comprises downsampling a representation of the first image and the second image to produce said representations of the first image and the second image at said plurality of resolutions. Optionally, the method comprises combining the pixel position information with one or more of said representations of the first image and the second image at said plurality of resolutions. Optionally, the method comprises reshaping the pixel position information before said combining. Optionally, said combining comprises performing an element-wise addition of the pixel position information with corresponding representations of the first image and the second image at said plurality of resolutions. According to a further aspect, there is provided a method for lossy image or video encoding and transmission,the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the optical flow information to a second computer system; wherein said step of producing optical flow information is based on a representation of pixel position information. According to a further aspect, there is provided a method for lossy image or video receipt and decoding, the method comprising the steps of: receiving at a second computer system information representing optical flow information indicative of a difference between a first image and a second image of an image sequence, the representation of optical flow information produced with a first neural network using the first and the second image and pixel position information; with a second neural network decoding the information representing the optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image. According to a further aspect, there is provided a data processing apparatus configured to perform any of the above methods. According to a further aspect, there is provided a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the optical flow information to a second computer system; with a second neural network decoding the information representing the optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image, wherein producing the optical flow information comprises producing, in parallel, feature maps from respective representations of the first image at different resolutions. Optionally, producing the optical flow information comprises producing, in parallel, feature maps from respective representations of the second image at different resolutions. Optionally, producing the optical flow information comprises combining the feature maps from the first image with the corresponding feature maps from the second image to estimate, at the plurality of resolutions, the difference between the first image and the second image. Optionally, said combining comprises producing one or more cost volumes indicative of the difference between the first image and the second image. Optionally, the method comprises using the estimated difference between the first image and the second image at a first resolution to warp the first image or the second image at a second resolution, and to use the warped first image or second image to estimate the difference between the first image and the second image at the second resolution. Optionally, the first neural network comprises a plurality of parallel process streams each defined by one or more non-sequential layers of the first neural network. Optionally, the method comprises producing the feature maps asynchronously. Optionally, the one or more layers of the first neural network together define a planar shaped feature extractor. Optionally, the method comprises producing the representations of the first image at the different resolutions by performing one or more downsampling operations on the first image, and feeding the respective outputs of the one or more downsampling operations directly into the respective process streams of the plurality of parallel process streams to produce the plurality of feature maps asynchronously. Optionally, each feature map is produced directly from the first image at a respective one of said different resolutions to increase the amount of first image information retained in each feature map compared to a method of producing feature maps using a pyramid shaped feature extractor. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the optical flow information to a second computer system; wherein producing the optical flow information comprises producing, in parallel, feature maps from respective representations of the first image at different resolutions. According to a further aspect, there is provided a method for lossy image or video receipt and decoding, the method comprising the steps of: receiving at a second computer system information representing optical flow information indicative of a difference between a first image and a second image of an image sequence, the representation of optical flow information produced with a first neural network and based on one or more feature maps produced using respective feature maps produced in parallel from respective representations of the first image at different resolutions; with a second neural network decoding the information representing the optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image. According to a further aspect, there is provided a data processing apparatus configured to perform any of the above methods. According to a further aspect, there is provided a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image, the second image, and a previously decoded representation of the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting a representation of the optical flow information to a second computer system; with a second neural network decoding the representation of optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image. Optionally, the step of producing the optical flow information using the first image, the second image, and the previously decoded representation of the second image comprises: (i) producing a blended image by combining the second image and the previously decoded representation of the second image, and (ii) producing the optical flow information by estimating a difference between the first image and the blended image. Optionally, the blended image comprises a convex combination of the second image and the previously decoded representation of the second image. Optionally, producing the blended image comprises, with a fourth neural network, producing a mask using the second image and the previously decoded representation of the second image, and performing a weighted combination of the second image and the previously decoded representation of the second image using the mask. Optionally, an amount of the second image and an amount of the previously decoded representation of the second image contributing to the blended image is correlated with an amount of motion in the image sequence. Optionally, producing the optical flow information further comprises using a third image of the image sequence and a previously decoded representation of the third image. Optionally, producing the optical flow information using the first image, the second image, the previously decoded representation of the second image, the third image, and the previously decoded representation of the third image comprises: (i) producing a blended image by combining the second image, the previously decoded representation of the second image, the third image, and the previously decoded representation of the third image, and (ii) producing the optical flow information by estimating a difference between the first image and the blended image. Optionally, the blended image comprises a convex combination of the second image, the previously decoded representation of the second image, the third image, and the previously decoded representation of the third image. Optionally, the first image, the second image, and / or the previously decoded representation thereof are represented in a YUV colour space, and wherein producing the optical flow information further comprises extracting a luma channel from one or more of the first image, the second image, and / or the previously decoded representation of the first image, and using the extracted luma channel(s) to produce the optical flow information. According to a further aspect, there is provided a method for lossy image or video encoding and transmission, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image, the second image, and a previously decoded representation of the second image, the optical flow information being indicative of a difference between the first image and the second image; and transmitting a representation of the optical flow information to a second computer system. According to a further aspect, there is provided a method for lossy image or video receipt and decoding, the method comprising the steps of: receiving at a second computer system a representation of optical flow information, the optical flow information being indicative of a difference between a first image and a second image of an image sequence and produced with a first neural network using the first image, the second image, and a previously decoded representation of the second image; with a second neural network decoding the representation of optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image. According to a further aspect, there is provided a data processing apparatus configured to perform any of the above methods. According to a further aspect, there is provided a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out any of the above methods. According to a further aspect, there is provided computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out any of the above methods. BRIEF DESCRIPTION OF THE DRAWINGS Aspects of the invention will now be described by way of examples, with reference to the following figures in which: Figure 1 illustrates an example of an image or video compression, transmission and decom- pression pipeline. Figure 2 illustrates a further example of an image or video compression, transmission and decompression pipeline including a hyper-network. Figure 3 illustrates an example of a video compression, transmission and decompression pipeline. Figure 4 illustrates an example of a video compression, transmission and decompression system. Figure 5 illustrates an example of a flow module. Figure 6a illustrates examples of two flow compression modules. Figure 6b illustrates an example of an image or video compression and decompression pipeline. Figure 7a illustrates an example of a flow module. Figure 7b illustrates an example of a warpless flow module. Figure 7c illustrates an example of a flow module with parallel upsampling steps. Figure 7d illustrates an example of a flow module with sequential upsampling steps. Figure 8 illustrates an example of a flow module with shaped pixel search blocks. Figure 9 illustrates an example of a multiscale flow module. Figure 10 illustrates a modification of the multiscale flow module of Figure 9. Figure 11 illustrates an example of a video compression, transmission and decompression pipeline. Figure 12 illustrates an example of a video compression, transmission and decompression pipeline. Figure 13 illustrates an example of a video compression, transmission and decompression pipeline. Figure 14 illustrates a sequence of frames of a video. Figure 15 illustrates an example of a video compression, transmission and decompression pipeline. Figure 16 illustrates an example of a video compression, transmission and decompression pipeline. Figure 17 illustrates an example of a video compression, transmission and decompression pipeline. Figure 18 illustrates an example of a video compression, transmission and decompression pipeline. Figure 19 illustrates an example of an optical flow compression network. Figure 20 illustrates an example of an optical flow compression network. Figure 21 illustrates an example of a video compression, transmission and decompression pipeline. DETAILED DESCRIPTION OF THE DRAWINGS Compression processes may be applied to any form of information to reduce the amount of data, or file size, required to store that information. Image and video information is an example of information that may be compressed. The file size required to store the information, particularly during a compression process when referring to the compressed file, may be referred to as the rate. In general, compression can be lossless or lossy. In both forms of compression, the file size is reduced. However, in lossless compression, no information is lost when the information is compressed and subsequently decompressed. This means that the original file storing the information is fully reconstructed during the decompression process. In contrast to this, in lossy compression information may be lost in the compression and decompression process and the reconstructed file may differ from the original file. Image and video files containing image and video data are common targets for compression. In a compression process involving an image, the input image may be represented as ^^. The data representing the image may be stored in a tensor of dimensions ^^ × ^^ × ^^, where ^^ represents the height of the image, ^^ represents the width of the image and ^^ represents the number of channels of the image. Each ^^ × ^^ data point of the image represents a pixel value of the image at the corresponding location. Each channel ^^ of the image represents a different component of the image for each pixel which are combined when the image file is displayed by a device. For example, an image file may have 3 channels with the channels representing the red, green and blue component of the image respectively. In this case, the image information is stored in the RGB colour space, which may also be referred to as a model or a format. Other examples of colour spaces or formats include the CMKY and the YCbCr colour models. However, the channels of an image file are not limited to storing colour information and other information may be represented in the channels. As a video may be considered a series of images in sequence, any compression process that may be applied to an image may also be applied to a video. Each image making up a video may be referred to as a frame of the video. The output image may differ from the input image and may be represented by ˆ. The difference between the input image and the output image may be referred to as distortion or a difference in image quality. The distortion can be measured using any distortion function which receives the input image and the output image and provides an output which represents the difference between input image and the output image in a numerical way. An example of such a method is using the mean square error (MSE) between the pixels of the input image and the output image, but there are many other ways of measuring distortion, as will be known to the person skilled in the art. The distortion function may comprise a trained neural network. Typically, the rate and distortion of a lossy compression process are related. An increase in the rate may result in a decrease in the distortion, and a decrease in the rate may result in an increase in the distortion. Changes to the distortion may affect the rate in a corresponding manner. A relation between these quantities for a given compression technique may be defined by a rate-distortion equation. AI based compression processes may involve the use of neural networks. A neural network is an operation that can be performed on an input to produce an output. A neural network may be made up of a plurality of layers. The first layer of the network receives the input. One or more operations may be performed on the input by the layer to produce an output of the first layer. The output of the first layer is then passed to the next layer of the network which may perform one or more operations in a similar way. The output of the final layer is the output of the neural network. Each layer of the neural network may be divided into nodes. Each node may receive at least part of the input from the previous layer and provide an output to one or more nodes in a subsequent layer. Each node of a layer may perform the one or more operations of the layer on at least part of the input to the layer. For example, a node may receive an input from one or more nodes of the previous layer. The one or more operations may include a convolution, a weight, a bias and an activation function. Convolution operations are used in convolutional neural networks. When a convolution operation is present, the convolution may be performed across the entire input to a layer. Alternatively, the convolution may be performed on at least part of the input to the layer. Each of the one or more operations is defined by one or more parameters that are associated with each operation. For example, the weight operation may be defined by a weight matrix defining the weight to be applied to each input from each node in the previous layer to each node in the present layer. In this example, each of the values in the weight matrix is a parameter of the neural network. The convolution may be defined by a convolution matrix, also known as a kernel. In this example, one or more of the values in the convolution matrix may be a parameter of the neural network. The activation function may also be defined by values which may be parameters of the neural network. The parameters of the network may be varied during training of the network. Other features of the neural network may be predetermined and therefore not varied during training of the network. For example, the number of layers of the network, the number of nodes of the network, the one or more operations performed in each layer and the connections between the layers may be predetermined and therefore fixed before the training process takes place. These features that are predetermined may be referred to as the hyperparameters of the network. These features are sometimes referred to as the architecture of the network. To train the neural network, a training set of inputs may be used for which the expected output, sometimes referred to as the ground truth, is known. The initial parameters of the neural network are randomized and the first training input is provided to the network. The output of the network is compared to the expected output, and based on a difference between the output and the expected output the parameters of the network are varied such that the difference between the output of the network and the expected output is reduced. This process is then repeated for a plurality of training inputs to train the network. The difference between the output of the network and the expected output may be defined by a loss function. The result of the loss function may be calculated using the difference between the output of the network and the expected output to determine the gradient of the loss function. Back-propagation of the gradient descent of the loss function may be used to update the parameters of the neural network using the gradients ^^ ^^ / ^^ ^^ of the loss function. A plurality of neural networks in a system may be trained simultaneously through back-propagation of the gradient of the loss function to each network. In the context of image or video compression, this type of system, where simultaneous training with back-propagation through each element or the whole network architecture may be referred to as end-to-end, learned image or video compression. Unlike in traditional compression algorithms that use primarily handcrafted, manually constructed steps, an end-to-end learned system learns itself during training what combination of parameters best achieves the goal of minimising the loss function. This approach is advantageous compared to systems that are not end-to-end learned because an end-to-end system has a greater flexibility to learn weights and parameters that might be counter-intuitive to someone handcrafting features. It will be appreciated that the term "training" or "learning" as used herein means the process of optimizing an artificial intelligence or machine learning model, based on a given set of data. This involves iteratively adjusting the parameters of the model to minimize the discrepancy between the model’s predictions and the actual data, represented by the above-described rate-distortion loss function. The training process may comprise multiple epochs. An epoch refers to one complete pass of the entire training dataset through the machine learning algorithm. During an epoch, the model’s parameters are updated in an effort to minimize the loss function. It is envisaged that multiple epochs may be used to train a model, with the exact number depending on various factors including the complexity of the model and the diversity of the training data. Within each epoch, the training data may be divided into smaller subsets known as batches. The size of a batch, referred to as the batch size, may influence the training process. A smaller batch size can lead to more frequent updates to the model’s parameters, potentially leading to faster convergence to the optimal solution, but at the cost of increased computational resources. Conversely, a larger batch size involves fewer updates, which can be more computationally efficient but might converge slower or even fail to converge to the optimal solution. The learnable parameters are updated by a specified amount each time, determined by the learning rate. The learning rate is a hyperparameter that decides how much the parameters are adjusted during the training process. A smaller learning rate implies smaller steps in the parameter space and a potentially more accurate solution, but it may require more epochs to reach that solution. On the other hand, a larger learning rate can expedite the training process but may risk overshooting the optimal solution or causing the training process to diverge. The training described herein may involve use of a validation set, which is a portion of the data not used in the initial training, which is used to evaluate the model’s performance and to prevent overfitting. Overfitting occurs when a model learns the training data too well, to the point that it fails to generalize to unseen data. Regularization techniques, such as dropout or L1 / L2 regularization, can also be used to mitigate overfitting. It will be appreciated that training a machine learning model is an iterative process that may comprise selection and tuning of various parameters and hyperparameters. As will be appreciated, the specific details, such as hyper parameters and so on, of the training process may vary and it is envisaged that producing a trained model in this way may achieved in a number of different ways with different epochs, batch sizes, learning rates, regularisations, and so on, the details of which are not essential to enabling the advantages and effects of the present disclosure, except where stated otherwise. The point at which an “untrained” neural network is considered be “trained” is envisaged to be case specific and depend on, for example, on a number of epochs, a plateauing of any further learning, or some other metric and is not considered to be essential in achieving the advantages described herein. More details of an end-to-end, learned compression process will now be described. It will be appreciated that in some cases, end-to-end, learned compression processes may be combined with one or more components that are handcrafted or trained separately. In the case of AI based image or video compression, the loss function may be defined by the rate distortion equation. The rate distortion equation may be represented by ^^ ^^ ^^ ^^ = ^^ + ^^ ∗ ^^, where ^^ is the distortion function, ^^ is a weighting factor, and ^^ is the rate loss. ^^ may be referred to as a lagrange multiplier. The langrange multiplier provides as weight for a particular term of the loss function in relation to each other term and can be used to control which terms of the loss function are favoured when training the network. In the case of AI based image or video compression, a training set of input images may be used. An example training set of input images is the KODAK image set (for example at www.cs.albany.edu / xypan / research / snr / Kodak.html). An example training set of input images is the IMAX image set. An example training set of input images is the Imagenet dataset (for example at www.image-net.org / download). An example training set of input images is the CLIC Training Dataset P (“professional”) and M (“mobile”) (for example at http: / / challenge.compression.cc / tasks / ). An example of an AI based compression, transmission and decompression process 100 is shown in Figure 1. As a first step in the AI based compression process, an input image 5 is provided. The input image 5 is provided to a trained neural network 110 characterized by a function ^^^^acting as an encoder. The encoder neural network 110 produces an output based on the input image. This output is referred to as a latent representation of the input image 5. In a second step, the latent representation is quantised in a quantisation process 140 characterised by the operation ^^, resulting in a quantized latent. The quantisation process transforms the continuous latent representation into a discrete quantized latent. An example of a quantization process is a rounding function. In a third step, the quantized latent is entropy encoded in an entropy encoding process 150 to produce a bitstream 130. The entropy encoding process may be for example, range or arithmetic encoding. In a fourth step, the bitstream 130 may be transmitted across a communication network. In a fifth step, the bitstream is entropy decoded in an entropy decoding process 160. The quantized latent is provided to another trained neural network 120 characterized by a function ^^^^acting as a decoder, which decodes the quantized latent. The trained neural network 120 produces an output based on the quantized latent. The output may be the output image of the AI based compression process 100. The encoder-decoder system may be referred to as an autoencoder. Entropy encoding processes such as range or arithmetic encoding are typically able to losslessly compress given input data up to close to the fundamental entropy limit of that data, as determined by the total entropy of the distribution of that data. Accordingly, one way in which end-to-end, learned compression can minimise the rate loss term of the rate-distortion loss function and thereby increase compression effectiveness is to learn autoencoder parameter values that produce low entropy latent representation distributions. Producing latent representations distributed with as low an entropy as possible allows entropy encoding to compress the latent distributions as close to or to the fundamental entropy limit for that distribution. The lower the entropy of the distribution, the more entropy encoding can losslessly compress it and the lower the amount of data in the corresponding bitstream. In some cases where the latent representation is distributed according to a gaussian or Laplacian distribution, this learning may comprise learning optimal location and scale parameters of the gaussian or Laplacian distributions, in other cases, it allows the learning of more flexible latent representation distributions which can further help to achieve the minimising of the rate-distortion loss function in ways that are not intuitive or possible to do with handcrafted features. Examples of these and other advantages are described in WO2021 / 220008A1, which is incorporated in its entirety by reference. Something which is closely linked to the entropy encoding of the latent distribution and which accordingly also has an effect on the effectiveness of compression of end-to-end learned approaches is the quantisation step. During inference, a rounding function may be used to quantise a latent representation distribution into bins of given sizes, a rounding function is not differentiable everywhere. Rather, a rounding function is effectively one or more step functions whose gradient is either zero (at the top of the steps) or infinity (at the boundary between steps). Back propagating a gradient of a loss function through a rounding function is challenging. Instead, during training, quantisation by rounding function is replaced by one or more other approaches. For example, the functions of a noise quantisation model are differentiable everywhere and accordingly do allow backpropagation of the gradient of the loss function through the quantisation parts of the end-to-end, learned system. Alternatively, a straight-through estimator (STE) quantisation model or one other quantisation models may be used. It is also envisaged that different quantisation models may be used for during evaluation of different term of the loss function. For example, noise quantisation may used to evaluate the rate or entropy loss term of the rate-distortion loss function while STE quantisation may be used to evaluate the distortion term. In a similar manner to how learning parameters top produce certain distributions of the latent representation facilitates achieving better rate loss term minimisation, end-to-end learning of the quantisation process achieves a similar effect. That is, learnable quantisation parameters provide the architecture with a further degree of freedom to achieve the goal of minimising the loss function. For example, parameters corresponding to quantisation bin sizes may be learned which is likely to result in an improved rate-distortion loss outcome compared to approaches using hand-crafted quantisation bin sizes. Further, as the rate-distortion loss function constantly has to balance a rate loss term against a distortion loss term, it has been found that the more degrees of freedom the system has during training, the better the architecture is at achieving optimal rate and distortion trade off. The system described above may be distributed across multiple locations and / or devices. For example, the encoder 110 may be located on a device such as a laptop computer, desktop computer, smart phone or server. The decoder 120 may be located on a separate device which may be referred to as a recipient device. The system used to encode, transmit and decode the input image 5 to obtain the output image 6 may be referred to as a compression pipeline. The AI based compression process may further comprise a hyper-network 105 for the transmission of meta-information that improves the compression process. The hyper-network 105 comprises a trained neural network 115 acting as a hyper-encoder ^^^^ℎand a trained neural network 125 acting as a hyper-decoder ^^^ℎ^ . An example of such a is shown in Figure 2. Components of the system not further discussed may be assumed to be the same as discussed above. The neural network 115 acting as a hyper-decoder receives the latent that is the output of the encoder 110. The hyper-encoder 115 produces an output based on the latent representation that may be referred to as a hyper-latent representation. The hyper-latent is then quantized in a quantization process 145 characterised by ^^ℎto produce a quantized hyper-latent. The quantization process 145 characterised by ^^ℎmay be the same as the quantisation process 140 characterised by ^^ discussed above. In a similar manner as discussed above for the quantized latent, the quantized hyper-latent is then entropy encoded in an entropy encoding process 155 to produce a bitstream 135. The bitstream 135 may be entropy decoded in an entropy decoding process 165 to retrieve the quantized hyper-latent. The quantized hyper-latent is then used as an input to trained neural network 125 acting as a hyper-decoder. However, in contrast to the compression pipeline 100, the output of the hyper-decoder may not be an approximation of the input to the hyper-decoder 115. Instead, the output of the hyper-decoder is used to provide parameters for use in the entropy encoding process 150 and entropy decoding process 160 in the main compression process 100. For example, the output of the hyper-decoder 125 can include one or more of the mean, standard deviation, variance or any other parameter used to describe a probability model for the entropy encoding process 150 and entropy decoding process 160 of the latent representation. In the example shown in Figure 2, only a single entropy decoding process 165 and hyper-decoder 125 is shown for simplicity. However, in practice, as the decompression process usually takes place on a separate device, duplicates of these processes will be present on the device used for encoding to provide the parameters to be used in the entropy encoding process 150. Further transformations may be applied to at least one of the latent and the hyper-latent at any stage in the AI based compression process 100. For example, at least one of the latent and the hyper latent may be converted to a residual value before the entropy encoding process 150,155 is performed. The residual value may be determined by subtracting the mean value of the distribution of latents or hyper-latents from each latent or hyper latent. The residual values may also be normalised. To perform training of the AI based compression process described above, a training set of input images may be used as described above. During the training process, the parameters of both the encoder 110 and the decoder 120 may be simultaneously updated in each training step. If a hyper-network 105 is also present, the parameters of both the hyper-encoder 115 and the hyper-decoder 125 may additionally be simultaneously updated in each training step. The training process may further include a generative adversarial network (GAN). When applied to an AI based compression process, in addition to the compression pipeline described above, an additional neutral network acting as a discriminator is included in the system. The discriminator receives an input and outputs a score based on the input providing an indication of whether the discriminator considers the input to be ground truth or fake. For example, the indicator may be a score, with a high score associated with a ground truth input and a low score associated with a fake input. For training of a discriminator, a loss function is used that maximizes the difference in the output indication between an input ground truth and input fake. When a GAN is incorporated into the training of the compression process, the output image 6 may be provided to the discriminator. The output of the discriminator may then be used in the loss function of the compression process as a measure of the distortion of the compression process. Alternatively, the discriminator may receive both the input image 5 and the output image 6 and the difference in output indication may then be used in the loss function of the compression process as a measure of the distortion of the compression process. Training of the neural network acting as a discriminator and the other neutral networks in the compression process may be performed simultaneously. During use of the trained compression pipeline for the compression and transmission of images or video, the discriminator neural network is removed from the system and the output of the compression pipeline is the output image 6. Incorporation of a GAN into the training process may cause the decoder 120 to perform hallucination. Hallucination is the process of adding information in the output image 6 that was not present in the input image 5. In an example, hallucination may add fine detail to the output image 6 that was not present in the input image 5 or received by the decoder 120. The hallucination performed may be based on information in the quantized latent received by decoder 120. Details of a video compression process will now be described. As discussed above, a video is made up of a series of images arranged in sequential order. AI based compression process 100 described above may be applied multiple times to perform compression, transmission and decompression of a video. For example, each frame of the video may be compressed, transmitted and decompressed individually. The received frames may then be grouped to obtain the original video. The frames in a video may be labelled based on the information from other frames that is used to decode the frame in a video compression, transmission and decompression process. As described above, frames which are decoded using no information from other frames may be referred to as I-frames. Frames which are decoded using information from past frames may be referred to as P-frames. Frames which are decoded using information from past frames and future frames may be referred to as B-frames. Frames may not be encoded and / or decoded in the order that they appear in the video. For example, a frame at a later time step in the video may be decoded before a frame at an earlier time. The images represented by each frame of a video may be related. For example, a number of frames in a video may show the same scene. In this case, a number of different parts of the scene may be shown in more than one of the frames. For example, objects or people in a scene may be shown in more than one of the frames. The background of the scene may also be shown in more than one of the frames. If an object or the perspective is in motion in the video, the position of the object or background in one frame may change relative to the position of the object or background in another frame. The transformation of a part of the image from a first position in a first frame to a second position in a second frame may be referred to as flow, warping or motion compensation. The flow may be represented by a vector. One or more flows that represent the transformation of at least part of one frame to another frame may be referred to as a flow map. An example AI based video compression, transmission, and decompression process 200 is shown in Figure 3. The process 200 shown in Figure 3 is divided into an I-frame part 201 for compressing and decompressing I-frames, and a P-frame part 202 for compressing and decompressing P-frames. It will be understood that these divisions into different parts are arbitrary and the process 200 may be also be considered as a single, end-to-end pipeline. As described above, I-frames do not rely on information from other frames so the I-frame part 201 may correspond to the compression, transmission, and decompression process illustrated in Figures 1 or 2. The specific details will not be repeated here but, in summary, an input image ^^0is passed into an encoder neural network 203 producing a latent representation which is quantised and entropy encoded into a bitstream 204. The subscript 0 in ^^0indicates the input image corresponds to a frame of a video stream at position t = 0. This may be the first frame of an entire video stream or the first frame of a chunk of a video stream made up of, for example, an I-frame and a plurality of subsequent P-frames and / or B-frames. The bitstream 204 is then entropy decoded and passed into a decoder neural network 205 to reproduce a reconstructed image ˆ0which in this case is an I-frame. The decoding step may be performed both locally at the same location as where the input image compression occurs as well as at the location where the decompression occurs. This allows the reconstructed image ˆ0to be available for later use by components, for example, a flow module and / or a residual module of both the encoding and decoding sides of the pipeline used for producing P- and / or B-frames. In contrast to I-frames, P-frames (and B-frames) do rely on information from other frames. Accordingly, the P-frame part 202 at the encoding side of the pipeline takes as input not only the input image ^^^^that is to be compressed (corresponding to a frame of a video stream at position t), but also one or more previously reconstructed images ˆ^^−1(and / or a ground truth representation thereof ^^^^−1when on the encode side) from an earlier frame t-1. As described above, the previously reconstructed ˆ^^−1is available at both the encode and decode side of the pipeline and can accordingly be used for various purposes at both the encode and decode sides, while only the encode side additionally has access to the ground truth frame ^^^^−1. At the encode side, previously reconstructed images may be used for generating a flow maps containing information indicative of inter-frame movement of pixels between frames. In the example of Figure 3, both the image being compressed ^^^^and the previously reconstructed image from an earlier frame ˆ^^−1are passed into a flow module part 206 of the pipeline. The flow module part 206 comprises an autoencoder such as that of the autoencoder systems of Figures 1 and 2 but where the encoder neural network 207 has been trained to produce a latent representation of a flow map from inputs ˆ^^−1and ^^^^, which is indicative of inter-frame movement of pixels or pixel groups between ˆ^^−1and ^^^^. The latent representation of the flow map is quantised and entropy encoded to compress it and then transmitted as a bitstream 208. On the decode side, the bitstream is entropy decoded and passed to a decoder neural network 209 to produce a reconstructed flow map ^^ . The reconstructed flow map ^^ is applied to the previously reconstructed image ˆ^^−1to generate a warped image ˆ^^−1, ^^. It is envisaged that any suitable warping technique may be used, for example bi-linear or tri-linear warping, as is described in Agustsson, E., Minnen, D., Johnston, N., Balle, J., Hwang, S. J., and Toderici, G. (2020), Scale-space flow for end-to-end optimized video compression. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (pp. 8503-8512), which is hereby incorporated by reference. It is further envisaged that a scale-space flow approach as described in the above paper may also optionally be used. The warped image ˆ^^−1, ^^is a prediction of how the previously reconstructed image ˆ^^−1might have changed between frame positions t-1 and t, based on the output flow map produced by the flow module part 206 autoencoder system from the inputs of ^^^^and ˆ^^−1. As with the I-frame, the reconstructed flow map ^^ and corresponding warped image ˆ^^−1, ^^may be produced both on the encode side and the decode side of the pipeline so they are available for use by other components of the pipeline on both the encode and decode sides. In the example of Figure 3, both the image being compressed ^^^^and the ˆ^^−1, ^^are passed into a residual module part 210 of the pipeline. The residual module part 210 comprises an autoencoder system such as that of the autoencoder systems of Figures 1 and 2 but where the encoder neural network 211 has been trained to produce a latent representation of a residual map indicative of differences between the input mage ^^^^and the warped image ˆ^^−1, ^^. The latent representation of the residual map is then quantised and entropy encoded into a bitstream 212 and transmitted. The bitstream 212 is then entropy decoded and passed into a decoder neural network 213 which reconstructs a residual map ^^ from the decoded latent representation. Alternatively, a residual map may first be pre-calculated between ^^^^and the ˆ^^−1, ^^and the pre-calculated residual map may be passed into an autoencoder for compression only. This hand-crafted residual map approach is computationally simpler, but reduces the degrees of freedom with which the architecture may learn weights and parameters to achieve its goal during training of minimising the rate-distortion loss function. Finally, on the decode side, the residual map ^^ is applied (e.g. combined by addition, subtraction or a different operation) to the warped image to produce a reconstructed image ˆ^^which is a reconstruction of image ^^^^and accordingly corresponds to a P-frame at position t in a sequence of frames of a video stream. It will be appreciated that the reconstructed image ˆ^^can then be used to process the next frame. That is, it can be used to compress, transmit and decompress ^^^^+1, and so on until an entire video stream or chunk of a video stream has been processed. Thus, for a block of video frames comprising an I-frame and ^^ subsequent P-frames, the bitstream may contain (i) a quantised, entropy encoded latent representation of the I-frame image, and (ii) a quantised, entropy encoded latent representation of a flow map and residual map of each P-frame image. For completeness, whilst not illustrated in Figure 3, any of the autoencoder systems of Figure 3 may comprise hyper and hyper-hyper networks such as those described in connection with Figure 2. Accordingly, the bitstream may also contain hyper and hyper-hyper parameters, their latent quantised, entropy encoded latent representations and so on, of those networks as applicable. Finally, the above approach may generally also be extended to B-frames, for example as is described in Pourreza, R., and Cohen, T. (2021). Extending neural p-frame codecs for b-frame coding. In Proceedings of the IEEE / CVF International Conference on Computer Vision (pp. 6680-6689). The above-described flow and residual based approach is highly effective at reducing the amount of data that needs to be transmitted because, as long as at least one reconstructed frame (e.g. I-frame ˆ^^−1) is available, the encode side only needs to compress and transmit a flow map and a residual map (and any hyper or hyper-hyper parameter information, as applicable) to reconstruct a subsequent frame. Figure 4 shows an example of an AI image or video compression process such as that described above in connection with Figures 1-3 implemented in a video streaming system 400. The system 400 comprises a first device 401 and a second device 402. The first and second devices 401, 402 may be user devices such as smartphones, tablets, AR / VR headsets or other portable devices. In contrast to known systems which primarily perform inference on GPUs such as Nvidia A100, GeForce 3090, GeFore 4090 GPU cards, the system 400 of Figure 4 performs the majority of operations during inference on a CPU and / or neural accelerator or NPU of the first and second devices respectively. That is, compute for performing both encoding and decoding are performed primarily by the respective CPUs and / or NPUs of the first and second devices 401, 402. This places very different power usage, memory and runtime constraints on the implementation of the above methods than when implementing AI-based compression methods wholly on GPUs with large VRAMs. In one example, the CPU of first and second devices 401, 402 may comprise a Qualcomm Snapdragon CPU. The first device 401 comprises a media capture device 403, such as a camera, arranged to capture a plurality of images, referred to hereafter as a video stream 404, of a scene 404. The video stream 404 is passed to a pre-processing module 406 which splits the video stream into blocks of frames, various frames of which will be designated as I-frames, P-frames, and / or B-frames. The blocks of frames are then compressed by an AI-compression module 407 comprising the encode side of the AI-based video compression pipeline of Figure 3. The output of the AI-compression module is accordingly a bitstream 408a which is transmitted from the first device 401, for example via a communications channel, for example over one or more of a WiFi, 3G, 4G or 5G channel, which may comprise internet or cloud-based 409 communications. The second device 402 receives the communicated bitstream 408b which is passed to an AI-decompression module 410 comprising the decode side of the AI-based video compression pipeline of Figure 3. The output of the AI-decompression module 402 is the reconstructed I-frames, P-frames and / or B-frames which are passed to a post-processing module 411 where they can prepared, for example passed into a buffer, in preparation for streaming 412 to and rendering on a display device 413 of the second device 402. It is envisaged that the system 400 of Figure 4 may be used for live video streaming at 30fps of a 1080p video stream, which means a cumulative latency of both the encode and decode side is below substantially 50ms, for example substantially 30ms or less. Achieving this level of runtime performance with only CPU and / or NPU compute on user devices presents challenges which are not addressed by known methods and systems or in the wider AI-compression literature. For example, execution of different parts of the compression pipeline during inference may be optimized by adjusting the order in which operations are performed using one or more known CPU scheduling methods. Efficient scheduling can allow for operations to be performed in parallel, thereby reducing the total execution time. It is also envisaged that efficient management of memory resources may be implemented, including optimising caching methods such as storing frequently-accessed data in faster memory locations, and memory reuse, which minimizes memory allocation and deallocation operations. A number of concepts related to the AI compression processes and / or their implementation in a hardware system discussed above will now be described. Although each concept is described separately, one or more of the concepts described below may be applied in an AI based compression process as described above. Concept 1: Temporal Consistency Loss Term In known flow-based approaches, and as described above, the loss function is typically represented by ^^ ^^ ^^ ^^ = ^^ + ^^ ∗ ^^, where ^^ is the distortion function, ^^ is a weighting factor, and ^^ is the rate loss. ^^ may be referred to as a lagrange multiplier. The langrange multiplier provides a weight for a particular term of the loss function in relation to each other term and can be used to control which terms of the loss function are favoured when training the network. Whilst flow information takes up part of the bitstream and is also used for reconstruction of the images neither the distortion term or rate term directly take account of flow contribution to the overall loss during training. Accordingly, present methods do not allow the optimization of a network during training to focus specifically on ensuring that the optical flow itself (e.g. the flow between an I-frame and a subsequent P- or B-frame) is reconstructed as accurately as possible during decoding. In other words, the known ^^ ^^ ^^ ^^ = ^^ + ^^ ∗ ^^ loss function fails to allow for flow to be explicitly taken into account during training. In order to address this problem, the present disclosure modifies the above loss function by including a term that depends directly on the difference between the optical flow between first and second input images, and the optical flow between corresponding reconstructed output images: ^^ ( ^^, ^ˆ^) where ^^ is information indicative of optical flow between a first image ^^^^−1at time ^^ − 1 and a second image ^^^^at time ^^, and ^ˆ^ is information indicative of optical flow between the corresponding reconstructed images ˆ^^−1and ˆ^^. Note that for ease of reference we refer to times t and t-1 but this can be generalised to any time points other than t and t-1, for example, the optical flow between t-2 and t, between t-2 and t-1, between t-n and t, and so on for any two or more images which are in some way temporally separated. This loss term is described herein as a temporal consistency loss ^^ . This is because training a network or system of networks using it in the loss function facilitates the learning of weights that force the reconstructed output images to be temporally consistent with the corresponding input images. That is, the flow between two input images is forced to match as closely the flow between the output images. Whilst it is envisaged that the loss function comprising solely ^^ may be used for training some or all parts of a compression pipeline, more flexibility and accordingly performance is obtained when the temporal consistency loss ^^ is included as a loss term in addition to one or more other loss terms, such as a rate term and / or distortion loss term: ^^ ^^ ^^ ^^ = ^^1^^ + ^^2^^ + ^^3^^ ( ^^, ^ˆ^) Optionally, the loss terms may be weighted with one or more regularization terms, here with weights ^^1, ^^2, ^^3to help focus the training towards a specific training objective based on one or more of reconstruction accuracy (distortion), bitstream size (rate), or flow (temporal consistency). Using the temporal consistency loss term in the loss function during training has a number of advantages. Firstly, this additional temporal-consistency-focused degree of freedom on which the network training objective can be based facilitates training the networks of the compression pipeline to be able to more accurately reconstruct images that contain objects that have a great deal of complex and intricate motion between the images. This can be intuitively understood as the temporal consistency loss term allowing the network to be trained to reconstruct this complexity and intricacy of the optical flow better and more accurately than a network whose loss term does not directly take temporal consistency into account. Secondly, it facilitates explicitly taking different types of information associated with optical flow into account when determining the loss during training. For example, one approach is to define the temporal consistency loss term as a function that depends on a flow map calculated in image space between the ground truth input images and the corresponding flow map calculated between the reconstructed output images:^^ ( ^^ ,ˆ)where ^^ defines the flow map between two input images, andˆdefines the flow between the reconstructed output images. However, if estimating the flow map involves one or more intermediate steps, for example estimating cost volumes at different levels of up or downsampling of the input image, the temporal consistency loss term may additionally or alternatively depend on the outputs of these intermediate steps. In this example, it may be defined as a function that depends on one or more cost volumes, for example calculated at different levels of up or downsampled feature pyramids, indicative of differences between pixels of a first input image and one or more pixels of a second input image at different levels of up or downsampling, and the corresponding cost volumes in the corresponding reconstructed first and second output images: ^^ ( ^^^^, ^ˆ^^^) where ^^^^, and ^ˆ^^^define cost volumes respectively associated with the matching costs for associating a pixel in a first image with its corresponding pixel or pixels in a second image for both the input images and the reconstructed output images. Effectively, estimating a cost volume is an example of estimating a pixel difference between a pixel in one image and one or more pixels in another image. For completeness, it is noted that the above temporal consistency loss term may be further generalised to be dependent on multiple cost volumes calculated from different resolutions of the image, more details of which are described in connection with the feature pyramids of Figure 5 onward, where cost volumes at different resolutions are used to estimate optical flow. For example, if the temporal consistency loss takes into account the cost volumes from pyramid levels 4 and 5 at the same time, the loss term would be: ^^4∗ ^^ ( ^^^4^ , ^ˆ^^4^ ) + ^^5∗ ^^ ( ^^^5^ , ^ˆ^^5^ ), where the index 4 and 5 indicates the level of the feature Various methods of calculating flow and / or associated cost volumes / pixel differences are known to the skilled person. For example, a handcrafted algorithm may be used to calculate the matchmaking cost between each pixel in the first image with pixels in a radius around the corresponding pixel in the second image, for example a 4x4 radius, a 7x7 radius or some other predetermined radius. This may also be applied to different levels of up or downsampling of the image or representations thereof at different spatial resolutions. The matchmaking cost may defined as closeness in pixel values by way of dot product, or by some other distance metric such as cosine similarity or mean absolute difference, and so on. The present disclosure envisages using mean absolute difference as this is substantially faster to calculate than other distance metrics which in turn helps to reduce runtimes. An illustrative method of estimating cost volumes and feature pyramids at different levels of resolution is described later in connection with Figure 5. It will be appreciated that flow maps, cost volumes, mean absolute differences and various representations thereof are only examples of the types of information indicative of temporal differences between first and second input or output images, and accordingly the temporal consistency loss term may depend on any other information indicative of temporal differences between images. Thirdly, the temporal consistency loss approach allows not only information associated with flow in image space to be taken into account explicitly in the loss, but also information associated with flow in one or more other spaces of the compression pipeline. For example, as described above in the general compression pipeline section, the first and second input images are said to be encoded into respective latent representations. If an object in the first image has moved to a different position in the second image, not only is there optical flow in the image space, but there is also corresponding flow in the latent space. The temporal consistency loss can accordingly additionally or alternatively be depend on information indicative of flow in the latent space. This is particularly advantageous because the latent representations are lower dimensional than ground truth images and calculating cost volumes and flow in this lower dimensional latent space is computationally cheaper, significantly contributing to reduced runtimes. This approach can be taken a step further where there is a hyper-network such as in Figure 2, or a hyper-hyper network, and so on, that further reduce dimensionality with each extra level of hyper. That is, movement between images will result in flow in image space, latent space, hyper-latent space and so on, each being lower-dimensional than the ground truth image, and accordingly in each of which flow can be estimated faster and computationally more cheaply. The temporal consistency loss can accordingly additionally or alternatively also depend on flow in these spaces. Moving onto exemplary details of the temporal consistency loss term ^^ , it may take the form of a mean squared error loss indicative of the mean squared error between the information associated with flow between the first and second input images, and the corresponding information associated flow between the first and second output images. However, this is exemplary only, and it may take on other forms as will be appreciated by the skilled person. Figure 5 illustrates an example of a flow module, in this case a network 500, configured to estimate information indicative of a difference between an image ^^^^−1and an image ^^^^, e.g. flow information. Figure 5 is provided as an example of how such flow information may be calculated between two input or output images, and accordingly which may be used when evaluating a loss function which incorporates a temporal consistency loss term. The network 500 comprises a set of layers 501a, 501b respectively for an image ^^^^−1and an image ^^^^from respective times or positions in a sequence ^^ − 1 and ^^ of a sequence of image frames. The set of layers 501a, 501b may define one or more convolution operations and / or nonlinear activations in the layers to sequentially downsample the input images to produce a pyramid of feature maps for different levels of coarseness or spatial resolution. This may comprise performing ℎ / 2 ^^ / 2 downsampling in a first layer, ℎ / 4 ^^ / 4 downsampling in a second layer ℎ / 8 ^^ / 8 downsampling in a third layer, ℎ / 16 ^^ / 16 downsampling in a fourth layer, ℎ / 32 ^^ / 32 downsampling in a fifth layer, ℎ / 64 ^^ / 64 downsampling in a sixth layer, and so on. It will of course be appreciated that these downsampling operations and levels of coarseness or spatial resolution of a pyramid feature map are exemplary only and others levels are also envisaged. With the downsampling operations performed and the corresponding pyramid of feature maps generated, a first cost volume 502 is calculated at the most coarse level between the feature map pixels of the first image ^^^^−1and the corresponding feature map pixels of the second image ^^^^. Cost volumes define the matchmaking cost of matching the pixels in the initial image with the pixels in the later image. That is, the closeness of each pixel, or a subset of all pixels, in the initial image to one or more pixels in the later image is determined with a measure of closeness such as a vector or dot product, a cosine similarity, a mean absolute difference, or some other measure of closeness. This metric may be calculated against all pixels in the later image, or only for pixels in a predetermined search radius such as a 1-10 pixel radius (preferably a 1, 2, 3, or 4 pixel radius), or some other radius as described in connection with concept 4 below, around the pixel coordinate corresponding to the pixel against which the measure of closeness is being calculated. Finally, a first flow 503 can be estimated from the first cost volume 502. This may be achieved using, for example a flow extractor network which may comprise a convolutional neural network comprising a plurality of layers trained to output a tensor defining a flow map from the input cost volumes. Other methods of calculating flow information from cost volumes will also be known to the skilled person. The same process is then repeated for the other levels of coarseness to calculate a second cost volume 504 and second flow 505, and so on for the cost volumes and flows associated with each of the levels of coarseness have been calculated, up to the final cost volume 506 and flow 507. The weights and / or biases of any activation layers in network 500 (e.g. optionally in the downsampling convolution layers and / or in a flow extractor network that produces the pyramid of feature maps and / or flow maps from the cost volumes) are trainable parameters and can accordingly be updated during training either alone, or in an end to end manner with the rest of the compression pipeline. The trainable nature of these parameters provides the network 500 with flexibility to produce feature maps at each level of spatial resolution (i.e. pyramid feature maps) and / or at the flow outputs that are forced into a distribution that best allows the network to meet its training objective (e.g. better compression, better reconstruction accuracy, more accurate reconstruction of flow, and so on). For example, it allows the network 500 to produce feature maps that, when cost volumes and / or flow are calculated therefrom, produce cost volumes or flows that are distributed roughly matching the latent representation distribution that would previously have been expected to be output by a dedicated flow encoder module. This effectively allows a dedicated flow encoder to be omitted entirely from the flow compression part of the pipeline. This will be explained in more detail below in concept 2 of the present disclosure. Optionally, for each level of coarseness or resolution, the flow of the previous level or levels of coarseness or resolution may be used to warp 508, 509, the feature maps before the cost volume is calculated. This has the effect of artificially reducing the amount of relative movement between the pixels of the t and t - 1 images or feature maps when calculating the cost volumes, reducing flow errors for high movement details. A warpless flow estimation is also envisaged in that the inventors have realized that removing warping entirely or in some or all levels of coarseness or resolution can substantially reduce runtime of flow calculation while maintaining good levels of flow accuracy, something which goes against the teachings of known flow estimation literature. As the warping process uses inputs from different levels of coarseness or spatial resolution, the flow estimation output may be upsampled 510, 511 first to match the coarseness resolution of the feature map to which the flow is being applied in the warping process. The outputs of the flow module may accordingly be one or more cost volumes or some representation thereof, and / or one or more flows or some representation thereof). Returning to the temporal consistency loss: ^^ ( ^^, ^ˆ^) The input images ^^^^−1and ^^^^are run through the flow module 500 to produce ^^, and the reconstructed output images ˆ^^−1and ˆ^^produced by the compression process are run through the flow module 500 to produce ^ˆ^. The variables ^^ and ^ˆ^ can accordingly comprise one or more of the cost volumes, and / or one or more flows output by the flow module 500 of Figure 5. The function ^^ may be a mean squared error or some other function which calculates the loss as a difference between ^^ and ^ˆ^ during evaluation of ^^ as a loss term during the training process. Whilst this concept has primarily been described in the context of training using a temporal consistency loss term. It will be appreciated that the flow module 500 of Figure 5 may also be used for calculating flow during inference. That is, applied to sequences of input images to compress them, transmit them in a bitstream, and decompress them, using a trained network or networks, whereby the decompression may be a flow-based process that uses flow to reconstruct one or more of the input images. As is apparent from the examples described above, it will be appreciated that the consistency loss term may depend on whatever representation of the cost volumes and / or flows is used in a given compression pipeline. This may accordingly encompass the temporal consistency loss term being dependent on a representation of cost volumes and / or flow directly output by whatever flow compression module is being used distributed according to whatever distribution such a module produces its outputs in. Additionally or alternatively, it also encompasses the temporal consistency loss term being dependent on some further encoding of those outputs into a different distribution, such as by a dedicated flow encoder configured to encode the produced cost volumes and / or flows into some encoded (optionally quantized) latent representation thereof distributed in a way that can be more efficiently entropy encoded than the flow information output by the flow module directly. It will be appreciated that the presently described temporal consistency loss concept may be used in a standalone manner or together with any of the other concepts described herein. Concept 2: Optical Flow Compression Network As has been described in Concept 1 above, training a learned compression network to explicitly take flow into account (whether in image space, latent space, hyper-latent space and so on) enables compression networks to be trained that are better suited to the specific task of video compression where harnessing temporal redundancy across a sequence of images can help to reduce bitstream sizes and reconstruction accuracy. It will be appreciated that running a flow-based compression pipeline once training has been completed also relies on estimation of flow, whether by handcrafted algorithm or through some trained network. The output estimated flow itself may be compressed and transmitted in a bitstream, which is typically done by a dedicated flow encoder network that encodes the flow information into a latent representation distributed according to a distribution that can be efficiently entropy encoded. Irrespective of the flow estimation approach taken, dedicated flow encoders increase run time and partly contribute to preventing learned compression codecs from running in real time or near real time. Consider for example the pipeline of Figure 3. At the encode side, a previously reconstructed image and a new, to-be-encoded image may be used for generating a flow map containing information indicative of inter-frame movement of pixels between frames. In the example of Figure 3, both the current image being compressed ^^^^and the previously reconstructed image from an earlier frame ˆ^^−1are passed into a flow module 207 that typically has two parts: a flow estimation part, and a dedicated encoder part that encodes the estimated flows into a latent representation that can be efficiently entropy encoded. The dedicated flow encoder is effectively a specialized network dedicated to producing latent representations of flow that are distributed as close to an optimally entropy encodable distribution as possible. In Figure 3, these two parts are shown as a single component. The flow estimation part 207 may comprise, for example, the flow module 500 of Figure 5. That is, the flow module 500 produces its cost volumes and flows, then passes one or more of these into the second part: the dedicated flow encoder network that is trained to produce the latent representation of flow that can be efficiently entropy encoded before being sent in a bitstream to the decoder. This approach is slow because we first have to calculate the cost volumes and flows before we can encode them into a latent representation, which itself is a slow process. We then do the entropy encoding of the latent representation of the cost volume(s) and / or flow(s) into a bitstream which is finally transmitted. This multi-step approach increases run time. Instead, concept 2 of the present disclosure proposes a different counter-intuitive approach: omit the dedicated flow encoder. Instead, the outputs of the flow module 500 output(s) are directly entropy encoded and fed into the bitstream without first encoding them into a latent representation. Given that the process of encoding flow module output(s) into a latent representation with a flow compression encoder is computationally expensive, removing this component entirely from the flow compression module 207 results in a significant decrease in runtime, thereby contributing to the goal of being able to run the pipeline in inference in real time or near real time. This is, in part, made possible by virtue of the trainable nature of the flow module 500, whereby the weights and biases of one or more of the activation layers of the flow module 500 result in cost volumes or flows that are already distributed according to a distribution that corresponds roughly to that which a dedicated flow encoder may produce. This is illustrated in more detail in Figure 6a. Figure 6a illustratively shows two flow compression modules 600a and 600b which may be used as flow module 207 in Figure 3. The same reference numerals are used for like-components. In the first module 600a, two input images ^^^^and ˆ^^−1are passed 601a, 601b are input into a flow module such as flow module 500 of Figure 5, which produces pyramid feature maps 602 of different levels of coarseness, and corresponding cost volumes and / or flows 603. The final flow estimation is then passed to a dedicated flow compression encoder 604 which encodes it to produce a latent representation of the final flow estimation. The output may thus be a latent representation of one or more of a H W, H / 2 W / 2, H / 4 W / 4, H / 8 W / 8, H / 16 W16. H / 32 W / 32, H / 64 W / 64, or some other resolution, cost volumes and / or flows. This is finally entropy encoded into a bitstream 605, and transmitted. The bitstream 605 is entropy decoded and the decoded bitstream is passed to a decoder 606 which reconstructs the cost volumes and / or flows which may be used in a flow-based approach as described above in the general concepts section. However, as described above, the approach of the flow compression module 600a with a dedicated flow compression encoder 604 is slow and computationally expensive. In contrast, the present inventors have realized that omitting the dedicated flow compression encoder 604 entirely and instead directly entropy encoding one or more outputs of the flow module 500 into the bitstream 605 without first passing it through the dedicated flow encoder 604 results in a substantial speed up at run time. Counter-intuitively, this removal of the dedicated flow compression encoder 604 does not appreciably appear to effect the performance of the rest of the compression pipeline both in terms of distortion and bit rate. This modified approach is illustrated with flow compression module 600b in Figure 6a (also described herein as an optical flow compression network) where it is apparent that the flow encoder 604 of flow compression module 600b has effectively been chopped out. Thus, instead of first producing a final flow map using one or more feature pyramids, and warping operations, and then producing an efficiently entropy encodable latent represention of that final flow map with a further encoder, the final flow map (and / or one or more of the cost volumes and / or flows into a latent representation before the entropy encoding step, the one or more of a H W, H / 2 W / 2, H / 4 W / 4, H / 8 W / 8, H / 16 W16. H / 32 W / 32, H / 64 W / 64, or some other resolution, cost volumes and / or flows) is natively produced with a distribution that is already optimised for a small compression rate based on the training of the weights of the optical flow compression network with a rate distortion loss function. That is, the flow map can natively already be efficiently entropy encoded into a small bitstream size without any additional encoding into a lanent representation being needed. This approach is based on the insight that flow modules such as flow module 500 have trainable parameters and accordingly have a great deal of flexibility in terms of the distribution of outputs they can be trained to produce. The inventors have found that when the dedicated flow compression encoder 604 is included, the flow module 500 has no need to produce outputs in a distribution that can be efficiently entropy encoded because the neural networks that make up the compression pipeline as a whole are simply able to rely on the dedicated flow encoder 604 to minimize any contribution to bitrate that including the flow information in the bitsream has. However, when the dedicated flow compression encoder 604 is removed, networks of the compression pipeline are no longer able to rely on the dedicated flow encoder 604. In its place, the inventors have found that the flow module 500 learns during training to compensate by outputting cost volumes and / or flows that are similarly distributed as those that would be output by a dedicated flow compression encoder 604. This can be understood as the flow module 500 (that is, the trainable networks within it such as the activation layers and / or convolution layers) effectively being forced to mimic the dedicated flow compression encoder 604 during training when the loss is being minimized because the system can no longer rely on the (now removed) dedicated flow compression encoder 604 to perform that task. It is envisaged that the training of the flowmodule 500may either be performed in an end-to-end manner together with the rest of the compression pipeline or alternatively in a student-teacher approach where the network of the flow module 500 is the student component, and a known optical flow model and / or pre-trained flow compression network is the teacher component. Additionally or alternatively, the training of the flow module 500 may be performed separately using data on which the groundtruth flow is known. For example, by using 3D animation video data whose groundtruth flow is known a priori through the animation program used to generate it, or using auto flow generation methods. This counter-intuitive removal of the dedicated flow compression encoder 604 from the flow compression module to force the other components to effectively take on the tasks previously performed by the dedicated flow compression encoder 604 contributes significantly to a speed up at run time speed on the encoding side of the compression pipeline. This approach accordingly makes a substantial stride forward towards the goal of achieving real time or near real time performance during inference. This concept may optionally be used together with concept 1, whereby the consistency loss term may depend on the representation of flow information directly output by the flow module 500 and distributed according to a first distribution that can be efficiently entropy encoded without first needing to encode it into a latent representation distributed according to a second distribution using a dedicated flow encoder 604. It will be appreciated that training the flow module may comprise using any of the techniques described herein in the general section above and elsewhere in the present disclosure, for example in connection with concept 1 and / or any of the other concepts described herein. For example, Figure 6b illustrates the introduction of the flow compression modules 600b into a flow-residual compression pipeline illustratively showing the encoding and decoding of p-frames of an image stream, whereby a groundtruth image and its previously encoded and decoded reconstruction at t-1 are available. This corresponds generally to the flow-residual compression pipeline shown in Figure 3 and accordingly uses the same reference numbers for corresponding features. However in this case, the output of the flow module 602 is natively in the distribution of a latent representation and can be immediately transmitted to the decoder, without needing a standalone dedicated flow compression encoder. In addition, it is envisaged that the warped reconstructed, previously decoded image from t-1 may be passed directly to the residual decoder, in addition to whatever is output by the residual encoder 211. This provides the residual decoder with the additional context of the warped reconstructed, previously decoded image. Concept 3: Variable pixel search radius and warpless flow estimation As has been described in connection with Figure 5 above, estimating pixel differences between two representations of different images facilitates the estimation of cost volumes and / or flows between those images. However, estimating pixel differences can introduce a very large compute overhead. This can be intuitively understood when considering an extreme case where a pixel of a first image having a 1920x1080 resolution is compared against every pixel of a second image of the same resolution, effectively, this requires comparing each of 2,073,600 pixels in the first image individually against that many pixels in the second image. Running this extreme case in real time is not possible in resource constrained environments. Even when images of this kind of resolution are downsampled to H / 64 W / 64 or further, the compute overhead for comparing each pixel of the first downsampled image against every pixel of the second downsampled image is very high. Instead, reducing the pixel search radius from covering every pixel in the second image to a smaller radius that covers only a subset thereof reduces the compute overhead to a manageable level. Thus, considering the example of cost volumes, one aspect of estimating cost volumes is determining what pixel search radius to use, for example, anywhere in the range of a 1 pixel search radius (effectively resulting in a 3x3 pixel block made up of a center pixel and the 8 pixels around it) up to max resolution pixel search radius covering the entire image. For example, with reference to Figure 5, one approach could be to calculate the cost volumes using a pixel search radius of 4, resulting in calculating measures of closenesss for each pixel in a 9x9 pixel search block (i.e. 81 pixels in total) in the second image for each pixel in the starting image. The advantage of increasing the search radius above this is that it is more likely to capture information indicative of flow for pixels associate with objects having very high movement albeit at the tradeoff of higher compute costs. Reducing the search radius below 4, e.g. to 1, 2, or 3 may result in errors in the finally estimated flow as, if a pixel of a high movement object moves outside of the search radius, its movement wont contribute to the cost volume and accordingly won’t be captured in the flow. The downsides of small search radii are partly compensated for by the above-described pyramid feature approach, whereby downsampling the images effectively forces movement information into a smaller number of pixels (which accordingly is more likely to be captured in a smaller search radius) at the cost of the detail of that movement information. Despite this, the pyramid feature approach described above can still require a large number of pixel measure of closeness calculations to be performed at the higher spatial resolutions of the feature pyramid. For example, at native H and W resolutions, or even at the H / 2 W / 2, or H / 4 W / 4 resolutions. Accordingly, even when using a reduced search radius together with a pyramid feature approach, run time can be slow. Concept 3 addresses this problem, at least in part, by using different pixel search radii for different levels of spatial resolution of a feature pyramid when calculating pixel differences between two images (where the term images may be either an image direclty or its feature map output at a given coarseness of a feature extractor such as that of Figure 5, 9, 10, 19 and / or 20. For example, rather than using a fixed pixel search radius of 4 (resulting in a 9x9 or 81 pixel search block) across all spatial resolutions, the pixel search radius may be dynamically adjusted depending on spatial resolution being considered, this means it may be, for example, decreased for higher spatial resolutions to speed up run time by reducing the number of pixel difference calculations that need to be performed for different spatial resolutions. In one implementation this may comprise using a first pixel search radius for one or more first spatial resolutions of a feature pyramid of an optical flow estimation method, and using a second pixel search radius for one or more second spatial resolutions. Alternatively, this may comprise using a different pixel search radius for each spatial resolution of a plurality of spatial resolutions. Figure 7a illustrates an example of a flow module, in this case a network 700, configured to estimate information indicative of a difference between an image ^^^^−1and an image ^^^^, e.g. flow information. Figure 7 is provided as an example of how such flow information may be calculated between two input or output images, and accordingly which may be used when evaluating a loss function which incorporates a temporal consistency loss term. The network 700 comprises a set of layers 701a, 701b respectively for an image ^^^^−1and an image ^^^^from respective times or positions in a sequence ^^ − 1 and ^^ of a sequence of image frames. The set of layers 701a, 701b may define one or more convolution operations and / or nonlinear activations in the layers to sequentially downsample the input images to produce a pyramid of feature maps for different levels of coarseness or spatial resolution. This may comprise performing ℎ / 2 ^^ / 2 downsampling in a first layer, ℎ / 4 ^^ / 4 downsampling in a second layer ℎ / 8 ^^ / 8 downsampling in a third layer, ℎ / 16 ^^ / 16 downsampling in a fourth layer, ℎ / 32 ^^ / 32 downsampling in a fifth layer, ℎ / 64 ^^ / 64 downsampling in a sixth layer, and so on. It will of course be appreciated that these downsampling operations and levels of coarseness or spatial resolution of a pyramid feature map are exemplary only and others levels are also envisaged. With the downsampling operations performed and the corresponding pyramid of feature maps generated, a first cost volume or pixel difference 702 (in this case a mean absolute difference or MAD, but in other cases it may be other metrics such as MSE, and / or other norms such as cosine distance and so on) is calculated at the most course level between the feature map pixels of the first image ^^^^−1and the corresponding feature map pixels of in the second image ^^^^. As described above in concept 1, this defines the matchmaking cost of matching the feature map pixels of the initial image with the feature map pixels of the later image. That is, the closeness of each feature map pixel, or a subset of all feature map pixels, of the initial image to one or more feature map pixels of the later image is determined with a measure of closeness such as a vector or dot product, a cosine similarity, a mean absolute difference, or some other measure of closeness. This metric may be calculated against all feature map pixels in the later image, or only for feature map pixels in a predetermined search radius such as a 1-10 pixel radius (preferably a 1, 2, 3, or 4 pixel radius), or some other radius as described in connection with concept 4 below, around the pixel coordinate corresponding to the pixel against which the measure of closeness is being calculated. However, in figure 7a, the pixel search radius is not the same for each layer of spatial resolution. Instead, the lower spatial resolutions e.g. H / 64 W / 64, H / 32 W / 32 etc. use a higher pixel search radius, in this case 3 (a 7x7 or 49 pixel search block), whereas the higher spatial resolutions e.g. H W, H / 2 W / 2, H / 4 W / 4 etc. use a lower pixel search radius, in this case 1 (a 3x3 or a 9 pixel search block). In this way, the total number of pixel difference operations that are performed is significantly reduced without significantly impacting accuracy. This is in part because a significant amount of flow information is already taken into account in the lower spatial resolutions and the inventors have realized that the contribution of the pixel differences of the higher spatial resolutions to the overall flow estimation may not be significant and accordingly, a very low pixel search radius of 1 (or less as will be described in concept 4) is possible in many cases without significantly affecting accuracy. Effectively, the very low pixel search radii of the higher spatial resolutions act as refinement search blocks, refining the flow for very granular details between immediately neighboring pixel(s), which can be performed very quickly compared to larger search radii. It is also envisaged that the decrease or increase in search radii across spatial resolutions may be gradual, for example starting off at e.g. 4 for the lowest spatial resolution, 3 for the one above that, 2 for the one above that, and so on. The pixel search radii may also be dynamically set based on some other determined parameter, for example, a user input or a result of an automatic classification of the type of image stream that is being encoded (e.g. a slow moving nature scene with very little movement versus a fast moving 3D animation with many fast moving objects). The pixel search radii for the different layers may also be a trainable parameter whereby in end to end training of the networks of the compression pipeline, the pixel search radius for each layer may be adjusted during training to find an optimal pixel search radius for each layer. Finally, a first flow 703 can be estimated from the first cost volume 702. This may be achieved using, for example a flow extractor network which may comprise a convolutional neural network comprising a plurality of layers trained to output a tensor defining a flow map from the input cost volumes. Other methods of calculating flow information from cost volumes will also be known to the skilled person. The same process is then repeated for the other levels of coarseness to calculate a second cost volume 704 and second flow 705, and so on for the cost volumes and flows associated with each of the levels of coarseness have been calculated, up to the final cost volume 706 and flow 707. As with the above described concepts, the weights and / or biases of the convolution and / or activation layers in network 700 (e.g. optionally in the downsampling convolution layers and / or in a flow extractor network that produces flow maps from the cost volumes) and optionally the pixel search radii are trainable parameters and can accordingly be updated during training either alone, or in an end to end manner with the rest of the compression pipeline. The trainable nature of these parameters provides the network 700 with flexibility to produce feature maps at each level of spatial resolution (i.e. pyramid feature maps) and / or at the flow outputs that are forced into a distribution that best allows the network to meet its training rate-distortion objective (e.g. optimising for better compression, better reconstruction accuracy, more accurate reconstruction of flow, and so on). As described above, optionally, for each level of spatial resolution, the flow of the previous level or levels of resolution may be used to warp 708, 709, the feature maps before the cost volume or other pixel difference metric is calculated. This has the effect of artificially reducing the amount of relative movement between the feature map pixels of the t and t - 1 images when calculating the cost volumes, reducing flow errors for high movement details. As the warping process uses inputs from different levels of spatial resolution, the flow estimation output may be upsampled 710, 711 first to match the coarseness resolution of the feature map to which the flow is being applied in the warping process. An interesting synergistic effect of the different search radii for different spatial resolutions is that it enables the above optional warping process to be omitted for one, some, or all spatial resolution pixel difference estimations. More specifically, as is described above in the concepts above, the concept of warping features at one layer of resolution based on the flow estimated from another layer of resolution such is as described in Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018, primarily aims to reduce errors in estimated flow for high movement objects. This warping effectively attempts to shift features that have moved great distances in the base image closer together again when calculating cost volumes. That is, if pixel matching is only being performed up to a predetermined radius that is less than the entire image, the matching is likely to be more accurate where a pixel of a high movement object has been warped from out of the search radius back into the search radius. However, something that known flow estimation processes, that focus on accuracy, do not take into account is that the warping process itself can have a substantial computational overhead and accordingly contributes to an increase in run time. A question accordingly becomes whether it is possible to remove the warping layers to increase run time. More specifically, consider estimating the difference between an input image and a previously decoded image using cost volumes or mean absolute differences. Estimating these with a pixel search radius or range of 4 pixels results in pixel matching for a block of 9x9 pixels (i.e. the center pixel plus 4 pixels in each direction around the center pixel) which is computationally expensive and slow to compute at run time as it requires the matching against 81 pixels. When this number of pixels is scaled up over e.g. a 1920x1080 resolution image, the cost volume computation quickly becomes an expensive operation to perform. The inventors have realized that warping can be removed from one, some, or all of the spatial resolutions without impacting flow estimation accuracy as long as at least one or some of the difference estimations of at least some of the spatial resolutions have a pixel search radius of 3 or more. Accordingly, setting the search radius for the difference estimation at the lowest spatial resolution to 3 or more, allows the search radius to be reduced to fewer than 3 (e.g. 1) for the final, highest spatial resolution difference estimation (which saves the most on compute cost), and allows warping to be removed entirely, also speeding up run time. It will accordingly be appreciated, that using pixel search blocks of different sizes contributes, at least in part, to facilitating the use of a warpless flow estimation network such as that shown in Figure 7b. All of the reference numerals in Figure 7b correspond to those shown in Figure 7a, except for the warping steps which are not present. Finally it will also be appreciated that a warpless flow estimation network such as that may also be used regardless of pixel search block size, e.g. where they are all bigger in size than 1, 2, 3, 4, etc. It will also be appreciated that where warping is present, the warping process enhances the ability of the flow estimator (e.g. a flow estimator network such as described in the pwc-net paper, or any other flow estimation method as will be known to the skilled person) to take into account the flow information from different levels of spatial resolution to pass into the next level of spatial resolution. However, for flow information for warpless flow estimation networks the flow information between different levels of resolution is passed into the next level of resolution simply by directly feeding the different spatial resolution representations of the inputs into the flow estimator. Accordingly, as the flow estimator network is trained, it learns to estimate flow simply from the unwarped representations of the previous level of spatial resolution. More generally, where no warping is performed, it will be appreciated that the difference estimations e.g. calculating the cost volumes or mean absolute differences no longer depends on the flow result of any earlier level of spatial resolution. Accordingly, it becomes possible to calculate all of these differences fully in parallel as long as the upsampling / downsampling to produce the different spatial resolution representations has already been performed. These cost volumes or mean absolute differences can then be passed into a flow estimator in one go. This ability to parallelize the difference estimations facilitated by the warpless approach, allows for a speed of up run time as long as hardware resources that can compute in parallel efficiently are available, such as a GPU, TPU, or some other parallel operation focused processor, which is increasingly the case on modern hardware devices such as smartphones, laptops and other mobile devices. Further, it is envisaged that the upsampling of the pixel difference estimations to get them into the same spatial resolution to perform a final flow estimation may be performed in parallel. This is shown in Figure 7c. In general terms, a difference estimation (e.g. a cost volume calculation 702, 704, 706b, 706a, 706 or mean absolute difference calculation) for each of the representations at the different spatial resolution levels 701a, 701b of the input images is performed. These may then be upsampled 710, 711, 712a, 712b, 712c (e.g. using a single layer convolution) in parallel to the same spatial resolution as applicable which is faster than a sequential approach where the compute resources for parallel calculations are available. It will be appreciated that the upsampling of the lowest spatial resolutions, may need to be performed additional times to be able to match the spatial resolution of the highest spatial resolutions. The amount of upsampling may also be a configurable and / or learnable parameter. The upsampled outputs are then combined, for example concatenated or combined in some other manner, before being past into a flow estimator e.g. a flow estimation network as will be appreciated by the skilled person. It will also be apparent from Figure 7c that the intermediate flow estimation steps have been omitted to further speed up run time. In general terms, the inventors have realized that the final flow estimation network is able to learn to output a final flow using only the different difference estimations from the different spatial resolutions (e.g. the cost volumes or mean absolute differences from each spatial resolution) without also needing the intermediate flow estimations. Effectively rendering the intermediate flow estimations superfluous and allowing a substantial speed up of run time – something which is made possible by omitting warping. The architecture shown in Figure 7c is in more general terms a high speed, parallelisable, low run time, warpless flow estimation network that synergistically cuts down on various components of known flow estimation approaches is able to output flow in the format of a latent representation that does not need a dedicated flow encoder. Its runtime speed is substantially faster than known methods. A variation of the architecture shown in Figure 7c is shown in Figure 7d, whereby the upsampling 710, 711, 712a, 712b, 712c of the difference estimations is instead performed sequentially, each time being combined with the next difference estimation of the next spatial resolution in a combination operation 713 such as a concatenation before the final flow estimation 707 is performed with the flow estimator. Finally, it will be appreciated that concept 3 may be used together with the concepts described herein or as a standalone concept. Concept 4: Shaped pixel search radii Continuing on from concept 3, the inventors have realized that a pixel search radius as used in the above-described flow modules need not be a search of pixels symmetrically around a center pixel. Instead, the pixel search radii may be shaped to, for example, skip one or more pixels at predetermined positions around a given search block. This effectively reduces the total number of pixel difference calculations that need to be performed during the flow estimation process and accordingly contributes to runtime speed up. Consider for example a 1 pixel search radius which corresponds to a 3x3 or 9 pixel search block (one center pixel and 1 pixel in each direction around it). In many cases, there is very little movement in pixel blocks that small. Accordingly, it is not necessary to perform the pixel difference calculations across the entire search block. Instead, it can be performed for only the pixel directly above, directly below, directly to the right and directly to the left of the center block e.g. in a diamond shape. Alternatively, the four corner pixels of the search block may be used instead, e.g. a cross shape. Thus, if the center block is also included, the total number of pixel difference calculations in these examples goes from 9 down to 5, an almost 50 per cent decrease in the number of pixel calculations to be performed within the context of the flow calculation. The same approach can be applied to larger search blocks, whereby the pixel difference calculations are not performed for a proportion of the search block. For example, consider a 4 pixel radius corresponding to a 9x9 search block or 81 pixels. Adopting the same “diamond” shape or “cross” shape described above, pixels above an angled line in each quadrant of the search block may be omitted from the pixel difference calculations, resulting in a diamond search block. Or alternatively, specifically those pixels in the corners may be included, resulting in a cross search block. This would again reduce the number of pixel difference calculations to be performed by approximately 50 per cent. This concept can be taken further by, for example using other shapes and patterns such as a checkerboard pattern, where every other pixel in the search block is omitted, or for example by adopting a more complex pattern which may be handcrafted or a learned shape or pattern. The patterns also need not be centered around a center pixel but may instead be positioned offset from the center pixel by some predetermined about in one or more directions. Further, the search radius, i.e. the indicating how big the search block will be need not be as small as the above described 1-10 pixel radii, but may now optionally be larger as the shaped or patterned search blocks allow a wider search radius to be implemented for the same compute cost as the total number of pixels in the shaped or patterned search blocks is smaller than a filled in search block. In the case of a learned shape or pattern, it is envisaged that the flow module may be initialized with a starting pixel search shape or pattern, such as a full search block or a randomized search block which omits certain pixels within the block randomly. A loss function is then evaluated whereby the loss function may comprise the above-described rate distortion loss or some and / or some other loss term such as the temporal consistency loss described above which takes flow directly into account and / or some other loss term. The search block shape or pattern is then updated based on the evaluation of the loss function, and the process is repeated until some training objective is met and / or a predetermined number of epochs have been completed. This training may take place in a standalone manner, with the rest of the weights and biases of the flow module and / or compression network remaining fixed, or it may be training in an end-to-end manner where the pixel search block shape for flow estimation is learned together with the other learnable parameters of the compression pipeline. Finally, the pixel search shape and / or pattern need not be fixed and may instead be dependent on the input image representations, whereby different pixel search shapes and / or patterns may be selected based on, for example, the spatial resolution of the input being searched over or some other property of the input. Figure 8 illustrates an example of a flow module, in this case a network 800, configured to estimate information indicative of a difference between an image ^^^^−1and an image ^^^^, e.g. flow information. Figure 8 is provided as an example of how such flow information may be calculated between two input or output images, and accordingly which may be used when evaluating a loss function which incorporates a temporal consistency loss term. The network 800 comprises a set of layers 801a, 801b respectively for an image ^^^^−1and an image ^^^^from respective times or positions in a sequence ^^ − 1 and ^^ of a sequence of image frames. The set of layers 801a, 801b may define one or more convolution operations and / or nonlinear activations in the layers to sequentially downsample the input images to produce a pyramid of feature maps for different levels of coarseness or spatial resolution. This may comprise performing ℎ / 2 ^^ / 2 downsampling in a first layer, ℎ / 4 ^^ / 4 downsampling in a second layer ℎ / 8 ^^ / 8 downsampling in a third layer, ℎ / 16 ^^ / 16 downsampling in a fourth layer, ℎ / 32 ^^ / 32 downsampling in a fifth layer, ℎ / 64 ^^ / 64 downsampling in a sixth layer, and so on. It will of course be appreciated that these downsampling operations and levels of coarseness or spatial resolution of a pyramid feature map are exemplary only and others levels are also envisaged. With the downsampling operations performed and the corresponding pyramid of feature maps generated, a first cost volume or pixel difference 802 (in this case a mean absolute difference or MAD) is calculated at the most course level between the feature map pixels of the first image ^^^^−1and the corresponding feature map pixels of the second image ^^^^. As described above in concept 1, this defines the matchmaking cost of matching the feature map pixels of the initial image with the feature map pixels in the later image. That is, the closeness of each feature map pixel, or a subset of all feature map pixels, of the initial image to one or more feature map pixels of the later image is determined with a measure of closeness such as a vector or dot product, a cosine similarity, a mean absolute difference, or some other measure of closeness. In the case of Figure 8, four illustrative different pixel search shapes or patterns are shown as illustrated in boxes 812a, 812b, 812c, and 812d. Shape 812a is a diamond pixel search block with a 3 pixel search radius. Shape 812b is a cross pixel search block with a 3 pixel search radius. Shape 812c is a checkerboard pixel search block with a 3 pixel search radius. Shape 812d is a diamond pixel search block with a 1 pixel search radius. It will be appreciated that these are exemplary only and other shapes, patterns and / or pixel search radii are also envisaged. For example, if there are additional spatial resolutions to be searched over during pixel cost matching (i.e. indicated by the three dots in Figure 8), these may use the same or other pixel search shapes and / or patterns and / or pixel search radii. For example the 1 pixel search radius of shape 812d, may instead be a 1 pixel search radius cross shape. Finally, a first flow 803 can be estimated from the first cost volume 802. This may be achieved using, for example a flow extractor network which may comprise a convolutional neural network comprising a plurality of layers trained to output a tensor defining a flow map from the input cost volumes. Other methods of calculating flow information from cost volumes will also be known to the skilled person. The same process is then repeated for the other levels of coarseness to calculate a second cost volume 804 and second flow 805, and so on for the cost volumes and flows associated with each of the levels of coarseness have been calculated, up to the final cost volume 806 and flow 807. It should also be noted that in the context of the final MAD pixel search radius being very small (i.e. a diamond block of 1 pixel search radius), this effectively acts as a diamond shaped refinement bloc used on the highest spatial resolution image representation. That is, the flow associated with large movements of objects between the inputs will have been captured in the lower spatial resolution layers and accordingly, the purpose of the final flow pixel matching cost (in this case MAD) is primarily to make small refinements between pixels that are immediately adjacent each other. The inventors have found that this final, low pixel radius, diamond refinement block is an effective way to increase accuracy of the final flow estimation 807 without resulting in a substantial compute cost increase. The diamond refinement block accordingly speeds up run time while maintaining flow estimation accuracy. It will be appreciated that concept 4 may be used together with any of the other concepts described herein and / or in a standalone manner. Concept 5: Multiscale Optical Flow Compression Network One approach to improving run time of an AI-based compression pipeline in real time or near real time is to perform one or more operations of the pipeline in integer data type precision, rather than floating point data type precision. For example, running the pipeline of Figure 3 in a floating point data type (e.g. fp32) format is computationally intensive and has a significantly longer run time than when running the pipeline in int8 precision. However, running the pipeline in int8 precision comes with its own challenges. One of which is that integer data type objects have significantly reduced dynamic range compared to floating point data type objects. That is, an int8 object only has 8 bits (256 unique values) to represent some datum. If the datum is e.g. a tensor comprising elements with more than 256 possible values, then an accuracy compromise must be made to fit the representation of all the elements into the 256 possible values. This can be easily illustrated with a toy example of two vectors. The first vector has 256 elements, each having an integer value between 0 to 256. This first vector can be represented in int8 without any accuracy loss. The second vector has 256 elements each having integer values ranging from 1 to 256, except one element which has the value 1024. Because of this single element having a value much greater than 256, we can only represent the full dynamic range (1-1024) with 256 unique value if we round / quantise / squeeze the lower values of the range into bins of width 4 (e.g. the original values 1, 2, 3, 4 are now all rounded into the int8 value 1, the original values 5, 6, 7, 8 are now all rounded into the int8 value 2, and so on), or if we clamp the 1024 value down to 256. This means any granular information that that might have been present in the lower end of the range or alternatively in the upper end of the range is destroyed when representing the second vector in int8. Put another way, representing fine information and coarse information at the same time in a single int8 object is non-trivial. This problem is particularly pronounced in the context of flow estimation because in most video sequences, there can be coarse, large scale, global motion at the same time as fine, small scale, local motion. Consider the following toy example: in one 1080p frame, an aeroplane in the sky in the background might be appear to be moving very slowly across the frame at a rate of 1 pixel per frame, while at the same time a car in the foreground might be moving very quickly across the frame at a rate of 540 pixels per frame (that is, it crosses 50% of the image in a single frame). Capturing both kinds of motion as optical flow information in a single int8 object that only has 256 bits to represent both the 1 pixel per frame motion and the 540 pixels per frame is difficult. That is, if the int8 object is configured to capture the fine motion, then it has to clamp all the coarse motion information into whatever flow value is represented by the end of the dynamic range of the int8 object. Conversely, if the int8 object is configured to capture the coarse motion more accurately than the fine motion, then it has to round all the motion at the fine end of the dynamic range into a single bin, effectively rounding its detail away entirely. In the context of Figure 7a, this problem can manifest itself in that the coarse flows output by the first flow estimation block 703 operating on the most downsampled input image end up being clamped away by the time they get to the fine flows output by the last flow estimation block 707. Or alternatively, if the int8 objects representing the flows are instead configured to maintain a high dynamic range to capture the coarse motion, the output of the final flow estimation block 707 has its fine motion rounded away at the lower end of the dynamic range. When the output flow is applied to the t-1 frame to produce the final ^^^^−1, ^^ ^^ ^^ ^^ ^^ ^^used by the residual module, either the fine motion information has been rounded away, or the coarse motion has been clamped away and so the resulting pipeline operating in integer precision struggles either with low motion and / or with high motion video sequences. The present inventors have realised that it is possible to harness the run time benefits of operating in integer precision without losing accuracy arising from dynamic range issues by (i) running an optical flow network such as that of Figure 7a first on a downsampled representation of the input image(s) to produce final, coarse optical flow information, (ii) using the final, coarse optical flow information to warp the t-1 input image ^^^^−1to produce ^^^^−1, ^^ ^^ ^^ ^^ ^^ ^^_ ^^ ^^ ^^ ^^ ^^ ^^, (iii) running another optical flow network such as that of Figure 7a but this time on a higher resolution input (e.g. the native input image resolution) in the form of the coarse-motion-warped t-1 input image ^^^^−1, ^^ ^^ ^^ ^^ ^^ ^^_ ^^ ^^ ^^ ^^ ^^ ^^to produce final, fine optical flow information, (iv) using the final, fine optical flow information to further warp ^^^^−1, ^^ ^^ ^^ ^^ ^^ ^^_ ^^ ^^ ^^ ^^ ^^ ^^to produce ^^^^−1, ^^ ^^ ^^ ^^ ^^ ^^_ ^^ ^^ ^^ ^^ ^^ ^^_ ^^ ^^ ^^ ^^, and then passing this coarse and fine warped image residual module of the pipeline. Effectively, two optical flow networks are run sequentially, first on a downsampled input, and then on a higher resolution, warped input. Each optical flow network producing its own integer represented optical flow data object with its own dynamic range to better match either the fine or coarse motion amounts than would be possible with a unified dynamic range int8 object. In the downsampled space in which the first optical flow network operates, fine motion details are, for all intents and purposes, rounded away by the initial downsampling steps. This means that the first optical flow network can store and fit all the coarse motion information (i.e. first optical flow information) in its integer data type object (e.g. an int8 object) without any concern for the lower end of the dynamic range in which fine motion would otherwise be represented. When this coarse motion information represented by this first optical flow information stored as an integer data type object is used to warp a (higher resolution) representation of the input image associated with t-1, it accurately captures coarse, global, large scale motion. In the toy example of the aeroplane and car described above, the warp based on the output of first optical flow network captures the high motion of the foreground car across 540 pixels of the frame, without effecting the low motion aeroplane in the background. When the coarse-motion-warped t-1 image ^^^^−1, ^^ ^^ ^^ ^^ ^^ ^^_ ^^ ^^ ^^ ^^ ^^ ^^is run through the the second optical flow network, which is operating in a higher resolution space, the coarse motion has already been captured and applied by warp to the pixels if the input t-1 image so the only additional movement left to capture in the optical flow information by the second optical flow network is fine movement. This means it can be entirely captured within the dynamic range of its own integer data type object without concern for coarse, large scale global motion. That is, the dynamic range of the integer data type flow object produced by the second optical flow network only has to accommodate fine motion, which fits into the dynamic range of the integer data object being used to store it. In the toy example of the aeroplane and car described above, the warp based on the output of the second optical flow network captures only the low motion of the aeroplane in the background, without effecting the high motion car in the foreground (which has already been taken care of by the coarse motion warp). By giving coarse flow its own integer data type object and fine flow its own integer data type object, and applying them in separate warping steps, the present methods facilitate a flow module of an AI-based compression pipeline that can run in integer precision for fast run times while at the same time having an effective, combined dynamic range that can capture both coarse and fine motion without a substantial drop in accuracy that might otherwise be expected from operating in integer precision. The above methods can also be generalised to any number of levels of motion coarseness, with any number of optical flow networks being used to produce their own integer type data object representation of flow. For example three optical flow networks may be arranged consecutively to produce integer representations of extremely coarse motion flow, average coarseness motion flow, and fine motion flow. Together these have an effective, combined dynamic range that can accurately capture high motion, medium motion, and fine motion. This can be extended to four optical flow networks, five optical flow networks, and so on. The term motion coarseness as used herein means how much fine motion detail is lost when representing that motion as a data object such as in a flow tensor. High coarseness means that information representing pixels that have only moved a small number of pixel coordinates (e.g. substantially 0-30%, 0-25%, 0-20%, 0-15%, 0-10%, 0-5%, or some other percentage of a frame’s resolution in an x or y direction) away from their original positions are not or substantially not captured in the information. Conversely low coarseness or fine means that this information is captured substantially in the information. The term dynamic range means a range of values that can be represented by a given number of bits in an integer format. Networks in which the above-described methods are implemented are collectively described herein as multiscale optical flow networks. Multiscale optical flow networks may be enhanced further by introducing one or more feed forward links between the separate optical flow networks, effectively conditioning one or more outputs of one or more layers of the later networks on one or more outputs of one or more layers of the earlier networks. For example, with reference to Figure 7a, the outputs of the intermediate flow estimation blocks 703, 705 and so on of the coarse motion optical flow network of a multiscale optical flow network implementaiton, may be fed forward and convolved with the inputs of (e.g. cost volumes) of one or more of the flow estimation blocks of a subsequent fine motion optical flow network. In general terms, this allows the layers of the pyramid feature extractors to learn during training what kinds of global, high motion flows end up resulting in different kinds of local, low motion flows, and so on, thus increasing training stability and resulting in convergence to lower losses. Further, a difference between the coarse-motion-warped t-1 image ^^^^−1, ^^ ^^ ^^ ^^ ^^ ^^_ ^^ ^^ ^^ ^^ ^^ ^^and the present t image ^^^^may be estimated (e.g. effectively calculating a residual after performing the coarse warp) and run through a set of pyramid feature extractors much in the same way as the pyramid feature extractors of the flow network, and the resulting tensors of different resolutions convolved with the inputs to the flow estimation blocks associated with that resolution of the fine motion optical flow network. Given that the coarse warp has the effect of applying large scale, coarse movements to the image ^^^^, the resulting residual tensors are mostly empty, save for those elements associated with pixels where fine movement is occurring. By conditioning the flow estimation blocks of the fine motion optical flow network on these fine movement residual tensors, the pyramid feature extractors learn during training to focus on the fine movement information, thereby increasing training stability and resulting in convergence to lower losses. Figure 9 illustrates an example of a multiscale flow module 900 implementation according to the present disclosure. The flow module 900 may be used as a flow module in, for example, the AI-based compression pipeline of Figure 3. The flow module 900 comprises two optical flow compression network blocks 901a, 901b, each having, for example, the optical flow compression network architecture such as that illustrated in Figures 5 to 8, and / or 19-20 and as described above in connection with Concepts 2 to 4 and 10. The architectural details of each optical flow compression network blocks 901a, 901b are omitted in Figure 9 for ease of viewing but, as in Figures 5 to 8, may comprise a pair of pyramid feature extractor layers applied to inputs from respective times ^^ and ^^ − 1 at different resolutions, which are sequentially applied to produce sequential flows for the different resolutions before producing a final output flow which is used to warp the ^^ − 1 input image. However, in the case of Figure 9, the input frames ^^^^−1and ^^^^are first downsampled 902 by a predetermined amount, for example 2x, 4x, 8x, 16x, 32x and so on. To produce downsampled frames in which fine motion details are effectively "rounded away" or lost as a result of the downsampling. These downsampled representations of the input frames ^^^^−1and ^^^^are passed through the first optical flow compression network block 901a in the manner described in connection with, for example, Figures 5 to 8, to produce a final flow ^^^^ ^^ ^^ ^^ ^^ ^^903 which is indicative of a difference between the downsampled representations of the input frames ^^^^−1and ^^^^, and thus which is indicative of only coarse, large scale motion between the two frames. As described above, this coarse flow information ^^^^ ^^ ^^ ^^ ^^ ^^903 can be accurately represented in integer data type format because it does need a dynamic range that can simultatenously represent small scale, local flow values and large scale, global flow values. Once the coarse flow information ^^^^ ^^ ^^ ^^ ^^ ^^903 has been estimated by the first optical flow compression network block 901a, it is used to warp 904 the initial t-1 frame ^^^^−1in its original resolution (this may mean ^^^^ ^^ ^^ ^^ ^^ ^^is upsampled to match the target resolution of ^^^^−1), thereby capturing large scale, global motion in a warped frame ^^^^−1, ^^_ ^^ ^^ ^^ ^^ ^^ ^^. If upsampling is performed, this may be, for example, performed with a static function such as a bilinear upsampler, or by a learned upsampler. This coarse-flow-warped frame is then used as the t-1 frame input to the second optical flow compression network 901b, together the t frame input ^^^^. Whilst not shown in the example of Figure 9, these two inputs may optionally be downsampled in the same way as for the first optical flow compression network 901b but in this case by a smaller amount. For example if the first optical flow compression network 901a input downsampling amount was 4x, then the downsampling amount for the second optical flow compression network 901b may be 2x, or indeed none at all. Given that all large scale, global motion has been taken into account and applied using the coarse flow warp 904 performed after the first optical flow compression network block 901a, the second optical flow compression network block 901b is able to focus on the fine motion which is taken into account and applied by performing the fine flow warp 905. After performing the fine flow warp 905, the resulting warped t-1 frame ^^^^−1, ^^_ ^^ ^^ ^^ ^^ ^^ ^^_ ^^ ^^ ^^ ^^takes into account both global motion and fine motion and is then passed on to the resiudal module of a flow-residual compression pipline, such as that shown in, for example, Figure 3 or Figure 6b. Figure 10 illustrates a modification of the multiscale flow module of Figure 9, in which a number of additional steps are introduced. Like-numbered reference numbers refer to features which are substantially the same in both figures. Each additional step may be used in a standalone manner, or together with each of the other steps. A first additional, optional step is that, in the second optical flow compression network 901b, the input frames from t and t-1 are themselves downsampled 1001 by a predetermined amount to reduce run time. The amount of downsampling is envisaged to be less than that performed in the downsampling steps 902 assocaited with the first optical flow compression network 901a. This can be understood in that the second optical flow compression network 901b is still intended to capture fine motion so the downsampling 1001 does round some of the fine motion detail away but as the amount of downsampling is less than the first optical flow compression network 901a downsampling steps 902, the second optical flow compression network 901b is still able to capture motion that is finer than the first optical flow compression network 901a. By way of example, if the first optical flow compression network downsampling amount 902 is a 32x downsample, then the second optical flow compression network 901b downsampling steps may be a 2x downsample. This amount has a significant effect on speeding up runtime while at the same time retaining more fine motion information than the 32x downsample of the first optical flow compression network 901a. A second additional, optional step is that one or more outputs 1002, 1003, 1004 of one or more layers of the first optical flow compression network 901a are fed forward into the second optical flow compression network 901b. For example, as illustrated in Figures 5 to 8, a flow is estimated for each layer of the feature pyramids. Thus, the first optical flow compression network 901a produces a plurality of intermediate outputs which are indicative of how the first optical flow compression network 901a is treating global, large-scale motion. Given that global, coarse, large-scale motion and fine, small-scale motion frequently occur with some dependence on the other, passing information indicative of the global, coarse, large-scale motion from the first optical flow compression network 901a to the second optical flow compression network 901b can help it to better capture fine, small-scale motion details. At a general level, feeding this information forward has the effect of conditioning the fine motion flows of feature pyramid of the second optical flow compression network 901b on their global, coarse motion flow counterparts of the first optical flow compression network 901a to improve the accuracy of the fine motion flow estimation. In practice, this may be implemented by convolving the one or more outputs 1002, 1003, 1004 with the corresponding outputs of the corresponding layers in the second optical flow compression network 901b. A third additional, optional step is performing a difference calculation Δ between the input frame ^^^^, and the coarse, warped frame ^^^^−1, ^^_ ^^ ^^ ^^ ^^ ^^ ^^at the resolutions corresponding to the resolutions of the feature pyramids of the second optical flow compression network 901b. These difference maps at the different levels may then be fed forward into the second optical flow compression network 901b, much in the same way as the outputs 1002, 1003, 1004 outlined above. For example, by convolving them with the corresponding outputs of the corresponding layers in the second optical flow compression network 901a. The difference calculation Δmay be a pixel-wise difference (e.g. subtracting pixel values of the channels from one frame from the corresponding pixel values of the channels of another frame) or some other difference, for example, calculating brightness value at each pixel coordinate and subtracting the brightness values of one frame from the other to produce a brightness difference map for resolution corresponding to the resolutions of the feature pyramid layers of the second optical flow compression network 901b. At a general level, the difference maps highlight to the second optical flow compression network 901b where the coarse warp 904 has not compensated for motion, thus conditioning the feature pyramid flow estimations on these difference maps. This in turn has the effect of improving the accuracy of the flow estimated by the second optical flow compression network 901b associated with these parts of the difference maps. When the second and third additional, optional steps outlined above are used together, the resulting effect is a second optical flow compression network 901b that estimates fine motion flows that are very strongly conditioned on the global, large-scale, coarse motion, which significantly increases the accuracy of the fine motion flows. The example multiscale optical flow compression network approach illustrated in Figure 10 accordingly enables accurate flow estimation across the full spectrum of low to high motion types. It will be appreciated that concept 5 may be used together with any of the other concepts described herein and / or in a standalone manner. Concept 6: Temporal Context in Flow In many sequences of frames, motion of objects depicted in those frames is correlated across the frame sequence. For example, if a vehicle is moving across a frame in a first direction then there is a high probability that it will continue to move in that direction in successive frames of the sequence. This means that there is temporally redundant information in the flow values (and flow feature maps) between frames of the sequence, in addition to any temporally redundant information that may be found in the pixel values of successive frames. Concept 6 is directed to taking advantage of the temporal redundancy between successive flows and flow feature maps of a frame sequence. Consider three frames ^^^^−2, ^^^^−1and ^^^^. There are two flows associated with these frames: (i) ^^^^−1indicative of a difference between ^^^^−2and ^^^^−1, and (ii) ^^^^indicative of a difference between ^^^^−1and ^^^^. In many frame sequences, depicted objects will continue to move consistently in the same direction. For example, for motion with a small magnitude, ^^^^−1≈ ^^^^(subject to some locally affine transform such as a starting position shift). Additionally, even if the flows change, depicted objects often follow natural movement patterns (e.g. an object falling under gravity onto an angled surface where it is deflected sideways) so, for a given depicted object, its future motion (and corresponding flow) is dependent on its past flow for many types of frame sequences, that is ^^^^−1← ^^^^. However, taking advantage of this temporal flow redundancy or temporal flow context is non-trivial. This is in part because flow is not a standalone concept in an AI-based compression pipeline but depends itself on the input frame pixels, the non-linear relationships between the encoder and decoder neural networks in the flow and residual modules, the weights of those neural networks and whatever training data resulted in those weights, and so on. The present inventors have identified that this temporal flow context is particularly effective at improving the compression rate (i.e. resulting in a smaller bitstream) of a flow module of an AI-based compression pipeline such as that of Figure 3. As described earlier in the present application, but repeated here for ease of reference, a hyper network serves the purposes of compressing and transmitting some arbitrary side information that may be used on the decode side for some purpose. One such purpose is to produce a hyper latent representation (of whatever is being entropy encoded into a bitstream) with the hyper encoder, and to decode this hyper latent representation into entropy parameters which are used for performing the entropy encoding of the latent representation itself. Two factors that effect compression rate are (i) the distribution of whatever is being entropy encoded into a bitstream and which is governed by the (trainable) weights of the base encoder and decoder, and (ii) the entropy parameters for entropy encoding those distributions into the bitstream, these entropy parameters being produced by the (trainable) weights of the hyper encoder and decoder. Because both the base encoder and decoder and the hyper encoder and decoder are trainable, they learn to produce whatever distributions and associated entropy parameters are optimal for achieving some desired rate distortion training objective (which is typically small bitstream size and high reconstruction accuracy). The present inventors have realised that giving the hyper network past flow information (in the form of an output flow map, and / or some intermediate flow feature maps output by one or more intermediate layers of the base flow encoder and / or decoder) when it encodes and decodes the hyper latent representation to produce the entropy parameters for entropy encoding (and decoding) the (flow) latent representation itself, it produces entropy parameters that entropy encode the (flow) latent representation into a smaller bitstream than without using the past flow information in this way. At a general level, this can be understood as conditioning the output of the hyper network encoder and / or decoder on past flow information and thereby allowing them to learn when and to what extent the current flow information is redundant and does not need to be sent in the bitstream to achieve a comparable reconstruction quality on the decode side. Thus, if we define the flow module as some function ^^ (e.g. a neural network flow module such as that shown in Figure 3) and we have as input ^^^^−1, ^^^^and a previous flow ^^^^−1(indicative of a difference between ^^^^−2, ^^^^−1), then the current flow indicative of a difference between ^^^^−1, ^^^^may be defined as an output of the function ^^ as follows: ^^^^= ^^( ^^^^−1, ^^^^| ^^^^−1), where ^^^^is conditioned on ^^^^−1. An exemplary implementation of this concept is illustrated with reference to Figures 11 and 12. Figure 11 shows flow encoder / decoder networks and residual encoder / decoder networks of an AI based compression pipeline. This may be, for example, similar to the AI based compression pipeline of the type shown in Figure 3. The flow encoder neural network takes a current image ^^^^and a previous image ^^^^−1, encodes these into a flow latent representation which is optionally quantised and entropy encoded and transmitted as a bit stream. On the decode side, the bitstream is received, entropy decoded into the flow latent representation that the flow decoder neural network uses as input to produce a representation of flow ,ˆ e.g. a flow map. The flow mapˆis applied to a previously decoded image ˆ^^−1to generate a warped version of that previously decoded image ˆ^^−1, ^^. The warped version of the previously decoded image ˆ^^−1, ^^is then fed into the residual encoder network, together with the current image ^^^^to produce a residual latent representation which is optionally quantised and entropy encoded and transmitted as a bit stream. On the decode side, the bitstream is received and entropy decoded back into the residual latent representation ^^ and used by the residual decoder neural network, in combination with information from the warped version of the previously decoded image ˆ^^−1, ^^, to produce the reconstructed image ˆ^^. In the case of Figure 11, the information associated with the warped previously decoded image ˆ^^−1, ^^is optionally first processed by module ^^ , referred to herein after as a composition adapter, for example to downsample and / or pad it before it is fed into the residual decoder together with the entropy decoded residual latent representation to produce the final reconstructed image ˆ^^. This process may then be repeated for ^^^^+1and so on to encode, transmit, and decode a sequence of frames. Whilst not shown in Figure 11, the flow encoder may comprise the architectures described with respect to any of the other concepts illustrated above in respect of optical flow compression networks of Figures 5 to 10, 19-20 and so on. As is further shown in Figure 11, the entropy encoding that produces the respective bitstreams 204, 208, 212 is supported by corresponding a hyper network which in turn is supported by a hyper hyper network. The hyper network for the I-frame module 201 comprises a neural network 1101 that is referred to as a hyper encoder, and a neural network 1102 that is referred to as a hyper decoder. As is described in connection with Figure 2, the hyper encoder neural network 1101 encodes the latent representation produced by the encoder neural network 203, and the hyper decoder neural network 1102 produces the entropy parameters that define the distribution of the latent representation. These parameters are used during entropy encoding to produce the bitstream 204. The encoded latent representation produced by the hyper encoder 1101 is itself entropy encoded into a bitstream using an entropy model using entropy parameters obtained by encoding the encoded latent representation using the hyper hyper encoder 1103, and decoding that encoded latent representation using the hyper hyper decoder 1104. This nested structure can be extended as desired but it has been found that a hyper and hyper hyper network achieve good performance for run time trade off. The entropy parameters that are used to produce the bitstream associated with the hyper hyper network (or whatever the highest level of the nested structure is) may optionally be fixed, set parametrically, or produced by an encoder / decoder neural network pair like in the hyper network. As can be seen in Figure 11, this nested hyper network and hyper hyper network approach is also provided in the optical flow module 206 in the form of hyper encoder neural network 1105, hyper decoder neural network 1106, hyper hyper encoder neural network 1107 and hyper hyper decoder neural network 1108. The same is also provided in the residual module 210 in the form of hyper encoder neural network 1109, hyper decoder neural network 1110, hyper hyper encoder neural network 1111 and hyper hyper decoder neural network 1112. Figure 12 corresponds to Figure 11 but illustrating an implementation of using temporal flow context for improving compression rates (i.e. smaller bitstream size) of the flow module. As illustrated in Figure 12, a past flow^ˆ^−1is fed as an input together with the latent representation ^ˆ^^^ ^^ ^^ ^^, ^^produced by the base flow encoder into the hyper encoder 507a. In order to do this, the past flow tensor^ˆ^−1may first be convolved with the latent representation tensor ^ˆ^^^ ^^ ^^ ^^, ^^to produce a combined input tensor to the hyper encoder 507a. One or more optional preprocessing steps may also be performed to upsample or downsample^ˆ^−1to match the shape requirements of the latent represntation tensor ^^^^ ^^ ^^ ^^, ^^before convolving the two inputs, or joining them together using a concatenation or other operation. Note that, in the case of operating in integer precision (e.g. int8) rather than in floating point, using one or more convolutions and / or activations to link the two inputs rather than a concatenation operation can be more efficient and faster on most commercially available chipsets. The output of the hyper encoder 507a is accordingly a hyper latent representation ^^^^ ^^ ^^ ^^, ^^that is conditioned on ^ˆ^−1. Optionally, on the decode side, a similar step may be performed in that the past flow tensor^ˆ^−1may be convolved (or joined using a concatenation or other operation) with the representation ^^^^ ^^ ^^ ^^, ^^before feeding it into the hyper decoder 509a. It will be appreciated that that one or more preprocessing steps may optionally be formed, for example upsampling or downsampling, to ensure the shape of ^^^^−1matches the shape requirements of ^^^^ ^^ ^^ ^^. Thus, the output of the hyper decoder 509a (i.e. reconstructed entropy parameters ^ˆ^^^ ^^ ^^ ^^, ^^) is also conditioned on the past flow tensor^ˆ^−1. Optionally, the past flow tensor^ˆ^−1be convolved with the output of one or intermediate layers of the hyper encoder and / or decoder after performing a suitable upsampling, downsampling and / or reshaping step as applicable. Finally, whilst not shown, it is also envisaged that one or more outputs of one or more intermediate layers of the base flow encoder and / or decoder may be used instead of or in addition to the base, past flow tensor^ˆ^−1. These intermediate outputs correspond to flow feature maps and are indicative of a the flow encoder and decoder’s abstracted representation of flow at intermediate layers so can accordingly contain useful information in a form that can be helpful for the hyper encoder and / or decoder to use when learning when and how to take advantage of temporal flow context. Further, whilst not shown in Figure 12, it is also envisaged that the past flow information may be fed into the hyper hyper encoder 1107 and / or hyper hyper decoder 1108 in a similar manner to that set out above for the hyper encoder 1105 and hyper decoder 1106. Indeed the above-described approach can be abstracted further to the general case where past flow can be fed into any of the hyper, hyper hyper (or where present, hyper hyper hyper) networks and so on. That is, the entropy parameters produced by these networks used for entropy encoding the various latent, hyper latent, hyper hyper latent and so on, may be conditioned on the past flow. That is, by conditioning the components of the flow pipeline that are largely responsible for the size of the bitstream through the entropy parameters on the past flows, the flow network is provided with an architectural mechanism to learn what information across a sequence of frames is important for flow, and what is not important, thus allowing the network to produce more accurate flows across the sequence that also compress to a smaller bitstream. In order to handle the first frame of a sequence where no past flow is available, for example for the first frame pair of a sequence of frames, it is envisaged that a dummy flow may be used for ^ˆ^−1instead of an actual flow. This dummy flow may be initialised heuristically based on input meta information (for example, based on video sequence type where same base average flow values may be set higher for action film video sequences, or lower for low motion nature scene video sequences, and so on), initialised randomly, for example set all to 0s or some other value, or initialised based on an output of a classifier operating on the first frame of the sequence to predict how much motion a given sequence may likely start with. It is also envisaged that the use of the past flow information in the hyper encoder and decoder may be skipped entirely by setting a frame type flag to be indicative of a first frame pair of a sequence. If this flag is received by the flow module, temporal flow context will not used for the first frame pair, and only started for frame pairs where a past flow is available and the flag is set differently. In this case, separate weights for the hyper encoder and decoder may be provided and hot-swapped based on whether or not the frame type flag is indicative of a past flow being available or not. This is because a set of weights learned using past flow information will be different to weights learned without using past flow. Accordingly, if no past flow information is available, a set of weights learned with past flow but inferencing without past flow, or with using dummmy flow information, may not operate as effectively as a set of weights learned without past flow. During training of the architecture of Figure 12, it is envisaged that the the training sequences of frames may include group of picture (GOP) sizes of at least 4. This is to ensure that there are at least two past flows available from which the hyper encoder and / or decoder may learn when and how to take advantage of temporal flow redundancy to produce improved compression rates. Figure 13 illustrates an additional and / or alternative approach to using past flow information for improving compression rates of the flow module. It may be used alone, or in conjunction with the approach illustrated with respect to Figure 12, and / or with any of the the other concepts described with reference to any of Figures 1 to 11, and 14-21. In the implementation of Figure 13, the hyper hyper encoder 1107 and decoder 1108 are replaced with a neural network 1113 comprising one or more layers configured for predicting entropy parameters specifically from the past flow information. In one example the neural network 1113 comprises a plurality of downsampling layers and one or more activation layers. The weights of the neural network 1113 may be trained end to end with the rest of the AI compression pipeline or in a standalone manner to predict the entropy parameters of the distribution of the hyper latent representation ˆ^^ ^^ ^^ ^^, ^^from the past flow information^ˆ^−1(and / or one or more feature maps from a flow encoder and / or decoder). As the past flow information^ˆ^−1is already present on the decode side, it is not necessary to include any additional side information such as a hyper hyper latent representation in the bitstream, thereby saving additional bits. At a very general level, the distribution of the hyper latent representation ˆ^^ ^^ ^^ ^^, ^^is highly correlated with the past flow information^ˆ^−1. Accordingly, the entropy parameters of the hyper latent representation ˆ^^ ^^ ^^ ^^, ^^ can be predicted effectively using neural network 1113 using this information for free on the decode side without using extra bits. Whilst not illustrated in Figures 11, 12, 13 or 14, it will be appreciated that the decode side operations are also performed on the encode side. For example, in order to produce the entropy parameters for entropy encoding, the hyper network encoder and decoder forward passes are run, and the resulting entropy parameters used on the encode side to produce the bitstream. The same applies for the hyper hyper network where present and so on. Thus, it will be appreciated that the "reconstructed" past flow information^ˆ^−1is available on both the encode and decode side of the pipeline. An example training method implementing concept 6 is set out below in pseudocode.

[0002] Algorithm 1 Temporal flow context training Inputs: Neural network ^^ ( ^^1) with optical flow compression network ^^( ^^2), training data with sequences of GOP size 4 Variables: Optimizer ^^, gradient computation flag ^^ ^^ ^^ ^^. Initialise: ^^ ^^ ^^ ^^ = ^^ ^^ ^^ ^^ for ^^ ( ^^1) and ^^( ^^2), and training loop counter ^^ = 0. while ^^ < ^^ do Forward pass through ^^ ( ^^1) including calculating and using ^^^^= ^^( ^^^^−1, ^^^^| ^^^^−1, ^^2) to produce ˆ^^, where ^^^^is a flow indicative of a difference between ^^^^and ^^^^−1Compute loss and perform backward pass with ^^ ^^ ^^ ^^. Apply optimizer ^^ to update ^^ ( ^^1) including ^^( ^^2). ^^ ← ^^ + 1 end while That is, a flow-residual neural compression pipeline is defined as a neural network(s) ^^ ( ^^1) where ^^1are the weights and / or biases of the network(s). The pipeline includes an optical flow compression network ^^( ^^2) where ^^2are the weights and / or biases of the optical flow compression network. A training data set comprising sequences of images with GOP size 4 or more is provided. An optimiser (e.g. stochastic gradient descent, adam, or any other optimiser algorithm) is defined and a gradient flag ^^ ^^ ^^ ^^ is set to true for the networks ^^ ( ^^1) and ^^( ^^2) indicating that they can be backpropagated through and their weights updated. For a number of training steps (e.g. ^^ steps) or while some other training condition is not yet satisfied, a loop is performed comprising a forward pass through ^^ ( ^^1) and ^^( ^^2) which includes calculating and using a current flow ^^^^= ^^( ^^^^−1, ^^^^| ^^^^−1) using the optical flow compression network ^^( ^^2) for training step ^^, with the hyper network and / or hyper hyper network of the optical flow compression network ^^( ^^2) conditioned on ^^^^−1, which is retained in memory for the purposes of feeding it into networks in the next training step. The calculated flow ^^^^is fed into the residual module of the network ^^ ( ^^1) to produce the final reconstructed frame. A loss comprising one or more loss terms is computed. The loss may comprise a rate term, a distortion term and / or one or a more other loss terms such as a temporal consistency loss term that is described in above in connection with concept 1. The gradients of the loss function are calculated and may be accumulated for a predetermined number of steps (e.g. a number of steps corresponding to a GOP size of the training data set), and the optimiser applied to update the weights ^^1and ^^2. The training loop is continued until some end of training condition is met, as e.g. after ^^ steps, after a loss has plateaued, after a validation loss starts to diverge from a training loss, or after some other requirement is met. It will be appreciated that concept 6 may be used together with any of the other concepts described herein and / or in a standalone manner. Concept 7: Future flow predictions The general idea of using past optical flow information can be extended beyond improving compression rates (i.e. beyond smaller bitstreams) to improving reconstruction accuracy by improving distortion metrics. As explained above in concept 6, flows that are indicative of differences between the frames of a frame sequence may be correlated and accordingly information from a past flow can contain useful information that can improve the accuracy of a flow predicted by a flow module in connection with a current frame, not just improve the compression rate. That is, giving one or more of the neural networks of a flow module information about what flow information is expected to be for the current frame based on what the flow information was for one or more past frames results in improvements in the accuracy of the flow information that is produced by the forward pass of the flow module for the current frame. In general terms, one or more neural networks of a flow module may be conditioned on a predicted (e.g. expected or hypothetical) current flow based on a past flow to improve the accuracy of the flow that is produced when the flow module runs its forward pass. In even more general terms, a past flow is used to predict a future flow, and this predicted future flow is combined with the other inputs into the optical flow compression network when running the forward pass. Thus, if we define the flow module as some function ^^ (e.g. a neural network flow module such as that shown in Figure 3) and we have as input ^^^^−1and ^^^^, we now also input a predicted current flow ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^that is predicting the actual flow associated with ^^^^−1and ^^^^based a previous flow ^^^^−1using some function ℎ where ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^= ℎ( ^^^^−1). The current flow indicative of a difference between ^^^^−1and ^^^^may then be defined as an output of the function ^^ as follows: ^^^^= ^^( ^^^^−1, ^^^^| ℎ( ^^^^−1)) ^^^^= ^^( ^^^^−1, ^^^^| ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^) where ^^^^is accordingly conditioned on ℎ( ^^^^−1) = ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^, where ℎ is a function that predicts a current flow from a past flow. In one example, the function ℎ may comprise a forward warping function. Forward warping is a known image processing technique for transforming some input set of pixels image to an output set of pixels by mapping all the points in the original image to their new positions in the output image based on some transformation function. In the context of optical flow information, forward warping transforms flow information pixels by mapping all the points from ^^^^−1to one or more points in ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^. One known forward warping-based optical flow prediction framework, referred to herein as soft splat, that may be used for ℎ is described in Wang et al, Splatflow: Learning Multi-frame Optical Flow via Splatting, https: / / arxiv.org / pdf / 2306.08887 and is incorporated herein by reference in its entirety. Other forward warping frameworks and functions are also envisaged. For example, different types of splatting (e.g. blending or normalization) such as average, linear, and / or softmax splatting are all envisaged. One or more non-splatting based approaches may also be used to perform the forward warping operation, including but not limited to linear interpolation, one or more phase based methods, and so on. Figure 14 illustrates the concept of forward warping applied to a toy example 1400 sequence of frames ^^^^−2, ^^^^−1, ^^^^, ^^^^+1. Motion between objects in frames may be described using flows ^ˆ^−1(indicative of a difference between ^^^^−2and ^^^^−1),^ˆ^(indicative of a difference between ^^^^−1and ^^^^), and^ˆ^+1(indicative of a difference between ^^^^and ^^^^+1). We denote the flows with a hat, indicative of a quantised flow, which may be used to perform the warping instead of the unquantised flows. The flows in general are "backwards looking" in that the flow^ˆ^−1associated with ^^^^−1tells us where to look in ^^^^−2to find out which pixels in ^^^^−2contribute to which pixels in ^^^^−1. Similarly, the flow^ˆ^associated with ^^^^tells us where to look in ^^^^to find out which pixels in ^^^^−1contribute to which pixels in ^^^^. Similarly, the flow^ˆ^+1associated with ^^^^+1tells us where to look in ^^^^to find out which pixels in ^^^^contribute to which pixels in ^^^^+1, and so on. This is indicated by the solid arrows in Figure 14 from each frame to the previous frame. However, we can transform these "backwards looking" flows to be "forwards looking" (or "backwards looking from a future frame") by performing a forward warping operation, effectively predicting what the future flow will be based on the current flow. These forward warping operations are illustrated by the curved dotted arrows 1401, 1402. The predicted flow ^ˆ^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^associated with the change from ^^^^−1to ^^^^is produced by performing a forward warp on the previous flow^ˆ^−1, in other words, by calculating^ˆ^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^= ℎ(^ˆ^−1). Similarly, the predicted flow^ˆ^+1, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^associatedwith the change from ^^^^to ^^^^+1is produced by performing a forward warp on the previous flow^ˆ^, in other words, by calculating^ˆ^+1, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^= ℎ(^ˆ^). It will be appreciated that explicit forward warping is only one example function that may be used for future future flow prediction. For example, the function ℎ may be a neural network with one or more convolution layers and / or activation layers with trainable weights. These may be pre-trained using supervised learning with a training dataset comprising example input sequences of frames and corresponding ground truth output flows, or using other learning methods known to those skilled in the art. Alternatively, the weights may be learned together with the other weights of the AI compression pipeline. As long as the function ℎ produces some representation of predicted future flow that the outputs of one or more of the other networks of the optical flow compression network can be conditioned on, the other networks of the compression pipeline are able to leverage that information during training to improve performance. Thus, when ^^^^is received at the start of the pipeline together with ^^^^−1, these inputs are fed into the optical flow compression network together with^ˆ^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^. Whilst^ˆ^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^is only a prediction, its presence conditions the optical flow compression network on a most likely future flow, thereby giving the optical flow compression network a clue as to what its output is likely to be and thereby making its task of producing a flow for use in the residual network an easier task to perform compared to producing an output optical flow^ˆ^solely from ^^^^and ^^^^−1without^ˆ^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^. The next (P- and / or B-) frames of a sequence may thus be produced by conditioning the output of the optical flow compression network on a past flow. Whilst not illustrated in Figure 14, this conditioning may be on multiple past flows. For example,^ˆ^+2, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^= ℎ(^ˆ^−1,^ˆ^),such that: ^^^^= ^^( ^^^^−1, ^^^^| ℎ(^ˆ^− ^^, ...,^ˆ^−2,^ˆ^−1)) and so on up to some arbitrary number ^^ of past frame flows. Also, whilst not illustrated in Figure 14, the flow information on which the conditioning is based may comprise the final output flow of the optical flow compression network, and / or some intermediate feature map of one or more layers of the neural networks that make up the encoder and / or decoder of the optical flow compression network. At a general level, it has been found that the flow feature maps comprise a lower level representation of past flow information (e.g. abstracted to some feature space that is lower in resolution than the final output flows). These lower level, lower resolution representations of flow are more efficient to perform computational operations with, and can be combined (e.g. by convolution and / or concatenation) with the outputs of one or more of the intermediate layers of the flow optical compression network having a corresponding resolution to achieve a same or similar improvement in accuracy as when using the full resolution flows which are computationally slower to work with. Figure 15 illustrates an example implementation of an AI-based compression pipeline using a predicted flow as input to the flow module. The architecture is similar to that illustrated in Figure 3 or Figure 11 and like-numbered reference numerals indicate like elements. However, in Figure 15, the previous flow information^ˆ^−1(and / or one or more intermediate flow feature maps taken from intermediate layers of the the flow decoder 209), is passed to a forward warping function ℎ to calculate^ˆ^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^1500 which is then fed together with the frames ^^^^−1and ^^^^into the optical flow compression network 207. Unlike in Figures 12 and 13, where the past optical flow information was primarily focused on improving compression rates (i.e. reducing bitstream size) by conditioning the outputs of the parts of the pipeline that are responsible for producing the entropy parameters used to entropy encode the outputs of the encoders into a bitstream, such as the hyper network and the hyper hyper network, the focus in Figure 15 is using the past optical flow information to improve reconstruction accuracy (i.e. reduce the distortion scores such as MSE, MS-SIM and so on). This distinction at a high level can be understood by considering a primary purpose of each of the components of the pipeline. That is, the hyper network and hyper hyper network produce respective hyper latent and hyper hyper latent representations that can be decoded into one or more entropy parameters that are used for entropy encoding and decoding to as close to a perfect Shannon entropy limit as possible. Thus, when these networks are given past flow information, they are able to learn what current flow information is redundant and accordingly produce hyper latents and hyper hyper latents that can be entropy encoded into smaller bitstreams than without past flow information. Conversely, the main encoding structure(s) of the optical flow compression network (such as that illustrated in Figures 5 to 10, and 19-20) have a purpose of reconstructing the flow as accurately as possible such that when the flow is subsequently used for warping a past image which is fed into the residual module, the residual decoder produces as accurate a reconstructed frame as possible because the flow that was used for the warping was more accurate. Accordingly, when the past flow information is used by the main encoder of the optical flow compression network, it learns which details of which parts of a given input when estimating flows contribute most to reconstruction accuracy and thus which parts to focus on when producing the final output flow information that can be used for subsequent warping. In the specific context of the described forward warping, the forward warped predicted flow ^ˆ^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^gives the main encoder of the optical flow compression network a clue as to what the future flow is likely to be, and thus conditions the encoder on where the most important flow details are likely to be. Whilst not illustrated in Figures 14 or 15, it will be appreciated that the decode side operations are also performed on the encode side. For example, in order to produce the entropy parameters for entropy encoding, the hyper network encoder and decoder forward passes are run, and the resulting entropy parameters used on the encode side to produce the bitstream. The same applies for the hyper hyper network where present and so on. Thus, it will be appreciated that the "reconstructed" past flow information^ˆ^−1is available on both the encode and decode side of the pipeline. As in the case of concept 7, it is envisaged that the image sequences on which the pipeline is trained includes a variety of different sequences with different group of picture (GOP) sizes. For example, this may be various sequences of GOP sizes of 4 or more to provide variety of short and long GOP sizes. Smaller GOP sizes may also be used, for specialised use cases associated with very short frame sequences. Taking the non-limiting example of GOP sizes of 4 or more, it can be understood from Figure 14 that by including at least 4 images, the network is able to accumulate gradients over three flows: a past flow, a current flow, and at least one future flow that can be predicted as ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^, allowing the network to usefully learn from temporal flow context. Increasing the GOP size higher than 4 can further increase this benefit as the gradients accumulate across a GOP, but this may come at the cost of reduced training stability. Reducing the GOP size below 4 can improve training stability. An example training method implementing concept 7 is set out below in pseudocode. Algorithm 2 Flow training with forward warping Inputs: Neural network ^^ ( ^^1) with optical flow compression network ^^( ^^2), training data with sequences of GOP size 4 Variables: Optimizer ^^, gradient computation flag ^^ ^^ ^^ ^^. Initialise: ^^ ^^ ^^ ^^ = ^^ ^^ ^^ ^^ for ^^ ( ^^1) and ^^( ^^2), and training loop counter ^^ = 0. while ^^ < ^^ do Forward pass through ^^ ( ^^1) including calculating ℎ( ^^^^−1) = ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^, where ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^is a predicted flow indicative of a difference between ^^^^and ^^^^−1, calculating ^^^^= ^^( ^^^^−1, ^^^^, ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^| ^^2), and using ^^^^to produce ˆ^^Compute loss and perform backward pass with ^^ ^^ ^^ ^^. Apply optimizer ^^ to update ^^ ( ^^1) including ^^( ^^2). ^^ ← ^^ + 1 end while That is, a flow-residual neural compression pipeline is defined as a neural network(s) ^^ ( ^^1) where ^^1are the weights and / or biases of the network(s). The pipeline includes an optical flow network ^^( ^^2) where ^^2are the weights and / or biases of the optical flow compression network. A training data set comprising sequences of images with GOP size 4 or more is provided. An optimiser (e.g. stochastic gradient descent, adam, or any other optimiser algorithm) is defined and a gradient flag ^^ ^^ ^^ ^^ is set to true for the networks ^^ ( ^^1) and ^^( ^^2) indicating that they can be backpropagated through and their weights updated. For a number of training steps (e.g. ^^ steps) or while some other training condition is not yet satisfied, a loop is performed comprising a forward pass through ^^ ( ^^1) and ^^( ^^2) which includes calculating a current flow ^^^^= ^^( ^^^^−1, ^^^^, ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^) using the optical flow compression network ^^( ^^2) for training step ^^. is fed into the residual module of the network ^^ ( ^^1) to produce the final reconstructed frame A loss comprising one or more loss terms is computed. The loss may comprise a rate term, a distortion term and / or one or a more other loss terms such as a temporal consistency loss term that is described in above in connection with concept 1. The gradients of the loss function are calculated and may be accumulated for a predetermined number of steps (e.g. a number of steps corresponding to a GOP size of the training data set), and the optimiser applied to update the weights ^^1and ^^2. The training loop is continued until some end of training condition is met, such as e.g. after ^^ steps, after a loss has plateaued, after a validation loss starts to diverge from a training loss, or after some other requirement is met. It will be appreciated that concept 7 may be used together with any of the other concepts described herein and / or in a standalone manner. For example, the past flow information may be fed not only into the base encoder 207 of the optical flow compression network after performing a transform ℎ as in Figure 15, but also at the same time into the hyper and / or hyper hyper network of the optical flow compression network as illustrated in Figures 12 and 13. Combining concept 6 and 7 in this way facilitates simultaneous compression rates (smaller bitstreams) reconstruction accuracy (lower distortion scores). Further, by implementing both concepts at the same time in training, the base encoder / decoder networks, the hyper encoder / decoder networks and the hyper hyper encoder decoder networks become jointly "past flow context aware" and accordingly learn a set of weights during training that are optimised for both training objectives simultaneously in a way that would not necessarily be the case when trained for only one or both objectives. The combination of concepts 6 and 7 thus produces a synergistic benefit when used together that is greater than the sum of its parts. An example training method implementing concept 6 and concept 7 together is set out below in pseudocode. Algorithm 3 Flow training with temporal context and forward warping Inputs: Neural network ^^ ( ^^1) with optical flow compression network ^^( ^^2), training data with sequences of GOP size 4 Variables: Optimizer ^^, gradient computation flag ^^ ^^ ^^ ^^. Initialise: ^^ ^^ ^^ ^^ = ^^ ^^ ^^ ^^ for ^^ ( ^^1) and ^^( ^^2), and training loop counter ^^ = 0. while ^^ < ^^ do Forward pass through ^^ ( ^^1) including calculating ℎ( ^^^^−1) = ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^, where ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^is a predicted flow indicative of a difference between ^^^^and ^^^^−1, ^^^^= ^^( ^^^^−1, ^^^^, ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^| ^^^^−1, ^^2), and using ^^^^to produce ˆ^^Compute loss and perform backward pass with ^^ ^^ ^^ ^^. Apply optimizer ^^ to update ^^ ( ^^1) including ^^( ^^2). ^^ ← ^^ + 1 end while

[0003] That is, a flow-residual neural compression pipeline is defined as a neural network(s) ^^ ( ^^1) where ^^1are the weights and / or biases of the network(s). The pipeline includes an optical flow compression network ^^( ^^2) where ^^2are the weights and / or biases of the optical flow compression network. A training data set comprising sequences of images with GOP size 4 or more is provided. An optimiser (e.g. stochastic gradient descent, adam, or any other optimiser algorithm) is defined and a gradient flag ^^ ^^ ^^ ^^ is set to true for the networks ^^ ( ^^1) and ^^( ^^2) indicating that they can be backpropagated through. For a number of training steps (e.g. ^^ steps) or while some other training condition is not yet satisfied, a loop is performed comprising a forward pass through ^^ ( ^^1) and ^^( ^^2) which includes calculating a current flow ^^^^= ^^( ^^^^−1, ^^^^, ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^| ^^^^−1) using the optical flow compression network ^^( ^^2) for training step ^^. Whereby forward warping ℎ is performed to produce ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^and the hyper network and / or hyper hyper network of the optical flow compression network ^^( ^^2) is conditioned on ^^^^−1. The calculated flow ^^^^is fed into the residual module of the network ^^ ( ^^1) to produce the final reconstructed frame A loss comprising one or more loss terms is computed. The loss may comprise a rate term, a distortion term and / or one or a more other loss terms such as a temporal consistency loss term that is described in above in connection with concept 1. The gradients of the loss function are calculated and may be accumulated for a predetermined number of steps (e.g. a number of steps corresponding to a GOP size of the training data set), and the optimiser applied to update the weights ^^1and ^^2. The training loop is continued until some end of training condition is met, such as e.g. after ^^ steps, after a loss has plateaued, after a validation loss starts to diverge from a training loss, or after some other requirement is met. Figure 16 illustrates an example implementation of anAI-based compression pipeline combining the concepts of 6 and 7 into a single pipeline, based on Figures 12, 13 and 15. Like-numbered elements are indicated by like numerals. Thus the implementation of Figure 16 comprises the steps of (i) calculating 1500 ℎ(^ˆ^−1) to produce ^^^^, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^which is combined with ^^^^−1and ^^^^and fed into the main encoder element 207 of the optical flow compression network (as in Figure 15), (ii) feeding the past flow information^ˆ^−1into the hyper network (encoder 1105 and / or decoder 1106 as in Figure 12), as well as (iii) feeding the past flow information^ˆ^−1into a hyper hyper network substitute network 1113 (as in Figure 13). Concept 8: Global pooling in the flow network As described earlier herein, one challenge that arises when estimating optical flow is how to simultaneously take into account small motion (where an object moves a few pixels between frames) and large motion (where an object moves many pixels between frames) as well as how to efficiently compress that estimated flow information. Concept 8 is directed to at least partially overcoming the problem of how to efficiently compress estimated flows. In general terms, concept 8 is directed to performing a global pooling operation on a representation of the flow information in the hyper or hyper hyper network before it is encoded. The pooling operation effectively produces a descriptor of global flow, allowing correlations between motion from across the entire flame to be exploited by the hyper or hyper hyper network when producing entropy parameters that are subsequently use to entropy encode and decode the flow information. In more general terms, the pooling allows the hyper and / or hyper hyper network to become "position aware" at a global, frame-wide level. To illustrate this, consider a toy example of a simple image sequence where a zoom-in is occurring. At a pixel level, a zoom-in motion causes pixels to move outwards and away from a central point. That is, the pixel motion in the up, down, left and right directions, is correlated. Wherever there is correlation in motion, there is an opportunity to improve compression rates as this correlation can be used to reduce information redundancy in the bitstream. However, without using pooling, the convolution kernels (with limited kernel size and receptive field) in the hyper or hyper hyper network do not capture any information about global motion in a frame. Thus, even if it would be apparent with full global information that a zoom-in motion is occurring and thus that much of the flow can be modelled with very little information, there is no architectural mechanism in the hyper or hyper hyper network to take advantage of the correlation to improve compression rates. If we take the same toy example but now introduce a global pooling step in association with the hyper and / or hyper hyper flow network, the object on which the hyper or hyper hyper flow encoder operate captures information taken from across the entire frame as a result of the pooling step. That is, the pooling step, for all intents and purposes condenses global flow information into a smaller dimensional object which is a descriptor of that global flow. The hyper and / or hyper hyper network thus no longer have the problem that the receptive field of the convolution kernels of its networks are unable to "see" global flows because the pooled object the networks are given captures global flows. The hyper and / or hyper hyper network are accordingly able to learn during training to take advantage of any global correlations in flow (such as correlations that arise from zooming in, panning, and so on) from the pooled object. Figure 17 illustratively shows an AI based compression pipeline. The pipeline of Figure 17 corresponds to that of Figure 11 in that like-numbered reference numerals refer to like-numbered features. The flow encoder neural network takes a current image ^^^^and a previous image ^^^^−1, encodes these into a flow latent representation which is optionally quantised and entropy encoded and transmitted as a bit stream. On the decode side, the bitstream is received, entropy decoded into the flow latent representation that the flow decoder neural network uses as input to produce a representation of flow ,ˆ e.g. a flow map. The flow mapˆis applied to a previously decoded image ˆ^^−1to generate a warped version of that previously decoded image ˆ^^−1, ^^. The warped version of the previously decoded image ˆ^^−1, ^^is then fed into the residual encoder network, together with the current image ^^^^to produce a residual latent representation which is optionally quantised and entropy encoded and transmitted as a bit stream. On the decode side, the bitstream is received and entropy decoded back into the residual latent representation ^^ and used by the residual decoder neural network, in combination with information from the warped version of the previously decoded image ˆ^^−1, ^^, to produce the reconstructed image ˆ^^. In the case of Figure 17, the information associated with the warped previously decoded image ˆ^^−1, ^^ is optionally first processed by module ^^ , referred to herein after as a composition adapter, for example to downsample and / or pad it before it is fed into the residual decoder together with the entropy decoded residual latent representation to produce the final reconstructed image ˆ^^. This process may then be repeated for ^^^^+1and so on to encode, transmit, and decode a sequence of frames. Whilst not shown in Figure 17, the flow encoder may comprise the architectures described with respect to any of the other concepts illustrated above in respect of optical flow compression networks of Figures 5 to 10. As is further shown in Figure 17, the entropy encoding that produces the respective bitstreams 204, 208, 212 is supported by corresponding a hyper network which in turn is supported by a hyper hyper network. The hyper network for the I-frame module 201 comprises a neural network 1101 that is referred to as a hyper encoder, and a neural network 1102 that is referred to as a hyper decoder. As is described in connection with Figure 2, the hyper encoder neural network 1101 encodes the latent representation produced by the encoder neural network 203, and the hyper decoder neural network 1102 produces the entropy parameters that define the distribution of the latent representation. These parameters are used during entropy encoding to produce the bitstream 204. The encoded latent representation produced by the hyper encoder 1101 (the "hyper" latent representation) is itself entropy encoded into a bitstream using an entropy model using entropy parameters obtained by encoding the encoded (i.e. "hyper") latent representation using the hyper hyper encoder 1103 to produce a hyper hyper latent representation, and decoding that hyper hyper latent representation using the hyper hyper decoder 1104 to produce the entropy parameters that are used to entropy encode the hyper latent representation into its bitstream 208. This nested structure can be extended as desired but it has been found that a hyper and hyper hyper network achieve good performance for run time trade off. Additionally, Figure 17 introduces a global pooling operation 1700 (for example a global average pooling or a global max pooling operation) applied to the output of the flow hyper encoder 1105 before it is fed into the flow hyper hyper encoder 1107. That is, the hyper latent representation of flow information produced by the hyper encoder 1105 in the form of a tensor is subject to a pooling operation, to produce a n-dimensional vector representation of the hyper latent representation, where the number of dimensions n may be set based on an expressiveness-to-compute efficiency trade off. That is, pooling the hyper latent representation flow tensor into a 10-dimensional vector destroys more information than pooling it into a 128-dimensional vector, which is more expressive, but the trade off is that the hyper hyper encoder now operates on a higher dimensional object, which may increase run time. Optionally, one or more reshaping operations may be performed to the hyper latent representation (a tensor) of flow information before the pooling operation is performed. The hyper hyper flow encoder 1107, encodes the pooled object (the n-dimensional vector) representing the hyper latent representation of flow into a hyper hyper latent representation of flow. The hyper hyper flow decoder 1108 entropy decodes the (global) hyper hyper flow latent, naively copying it out to the correct shape (i.e. effectively performing an unpooling or de-pooling operation 1705), and further processes that to produce the entropy parameters as expected as output from the hyper hyper flow decoder 1108. The output of running the hyper hyper flow decoder 1108 with an unpooled object is thus the entropy parameters, these are then used to entropy encode the hyper latent representation of flow output by the hyper flow encoder 1105. Note that because some information is destroyed by the pooling operation 1700 and cannot be retrieved by the above-described naive unpooling, the reconstruction of the hyper latent representation is not an exact reconstruction of the object as it existed before the pooling operation. However, this does not matter because the hyper latent representation’s primary purpose is to facilitate the production of optimal entropy parameters that produce a bitstream that is as small as possible. The exact form of the hyper latent representation of flow information (and the hyper hyper latent representation of flow information) thus does not matter as long as the hyper network and hyper hyper network are able to use it to achieve smaller bitstream sizes. Put in very general terms, the pooling of the hyper latent representation before it is encoded by the hyper hyper encoder 1107, has the effect of destroying some spatial information but at the same time retaining and condensing global flow information into a form that the hyper network and hyper hyper network are better able to use as a global flow descriptor than the hyper latent representation without pooling. Whilst not shown in Figure 17, the pooling operation may additionally or alternatively be applied to the output of the optical flow compression network (i.e. to the latent representation of flow information) before it is fed into the hyper flow encoder 1105, to also make the hyper flow network (i.e. the flow encoder 1105 and decoder 1106) "global flow aware", allowing it to also learn to take advantage of global flow correlations to improve compression rates through the entropy parameters produced by the hyper flow network. Optionally, we can further enhance the effect that pooling has on making the hyper hyper flow network (or hyper flow network) position aware by combining the optical flow information (and / or its latent, hyper latent or hyper hyper latent representation) with learned positional embeddings. This may accordingly be at the base optical flow encoder and / or decoder level, but also or instead at the hyper encoder and / or decoder level, and / or even at the hyper hyper encoder and / or decoder level. Figure 17 further illustrates the use of positional embeddings at the hyper hyper network level. For example, consider the the hyper hyper decoder 1108 and the output of the unpooling operation 1701. As shown in Figure 17, a neural network 1702 is introduced that produces a learned positional embedding from an input coordinate grid 1703 based on the resolution of the images on which the compression pipeline is being used. In more detail, the neural network 1702 takes as input a coordinate grid tensor and outputs a positional embedding based on that tensor. The input tensor may be as simple as an arbitrary h x w matrix of values initialised between 0 and 1 where h x w correspond to the resolution of the input images of the image sequence, or the resolution of the flow latent representation, flow hyper latent representation, or flow hyper hyper latent representation. Alternatively it may be a more complex representation of position information, for example a tensor of h x w x c, where c is now a channel dimension such that the position embedding may be estimated from different grids for respective channels of an input. The positional embedding tensor is then combined with the output of the unpooled object (i.e. combined with the output of the unpooling operation 1701 performed on the hyper hyper decoder 1108). This combination may be a simple element-wise addition or some other combination operation. Additional pre-processing may be necessary to facilitate such a combination, for example one or more tensor shape modification operations. The resulting combined object is then fed into the hyper decoder 1106 which produces the entropy parameters used to entropy encode (and decode) the hyper latent representation of flow into its bitstream. At a general level, the effect of the positional embeddings is to allow the unpooled object to be spatially varying. That is, the positional embeddings facilitate producing a hyper flow latent representation with values that can differ depending on pixel location. To illustrate this, consider again the zoom in / out toy example. The decoded hyper hyper latent representation of flow will be indicative of a motion direction pointing precisely away from (or towards) the center of the zoom, but if the networks don’t know where the motion is in relation to that center (i.e. because the unpooled object without the positional embedding is the same in every location as the original pooling operation destroyed the spatial information) then the networks won’t be able to produce a flow that "points" in the right direction i.e. it the produced flow will be spatially uniform. In contrast, the positional embedding allows the values of the othersise spatially uniform unpooled object to vary by position and thus allow a better modelling of flow corresponding to the zoom in / out motion. Put simply, the positional embedding modifies the non-spatially varying values of the unpooled object based on pixel location such that they become spatially varying. Indeed, without positional embeddings, global pooling alone is less effective at modelling anything other than spatially-constant global motion (i.e. panning). Put even more simply, combining a positional embedding with the unpooled object "reminds it" about the coordinate system it exists in by modifying it element-wise with the values of the positional embedding. Positional embeddings can also optionally or alternatively be introduced elsewhere in the flow network. For example, we can produce a positional embedding with a further neural network 1704 (or with the same network 1702), and combine it with the hyper latent representation before performing the pooling operation 1700. In the same way as described above, the further neural network 1704 takes as input a coordinate grid tensor and outputs a positional embedding based on that tensor. The input tensor may be as simple as an arbitrary h x w matrix of values initialised between 0 and 1 where h x w correspond to the resolution of the input images of the image sequence, or the resolution of the flow latent representation, flow hyper latent representation, or flow hyper hyper latent representation. Alternatively it may be a more complex representation of position information, for example a tensor of h x w x c, where c is now a channel dimension such that the position embedding may be estimated from different grids for respective channels of an input. It will be appreciated that for training, the neural network(s) 1702, 1704 that produce the positional embedding(s), may be trained in an end to end manner with one or more of the other neural networks of the compression pipeline (e.g. such as those shown in Figure 17), or trained in a standalone manner. That is, the weights of the networks are updated based on the same loss function (e.g. rate distortion loss function, or some other loss function) whereby the training objective is to learn a set of weights that helps to minimise this loss by producing positional embeddings that optimise the compression rate and reconstruction accuracy of the AI-compression pipeline. In this specific case, the positional embeddings, together with the global pooling, operate as an architectural mechanism to allow the flow hyper network and hyper hyper network to become aware of global flow. The inventors have also unexpectedly found that the above-described approach of using posi- tional embeddings are particularly effective at achieving improved rate distortion performance when running the compression pipeline in integer (e.g. int8) precision. When the data objects input and output by the neural networks of the pipeline are int8 objects, each object has a set of quantisation parameters that define how to interpret the integer values of the data in terms of how wide are the bin sizes of the data object, and where they are distributed across the 256 possible int8 integer values of the data object. By conditioning the entropy parameters (which model the distribution of the values of the flow latent, hyper flow latent and hyper hyper flow latent representations) on the positional embeddings, the associated quantisation parameters used when quantising the flow inputs, intermediate values in the neural networks of the pipeline, and the outputs also become conditioned on the positional embedding, thus allowing the networks to learn to produce distributions of flow latent, flow hyper latent and flow hyper hyper latent representations that quantise well into int8 with a smaller quantisation error than when a positional embedding is not used. The above concepts may be abstracted with the following pseudocode for encode and decode operations. On encode side we can denote the flow hyper hyper latent representation as ^^, the flow hyper hyper encoder 1107 as ^^ ^^^^ ^^ ^^ ^^ ^^ ^^ ^^and the hyper latent representation (output by the flow hyper encoder 1105) as ^^, then we can calculate ^^ by: ^^ = ^^ ^^^^ ^^ ^^ ^^ ^^ ^^ ^^( ^^) = ^^( ^^( ^^ ( ^^)) where ^^ is neural network 1704 producing a position embedded latent based on the hyper latent representation ^^, ^^ is pooling operation that contracts each channel of an input into a single number, for example a global average pooling or global max pooling operation, and ^^ represents hyper hyper encoder neural network 1107 (for example a multilayer perceptron or some other neural network) that receives the positional embedding combined, global pooled hyper latent representation. On decode side we can denote the hyper hyper latent representation as ^ˆ^ (where the hat notation is indicative of a quantised representation), the flow hyper hyper decoder 1108 as ^^ ^^^^ ^^ ^^ ^^ ^^ ^^ ^^and then produce the entropy parameters ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^^^from ^ˆ^, and entropy decode the hyper latent representation bitstream using those entropy parameters to produce the hyper latent representation ˆ from the bitstream by calculating: ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^^^= ^^ ^^^^ ^^ ^^ ^^ ^^ ^^ ^^( ^ˆ^) = ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ (ℎ( ^ˆ^)) ˆ = ^^ ^^ ( ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^)^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^where ℎ is a neural network representing the hyper hyper decoder 1108, for example a multilayer perceptron or any other neural network, ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ represents the naive "unpooling" operation, copying the output of ℎ into the shape matching the the initial hyper latent representation ^^ before it was pooled, and ^^ ^^ represents an entropy decoding operation. It will be appreciated that concept 8 may be used as a standalone method or together with any of the methods described herein in connection with the other concepts. Concept 9: Positional embeddings in the flow network The inventors have also unexpectedly identified that including positional embeddings as an input to one or more layers of the optical compression flow network 207 may be used as a standalone architectural mechanism in the optical compression flow network 207 to help preserve the precise locations of features when any downsampling is being performed, for example as part of an implementation of one or more feature pyramids such as those shown in Figures 5-10, or 19-20. Figure 18 illustrates an AI-based compression pipeline corresponding to that of Figure 11. Like numbered reference numerals indicate like-numbered features. The flow encoder neural network takes a current image ^^^^and a previous image ^^^^−1, encodes these into a flow latent representation which is optionally quantised and entropy encoded and transmitted as a bit stream. On the decode side, the bitstream is received, entropy decoded into the flow latent representation that the flow decoder neural network uses as input to produce a representation of flow ,ˆ e.g. a flow map. The flow mapˆis applied to a previously decoded image ˆ^^−1to generate a warped version of that previously decoded image ˆ^^−1, ^^. The warped version of the previously decoded image ˆ^^−1, ^^is then fed into the residual encoder network, together with the current image ^^^^to produce a residual latent representation which is optionally quantised and entropy encoded and transmitted as a bit stream. On the decode side, the bitstream is received and entropy decoded back into the residual latent representation ^^ and used by the residual decoder neural network, in combination with information from the warped version of the previously decoded image ˆ^^−1, ^^, to produce the reconstructed image ˆ^^. In the case of Figure 17, the information associated with the warped previously decoded image ˆ^^−1, ^^is optionally first processed by module ^^ , referred to herein after as a composition adapter, for example to downsample and / or pad it before it is fed into the residual decoder together with the entropy decoded residual latent representation to produce the final reconstructed image ˆ^^. This process may then be repeated for ^^^^+1and so on to encode, transmit, and decode a sequence of frames. Whilst not shown in Figure 18, the flow encoder may comprise the architectures described with respect to any of the other concepts illustrated above in respect of optical flow compression networks of Figures 5 to 10. Additionally a neural network 1801 is introduced that produces a learned positional embedding from an input coordinate grid 1802 based on the resolution of the images on which the compression pipeline is being used. In more detail, the neural network 1801 takes as input a coordinate grid tensor and outputs a positional embedding based on that tensor. The input tensor may be as simple as an arbitrary h x w matrix of values initialised between 0 and 1 where h x w correspond to the resolution of the input images of the image sequence, or the resolution of the flow latent representation, flow hyper latent representation, or flow hyper hyper latent representation. Alternatively it may be a more complex representation of position information, for example a tensor of h x w x c, where c is now a channel dimension such that the position embedding may be estimated from different grids for respective channels of an input. With reference to Figure 5 and Figure 10, the positional embedding produced by neural network 1801 may be combined, e.g. element-wise with or without reshaping, to the outputs of one or more of the pyramid feature extractor layers of the optical flow compression networks 500, 1000 between downsampling operations. The effect of this is to "remind" the layers of the optical flow compression network of the coordinate system in which it is operating, even at the downsampled levels. This means that the precise locations of features can be preserved while the features are downsampled in the higher levels of the pyramid feature extractor layers. This is useful because downsampling normally destroys fine-grained position information while the introduction of the positional embeddings is able to partially restore or retain some of this fine-grained position information. More fine-grained position information results in more accurate feature positions and thus more accurate flow detection. It will be appreciated that for training, the neural network 1801 that produce the positional embedding, may be trained in an end to end manner with one or more of the other neural networks of the compression pipeline (e.g. such as those shown in Figure 18), or trained in a standalone manner. That is, the weights of the networks are updated based on the same loss function (e.g. rate distortion loss function, or some other loss function) whereby the training objective is to learn a set of weights that helps to minimise this loss by producing positional embeddings that optimise the compression rate and reconstruction accuracy of the AI-compression pipeline. It will be appreciated that concept 9 may be used as a standalone method or together with any of the methods described herein in connection with the other concepts. Concept 10: Planar feature extractor The optical flow compression networks described in concepts 1 to 9 above have generally been illustrated using a pyramid feature extractor. For example, Figure 5 illustrates a flow module comprising a neural network 500 with various pyramid layers 501a, 501b for sequentially producing feature maps based on a first image ^^^^−1and a second image ^^^^of an image sequence. That is, the feature map output by the the first layer of the pyramid is fed into the next layer of the pyramid and so on. A consequence of this architecture is that the task of extracting coarse and fine flows from the different features from the input images is shared across the layers of the pyramid, in theory making the task easier to learn due to the next layers being able to take advantage of the work already performed by the previous layers in extracting features that are potentially important for accurate flow estimation. However, the inventors have identified that sequential pyramid structures have a number of unexpected drawbacks when used for flow estimation in AI-based compression pipelines. The first drawback can be understood by considering that features of a feature map that might be important for accurately estimating coarse, global flows may not be as important for accurately estimating fine, local flows, and vice versa. Consider, for example, a pyramid feature extractor such as that of Figure 5. The feature map produced by the first, highest resolution pyramid layer produces a feature map that may emphasise features that are associated with fine, local flows where this fine level of granularity is still present in the input. This feature map emphasising fine, local flow features is then passed to the next layer which has a smaller resolution and considers the fine, local flow features to be less important so tries instead to emphasise in its output feature map different features that it considers to be more useful for accurate flow estimation at its resolution, and so on. Each pyramid layer after the first never actually sees the input image and instead only gets a heavily modified representation of the input image in the form of a feature map that the previous layer considered to be useful at its resolution but that may not be useful for the current layer at its resolution. This means that, by the time we get to the coarsest layer of the pyramid, the information present in the input image has already been through a series of sequential transformations, each resulting in information being lost that may have been useful to the coarsest layer. One consequence of this is that the resulting flows that are produced from the feature maps extracted by the pyramid architecture (for example as described in connection with Figure 5, and / or any of the other concepts described herein) struggle to accurately capture fine, local flows and coarse, global flows simultaneously. The second drawback of a pyramid feature extractor where output feature maps from one layer are past to the next layer is that its operation is synchronous and accordingly can have a significant runtime overhead. That is, the operations defining a next layer cannot be performed until the operations of the current layer are complete and have produced an output to feed into the next layer. This means that pyramid shaped feature extractors can be a runtime bottleneck in AI-based compression pipelines that use a flow estimation in their architecture. The present inventors have realised that a planar feature extractor at least partially overcomes both of these drawbacks. Figure 19 illustrates a flow module of an AI-based compression pipeline such as that of any of the other concepts described herein but with the pyramid shaped feature extractor architecture replaced with a planar shaped architecture. That is, each of the layers of the respective feature extractors are arranged in parallel to each other in that they do not depend on each others’ outputs as inputs. Instead, they each receive as input a representation of the input image(s) (downsampled to the resolution that each layer expects to receive but otherwise not transformed into a feature map). This means each layer of the feature extractor gets to see as close to an unmodified representation of the input image(s) (i.e. ^^^^and ^^^^−1) as possible, ensuring that each layer is able to extract those features which they respectively consider to be most important for accurately modelling flow at their respective resolutions, and without information being lost through the successive generation of feature maps in a pyramid architecture. When these independently produced feature maps are used to estimate flows at the various resolutions, the flows are better able to model both fine, local flows and coarse, global flows more accurately than flows produced using a pyramid feature extractor (note accuracy in this context may be bench marked against a known ground truth flow). In very general terms, transforming a deep, narrow pyramid shaped feature extractor into a shallow and wide feature extractor with each layer directly operating on the input image instead of on some intermediate feature map from an earlier layer results in an increase in the amount of useful information from the input image being retained in each feature map, thereby producing more accurate flows at each resolution, and thereby resulting in a more accurate overall output flow from the optical flow compression network. A second advantage of a planar shaped feature extractor architecture is that it allows for asynchronous operation because none of the layers have to wait for the output of any of the other layers. This entirely mitigates the runtime bottleneck of the synchronous operation of pyramid shaped feature extractor architectures and thereby reduces total runtime of the flow module. Figure 19 will now be described in more detail. Figure 19 illustrates an example of an optical flow compression network 1900 implementation using a planar feature extractor block 1901 according to the present disclosure. The optical flow compression network 1900 may be used as or in a flow module in, for example, the AI-based compression pipeline of Figure 3. The optical flow compression network 1900 in Figure 19 is based on the optical flow compression network of Figure 5 for illustrative purposes but it will be appreciated that the planar feature extractor block 1901 may be used as a drop-in replacement in connection with any of the other concepts described herein to replace fully or partially a pyramid feature extractor architecture. Continuing for now using the illustrative example based on Figure 5, like-numbered reference numerals refer to like-numbered elements in Figure 5. The optical flow compression network 1900 in Figure 19 comprises a neural network comprising a set of layers 1902a, 1902b that are configured to output feature maps at a respective set of resolutions. This may be, for example, at resolutions relative to the input image height and width (ℎ, ^^) of ℎ / 2 ^^ / 2 in a first layer, ℎ / 4 ^^ / 4 in a second layer, ℎ / 8 ^^ / 8 in a third layer, ℎ / 16 ^^ / 16 in a fourth layer, ℎ / 32 ^^ / 32 in a fifth layer, ℎ / 64 ^^ / 64 in a sixth layer, and so on. It will of course be appreciated that these spatial resolution are exemplary only and others are also envisaged. Unlike in a pyramid feature extractor where the inputs to each layer are an output feature map of a lower resolution from a previous layer, the inputs to each layer of a planar feature extractor 1901 are not feature maps but are instead simple downsampled representations of the input. For example, the result of performing one or more naive downsampling operations 1903a, 1903b such as bilinear, or cubic interpolation, nearest neighbour interpolation, strided convolutions followed by subsampling, or any other downsampling methods known to the skilled person. It is also envisaged that the one or more naive downsampling operations 1903a, 1903b may be implemented using one or more convolutional layers integrated into the layers 1902a, 1902b of the neural networks of the optical flow compression network 1900. The downsampled input image at a each resolution is then fed into a respective layer for receiving that resolution which then outputs the feature map. In this way, the inputs to each layer, particularly the layers processing the lowest resolution inputs, retain more of the information of the input image as they are "closer" or "fewer steps removed from" the input image(s) (and thus less modified) than is the case for inputs which themselves are feature maps output by a previous layer. In addition, in Figure 19, the layers of the neural network of the optical flow compression network shown in a planar or parallel arrangement as each layer is can operate entirely independently of the other layers. This means the planar feature extractor block in Figure 19 can be run entirely or partially in parallel to reduce the overall runtime of the feature extraction steps. Once the layers have produced their respective outputs (feature maps) at the respective resolutions, cost volumes (e.g. MAD or some other difference estimation as described herein) are calculated and a representation of flow is then estimated for that resolution. The representation of flow is then used to warp the next resolution up, before the same steps are repeated until a final cost volume and final representation of flow is estimated and used in the rest of the AI-based compression pipeline as illustrated and described in connection with the other concepts described herein. It will be appreciated that the fully planar architecture described above is illustrative only and a hybrid architecture that is a mix of a planar and a pyramid architecture is also envisaged. This is illustrated in more detail in Figure 20. Figure 20 illustrates an example optical flow compression network 2000 implementation using a hybrid, planar-pyramid feature extractor block 2001 according to the present disclosure. The optical flow compression network 2000 may be used as or in a flow module in, for example, the AI-based compression pipeline of Figure 3. The optical flow compression network 2000 is based on the optical flow compression network of Figure 5 for illustrative purposes but it will be appreciated that the hybrid, planar-pyramid feature extractor block 2001 may be used as a drop-in replacement in connection with any of the other concepts described herein to replace fully or partially a pyramid feature extractor architecture. Continuing for now using the illustrative example based on Figure 5, like-numbered reference numerals refer to like-numbered elements in Figure 5. The optical flow compression network 2000 in Figure 20 comprises a neural network comprising a set of layers 2002a, 2002b that are configured to output feature maps at a respective set of resolutions. This may be, for example, at resolutions relative to the input image height and width (ℎ, ^^) of ℎ / 2 ^^ / 2 in a first layer, ℎ / 4 ^^ / 4 in a second layer, ℎ / 8 ^^ / 8 in a third layer, ℎ / 16 ^^ / 16 in a fourth layer, ℎ / 32 ^^ / 32 in a fifth layer, ℎ / 64 ^^ / 64 in a sixth layer, and so on. It will of course be appreciated that these spatial resolution are exemplary only and others are also envisaged. Unlike in a pyramid feature extractor where the inputs to each layer are an output feature map of a lower resolution from a previous layer, the inputs to some layers of a hybrid, planar-pyramid feature extractor architecture are simple downsampled representations of the input, while the inputs to other layers are the output feature maps of a previous layer. Effectively, in the hybrid architecture, the pyramid architecture is split up into a number of smaller sub-pyramids. In the case of Figure 20, there are two sub-pyramids, however it is envisaged that any number may be used. By introducing multiple sub-pyramids instead of a single pyramid, the information propagating through each sub-pyramid is "closer to" or "fewer processing steps away from" the input, thereby helping to retain more information from the input that might otherwise be lost with each successive processing of the feature maps by the next layers. At the same time, the sub-pyramid structures provide the hybrid architecture with some of the expressiveness of a deeper architecture which is not present in a the pure, shallow planar feature extractor. For the first layer of each sub-pyramid, the input may be a simple downsampled representation of the input image, for example the result of performing one or more naive downsampling operations 2003a, 2003b such as bilinear, or cubic interpolation, nearest neighbour interpolation, strided convolutions followed by subsampling, or any other downsampling methods known to the skilled person. It is also envisaged that the one or more naive downsampling operations 2003a, 2003b may be implemented using one or more convolutional layers integrated into the layers 1902a, 1902b of the neural networks of the optical flow compression network 1900Then, once the first layer of the sub-pyramids have produced their respective feature maps, the feature maps can be fed as inputs into next layers of each sub-pyramid in the same way as is the case for a feature pyramid extractor such as that of Figure 5. Further, as was the case for Figure 19, a runtime speed up can be achieved with a hybrid, planar-pyramid architecture because each sub-pyramid is independent of each other so each sub-pyramid can be run in asynchronously in parallel. Once the layers have produced their respective outputs (feature maps) at the respective resolutions, cost volumes (e.g. MAD or some other difference estimation as described herein) are calculated and a representation of flow is then estimated for that resolution. The representation of flow is then used to warp the next resolution up, before the same steps are repeated until a final cost volume and final representation of flow is estimated and used in the rest of the AI-based compression pipeline as illustrated and described in connection with the other concepts described herein. The number of sub-pyramids may be based on a trade-off between runtime, expressiveness, and input information retention at each resolution. For example, for input images where there is significant amounts of both local and global motion, input information at each resolution is important and a higher number of sub-pyramids may be advantageous (or indeed in the extreme case, a fully planar architecture may be used). Conversely, if there is only very fine-grained, local motion in a small part of the image that is complex, then a more expressive architecture may be advantageous and a smaller number of sub-pyramids may be optimal. Similarly, if runtime is a priority, then asynchronous running is prioritised and the total number of sub-pyramids may be increased. Finally, it is envisaged that the number of sub-pyramids may be dynamically changed or hot-swapped during inference. For example, the weights of the neural network for different numbers of sub-pyramids may be trained separately and the number of sub-pyramids used for a given frame sequence may be determined based on a property of the frame sequence (for example, a result of a classification of the frame sequence into specific type of video such as high-motion animation, static nature scenes, and so on). During inference the most optimal number of sub-pyramids may be selected for that type of frame sequence for a given runtime, expressivness and input information retention trade off. For completeness, whilst Figures 19 and 20 illustrate the use of cost volumes for difference estimation, any difference estimation methods including cost volumes, MAD, and others described above or known to the skilled person are also envisaged. The training of the weights of the neural networks in Figure 19 and / or 20 may be performed in an end-to-end manner with the rest of the neural networks of an AI-based compression pipeline (e.g. such as that of Figure 3) using a rate distortion loss. Alternatively, they may be pre-trained in a standalone, supervised manner again using a rate distortion loss (with distortion being based on against a ground truth flow) and / or using a temporal consistency loss or some other loss term. It will be appreciated that concept 10 may be used in a standalone manner in an AI-based compression pipeline or in connection with any of the other concepts described herein. Concept 11: Flows using ground truth frame blend As described above, optical compression flow networks in some cases struggle to compress very low motion scenes (such as barely changing, static nature scenes) and very high motion scenes (such as action-heavy movie sequences) well. For example, low motion scenes that should be easy to compress into a very small bitstream can sometimes produce bitstreams that contain significant amounts of redundant information, and that produce inaccurate flow information. In other cases, high motion scenes result in flow information that does not model the flow of that high motion accurately. The inventors have unexpectedly found that these performance issues can be attributed in part to which frame is used as a reference frame when estimating optical flow information. That is, there is noticeable optical flow compression network performance difference between using a ground truth frame ^^^^−1versus a previously decoded frame ˆ^^−1as the reference frame against which to compare the current frame ^^^^when calculating optical flow information indicative of a difference between the current frame and the reference frame. More specifically, when the reference frame against which the current frame ^^^^is compared is the ground truth frame ^^^^−1(that is, the next frame of the image sequence being encoded on the encode side of the compression pipeline), the optical flow compression network is able to successfully compress high motion image sequences into small sized bitstreams that can be decoded into accurate optical flow information while at the same time struggling to perform equally well on simple, low motion image sequences. Conversely, if the reference frame is the previously decoded representation of the ground truth frame ˆ^^−1(i.e. a representation of the ground truth frame ^^^^−1that has been passed through the optical flow compression network and the residual network on the encode side as part of producing the entropy parameters using the hyper and / or hyper hyper networks), then the optical flow compression network is able to compress simple, low motion sequences into small sized bitstreams that can be decoded into accurate optical flow information while at the same time struggling to perform equally well on high motion image sequences. Formally, if we define ^^ ^^ ^^ ^^ as the output of a function ^^ , for example the optical flow compression network, then, using the ground truth frame ^^^^−1as the reference frame gives us: ^^ ^^ ^^ ^^^^ ^^= ^^ ( ^^^^, ^^^^−1) or, using the previously decoded frame as the reference frame, gives us: ^^ ^^ ^^ ^^^^ ^^= ^^ ( ^^^^, ˆ^^−1) where the ^^ ^^ indicates the reference frame is the ground truth frame ^^^^−1and ^^ ^^ indicates the reference frame is a previously decoded representation of the ground truth frame ˆ^^−1. The inventors have realised that that the overall performance of an AI-based compression pipeline (defined in, for example bits per pixel, or other metric) when using ^^ ^^ ^^ ^^^^ ^^is better on certain image sequence types where ^^ ^^ ^^ ^^^^ ^^performs worse, with the opposite relationship holding true on other image sequence types. The inventors have further realised that this unexpected difference in performance between ^^^^−1versus ˆ^^−1as the reference frame for different image sequence types can be leveraged to improve performance on the different image sequence types, for example on both high motion, and simple, low motion scene types. Specifically, using a blend of ^^^^−1and ˆ^^−1as the reference frame results in the best of both worlds: improved performance of the optical flow compression network on both both high motion, and simple, low motion scene types. In very general terms, using a blend appears to cancel out the performance drops that arise when only one or the other of ^^^^−1or ˆ^^−1is used as the reference frame. Thus, if we again define the optical flow compression network as a function ^^ , we can now instead estimate the optical flow information from a combination of the ground truth frame and the previously decoded frame using: ^^ ^^ ^^ ^^^^ ^^ ^^ ^^ ^^= ^^ ( ^^^^, ^^^^−1, ˆ^^−1) where ^^ ^^ ^^ ^^ ^^ indicates the reference frame is a combination of the ground truth frame and the previously decoded frame. More specifically, we can define a function ℎ that combines the ground truth frame and the previously decoded frame, giving: ^^ ^^ ^^ ^^^^ ^^ ^^ ^^ ^^= ^^ ( ^^^^, ℎ( ^^^^−1, ˆ^^−1)) The function ℎ may take any suitable form as long as it is able to combine two frames in a controllable way, for example, ℎ may comprise a neural network comprising one or more convolution layers and / or activation layers with learnable weights that outputs a pixel-wise mask (for example a mask made up of values of between 0 to 1 for each pixel coordinate) based on the inputs ^^^^−1and ˆ^^−1that can be applied to ^^^^−1and ˆ^^−1, for example, the final output of ℎ may be: ℎ( ^^^^−1, ˆ^^−1) = ^^^^−1, ^^ ^^ ^^ ^^ ^^= ^^^^ ^^^^^^−1+ ^^^^ ^^ˆ^^−1 where ^^^^ ^^and ^^^^ ^^are tensors of mask values for each pixel coordinate i, j of ^^^^−1and ˆ^^−1, and ^^^^−1, ^^ ^^ ^^ ^^ ^^is the blended image used as the reference frame by the optical flow compression network. This formulation is of course exemplary only and the blending function may take any suitable form. That is, ^^^^ ^^and ^^^^ ^^may, but need not, add up to 1, for example, optionally ^^^^ ^^= 1 − ^^^^ ^^, resulting in a convex combination. Further, other formulations that do not use explicit pixel-wise masks are also envisaged, for example any arbitrary neural combination of the ground truth frame and previously decoded representation thereof. Thus we get: ^^ ^^ ^^ ^^^^ ^^ ^^ ^^ ^^= ^^ ( ^^^^, ^^^^−1, ^^ ^^ ^^ ^^ ^^) In very general terms, ℎ operates as an architectural mechanism with learnable weights that the optical flow compression can use to decide how much of ^^^^−1and how much of ˆ^^−1should be used for a pair of input frames at ^^ and ^^ − 1 optimised for a given objective function (e.g. the weights are learned based on minimising a rate distortion loss so the weights that are learned result, in inference, in a given blend of ^^^^−1and ˆ^^−1that minimises this loss for this input pair of frames). For completeness, it is noted that training of the weights of the neural network of ℎ may be performed, at least partially, together with the training of the weights of the neural networks of the optical flow compression network as it is the optical flow compression network which ℎ is intended to improve. It will be appreciated that either ℎ and / or the optical flow can also be frozen at different times and trained in a standalone manner for a predetermined number of steps to improve training stability of one or both components. An optional addition to the above architectural mechanism is to condition the output of ℎ on the current frame ^^^^by introducing it (e.g. by convolving or by concatenation) to one or more of the initial or intermediate inputs of one or more layers of the neural network of ℎ, that is: ℎ( ^^^^−1, ˆ^^−1| ^^^^) = ^^^^−1, ^^ ^^ ^^ ^^ ^^The effect of introducing ^^^^is to ensure the neural network of ℎ is "current frame aware", that is, it makes ℎ aware how much motion has occurred even if the input has been downsampled from some higher resolution. For example, if there is downsampling by a factor of 4, any features that moved by for example 1 pixel are will be approximately in the same position in both ^^^^−1and ^^^^in downsampled space, whereas features that moved by, for example, 30 pixels are not going to be in approximately the same position in the downsampled space. In this way, there’s a clear mechanism for ℎ (the reference frame generator) to stratify high versus low motion. Further, optionally, the neural network of ℎ may comprise one or more layers that downsample the input ^^^^−1and ˆ^^−1into some smaller resolution space before passing the downsampled inputs into further layers, for example, downsampling into a ℎ / 4, ^^ / 4 space, to increase the receptive field of ℎ. Where the formulation of ℎ produces a mask, then the output mask may be upsampled back into the original resolution to produce the final output ^^^^−1, ^^ ^^ ^^ ^^ ^^. Figure 21 illustrates an example implementation of an AI-based compression pipeline 2100 where ^^^^−1, ^^ ^^ ^^ ^^ ^^is used as a reference frame for calculating flow. For the purposes of illustration, the pipeline shown in Figure 21 is based on Figures 3 and 11, and like-numbered reference numerals refer to like-numbered elements in those figures. However, it will be appreciated that the concept of using a blended image as a reference frame may be used in any AI-based compression pipeline architecture where flow is calculated from two frames where both a ground truth and a previously decoded frame are available. As in Figures 3 and 11, the optical flow compression network 207 takes a current image ^^^^and a reference image as input. However unlike in Figures 3 and 11, the reference image in Figure 21 is a blended image produced using a blending function ℎ 2101 applied to the ground truth previous image ^^^^−1and a previously decoded representation of that ground truth previous image ˆ^^−1. Also shown in Figure 21 is the optional conditioning of the blended image on the current frame ^^^^which is introduced into the blending function 2101 as side information. The optical flow compression network 207 encodes these inputs into a flow latent representation which is optionally quantised and entropy encoded and transmitted as a bit stream. On the decode side, the bitstream is received, entropy decoded into the flow latent representation that the flow decoder neural network uses as input to produce a representation of flow ,ˆ e.g. a flow map. The flow mapˆis applied to a previously decoded image ˆ^^−1to generate a warped version of that previously decoded image ˆ^^−1, ^^. The warped version of the previously decoded image ˆ^^−1, ^^is then fed into the residual encoder network, together with the current image ^^^^to produce a residual latent representation which is optionally quantised and entropy encoded and transmitted as a bit stream. On the decode side, the bitstream is received and entropy decoded back into the residual latent representation ^^ and used by the residual decoder neural network, in combination with information from the warped version of the previously decoded image ˆ^^−1, ^^, to produce the reconstructed image ˆ^^. In the case of Figure 21, as in Figure 11, the information associated with the warped previously decoded image ˆ^^−1, ^^is optionally first processed by module ^^ , referred to herein after as a composition adapter, for example to downsample and / or pad it before it is fed into the residual decoder together with the entropy decoded residual latent representation to produce the final reconstructed image ˆ^^. This process may then be repeated for ^^^^+1and so on to encode, transmit, and decode a sequence of frames. Whilst not shown in Figure 21, optical flow compression network may comprise any of the architectural components described with respect to any of the other concepts illustrated above in respect of optical flow compression networks of Figures 5 to 20. As is further shown in Figure 21, the entropy encoding that produces the respective bitstreams 204, 208, 212 is supported by corresponding a hyper network which in turn is supported by a hyper hyper network. The hyper network for the I-frame module 201 comprises a neural network 1101 that is referred to as a hyper encoder, and a neural network 1102 that is referred to as a hyper decoder. As is described in connection with Figure 2, the hyper encoder neural network 1101 encodes the latent representation produced by the encoder neural network 203, and the hyper decoder neural network 1102 produces the entropy parameters that define the distribution of the latent representation. These parameters are used during entropy encoding to produce the bitstream 204. The encoded latent representation produced by the hyper encoder 1101 is itself entropy encoded into a bitstream using an entropy model using entropy parameters obtained by encoding the encoded latent representation using the hyper hyper encoder 1103, and decoding that encoded latent representation using the hyper hyper decoder 1104. This nested structure can be extended as desired but it has been found that a hyper and hyper hyper network achieve good performance for run time trade off. The entropy parameters that are used to produce the bitstream associated with the hyper hyper network (or whatever the highest level of the nested structure is) may optionally be fixed, set parametrically, or produced by an encoder / decoder neural network pair like in the hyper network. As can be seen in Figure 21, this nested hyper network and hyper hyper network approach is also provided in the optical flow module 206 in the form of hyper encoder neural network 1105, hyper decoder neural network 1106, hyper hyper encoder neural network 1107 and hyper hyper decoder neural network 1108. The same is also provided in the residual module 210 in the form of hyper encoder neural network 1109, hyper decoder neural network 1110, hyper hyper encoder neural network 1111 and hyper hyper decoder neural network 1112. The above-described approach can also be generalised further to any number of previous ground truth frames and / or previously decoded frames. For example, ground truth frames and previously decoded frames from positions ^^ − 1, ^^ − 2. ^^ − 3, ..., ^^ − ^^. That is, we can produce a blended image using: ^^ ^^ ^^ ^^^^ ^^ ^^ ^^ ^^= ^^ ( ^^^^, ^^^^−1, ˆ^^−1, ^^^^−2, ˆ^^−2, ..., ^^^^− ^^, ˆ^^− ^^) where ^^ ^^ ^^ ^^ ^^ indicates the reference frame is a combination of the ground truth frame and the previously decoded frame from any number of frames other than the current frame. As before, we can define a function ℎ that combines the ground truth frame and the previously decoded frame, giving: ^^ ^^ ^^ ^^^^ ^^ ^^ ^^ ^^= ^^ ( ^^^^, ℎ( ^^^^−1, ˆ^^−1, ^^^^−2, ˆ^^−2, ..., ^^^^− ^^, ˆ^^− ^^)) We then get: ℎ( ^^^^−1, ˆ^^−1, ..., ^^^^− ^^, ˆ^^− ^^) = ^^^^ ^^ ^^ ^^ ^^giving: ^^ ^^ ^^ ^^^^ ^^ ^^ ^^ ^^= ^^ ( ^^^^, ^^^^ ^^ ^^ ^^ ^^) As above, the output of ℎ can also be conditioned on the current frame ^^^^in the same way to make ℎ "current frame aware" by providing a mechansim to stratify high versus low motion even in downsampled space. Generalising in this way provides the optical flow compression network with a mechanism to overcome the kind of performance mismatches between different scene types (like high and low motion scenes, like high blur and low blur scenes, and any other scene types where such differences result in a performance mismatch) that can arise from using a ground truth frame, a previously decoded representation thereof, or some combination thereof as a reference frame when estimating optical flows. That is, the neural network of ℎ during training learns how much of each of the ^^ frames and / or previously decoded representations should contribute to a blend, masking out less useful information for achieving better performance on a rate distortion objective function while leaving in more useful information for increasing performance. It will appreciated that only some of the channels of ^^^^− ^^and / or ˆ^^−1may be included as input into ℎ. For example, when working in YUV space, optionally only the luma channel, or only the chroma channels, may be included in the blend. It will be understood that using only the luma channel information reduces the computational burden while still maintaining reasonable accuracy as motion information is typically over represented in luma channels of a YUV colour space. Thus, ^^^^ ^^ ^^ ^^ ^^may be a single channel luma image used as a reference frame for estimating flow relative to ^^^^, which may optionally also a be single channel luma image to reduce flow run times compared to estimating flows across all YUV channels. It will also be appreciated that concept 11 may be used in a standalone manner in an AI- based compression pipeline or in connection with any of the other concepts described herein. Specifically, concept 11 may be used as a drop-in replacement for a reference frame in a flow module where flow is being estimated from a current frame and a reference frame. The subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. The subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory program carrier for execution by, or to control the operation of, data processing apparatus. Alternatively or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. The computer storage medium is not, however, a propagated signal. The term “data processing apparatus” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A computer program (which may also be referred to or described as a program, software, a software application, a module, a software module, a script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network. The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). Computers suitable for the execution of a computer program include, by way of example, can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a VR headset, a game console, a Global Positioning System (GPS) receiver, a server, a mobile phones, a tablet computer, a notebook computer, a music player, an e-book reader, a laptop or desktop computer, a PDAs, a smart phone, or other stationary or portable devices, that includes one or more processors and computer readable media, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. The subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet. The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. While this specification contains many specific implementation details, these should be construed as descriptions of features that may be specific to particular examples of particular inventions. Certain features that are described in this specification in the context of separate examples can also be implemented in combination in a single example. Conversely, various features that are described in the context of a single example can also be implemented in multiple examples separately or in any suitable subcombination. Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the examples described above should not be understood as requiring such separation in all examples, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products. Where the term "previously decoded image" or "previously decoded representation of an image" is used herein, typically but not exclusively in the context of its use as a reference frame against which a current image is being compared when estimating flow, it will be appreciated that, on the encode side, this additionally or alternatively may be a ground truth representation of the previously decoded image or representation thereof, for example simply an earlier or later image of the input image sequence, rather than a representation thereof that is created by passing that image first through the various encoder and decoder structures.

Claims

CLAIMS 1. A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving first and second input images; estimating optical flow information indicative of a difference between representations of the first and second input images; receiving first and second output images, wherein the first and second output images comprise approximations of the first and second input images encoded and decoded by one or more neural networks; estimating optical flow information indicative of a difference between representations of the first output image and the second output image; evaluating a function based on a difference between the estimated optical flow information associated with the first and second input images and the estimated optical flow information associated with the first and second output images; updating the parameters of the one or more neural networks based on the evaluated function; and repeating the above steps using a first set of input images to produce one or more trained neural network.

2. The method of claim 1, wherein estimating optical flow information between representations of the first and second input images comprises estimating a pixel difference between a pixel inthe representation of the first input image and one or more pixels in the representation of the second input image.

3. The method of claim 1, wherein estimating optical flow information between representations of the first and second output images comprises estimating a pixel difference between a pixel in the representation of the first output image and one or more pixels in the representation of the second output image.

4. The method of claim 3, wherein said pixel difference comprises a representation of one or more cost volumes.

5. The method of claim 4, wherein evaluating the function comprises estimating a difference between (i) the representation of the one or more cost volumes associated with the first and second input images, and (ii) the representation of one or more cost volumes associated with the first and second output images.

6. The method of any of claims 2-5, wherein the difference between a pixel in the representation of the first input image and one or more pixels in the representation of the second input image, and / or the difference between a pixel in the representation of the first output image and one or more pixels in the representation of the second output image, is estimated for pixels in a predetermined pixel radius.

7. The method of any of claims 2-6, comprising performing a mean absolute difference calculation to estimate said pixel difference.

8. The method of any of claims 2-6, comprising performing a dot product calculation to estimate said pixel difference.

9. The method of any preceding claim, comprising producing the representations of the first and second input images for a plurality of spatial resolutions with said one or more neural networks.

10. The method of claim 9, comprising estimating a difference between (i) the representation of the one or more cost volumes associated with the first and second input images at each of said plurality of different spatial resolutions, and (ii) the representation of one or more cost volumes associated with the first and second output images at each said plurality of different spatial resolutions.

11. The method of any preceding claim, comprising: (i) encoding the first input image to produce a latent representation using a first neural network of the one or more neural networks; (ii) decoding the latent representation to produce the first output image using a second neural network of the one or more neural networks.

12. The method of claim 11, comprising: (iii) encoding the estimated optical flow information indicative of a difference between representations of the first and second input images to produce a latent representation using a third neural network of the one or more neural networks; (iv) decoding the latent representation to produce the estimated optical flow informationusing a fourth neural network of the one or more neural networks; and (v) using the optical flow information and the first output image to produce the second output image.

14. The method of claim 12 or 13, comprising: entropy encoding estimated optical flow information and / or the latent representation of the optical flow information into a bitstream, and entropy decoding the bitstream to produce the estimated optical flow information and / or latent representation of the optical flow information.

15. The method of claim 14, wherein the function further comprises a term based on a size of the bitstream.

16. The method of any preceding claim, wherein the estimated optical flow information indicative of a difference between a representation of the first and second input images comprises a difference between the latent representation of the first input image and a latent representation of the second input image.

17. The method of any preceding claim, wherein the estimated optical flow information indicative of a difference between a representation of the first and second output images comprises a difference between a latent representation of the first output image and a latent representation of the second output image.

18. A data processing system configured to perform the method of any one of claims 1 to 17.

19. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of claims 1 to 17.

20. A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information distributed according to a first distribution from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and a representation of the input image; transmitting the optical flow information to a second computer system; decoding the optical flow information distributed according to the first distribution using a second neural network; and using the previously decoded image and the decoded optical flow information to produce an output image, wherein the output image is an approximation of the input image.

21. The method of claim 20, wherein the transmitting of the optical flow information is performed without encoding the optical flow information into a latent representation.

22. The method of claim 20 or 21, comprising: entropy encoding the optical flow information distributed according to the first distribution into a bitstream, and transmitting the bitstream to the second computer system.

23. The method of claim 22, comprising: entropy decoding the bitstream at the second computer system to produce the optical flow information distributed according to the first distribution.

24. The method of any of claims 20-23, wherein estimating optical flow information comprises using the first neural network to produce the representation of the previously decoded image and the representation of the input image.

25. The method of any of claims 20-24, wherein estimating optical flow information comprises estimating a pixel difference between a pixel in the representation of the previously decoded image and one or more pixels in the representation of the input image.

26. The method of claim 25, wherein said pixel difference comprises a representation of one or more cost volumes.

27. The method of claim 26, wherein the one or more cost volumes are associated with representations the previously decoded image and the input image at a plurality of spatial resolutions.

28. The method of claims 26 or 27, comprising estimating the optical flow information from the representation of the one or more cost volumes.

29. The method of claim 28, wherein estimating the optical flow information from the representation of the one or more cost volumes is performed with the first neural network.

30. The method of any of claims 20-29, comprising producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation; transmitting the latent representation to the first computer system; and decoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image.

31. A method for lossy image or video encoding and transmission, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information distributed according to a first distribution from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and the representation of the input image; and transmitting the optical flow information to a second computer system.

32. The method of claim 31, wherein the transmitting of the optical flow information is performed without encoding the optical flow information into a latent representation.

33. The method of claim 31 or 32, comprising producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation; and decoding the latent representation using a fourth neural network to produce the previouslydecoded image, wherein the previously decoded image is an approximation of the starting image.

34. A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image and a previously decoded image; estimating optical flow information distributed according to a first distribution from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and a representation of the input image; decoding the optical flow information distributed according to the first distribution using a second neural network; and using the previously decoded image and the decoded optical flow information to produce an output image, wherein the output image is an approximation of the input image. evaluating a loss function; updating the parameters of the first and / or second neural networks based on the evaluated loss function; and repeating the above steps using a set of input images to produce first and / or second trained neural networks.

35. The method of claim 34, wherein the decoding of the optical flow information is performed without encoding the optical flow information into a latent representation.

36. The method of claim 34 or 35, comprising producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation; decoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image; and updating the parameters of the third and / or fourth neural networks based on the evaluated loss function.

37. A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and a representation of the input image; transmitting the optical flow information to a second computer system; decoding the optical flow information using a second neural network;and using the previously decoded image and the decoded optical flow information to produce an output image, wherein the output image is an approximation of the input image; wherein estimating the optical flow information comprises estimating differences between representations of the previously decoded image and representations of the input image in pixel search blocks of a plurality of sizes.

38. The method of claim 37, comprising estimating said differences for a plurality of spatial resolutions of the representations of the input image and the previously decoded image, wherein a pixel search block of a first size is used for a first spatial resolution, and a pixel search block of a second size is used for a second spatial resolution.

39. The method of claim 38, wherein the first size is larger than the second size, and the first spatial resolution is a lower spatial resolution than the second spatial resolution.

40. The method of claim 38 or 39, wherein the first and second sizes are defined by a pixel search radius around a center pixel coordinate of the pixel search block.

41. The method of claim 40, wherein the first size comprises a pixel search radius of 3.

42. The method of claim 41, wherein the second size comprises a pixel search radius of 1.

43. The method of any of claims 37-42, wherein the differences between the representations of the previously decoded image and representations of the input image comprise mean absolute differences.

44. The method of any of claims 37-43, wherein the difference between the representations of the previously decoded image and representations of the input image comprise cost volumes.

45. The method ofany of claims 37-44, comprising producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation;and decoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image.

46. The method of any of claims 37-45, comprising estimating said differences for a plurality of spatial resolutions of representations of the input image and the previously decoded image by sequentially performing the steps of: (i) estimating a difference associated with a first spatial resolution; (ii) upsampling the estimated difference to a second spatial resolution; (iii) estimating a difference associated with the second spatial resolution; and (iv) combining the upsampled difference estimation with the estimated difference.

47. The method of any of claims 37-46, comprising estimating said differences for a plurality of spatial resolutions of representations of the input image and the previously decoded image by performing the steps of: (i) estimating a difference associated with a first spatial resolution; (ii) estimating a difference associated with a second spatial resolution; (iii) upsampling, in parallel, the estimated differences associated with the first and second spatial resolutions; and (iv) combining the upsampled differences.

48. The method of any of claims 37-47, wherein estimating said differences for a plurality of spatial resolutions of representations of the input image and the previously decoded image comprises performing a warpless difference estimation.

49. A method for lossy image or video encoding and transmission, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information distributed according to a first distribution from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and the representation of the input image; and transmitting the optical flow information to a second computer system; wherein estimating the optical flow information comprises estimating differences between representations of the previously decoded image and representations of the input image in pixel search blocks of a plurality of sizes.

50. The method of claim 49, comprising estimating said differences for a plurality of spatial resolutions of the input image and the previously decoded image, wherein a pixel search block of a first size is used for a first spatial resolution, and a pixel search block of a second size is used for a second spatial resolution.

51. The method of claim 49 or 50, comprising producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation; anddecoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image.

52. A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and a representation of the input image, and said estimating the optical flow information comprising performing a warpless difference estimation; transmitting the optical flow information to a second computer system; decoding the optical flow information using a second neural network; and using the previously decoded image and the decoded optical flow information to produce an output image, wherein the output image is an approximation of the input image.

53. A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information from the previously decoded image and the input image using a first neural network, the optical flow information being indicative of a difference between a representation of the previously decoded image and a representation of the inputimage; transmitting the optical flow information to a second computer system; decoding the optical flow information using a second neural network;and using the previously decoded image and the decoded optical flow information to produce an output image, wherein the output image is an approximation of the input image; wherein estimating the optical flow information comprises estimating differences between representations of the previously decoded image and representations of the input image in at least one shaped pixel search block.

54. The method of claim 53, wherein estimating the optical flow information comprises estimat- ing differences between representations of the previously decoded image and representations of the input image in a plurality of pixel search blocks having different shapes.

55. The method of claim 54, wherein at least one of said shapes comprises a diamond shape.

56. The method of any of claims 53-55, comprising estimating said differences for a plurality of spatial resolutions of the input image and the previously decoded image, wherein a pixel search block of a first shape is used for a first spatial resolution, and a pixel search block of a second shape is used for a second spatial resolution.

57. The method of claim 56, wherein the shape of the pixel search block used for a highest spatial resolution comprises a diamond shape.

58. The method of claim 57, wherein the diamond shaped pixel search block used for a highest spatial resolution define a pixel search radius of 1.

59. The method of claim 58, wherein one or more shaped pixel search blocks other than the pixel search block used for the highest spatial resolution defines a pixel search radius of 3.

60. The method of any of claims 53-59, wherein the differences between the representations of the previously decoded image and representations of the input image comprise mean absolute differences.

61. The method of any of claims 53-60, wherein the difference between the representations of the previously decoded image and representations of the input image comprise cost volumes.

62. The method of any of claims 53-61, comprising producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation; and decoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image.

63. A method for lossy image or video encoding and transmission, the method comprising the steps of: receiving an input image and a previously decoded image at a first computer system; estimating optical flow information distributed according to a first distribution from the previously decoded image and the input image using a first neural network, the opticalflow information being indicative of a difference between a representation of the previously decoded image and the representation of the input image; and transmitting the optical flow information to a second computer system; wherein estimating the optical flow information comprises estimating differences between representations of the previously decoded image and representations of the input image in at least one shaped pixel search block.

64. The method of claim 63, wherein estimating the optical flow information comprises estimat- ing differences between representations of the previously decoded image and representations of the input image in a plurality of pixel search blocks having different shapes.

65. The method of claim 64, wherein at least one of said shapes comprises a diamond shape.

65. The method of any of claims 63 to 65, comprising estimating said differences for a plurality of spatial resolutions of the input image and the previously decoded image, wherein a pixel search block of a first shape is used for a first spatial resolution, and a pixel search block of a second shape is used for a second spatial resolution.

66. The method of claim 65, wherein the shape of the pixel search block used for a highest spatial resolution comprises a diamond shape.

67. The method of any of claims 63 to 66, comprising producing the previously decoded image by: encoding a starting image using a third neural network to produce a latent representation;and decoding the latent representation using a fourth neural network to produce the previously decoded image, wherein the previously decoded image is an approximation of the starting image.

68. A data processing system configured to perform the method of any one of claims 53 to 67.

69. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of claims 53 to 67.

70. A method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image at a first computer system; with a first neural network, producing first optical flow information using the first image and the second image, each represented at a first resolution, the first optical flow information being indicative of a difference between the first image and the second image; with a second neural network, producing second optical flow information using the first image and the second image, each represented at a second resolution, and using the first optical flow information, the second optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the second optical flow information to a second computer system; with a third neural network, decoding the information representing the second optical flow information to produce an approximation of the second optical flow information; andwith a fourth neural network, producing an output image using the second optical flow information, wherein the output image is an approximation of the first image.

71. The method of claim 70, comprising downsampling the first image and the second image to the first resolution.

72. The method of claim 70 or 71, wherein the second resolution is greater than the first resolution.

73. The method of any of claims 70 to 72, wherein the first optical flow information is stored as an integer data type having a dynamic range corresponding to a coarse motion range, and wherein the second optical flow information is stored as an integer data type having a dynamic range corresponding to a fine motion range.

74. The method claim 73, wherein the coarse information range is greater than the fine motion range.

75. The method of claim 73 or 74, wherein the integer data type is an int8 data type.

76. The method of any of claims 70 to 75, wherein using the first optical flow information comprises warping the second image with the first optical flow information, and wherein producing the second optical flow information comprises using the warped second image, the second optical flow information being indicative of a difference between the first image and the warped second image.

77. The method of claim 76, comprising estimating a difference between the first image and the warped second image, and using the difference to produce the second optical flow information.

78. The method of claim 77, wherein the difference comprises a brightness difference.

79. The method of claim 77 or 78, wherein using the difference comprises convolving a representation of the difference with one or more outputs of one or more layers of the second neural network.

80. The method of any of claims 70 to 79, comprising downsampling the first image and the second image to the second resolution.

81. The method of any of claims 70 to 80, comprising feeding one or more outputs of one or more layers of the first neural network forward to the second neural network, and wherein producing the second optical flow information comprises using the one or more outputs of one or more layers of the first neural network.

82. The method of claim 81, wherein using the one or more outputs of one or more layers of the first neural network comprises convolving the one or more outputs with one or more outputs of one or more layers of the second neural network.

83. A method for lossy image or video encoding and transmission, the method comprising the steps of: receiving a first image and a second image at a first computer system; with a first neural network, producing first optical flow information using the first imageand the second image, each represented at a first resolution, the first optical flow information being indicative of a difference between the first image and the second image; with a second neural network, producing second optical flow information using the first image and the second image, each represented at a second resolution, and using the first optical flow information, the second optical flow information being indicative of a difference between the first image and the second image; and transmitting information representing the second optical flow information to a second computer system.

84. A method for lossy image or video receipt and decoding, the method comprising the steps of: receiving information representing second optical flow information at a second computer system, the second optical flow information produced with a second neural network using a first and a second image, each represented at a second resolution, and using first optical flow information, the second optical flow information being indicative of a difference between the first image and the second image; with a third neural network, decoding the information representing the second optical flow information to produce an approximation of the second optical flow information; and with a fourth neural network, producing an output image using the second optical flow information, wherein the output image is an approximation of the first image.

85. A method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of:receiving a first image and a second image at a first computer system; with a first neural network, producing first optical flow information using the first image and the second image, each represented at a first resolution, the first optical flow information being indicative of a difference between the first image and the second image; with a second neural network, producing second optical flow information using the first image and the second image, each represented at a second resolution, and using the first optical flow information, the second optical flow information being indicative of a difference between the first image and the second image; with a fifth neural network, producing third optical flow information using the first image and the second image, each represented at a third resolution, and using the second optical flow information, the third optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the third optical flow information to a second computer system; transmitting information representing the third optical flow information to a second computer system; with a third neural network, decoding the information representing the third optical flow information to produce an approximation of the third optical flow information; and with a fourth neural network, producing an output image using the third optical flow information, wherein the output image is an approximation of the first image.

86. The method of claim 85, wherein the third resolution is greater than the first resolution and second resolution.

87. The method of any of claims 85 to 86, wherein the first optical flow information is stored as an integer data type having a dynamic range corresponding to a first coarseness of motion range, wherein the second optical flow information is stored as an integer data type having a dynamic range corresponding to second coarseness of motion range, and wherein the third optical flow information is stored as an integer data type having a dynamic range corresponding to a third coarseness of motion range.

88. The method of claim 87, wherein the integer data type is an int8 data type.

89. The method of any of claims 85 to 88, wherein the first coarseness of motion range is greater than the second coarseness of motion range, and wherein the second coarseness of motion range is greater than the third coarseness of motion range.

90. A data processing apparatus configured to perform the method of any of claims 70 to 89.

91. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of claims 70 to 89.

92. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any of claims 70 to 89.

93. A method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computersystem; with a first neural network, producing first optical flow information using the first image and the second image, the first optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the first optical flow information to a second computer system; with a second neural network and using second optical flow information indicative of difference between the second image and a third image of the sequence, decoding the information representing the first optical flow information to produce an approximation of the first optical flow information; and with a third neural network, producing an output image using the first optical flow information, wherein the output image is an approximation of the first image.

94. The method of claim 93, wherein using the second optical flow information comprises producing first entropy parameters indicative of a distribution of the information representing the first optical flow information.

95. The method of any of claims 93 to 94, comprising entropy encoding and entropy decoding the information representing the first optical flow information using the first entropy parameters 96. The method of any of claims to 93 to 95, comprising, with a fourth neural network, producing a hyper latent representation of the first optical flow information using the first optical flow information and using the second optical flow information.

97. The method of claim 96, with a fifth neural network producing the first entropy parameters by decoding the hyper latent representation.

98. The method of any of claims 96 to 97, comprising: with a sixth neural network producing a hyper hyper latent representation of the first optical flow information using the hyper latent representation of the first optical flow information and using the second optical flow information.

99. The method of any of claim 96 to 98, wherein using the second optical flow information comprises producing second entropy parameters indicative of a distribution of the hyper latent representation.

100. The method of claim 99, comprising entropy encoding and entropy decoding the hyper latent representation using the second entropy parameters.

101. The method of any of claims 96 to 101, wherein using the second optical flow information comprises conditioning the hyper latent representation produced by the fourth neural network on the second optical flow information.

103. The method of claim 102, wherein said conditioning comprises combining the second optical flow information and the information representing the first optical flow information using one or more convolution operations and / or one or more concatenation operations, and inputting the combined information into the fourth neural network.

104. The method of claim 98, wherein using the second optical flow information comprises conditioning the hyper hyper latent representation produced by the sixth neural network on the second optical flow information.

105. The method of claim 104 wherein said conditioning comprises combining the second optical flow information and the hyper latent representation using one or more convolution operations and / or one ormore concatenation operations, and inputting the combined information into the sixth neural network.

106. The method of 99 comprising producing the second entropy parameters using the second optical flow information without using a hyper hyper latent representation.

107. The method of any of claims 93 to 106, wherein the second optical flow information comprises one or more outputs of one or more intermediate layers of the first neural network.

108. The method of any of claims 93 to 107, comprising identifying the first image and the second image as a chronologically first frame pair of the image sequence, and using synthetic second optical flow information as the second optical flow information.

109. The method of any of claims 93 to 108, comprising identifying the first image and the second image as a chronologically first frame pair of the image sequence, and encoding, transmitting and decoding the first image without using second optical flow information.

110. A method for lossy image or video receipt and decoding, the method comprising the steps of:receiving information representing first optical flow information at a second computer system, the first optical flow information produced using a first neural network and being indicative of a difference between a first image and a second image of an image sequence; with a second neural network and using second optical flow information indicative of difference between the second image and a third image of the sequence, decoding the information representing the first optical flow information to produce an approximation of the first optical flow information; and with a third neural network, producing an output image using the first optical flow information, wherein the output image is an approximation of the first image.

111. A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence; with a first neural network, producing first optical flow information using the first image and the second image, the first optical flow information being indicative of a difference between the first image and the second image; with a second neural network and using second optical flow information indicative of difference between the second image and a third image of the sequence, decoding the information representing the first optical flow information to produce an approximation of the first optical flow information; with a third neural network, producing an output image using the first optical flow information, wherein the output image is an approximation of the first image;evaluating a function based on a difference between the estimated first image and the output image; updating the parameters of the first, second and / or third networks based on the evaluated function; and repeating the above steps using a first set of image sequences to produce trained first, second and third neural networks.

110. The method of claim 109, wherein the first set of image sequences comprises at least one image sequence with a group of picture (GOP) size of at least four frames.

111. A data processing apparatus configured to perform the method of any of claims 93 to 110.

112. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of claims 93 to 110.

113. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any of claims 93 to 110.

114. A method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing actual first optical flow information using the first image, the second image, and predicted optical flow information, the actual first optical flowinformation being indicative of a difference between the first image and the second image, and the predicted optical flow information being based on second optical flow information indicative of a difference between the second image and a third image of the sequence; transmitting information representing the actual first optical flow information to a second computer system; with a second neural network, decoding the information representing the actual first optical flow information to produce an approximation of the actual first optical flow information; and with a third neural network, producing an output image using the actual first optical flow information, wherein the output image is an approximation of the first image.

115. The method of claim 114, comprising producing the predicted optical flow information by performing a forward warping operation on the second optical flow information.

116. The method of claim 115, wherein the forward warping operation comprises a soft splat operation.

117. The method of any of claims 114 to 116, wherein the second optical flow information is further indicative of a difference between the third image of the sequence and a fourth image of the sequence.

118. The method of any of claims 114 to 117, comprising conditioning the first actual optical flow information produced by the first neural network on the predicted optical flow information.

119. The method of claim 118, wherein said conditioning comprises combining the predicted optical flow information with the first image and / or the second image by performing a convolution operation and / or a concatenation operation before performing a forward pass of the first neural network.

120. A method for lossy image or video encoding and transmission, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing actual first optical flow information using the first image, the second image, and predicted optical flow information, the actual first optical flow information being indicative of a difference between the first image and the second image, and the predicted optical flow information being based on second optical flow information indicative of a difference between the second image and a third image of the sequence; transmitting information representing the actual first optical flow information to a second computer system.

121. A method for lossy image or video receipt and decoding, the method comprising the steps of: receiving information representing actual first optical flow information at a second computer system, the actual first optical flow information produced using a first image, a second image, and predicted optical flow information, the actual first optical flow information being indicative of a difference between the first image and the second image, and the predictedoptical flow information being based on second optical flow information indicative of a difference between the second image and a third image of the sequence; with a second neural network, decoding the information representing the actual first optical flow information to produce an approximation of the actual first optical flow information; and with a third neural network, producing an output image using the actual first optical flow information, wherein the output image is an approximation of the first image.

122. A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence; with a first neural network, producing actual first optical flow information using the first image, the second image, and predicted optical flow information, the actual first optical flow information being indicative of a difference between the first image and the second image, and the predicted optical flow information being based on second optical flow information indicative of a difference between the second image and a third image of the sequence; with a second neural network, decoding the information representing the actual first optical flow information to produce an approximation of the actual first optical flow information; and with a third neural network, producing an output image using the actual first optical flow information, wherein the output image is an approximation of the first image. evaluating a function based on a difference between the estimated first image and theoutput image; updating the parameters of the first, second and / or third networks based on the evaluated function; and repeating the above steps using a first set of image sequences to produce trained first, second and third neural networks.

123. The method of claim 122, wherein the first set of image sequences comprises at least one image sequence with a group of picture (GOP) size of at least four frames.

124. A data processing apparatus configured to perform the method of any of claims 114 to 123.

125. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of claims 114 to 123.

126. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any of claims 114 to 123.

127. A method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image and the second image, the optical flow information being indicative of a difference between thefirst image and the second image; transmitting information representing the optical flow information to a second computer system; with a second neural network decoding the information representing the optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image, wherein said step of producing optical flow information comprises performing a pooling operation on a representation of said optical flow information.

128. The method of claim 127, wherein the pooling operation converts the representation of said optical flow information into a motion descriptor indicative of global motion between the first image and the second image.

129. The method of claim 127 or 128, comprising performing the pooling operation on a hyper latent representation of the optical flow information.

130. The method of claim 129, comprising combining the hyper latent representation of the optical flow information with a representation of pixel location information before performing said pooling operation.

131. The method of any of claims 129-130, comprising performing an unpooling operation on a reconstruction of the hyper latent representation of the optical flow information.

132. The method of claim 131, comprising combining the reconstruction of the hyper latent representation of the optical flow information with a representation of pixel location information after performing the unpooling operation.

133. The method of claim 130 or 132, wherein the representation of pixel position information comprises a positional embedding. 134.The method of claim 133, comprising producing the positional embedding by embedding a tensor representing pixel coordinates associated with the first image or second image with a fourth neural network.

135. A method for lossy image or video encoding and transmission, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the optical flow information to a second computer system; wherein said step of producing optical flow information comprises performing a pooling operation on a representation of said optical flow information.

136. A method for lossy image or video receipt and decoding, the method comprising the steps of: receiving at a second computer system information representing optical flow information indicative of a difference between a first image and a second image of an image sequence, the representation of optical flow information produced with a first neural network using the first and the second image; with a second neural network decoding the information representing the optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image, wherein said step of decoding comprises performing an unpooling operation on a representation of said optical flow information.

137. A data processing apparatus configured to perform the method of any of claims 127 to 136.

138. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of claims 127 to 136.

139. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any of claims 127 to 136.

140. A method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of:receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the optical flow information to a second computer system; with a second neural network decoding the information representing the optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image, wherein said step of producing optical flow information is based on a representation of pixel position information.

141. The method of claim 140, wherein the representation of pixel position information comprises a positional embedding. 142.The method of claim 141, comprising producing the positional embedding by embedding a tensor representing pixel coordinates with a fourth neural network.

143. The method of any of claims 140 to 142, producing said optical flow information from representations of the first image and the second image downsampled to a plurality of resolutions.

144. The method of claim 143, comprising downsampling a representation of the first image and the second image to produce said representations of the first image and the second image at said plurality of resolutions.

145. The method of claim 144, comprising combining the pixel position information with one or more of said representations of the first image and the second image at said plurality of resolutions.

146. The method of claim 145, comprising reshaping the pixel position information before said combining.

147. The method of any of claims 145 to 146, wherein said combining comprises performing an element-wise addition of the pixel position information with corresponding representations of the first image and the second image at said plurality of resolutions.

148. A method for lossy image or video encoding and transmission,the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the optical flow information to a second computer system;wherein said step of producing optical flow information is based on a representation of pixel position information.

149. A method for lossy image or video receipt and decoding, the method comprising the steps of: receiving at a second computer system information representing optical flow information indicative of a difference between a first image and a second image of an image sequence, the representation of optical flow information produced with a first neural network using the first and the second image and pixel position information; with a second neural network decoding the information representing the optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image.

150. A data processing apparatus configured to perform the method of any of claims 140 to 149.

151. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of claims 140 to 149.

152. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any of claims 140 to 149.

153. A method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the optical flow information to a second computer system; with a second neural network decoding the information representing the optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image, wherein producing the optical flow information comprises producing, in parallel, feature maps from respective representations of the first image at different resolutions.

154. The method of claim 153, wherein producing the optical flow information comprises producing, in parallel, feature maps from respective representations of the second image at different resolutions.

155. The method of claim 154, wherein producing the optical flow information comprises combining the feature maps from the first image with the corresponding feature maps from thesecond image to estimate, at the plurality of resolutions, the difference between the first image and the second image.

156. The method of claim 155, wherein said combining comprises producing one or more cost volumes indicative of the difference between the first image and the second image.

157. The method of claim 156, comprising using the estimated difference between the first image and the second image at a first resolution to warp the first image or the second image at a second resolution, and to use the warped first image or second image to estimate the difference between the first image and the second image at the second resolution.

158. The method of any of claims 153 to 157, wherein the first neural network comprises a plurality of parallel process streams each defined by one or more non-sequential layers of the first neural network.

159. The method of any of claims 153 to 158, comprising producing the feature maps asynchronously.

160. The method of claim 158 or 159, wherein the one or more layers of the first neural network together define a planar shaped feature extractor.

161. The method of any of claims 158 to 160, comprising producing the representations of the first image at the different resolutions by performing one or more downsampling operations on the first image, and feeding the respective outputs of the one or more downsampling operationsdirectly into the respective process streams of the plurality of parallel process streams to produce the plurality of feature maps asynchronously.

162. The method of any of claims 153 to 161, wherein each feature map is produced directly from the first image at a respective one of said different resolutions to increase the amount of first image information retained in each feature map compared to a method of producing feature maps using a pyramid shaped feature extractor.

163. A method for lossy image or video encoding and transmission, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting information representing the optical flow information to a second computer system; wherein producing the optical flow information comprises producing, in parallel, feature maps from respective representations of the first image at different resolutions.

164. A method for lossy image or video receipt and decoding, the method comprising the steps of: receiving at a second computer system information representing optical flow information indicative of a difference between a first image and a second image of an image sequence, therepresentation of optical flow information produced with a first neural network and based on one or more feature maps produced using respective feature maps produced in parallel from respective representations of the first image at different resolutions; with a second neural network decoding the information representing the optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image.

165. A data processing apparatus configured to perform the method of any of claims 153 to 164.

166. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of claims 153 to 164.

167. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any of claims 153 to 164.

168. A method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image, the second image, and a previously decoded representation of the second image, the optical flow information being indicative of a difference between the first image and the second image;transmitting a representation of the optical flow information to a second computer system; with a second neural network decoding the representation of optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image.

169. The method of claim 168, wherein the step of producing the optical flow information using the first image, the second image, and the previously decoded representation of the second image comprises: (i) producing a blended image by combining the second image and the previously decoded representation of the second image, and (ii) producing the optical flow information by estimating a difference between the first image and the blended image.

170. The method of claim 168 or 169, wherein the blended image comprises a convex combination of the second image and the previously decoded representation of the second image.

171. The method of claim any of claims 168 to 170, wherein producing the blended image comprises, with a fourth neural network, producing a mask using the second image and the previously decoded representation of the second image, and performing a weighted combination of the second image and the previously decoded representation of the second image using the mask.

172. The method of any of claims 168 to 171, wherein an amount of the second image and an amount of the previously decoded representation of the second image contributing to the blended image is correlated with an amount of motion in the image sequence.

173. The method of any claims 168 to 171, wherein producing the optical flow information further comprises using a third image of the image sequence and a previously decoded representation of the third image.

173. The method of claim 172, wherein the step of producing the optical flow information using the first image, the second image, the previously decoded representation of the second image, the third image, and the previously decoded representation of the third image comprises: (i) producing a blended image by combining the second image, the previously decoded representation of the second image, the third image, and the previously decoded representation of the third image, and (ii) producing the optical flow information by estimating a difference between the first image and the blended image.

174. The method of claim 173, wherein the blended image comprises a convex combination of the second image, the previously decoded representation of the second image, the third image, and the previously decoded representation of the third image.

175. The method of any of claims 168 to 174, wherein the first image, the second image, and / or the previously decoded representation thereof are represented in a YUV colour space, and wherein producing the optical flow information further comprises extracting a luma channel from one or more of the first image, the second image, and / or the previously decodedrepresentation of the first image, and using the extracted luma channel(s) to produce the optical flow information.

176. A method for lossy image or video encoding and transmission, the method comprising the steps of: receiving a first image and a second image of an image sequence at a first computer system; with a first neural network, producing optical flow information using the first image, the second image, and a previously decoded representation of the second image, the optical flow information being indicative of a difference between the first image and the second image; and transmitting a representation of the optical flow information to a second computer system.

177. A method for lossy image or video receipt and decoding, the method comprising the steps of: receiving at a second computer system a representation of optical flow information, the optical flow information being indicative of a difference between a first image and a second image of an image sequence and produced with a first neural network using the first image, the second image, and a previously decoded representation of the second image; with a second neural network decoding the representation of optical flow information to produce an approximation of the optical flow information; and with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image.

178. A data processing apparatus configured to perform the method of any of claims 168 to 177.

179. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of claims 168 to 177.

180. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method of any of claims 168 to 177.

Images