Optical flow guided knowledge distillation video prediction model compression method and system

Abstract

This invention discloses a video prediction model compression method and system based on optical flow-guided knowledge distillation, belonging to the field of computer vision technology. The method includes: generating optical flow distillation loss by constraining the optical flow prediction distribution of the student network to be consistent with that of the teacher network; generating channel alignment loss by aligning the channel attention distributions of the teacher and student networks using divergence; generating pixel-level reconstruction loss based on the difference in pixel values ​​between the generated image predicted by the student network and the real image; generating perceptual loss based on the difference in high-level semantic feature space between the generated image predicted by the student network and the real image; and obtaining the trained student network based on the optical flow distillation loss, channel alignment loss, pixel-level reconstruction loss, and perceptual loss. This invention can solve the performance degradation problem caused by neglecting spatiotemporal characteristics in existing compression methods for video prediction tasks.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a knowledge distillation video prediction model compression method and system based on optical flow guidance. Background Technology

[0002] With the development of deep learning, video prediction models play a crucial role in real-time systems such as autonomous driving, video surveillance, and remote collaboration. However, existing high-performance video prediction models (such as convolutional models based on recurrent neural networks or optical flow prediction) typically have a large number of parameters and complex temporal dependencies, resulting in high computational overhead and inference latency, making them difficult to deploy efficiently on resource-constrained devices. For example, in remote collaboration systems, the real-time performance of video prediction directly impacts the interactive experience, and the inference latency of traditional models often cannot meet the demands of instant transmission. Therefore, how to effectively compress video prediction models without significantly reducing prediction accuracy has become a critical problem that urgently needs to be solved.

[0003] Existing model compression methods (such as quantization, pruning, knowledge distillation, etc.) have achieved significant results in tasks such as image classification and object detection, but their applicability in video prediction tasks is still significantly insufficient. Video prediction models need to model the motion information (such as optical flow) between frames and the temporal dependencies of multiple frames, and their computation process is fundamentally different from that of static image tasks. If general compression methods are directly applied, it often leads to a decrease in the model's ability to capture motion features, and problems such as fuzzy prediction results or temporal inconsistencies. For example, traditional knowledge distillation methods are mainly designed for classification tasks, and compression is achieved by matching the output distribution or intermediate features of teacher and student networks, but the spatiotemporal dynamic characteristics of video data are ignored. Specifically, existing methods have the following limitations: (1) lack of targeted distillation of motion information such as optical flow, which makes it difficult for the compressed model to accurately predict changes between frames; (2) not fully considering the multi-scale temporal dependency structure of video prediction models, which may destroy the key motion modeling ability during the compression process; (3) the distillation target is singular, usually only focusing on pixel-level matching of output frames, while ignoring the constraints on motion consistency and temporal coherence.

[0004] To address the aforementioned issues, there is an urgent need for a model compression method specifically designed for video prediction tasks, which can reduce model complexity while maintaining its efficient ability to model temporal dynamics. Summary of the Invention

[0005] This invention provides a knowledge distillation-based video prediction model compression method and system guided by optical flow. By introducing optical flow information and channel distribution alignment strategies, it achieves efficient compression of the video prediction model, reducing the number of model parameters while maintaining its ability to predict temporal dynamics. This invention can be applied to scenarios requiring real-time video prediction, such as autonomous driving, video surveillance, and remote collaboration, solving the performance degradation problem caused by neglecting spatiotemporal characteristics in existing compression methods for video prediction tasks.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A knowledge distillation video prediction model compression method based on optical flow guidance, the method comprising: By constraining the optical flow prediction distribution of the student network to be consistent with that of the teacher network, optical flow distillation loss is generated. use Divergence aligns the channel attention distributions of the teacher and student networks, generating channel alignment loss; Pixel-level reconstruction loss is generated based on the difference in pixel values ​​between the generated image predicted by the student network and the real image. Generate perceptual loss based on the difference between generated images and real images in the high-level semantic feature space predicted by the student network; The trained student network is obtained based on optical flow distillation loss, channel alignment loss, pixel-level reconstruction loss, and perceptual loss.

[0007] Furthermore, the teacher network includes The student network includes a basic module. A basic module, The basic module includes an optical flow feature encoding input branch, an image feature encoding input branch, a downsampling unit, a stitching unit, a first convolution module, a second convolution module, a deconvolution module, a splitting unit, an optical flow feature encoding output branch, and a mask output branch.

[0008] Furthermore, the process of video prediction based on the trained student network includes: Get the Frame image, the first Frame image, the first Frame image and the first Frame image to the Optical flow changes in frame images, the first Frame image to the Optical flow changes in frame images, the first Frame image to the Optical flow changes in frame images, the first Frame image to the Optical flow changes in the frame image are input; The first Frame image, the first Frame image and the first Frame image to the Optical flow changes in frame images, the first Frame image to the The optical flow changes of the frame image are input into the first basic module to obtain the second... Image prediction results for the frame image; The first Frame image, the first The output of the first basic module Image prediction results of frame image, the first Frame image to the Optical flow changes in frame images, the first Frame image to the Optical flow changes of the frame image input number The first basic module, resulting in the first... The first output of the basic module Image prediction results for frame images; where, ; The first Frame image, the first The output of the first basic module Image prediction results of frame image, the first Frame image to the Optical flow changes in frame images, the first Frame image to the Optical flow changes of the frame image input number The first basic module, resulting in the first... Image prediction results for frame images.

[0009] Furthermore, by constraining the optical flow prediction distribution of the student network to be consistent with that of the teacher network, an optical flow distillation loss is generated, including: Obtain the long-term optical flow generated by the teacher network respectively , short-term optical flow and long-term optical flow generated by student networks , short-term optical flow ; Long-term optical flow , short-term optical flow Long-term optical flow and short-term optical flow The first in Each channel is spatially normalized to obtain the corresponding spatial probability map. Spatial probability graph Spatial probability graph Spatial probability graph ; Based on spatial probability graphs Spatial Probability Map Calculate the first Short-term optical flow of each channel Divergence; Based on spatial probability graphs Spatial Probability Map Calculate the first Long-term optical flow of each channel Divergence; Short-term optical flow of all channels Divergence and Long-Term Optical Flow The average divergence is used to obtain the optical flow distillation loss.

[0010] Furthermore, the channel alignment loss ;in, Indicates the total number of channels. This indicates that the teacher network is in the channel. probability distribution on, Indicates student network in channel probability distribution on, probability distribution With probability distribution divergence between , , These represent the height and width of the feature map, respectively. , These represent the coordinate indices of the feature map in the horizontal and vertical directions, respectively.

[0011] Furthermore, the pixel-level reconstruction loss ;in, This represents the generated image based on student network predictions. Represents a real image. Indicates the height of the generated image and the real image. This represents the width of the generated image and the real image. Describing the L1 norm, This represents the normalization factor.

[0012] Furthermore, the perceived loss ;in, Indicates the first Feature extraction network of layer feature maps, Indicates the first The number of channels in the layer feature map. Indicates the first The height of the layer feature map, Indicates the first The width of the layer feature map, Denotes the square of the L2 norm. This represents the generated image based on student network predictions. Represents a real image.

[0013] A knowledge distillation video prediction model compression system based on optical flow guidance, the system comprising: The optical flow distillation loss generation module is used to generate optical flow distillation loss by constraining the optical flow prediction distribution of the student network to be consistent with that of the teacher network. The channel alignment loss generation module is used to utilize... Divergence aligns the channel attention distributions of the teacher and student networks, generating channel alignment loss; The pixel-level reconstruction loss generation module is used to generate pixel-level reconstruction loss based on the difference in pixel values ​​between the generated image predicted by the student network and the real image. The perceptual loss generation module is used to generate perceptual loss based on the difference between the generated image predicted by the student network and the real image in the high-level semantic feature space. The multi-loss joint optimization module is used to obtain the trained student network based on optical flow distillation loss, channel alignment loss, pixel-level reconstruction loss, and perceptual loss.

[0014] A computer device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the optical flow-guided knowledge distillation video prediction model compression method described above.

[0015] A computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the optical flow-guided knowledge distillation video prediction model compression method described above.

[0016] Compared with the prior art, the present invention has at least the following technical effects.

[0017] 1) This invention explicitly models inter-frame motion information in video through an optical flow-guided distillation mechanism, thus solving the problem of motion feature loss in video tasks by general distillation methods; 2) The channel distribution alignment strategy of the present invention enables efficient transfer of spatial attention knowledge, allowing the student network to accurately focus on key motion regions; 3) The compressed student model has 60-75% fewer parameters and 2-3 times faster inference speed, while maintaining over 90% prediction accuracy. 4) This invention is applicable to various deep learning-based video prediction models and has wide applicability and scalability. Attached Figure Description

[0018] Figure 1 This is the network architecture diagram of the present invention.

[0019] Figure 2 This is a basic module structure diagram of the video prediction algorithm.

[0020] Figure 3 This is a flowchart of a knowledge distillation video prediction model compression method based on optical flow guidance.

[0021] Figure 4 This is a block diagram of a knowledge distillation video prediction model compression system based on optical flow guidance.

[0022] Figure 5 It is a block diagram of computer equipment. Detailed Implementation

[0023] The present invention will now be described in further detail with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0024] The knowledge distillation model compression system based on optical flow guidance proposed in this invention adopts a "teacher-student" network architecture, such as... Figure 1 As shown, the teacher network is a high-performance but structurally complex video prediction model, while the student network is a lightweight model with approximately one-quarter of the number of parameters as the teacher network. During training, the parameters of the teacher network remain constant, while the student network learns from random initialization.

[0025] The input to the video prediction network is the first... Frame, First Frame and the Three consecutive frames of images, and the first... The H.264 encoded information of the frame image is processed by a convolutional module and converted into optical flow feature codes. The three consecutive frame images and the optical flow feature codes are used as network input to generate the first... The predicted image of the frame. The teacher network consists of 9 cascaded basic modules, and the student network consists of 5 cascaded basic modules.

[0026] Figure 2 The diagram illustrates the basic module structure that constitutes the teacher and student models. The basic modules shown are used to... Frame image and the first Frame images, predicting the two frames relative to the first Light motion offset of frame image ( The image is then fused using a reverse warp operation to generate the first... The predicted image of the frame.

[0027] The inputs to the basic module include: optical flow feature encoding. and , used to indicate the first Frame image and the first Optical flow prediction components in the horizontal and vertical directions of the frame image; and image feature encoding for representing the first frame. Frame image, initial prediction of the first The frame image, and the first frame obtained after motion compensation of the two. Frame prediction results. The above input features are sequentially processed by the first convolutional module to extract two-dimensional features, then by the second convolutional module to further extract two-dimensional features, and finally by the deconvolutional module to generate a two-dimensional feature map with 5 channels. The two-dimensional feature map is split along the channel dimension, and optical flow prediction codes are output separately. and And the corresponding fusion mask.

[0028] The video prediction network uses the first basic module based on the... Frame and the Frame image and the two frames to the first The optical flow changes of the frame image predict the first Frame image; subsequent basic modules are based on the first Frame image and the first Frame image and the two frames to the first The optical flow changes of the frame image predict the first Frame image.

[0029] The video prediction network employs multiple cascaded basic modules for progressive prediction. The first basic module is based on the... Frame and the Frame image, and combined with the two frames relative to the first The optical flow changes of the frame image generate the first The initial predicted image of the frame; subsequent basic modules are based on the output of the previous basic module. Frame prediction image and the first The original image of frame 1 is used as input, and the two images are combined to form the second frame 2. The optical flow changes of the frame image, for the first The frame-by-frame predicted image is progressively corrected and optimized to finally obtain the first frame. The prediction results for the frames.

[0030] During the training phase, both the teacher and student networks perform forward inference simultaneously and output the corresponding optical flow component prediction results at each basic module. The predicted optical flow field output by the teacher network serves as a soft objective, guiding the training process of the student network through a pre-designed distillation loss function.

[0031] Specifically, the knowledge distillation video prediction model compression method based on optical flow guidance of the present invention, such as... Figure 3 As shown, it includes the following steps.

[0032] Step 1: Generate optical flow distillation loss by constraining the optical flow prediction distribution of the student network to be consistent with that of the teacher network.

[0033] The calculation of optical flow distillation loss aims to make the optical flow prediction of the student network approximate the teacher network in terms of spatial probability distribution through knowledge distillation, thereby conveying accurate motion pattern knowledge.

[0034] (1) Obtain the optical flow output of the teacher and student network.

[0035] Obtaining the long-term optical flow generated by the teacher network With short-term optical flow and long-term optical flow generated by student networks With short-term optical flow In one embodiment, each optical flow field has 2 channels (channel 1: horizontal displacement component; channel 2: vertical displacement component) and a spatial dimension of H×W. Therefore, the number of channels C=2.

[0036] (2) Perform spatial probability normalization on each optical flow channel.

[0037] This invention addresses the short-term and long-term optical flow of each channel for both teachers and students. Spatial Softmax normalization is performed to convert the activation values ​​within the channels into a spatial probability distribution map.

[0038] For teacher networks, their channels Spatial probability distribution of short- and medium-term optical flow for: For teacher networks, their channels Spatial probability distribution of medium- and long-term optical flow for: For student networks, their channels Spatial probability distribution of short- and medium-term optical flow for: For teacher networks, their channels Spatial probability distribution of medium- and long-term optical flow for: in, This is the temperature coefficient.

[0039] (3) Calculate the KL divergence.

[0040] Calculate the KL divergence between the teacher and student distributions in the corresponding channels for short-term and long-term optical flow, respectively.

[0041] Regarding short-term optical flow, based on spatial probability distribution maps Spatial probability distribution map To calculate the KL divergence of the short-term optical flow probability distribution : Regarding long-term optical flow, based on spatial probability distribution maps Spatial probability distribution map To calculate the KL divergence of the short-term optical flow probability distribution : (4) Generation of optical flow distillation loss.

[0042] The optical flow distillation loss is obtained by averaging the KL divergence across all channels. : in, Indicates the total number of channels.

[0043] In summary, the optical flow-guided distillation mechanism of this invention constitutes the core innovation of this invention, and its specific implementation steps are as follows: After processing the input frame sequence, the teacher network generates an optical flow field for reconstructing the current frame during the intermediate decoding stage. The optical flow field includes short-term optical flow and long-term optical flow. The student network accordingly estimates the corresponding optical flow field for the same input data. For each optical flow component in the optical flow field, normalization operations are applied to the horizontal and vertical components respectively, converting the feature values ​​in each channel into a probability distribution, and introducing a temperature coefficient... The smoothness of the probability distribution is adjusted to control the degree of sharpening of the distribution.

[0044] Step 2: Utilize Divergence aligns the channel attention distributions of the teacher and student networks, generating a channel alignment loss.

[0045] This invention calculates the divergence between the probability distributions of the teacher network and the student network on corresponding channels, and then averages the divergence across all channels, defining it as the channel alignment distillation loss. By minimizing this loss function, the student network learns the attention distribution characteristics of the teacher network in the spatial dimension, thereby focusing on key image regions consistent with the teacher network, while avoiding the optimization instability problem caused by directly matching feature values.

[0046] in, Denotes KL divergence, This indicates that the teacher network is in the channel. probability distribution on, Indicates student network in channel probability distribution on, , These represent the height and width of the feature map, respectively. , These represent the coordinate indices of the feature map in the horizontal and vertical directions, respectively. Indicates the total number of channels.

[0047] Step 3: Generate pixel-level reconstruction loss based on the difference in pixel values ​​between the generated image predicted by the student network and the real image.

[0048] Pixel-level reconstruction loss of the present invention ;in, This represents the generated image based on student network predictions. Represents a real image. Indicates the height of the generated image and the real image. This represents the width of the generated image and the real image. Describing the L1 norm, This represents the normalization factor.

[0049] Step 4: Generate perceptual loss based on the difference between the generated image predicted by the student network and the real image in the high-level semantic feature space.

[0050] The perceived loss of the present invention ;in, Indicates the first Feature extraction network of layer feature maps, Indicates the first The number of channels in the layer feature map. Indicates the first The height of the layer feature map, Indicates the first The width of the layer feature map, Denotes the square of the L2 norm. This represents the generated image based on student network predictions. Represents a real image.

[0051] Step 5: Based on optical flow distillation loss, channel alignment loss, pixel-level reconstruction loss, and perceptual loss, the trained student network is obtained.

[0052] Because this trained student network is trained using optical flow distillation loss and channel alignment loss, it not only solves the problem of motion feature loss in video tasks caused by general distillation methods, but also enables the student network to accurately focus on key motion regions. Compared to the teacher network, this student network reduces the number of parameters by 60-75%, increases inference speed by 2-3 times, and maintains prediction accuracy of over 90%.

[0053] Table 1 shows the experimental results on the Cityscape dataset. This experiment evaluates the model performance from two aspects: image quality and efficiency. For image quality evaluation, four commonly used image quality metrics were used to quantitatively assess the similarity between predicted and ground truth frames: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), Multi-Scale Structural Similarity Index Measure (MS-SSIM), and Learned Perceptual Image Patch Similarity (LPIPS).

[0054] Table 1 shows the experimental results. The experimental results show that the optical flow-guided distillation compression method significantly improves the prediction quality of the student model with fewer parameters.

[0055] Based on the same concept, this invention also discloses a knowledge distillation video prediction model compression system guided by optical flow, such as... Figure 4 As shown, the system includes: The optical flow distillation loss generation module is used to generate optical flow distillation loss by constraining the optical flow prediction distribution of the student network to be consistent with that of the teacher network. The channel alignment loss generation module is used to utilize... Divergence aligns the channel attention distributions of the teacher and student networks, generating channel alignment loss; The pixel-level reconstruction loss generation module is used to generate pixel-level reconstruction loss based on the difference in pixel values ​​between the generated image predicted by the student network and the real image. The perceptual loss generation module is used to generate perceptual loss based on the difference between the generated image predicted by the student network and the real image in the high-level semantic feature space. The multi-loss joint optimization module is used to obtain the trained student network based on optical flow distillation loss, channel alignment loss, pixel-level reconstruction loss, and perceptual loss.

[0056] Based on the same concept, this invention also discloses a computer device, which may be a terminal, a laptop computer, a desktop computer, a server, a computer cluster, or other types of computer devices. For example... Figure 5 As shown, the computer device may include at least one processor and memory. The processor can execute instructions stored in the memory. The processor is communicatively connected to the memory via a data bus. In addition to the memory, the processor can also be communicatively connected to input devices, output devices, and communication devices via the data bus.

[0057] The processor can be any conventional processor. Processors may include central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), systems on chips (SoCs), application-specific integrated circuits (ASICs), or combinations thereof.

[0058] Memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0059] In this embodiment of the invention, an executable instruction is stored in a memory. The processor can read the executable instruction from the memory and execute the instruction to implement all or part of the steps of the method of the invention.

[0060] Based on the same concept, the present invention also discloses a computer-readable storage medium including a computer program product or storing the computer program product. The computer product includes computer program instructions that can be executed by a processor to perform all or part of the steps described in the exemplary embodiments above.

[0061] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages, and scripting languages ​​(e.g., Python). The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0062] Computer-readable storage media can take the form of any combination of one or more readable media. A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: static random access memory (SRAM) having one or more electrically connected wires; electrically erasable programmable read-only memory (EEPROM); erasable programmable read-only memory (EPROM); programmable read-only memory (PROM); read-only memory (ROM); magnetic storage; flash memory; magnetic disk or optical disk; or any suitable combination thereof.

[0063] Although specific embodiments of the invention have been disclosed for illustrative purposes to aid in understanding and implementing the invention, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the invention should not be limited to the content disclosed in the preferred embodiments, and the scope of protection claimed by the invention is defined by the claims.

Claims

1. A video prediction model compression method based on optical flow-guided knowledge distillation, characterized in that, The method includes: By constraining the optical flow prediction distribution of the student network to be consistent with that of the teacher network, optical flow distillation loss is generated. use Divergence aligns the channel attention distributions of the teacher and student networks, generating channel alignment loss; Pixel-level reconstruction loss is generated based on the difference in pixel values ​​between the generated image predicted by the student network and the real image. Generate perceptual loss based on the difference between generated images and real images in the high-level semantic feature space predicted by the student network; The trained student network is obtained based on optical flow distillation loss, channel alignment loss, pixel-level reconstruction loss, and perceptual loss.

2. The method according to claim 1, characterized in that, The teacher network includes The student network includes a basic module. A basic module, The basic module includes an optical flow feature encoding input branch, an image feature encoding input branch, a downsampling unit, a stitching unit, a first convolution module, a second convolution module, a deconvolution module, a splitting unit, an optical flow feature encoding output branch, and a mask output branch.

3. The method according to claim 2, characterized in that, The process of video prediction based on a trained student network includes: Get the Frame image, the first Frame image, the first Frame image and the first Frame image to the Optical flow changes in frame images, the first Frame image to the Optical flow changes in frame images, the first Frame image to the Optical flow changes in frame images, the first Frame image to the Optical flow changes in the frame image are input; The first Frame image, the first Frame image and the first Frame image to the Optical flow changes in frame images, the first Frame image to the The optical flow changes of the frame image are input into the first basic module to obtain the second... Image prediction results for the frame image; The first Frame image, the first The first output of the basic module Image prediction results of frame image, the first Frame image to the Optical flow changes in frame images, the first Frame image to the Optical flow changes of the frame image input number The first basic module, resulting in the first... The first output of the basic module Image prediction results for frame images; where, ; The first Frame image, the first The first output of the basic module Image prediction results of frame image, the first Frame image to the Optical flow changes in frame images, the first Frame image to the Optical flow changes of the frame image input number The first basic module, resulting in the first... Image prediction results for frame images.

4. The method according to claim 1, characterized in that, By constraining the optical flow prediction distribution of the student network to be consistent with that of the teacher network, an optical flow distillation loss is generated, including: Obtain the long-term optical flow generated by the teacher network respectively , short-term optical flow and long-term optical flow generated by student networks , short-term optical flow ; Long-term optical flow , short-term optical flow Long-term optical flow and short-term optical flow The first in Each channel is spatially normalized to obtain the corresponding spatial probability map. Spatial probability graph Spatial probability graph Spatial probability graph ; Based on spatial probability graphs Spatial Probability Map Calculate the first Short-term optical flow of each channel Divergence; Based on spatial probability graphs Spatial Probability Map Calculate the first Long-term optical flow of each channel Divergence; Short-term optical flow of all channels Divergence and Long-Term Optical Flow The average divergence is used to obtain the optical flow distillation loss.

5. The method according to claim 1, characterized in that, The channel alignment loss ;in, Indicates the total number of channels. This indicates that the teacher network is in the channel. probability distribution on, Indicates student network in channel probability distribution on, probability distribution With probability distribution divergence between , , These represent the height and width of the feature map, respectively. , These represent the coordinate indices of the feature map in the horizontal and vertical directions, respectively.

6. The method according to claim 1, characterized in that, The pixel-level reconstruction loss ;in, This represents the generated image based on student network predictions. Represents a real image. Indicates the height of the generated image and the real image. This represents the width of the generated image and the real image. Describing the L1 norm, This represents the normalization factor.

7. The method according to claim 1, characterized in that, The perceived loss ;in, Indicates the first Feature extraction network of layer feature maps, Indicates the first The number of channels in the layer feature map. Indicates the first The height of the layer feature map, Indicates the first The width of the layer feature map, Denotes the square of the L2 norm. This represents the generated image based on student network predictions. Represents a real image.

8. A knowledge distillation video prediction model compression system based on optical flow guidance, characterized in that, The system includes: The optical flow distillation loss generation module is used to generate optical flow distillation loss by constraining the optical flow prediction distribution of the student network to be consistent with that of the teacher network. The channel alignment loss generation module is used to utilize... Divergence aligns the channel attention distributions of the teacher and student networks, generating channel alignment loss; The pixel-level reconstruction loss generation module is used to generate pixel-level reconstruction loss based on the difference in pixel values ​​between the generated image predicted by the student network and the real image. The perceptual loss generation module is used to generate perceptual loss based on the difference between the generated image predicted by the student network and the real image in the high-level semantic feature space. The multi-loss joint optimization module is used to obtain the trained student network based on optical flow distillation loss, channel alignment loss, pixel-level reconstruction loss, and perceptual loss.

9. A computer device, characterized in that, The computer device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the knowledge distillation video prediction model compression method based on optical flow guidance as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the knowledge distillation video prediction model compression method based on optical flow guidance as described in any one of claims 1-7.

Images