Football match collection generation method based on cloud collaborative multi-modal large model

Abstract

The invention belongs to the technical field of artificial intelligence, and particularly relates to a cloud collaborative multi-modal large model-based football match collection generation method, which comprises the following steps that: an edge end performs standardization processing on a football match multi-view video uploaded by a user; the edge end calls a large model to carry out event identification on the standardized multi-view videos, the goal occurrence moments of goal events in the videos are extracted, and corresponding target timestamps and frame numbers are output; the edge end cuts the standardized multi-view video according to the target timestamp to obtain a pre-fragment and a post-fragment; the edge end extracts a multi-view key frame from the standardized multi-view video according to the target timestamp and uploads the multi-view key frame to the cloud end; the cloud processes the multi-view key frame through the key model, generates a lens conveying segment and returns the lens conveying segment to the edge end; the edge end uses an FFemg tool to carry out splicing processing on the front segment, the lens segment and the rear segment, and a football match collection video is obtained; according to the method, the time delay and calculation cost of whole-process reasoning are remarkably reduced.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a method for generating football match highlights based on a cloud-based collaborative multimodal large model. Background Technology

[0002] In recent years, the live streaming and short video dissemination of sports events have developed rapidly. Users have placed higher demands on the real-time, high-frequency, and personalized nature of key event clips, such as goals in football matches. To improve content production efficiency, the industry is trending towards the adoption of automation technologies to perform event detection, clip trimming, and content synthesis on sports videos, thereby automating the generation of match highlights and replays.

[0003] Currently, mainstream technologies typically analyze the entire match video, locate key events such as goals, and stitch together segments before and after the events according to preset rules. With the rise of large-scale visual models, their powerful scene semantic understanding capabilities are expected to further improve the robustness of event recognition. However, several challenges remain in practical implementation: First, match videos are usually captured from multiple cameras and perspectives. Videos from different perspectives differ in encoding parameters, frame rates, resolutions, and durations, and their timelines are often not strictly synchronized, making it difficult to accurately map the identified event moments to other perspectives, thus affecting the quality of segment trimming and content synthesis. Second, relying entirely on high-performance cloud models for inference on the entire video stream leads to high bandwidth consumption, high processing latency, and increased costs. If lightweight edge models are used entirely, false negatives or missed detections are likely to occur in complex scenarios such as camera occlusion, rapid switching, and replay footage, making it difficult to achieve an effective balance between real-time performance and accuracy.

[0004] Furthermore, traditional highlight generation often employs a "crop-and-stitch" approach, resulting in a relatively limited range of presentation. While bullet time and other camera movement effects can significantly enhance the visual impact of key moments, their traditional implementation relies on multi-camera arrays or 3D reconstruction technology, leading to complex deployments and high costs. The rise of generative video technology has provided new ideas for generating dynamic camera movement clips based on keyframes, but existing solutions still face challenges in engineering implementation regarding subject consistency, temporal stability, and natural integration with real video clips. Moreover, they lack an integrated design that coordinates with edge-side event detection processes. Summary of the Invention

[0005] To address the above problems, this invention provides a method for generating football match highlights based on a cloud-based collaborative multimodal large model, comprising the following steps:

[0006] S1. The edge device performs standardized processing on the multi-view videos of football matches uploaded by users; the multi-view videos of football matches include N videos from different perspectives, where N≥2;

[0007] S2. The edge device calls the large model to perform event recognition on standardized multi-view videos, extracts the time of the goal in each video, and outputs the corresponding target timestamp and frame number;

[0008] S3. The edge end crops the standardized multi-view video according to the target timestamp to obtain the front segment and the back segment;

[0009] S4. The edge device extracts multi-view keyframes from the standardized multi-view video based on the target timestamp and uploads them to the cloud; the multi-view keyframes include keyframes from N standardized videos;

[0010] S5. The cloud processes multi-view keyframes through a key model to generate N-1 camera movement clips and returns them to the edge; the key model includes a diffusion image-to-video generation network, a text encoder, and a variational autoencoder.

[0011] S6. At the edge, the FFemg tool is used to stitch together the foreground clip, N-1 camera movement clips, and background clip to obtain a football match highlights video.

[0012] The beneficial effects of this invention are:

[0013] By standardizing and aligning the timing of multi-view match videos, the system ensures accurate mapping of goal moments across different perspectives, improving the stability of cropping and compositing. A lightweight goal recognition model is deployed at the edge to achieve highly accurate goal positioning under low latency and low bandwidth overhead. A cloud-based generative AI camera movement workflow is triggered based on goal keyframes as the start and end frames, utilizing first and last frame constraints and positive / negative cue constraints to generate camera movement segments, reducing jitter, distortion, and subject drift while enhancing the viewing experience. Finally, the pre-goal segment, camera movement segment, and post-goal segment are spliced ​​together for output, resulting in a structurally complete and more expressive video.

[0014] Compared to existing technologies, this invention effectively solves the event alignment problem caused by content inconsistencies and time asynchronies in multi-view videos, significantly reducing the latency and computational cost of the entire inference process. It achieves high-quality bullet-time effects without relying on multi-camera arrays or complex 3D reconstruction, thereby greatly improving the real-time performance, accuracy, and visual expressiveness of exciting clip generation. Attached Figure Description

[0015] Figure 1 This is a flowchart of a method for generating football match highlights based on a cloud-based collaborative multimodal large model, according to the present invention.

[0016] Figure 2 This is a flowchart of cloud-based AI camera movement generation in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figure 1 This invention provides a method for generating football match highlights based on a cloud-based collaborative multimodal large model, comprising the following steps:

[0019] S1. The edge device performs standardized processing on the multi-view videos of football matches uploaded by users; the multi-view videos of football matches include videos of the same scene from N different perspectives, where N≥2.

[0020] Specifically, in a football match, multiple cameras are set up on the same field, each with a different shooting angle. This results in videos of the same scene from different perspectives, such as frontal and rear views. In this embodiment of the invention, the videos of the same scene from different perspectives are numbered from 1 to N.

[0021] In some embodiments, step S1, which standardizes the multi-view video, includes:

[0022] S11. Read the metadata of N video streams and execute step S12; the metadata includes the encapsulation format, encoding parameters, frame rate (fps), resolution ((H,W), where H and W represent the image height and width respectively), and pixel format (pix_fmt); the encoding parameters include the video encoder (vcodec), audio encoder (acodec), audio sampling rate (audio_sr), and duration.

[0023] S12. Using FFmpeg, the container format and encoding parameters of the N video streams are uniformly converted to a preset standard to ensure that the video encoder, audio encoder, pixel format and audio sampling rate of the two video streams output from different perspectives are consistent. Then, step S13 is executed.

[0024] S13. Normalize the frame rates of the N video streams to the target frame rate (fps). t And scaling and padding operations are used to keep the resolution of the N video outputs uniform, and then step S14 is executed;

[0025] Specifically, in this embodiment of the invention, the resolution of N video streams is unified to (H t W t ), H t W tThese represent height and width, respectively. By standardizing the frame rate and resolution, the goal moments can be mapped and cropped at a uniform frame rate and spatial scale during subsequent processing.

[0026] S14. Perform duration alignment processing on the N video streams to generate standardized multi-view video. The standardized multi-view video includes N standardized video streams.

[0027] In some embodiments, time-length alignment processing of N video streams includes:

[0028] Using the shortest duration among the N video streams as a reference, the remaining N-1 video streams are cropped to the same length and aligned with the start time points of the N video streams.

[0029] Alternatively, using the longest duration among the N video streams as a reference, fill the remaining N-1 video streams with the same length and align them with the start time points of the N video streams.

[0030] S2. The edge device calls a large model to perform event recognition on standardized multi-view videos, extracts the moment of the goal in each video, and outputs the corresponding target timestamp and frame number.

[0031] In this embodiment of the invention, a large model is deployed at the edge as a lightweight event recognition model for goal detection in video.

[0032] In some embodiments, step S2, where the edge terminal invokes a large model to perform event recognition on each standardized video stream, includes:

[0033] S21. To reduce computational overhead and improve recognition efficiency, a frame extraction strategy based on a fixed sampling rate is used to convert the current standardized video into a sequence of frames arranged in chronological order, and retain the time information corresponding to each frame to establish a mapping relationship between frames and timestamps.

[0034] S22. Construct recognition prompts to guide the large model to focus on key visual evidence of the goal event and constrain the large model to output recognition results in a structured form.

[0035] In some embodiments, the identification prompt may include a two-stage judgment logic and output field constraints; wherein, the two-stage judgment logic includes first recalling suspected goal segments, and then performing a review and confirmation; the output field constraints include whether a goal event occurred, the time when the goal occurred, confidence level / team color, etc.

[0036] S23. Input the frame sequence and the recognition prompt words into the large model for inference to obtain the output result; the output result includes at least two types of information: whether a goal event has occurred and the time when the goal occurred.

[0037] S24. Use the moment the goal occurs as the target timestamp t.goal And extract the corresponding frame number f goal In some embodiments, the large model deployed at the edge uses the Qwen2.5-vl-2B model, and performs frame extraction on the video using a fixed sampling rate of 2 frames / second. Specifically, the output of the large model is in the following format:

[0038] More goals: {"goal": true, "moment": XX}

[0039] No goals scored: {"goal": false}

[0040] S3. The edge end performs cropping on the standardized multi-view video based on the target timestamp to obtain the front segment and the back segment.

[0041] In some embodiments, step S3 includes:

[0042] S31. Determine the cropping time windows for the first and Nth normalized video streams:

[0043] The cropping time window for the first standardized video stream is [t]. goal,1 -T pre ,t goal,1 ], where t goal,1 T represents the target timestamp of the first standardized video stream. pre Indicates the pre-load duration;

[0044] The cropping time window for the Nth standardized video is [t]. goal,N ,t goal,N +T post ], where t goal,N T represents the target timestamp of the Nth standardized video stream. post Indicates the duration after the last frame;

[0045] When t goal,A <T pre At that time, the start time of the cropping interval of the first-channel standardized video is corrected to the start time of the first-channel standardized video, that is, let t goal,A -T pre =0; when t goal,B +T post When the total duration of the Nth standardized video exceeds the limit, the end time of the cropping interval of the Nth standardized video will be corrected to the end time of the Nth standardized video.

[0046] In some embodiments, T is set pre =10 seconds, T post =10 seconds.

[0047] S32. Based on the cropping time window, use FFmpeg to crop the first and Nth normalized video streams respectively to obtain the pre-video and post-video segments.

[0048] S4. The edge device extracts multi-view keyframes from the standardized multi-view video based on the target timestamp and uploads them to the cloud; the multi-view keyframes include keyframes from N standardized videos.

[0049] In some embodiments, the process of extracting keyframes from the n=1,2,…,N normalized video streams includes: in the nth normalized video stream, in window [t goal,n -△, t goal,n Select the frame with the highest resolution from the +△] as the keyframe K. n ; where t goal,n This represents the target timestamp of the nth standardized video stream, and △ represents the preset time interval.

[0050] When a more stable image is required, the second method is preferred.

[0051] S5. The cloud processes multi-view keyframes through key models, generates camera movement clips, and returns them to the edge; the key models include a diffusion image-to-video generation network, a text encoder, and a variational autoencoder.

[0052] In some embodiments, the standardized video streams n=1,2,…,N and n+1 are divided into a group of samples, resulting in N-1 groups of samples; for each group of samples, the cloud extracts the keyframe K of the nth standardized video stream. n and the (n+1)th normalized video keyframe K n+1 These serve as the start and end frames, respectively. The key model is invoked, and a node-based workflow-driven generation engine executes the camera movement generation process, ultimately outputting the corresponding camera movement clips, including:

[0053] S51. A positive condition vector and a negative condition vector are generated by a text encoder, thereby imposing semantic and stylistic constraints on the update direction of latent variables during the diffusion sampling process.

[0054] In some embodiments, the present invention effectively constrains the content and direction of camera movement segments by introducing positive and negative cue words. Simultaneously, to enable the effective application of positive and negative cue words in the diffusion image-to-video generation network, a text encoder is used to encode the positive and negative cue words into conditional vectors that can be used by the diffusion image-to-video generation network.

[0055] Specifically, positive prompts are used to clearly define generation goals such as "camera movement time", "camera surround", "subject time freeze" and "controllable changes in the first and last frame actions", thereby guiding the model to converge towards the expected effect in terms of motion camera design and time effects; negative prompts are used to suppress undesirable phenomena such as motion blur, camera shake, screen flicker, background jump, character structure deformation, facial and hand distortion, and identity consistency drift.

[0056] Specifically, the text encoder may employ a CLIP-type encoder, a T5-type encoder, or other equivalent text conditional encoding structures.

[0057] S52. The Variational Autoencoder (VAE) generates an initial sequence of latent variables based on the start frame and the end frame, and establishes temporal boundary constraints from the start frame to the end frame in the latent space based on the positive condition vector and the negative condition vector.

[0058] VAE is a deep learning-based generative model consisting of an encoder and a decoder. Its core function is to achieve a bidirectional mapping between the image pixel space and the latent space. This invention establishes a mapping relationship between the pixel space and the latent variable space using VAE, transferring the generation process to the compressed latent space to improve computational efficiency. Finally, the latent variables are reconstructed into a visible sequence of image frames.

[0059] S53. The diffusion-based image-to-video generation network receives an initial latent variable sequence and iterates to generate a final latent variable sequence. The diffusion-based image-to-video generation network includes a video generation network and an algorithm executor.

[0060] Specifically, the core function of the video generation network during the diffusion sampling iteration is to predict the noise residual and update the latent variables based on the prediction results. This process is mainly implemented by a noise prediction network (i.e., the main video diffusion network). The main video diffusion network can adopt any noise prediction structure suitable for video generation tasks, with common implementations including 3D-UNet, spatiotemporal attention networks, or other functionally equivalent architectures. The algorithm executor is responsible for executing the diffusion sampling iteration process. Its function is to cyclically call the aforementioned video generation network according to the specified sampling algorithm and step scheduling strategy, and iteratively update the latent variables during this process.

[0061] In some embodiments, to adapt to the generation characteristics of different noise stages, the present invention employs two sets of video diffusion generation main networks, optimized for high-noise and low-noise stages respectively. To achieve a surround camera effect and enhance temporal consistency with a lower sampling number, this embodiment further introduces LoRA (Low-Rank Adaptation). Without changing the structure of the video diffusion generation main network, controllable constraints on the consistency of motion trajectory and content are achieved through low-rank incremental weight injection. In a preferred embodiment, LoRA is also divided into two groups according to the noise stage to adapt to different generation processes. Simultaneously, the algorithm executor employs a two-stage diffusion sampling link, containing two sampler nodes.

[0062] This design decouples the sampling process into two stages: high-noise coarse shaping and low-noise fine shaping. This allows for maintaining structural stability and improving detail quality with fewer sampling steps, thereby enhancing the controllability and stability of camera movement video generation.

[0063] S54. The decoder of the variational autoencoder decodes the final latent variable sequence to obtain the camera movement segment.

[0064] In some embodiments, camera movement clips can be constructed based on a bullet time generation model to generate bullet time effect shots; other large visual generation models can also be used to generate camera movement clips with different styles.

[0065] In some embodiments, the present invention employs, as follows Figure 2 The architecture shown generates camera movement clips. Among them:

[0066] Video generation networks include:

[0067] UNETLoader-1 is used to load the main network for high-noise video diffusion generation; it is mainly used in the first half of diffusion sampling, aiming to achieve the initial construction of the overall video structure and the rough generation of motion trajectories.

[0068] UNETLoader-2 is used to load the main network for low-noise video diffusion generation; it is mainly used in the post-diffusion sampling stage, aiming to refine the model and effectively suppress artifacts to improve the detail quality and visual fidelity of the generated results.

[0069] LoraLoaderModelOnly-1 is used to inject high_noise LoRA into the main network for video diffusion generation loaded by UNETLoader-1 to achieve specific AI camera movement effects and adapt to the 4-step inference process.

[0070] LoraLoaderModelOnly-2 is used to inject low_noise LoRA into the main network for video diffusion generation loaded by UNETLoader-2, which is used to further suppress jitter, distortion and detail fragmentation during the refinement stage.

[0071] In some embodiments, the high-noise video diffusion generation master network (i.e., the video diffusion generation master network for the high-noise coarse shaping stage) can use wan2.2_i2v_A14b_high_noise; the low-noise video diffusion generation master network (i.e., the video diffusion generation master network for the low-noise refinement and shaping stage) can use wan2.2_i2v_A14b_low_noise.

[0072] In some embodiments, high_noise LoRA can be wan2.2bullet_time_high_noise; low_noise LoRA can be wan2.2_i2v_lightx2v_4steps_lora_v1_low_noise.

[0073] In some embodiments, such as Figure 2 As shown, the SD3 sampling node can be used to standardize and encapsulate the sampling configuration of the network after loading LoRA, transforming its output into a model object that can be directly called by the sampler. This process integrates network weights, sampling strategy parameters, and noise scheduling parameters into a unified diffusion sampling framework, achieving structured parameter integration and systematic management of the sampling process.

[0074] The algorithm executor includes:

[0075] KSampler-1 is used to receive the initial latent variable sequence output by AVE and call the network output by LoraLoaderModelOnly-1 to perform diffusion sampling iteration in the high-noise coarse-forming stage to generate the intermediate latent variable sequence.

[0076] KSampler-2 is used to receive the intermediate latent variable sequence output by KSampler-1 and call the network output by LoraLoaderModelOnly-2 to perform diffusion sampling iteration in the low-noise refinement and shaping stage to generate the final latent variable sequence.

[0077] In particular, UNETLoader, LoraLoaderModelOnly, and KSampler are all cloud nodes. The suffixes "-1" and "-2" are only used for identification and do not limit the node implementation method or specific deployment platform.

[0078] The text encoder uses a CLIP-class encoder, which specifically includes:

[0079] CLIPLoader is used to load CLIP models;

[0080] CLIPTextEncode-1 is used to receive positive prompt words, call the CLIP model loaded by CLIPLoader to encode the positive prompt words, and generate a positive condition vector;

[0081] CLIPTextEncode-2 is used to receive negative prompt words, call the CLIP model loaded by CLIPLoader to encode the negative prompt words, and generate a negative condition vector.

[0082] It should be noted that CLIPTextEncode-1 and CLIPTextEncode-2 are the same cloud node, and the "-1" and "-2" are only used for annotation and differentiation.

[0083] In particular, the positive and negative condition vectors are used together as conditional inputs to WanFirstLastFrameToVideo, thereby imposing constraints on the latent variable update direction at each step of the sampling iteration, improving the stability and consistency of the generated results.

[0084] Variational autoencoders include:

[0085] VAELoader is used to load VAE models (such as wan_2.1_vae.safetensors) and provides encoder and decoder functions;

[0086] WanFirstLastFrameToVideo is used to access the VAE model loaded by VAELoader. Its core function is to generate continuous transition frames between the specified start and end frames and construct the corresponding initial latent variables. At the same time, it uses the input positive and negative condition vectors to establish temporal boundary constraints from the start frame to the end frame in the latent space, thereby ensuring that the generated video sequence maintains a high degree of consistency with the input keyframes at the start and end positions.

[0087] VAEDecode is used to connect to the VAE decoder of VAELoader and decode the final latent variable sequence output by KSampler-2.

[0088] After the composite of the camera movement video clips is completed, the generated frame sequence is packaged into an independent video file and saved, and finally the complete camera movement video clip is output.

[0089] S6. At the edge, the FFemg tool is used to stitch together the foreground clip, N-1 bullet-time camera clips, and background clip to obtain a football match highlights video.

[0090] Before splicing video clips, it is recommended to perform uniform encoding parameters and time base verification on the clips to be synthesized. If there are inconsistencies in frame rate, resolution, pixel format, or audio parameters, transcoding should be performed beforehand to ensure that the parameters of each clip are consistent, thereby avoiding screen tearing, audio-visual asynchrony, or encoding compatibility issues after splicing. After processing, the above clips are synthesized into an output video in chronological order and then saved or published. Due to the characteristics of the workflow, the last frame of the video corresponding to the nth sample group is the first frame of the video clip corresponding to the n+1th sample group, thus achieving a seamless transition.

[0091] In summary, this invention addresses the need for automatic generation of highlight clips from multi-view football match videos. First, it standardizes and aligns the multi-view videos uploaded by users to ensure consistency in cross-view event localization and cropping. Then, it deploys lightweight goal recognition capabilities at the edge and combines them with a large visual model to detect goals and determine their timing, thus balancing recognition efficiency and accuracy under resource constraints. After determining the goal time, it triggers a cloud-based generative bullet-time camera workflow by inputting the number of frames representing the goal time from multiple perspectives. Stable camera movement video clips are generated through first and last frame constraints, positive and negative cue constraints, and two-stage diffusion sampling. Finally, the preceding clips, camera movement clips, and subsequent clips are stitched together to output the target video. Through this process, this invention solves the problem of event alignment difficulties caused by inconsistent and asynchronous multi-view video parameters, reduces latency and cost associated with end-to-end cloud inference, and achieves bullet-time camera movement effects without the need for multi-camera arrays or 3D reconstruction. This significantly improves the real-time performance, stability, and visual expressiveness of goal highlight clip generation, demonstrating promising application prospects.

[0092] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for generating football match highlights based on a cloud-based collaborative multimodal large model, characterized in that, Includes the following steps: S1. The edge device performs standardized processing on the multi-view videos of football matches uploaded by users; the multi-view videos of football matches include N videos from different perspectives, where N≥2; S2. The edge device calls the large model to perform event recognition on standardized multi-view videos, extracts the time of the goal in each video, and outputs the corresponding target timestamp and frame number; S3. The edge end crops the standardized multi-view video according to the target timestamp to obtain the front segment and the back segment; S4. The edge device extracts multi-view keyframes from the standardized multi-view video based on the target timestamp and uploads them to the cloud; the multi-view keyframes include keyframes from N standardized videos; S5. The cloud processes multi-view keyframes through a key model to generate N-1 camera movement clips and returns them to the edge; the key model includes a diffusion image-to-video generation network, a text encoder, and a variational autoencoder. S6. At the edge, the FFemg tool is used to stitch together the foreground clip, N-1 camera movement clips, and background clip to obtain a football match highlights video.

2. The method for generating football match highlights based on a cloud-based collaborative multimodal large model according to claim 1, characterized in that, Step S1, standardizing the multi-view video, includes: S11. Read the metadata of N video streams and execute step S12; the metadata includes the encapsulation format, encoding parameters, frame rate, resolution, and pixel format, and the encoding parameters include the video encoder, audio encoder, audio sampling rate, and duration; S12. Using FFmpeg, the container format and encoding parameters of the N video streams are uniformly converted to a preset standard, and then step S13 is executed; S13. Normalize the frame rate of N video streams to the target frame rate, and use scaling and padding operations to keep the resolution of N video streams uniform, and then execute step S14. S14. Perform duration alignment processing on the N video streams to generate standardized multi-view videos.

3. The method for generating football match highlights based on a cloud-based collaborative multimodal large model according to claim 1, characterized in that, Step S2, where the edge device calls the large model to perform event recognition on each standardized video stream, includes: S21. Based on a fixed sampling rate frame extraction strategy, the current standardized video is converted into a sequence of frames arranged in chronological order, and the time information corresponding to each frame is retained to establish a mapping relationship between frames and timestamps. S22. Construct recognition prompts to guide the large model to focus on key visual evidence of the goal event and constrain the large model to output recognition results in a structured form; S23. Input the frame sequence and the recognition prompt words into the large model for inference to obtain the output result; the output result includes at least two types of information: whether a goal event has occurred and the time when the goal occurred. S24. Use the moment the goal occurs as the target timestamp and extract the corresponding frame number.

4. The method for generating football match highlights based on a cloud-based collaborative multimodal large model according to claim 1, characterized in that, Step S3 includes: S31. Determine the cropping time windows for the first and Nth normalized video streams: The cropping time window for the first standardized video stream is [t]. goal,1 -T pre ,t goal,1 ], where t goal,1 T represents the target timestamp of the first standardized video stream. pre Indicates the pre-load duration; The cropping time window for the Nth standardized video is [t]. goal,N ,t goal,N +T post ], where t goal,N T represents the target timestamp of the Nth standardized video stream. post Indicates the duration after the last frame; When t goal,A <T pre At that time, the start time of the cropping interval of the first-channel standardized video is corrected to the start time of the first-channel standardized video, that is, let t goal,A -T pre =0; when t goal,B +T post When the total duration of the Nth standardized video exceeds the limit, the end time of the cropping interval of the Nth standardized video will be corrected to the end time of the Nth standardized video. S32. Based on the cropping time window, use FFmpeg to crop the first and Nth normalized video streams respectively to obtain the pre-video and post-video segments.

5. The method for generating football match highlights based on a cloud-based collaborative multimodal large model according to claim 1, characterized in that, The process of extracting keyframes from the n=1,2,…,N-th normalized video stream includes: In the nth normalized video, in window [t] goal,n -△, t goal,n Select the frame with the highest resolution from the +△] as the keyframe K. n ; where t goal,n This represents the target timestamp of the nth standardized video stream, and △ represents the preset time interval.

6. The method for generating football match highlights based on a cloud-based collaborative multimodal large model according to claim 1, characterized in that, The standardized video streams n=1,2,…,N are grouped together with the (n+1)th standardized video stream, resulting in N-1 groups of samples. For each group of samples, the cloud then extracts the keyframe K from the nth standardized video stream. n and the (n+1)th normalized video keyframe K n+1 Using these as the start and end frames respectively, the key model is used to generate corresponding camera movement segments, including: S51. Generate positive and negative conditional vectors using a text encoder; S52. The variational autoencoder generates an initial latent variable sequence based on the start frame and the end frame, and establishes temporal boundary constraints from the start frame to the end frame in the latent space based on the positive condition vector and the negative condition vector. S53. The diffusion-based image-to-video generation network receives an initial latent variable sequence and iterates to generate a final latent variable sequence; the diffusion-based image-to-video generation network includes a video generation network and an algorithm executor; S54. The decoder of the variational autoencoder decodes the final latent variable sequence to obtain the camera movement segment.

7. The method for generating football match highlights based on a cloud-based collaborative multimodal large model according to claim 6, characterized in that, Video generation networks include: UNETLoader-1 is used to load high-noise video diffusion generation main network; UNETLoader-2 is used to load low-noise video diffusion generation main network; LoraLoaderModelOnly-1 is used to inject high-noise LoRA into the video diffusion generated by UNETLoader-1 to produce the main network; LoraLoaderModelOnly-2 is used to inject low_noise LoRA into the video loaded by UNETLoader-2 to generate the main network; The algorithm executor includes: KSampler-1 is used to receive the initial latent variable sequence output by AVE and call the network output by LoraLoaderModelOnly-1 to generate the intermediate latent variable sequence. KSampler-2 is used to receive the intermediate latent variable sequence output by KSampler-1 and call the network output by LoraLoaderModelOnly-2 to generate the final latent variable sequence. The text encoder uses a CLIP-class encoder, which specifically includes: CLIPLoader is used to load CLIP models; CLIPTextEncode-1 is used to receive positive prompt words, call the CLIP model loaded by CLIPLoader to encode the positive prompt words, and generate a positive condition vector; CLIPTextEncode-2 is used to receive negative prompt words, call the CLIP model loaded by CLIPLoader to encode the negative prompt words, and generate a negative condition vector; Variational autoencoders include: VAELoader is used to load VAE models; WanFirstLastFrameToVideo is used to access the VAE model loaded by VAELoader. Its core function is to generate continuous transition frames between the specified start and end frames and construct the corresponding initial latent variables. At the same time, it uses the input positive and negative condition vectors to establish temporal boundary constraints from the start frame to the end frame in the latent space. VAEDecode is used to connect to the VAELoader's VAE decoder and decode the final latent variable sequence output by KSampler-2.

Images