Intelligent narrative video generation method based on fusion of first-person view angle and third-person view angle

Abstract

The invention relates to the technical field of video generation, in particular to an intelligent narrative video generation method for fusion of first-person and third-person view angles. The method comprises the following steps: performing event capture processing on a global picture of a third-person view angle; when a high-frequency action is detected, determining the moment as a narration trigger point, and outputting a trigger identifier; performing time alignment on the third-person picture and the first-person view angle picture by taking the moment of triggering the identifier as a center to form a synchronous fragment; in the synchronous fragments, performing main body action analysis on the first person fragments, and taking two types of analysis results as fragment attributes; performing main and auxiliary lens judgment on the synchronization segment based on the segment attribute and outputting a lens retention segment; performing link arrangement on the shot retention fragments, and constructing a fragment sequence of narrative mobility; according to the method, the intelligent competition collection which is clear in structure and coherent in narration and has main body action details and global scene information is automatically generated by fusing the double-view-angle content.

Description

Technical Field

[0001] This invention relates to the field of video generation technology, and in particular to an intelligent narrative video generation method that integrates first-person and third-person perspectives. Background Technology

[0002] Traditional video recordings typically use a third-person perspective, employing a fixed camera to observe the entire competition process. This clearly presents the competition scene and the overall movement trajectories of the participants, but struggles to capture the detailed movements and subjective experiences of the athletes. First-person perspective videos, on the other hand, record the real-time view of the athletes using wearable cameras, directly reflecting their actions and details. However, they have limitations in perspective, easily losing overall scene information and the dynamics of other subjects. Current technologies primarily rely on manual editing or simple synchronous splicing methods for combining multi-view videos. This is not only time-consuming and labor-intensive but also struggles to ensure narrative coherence and visual rhythm consistency. It fails to automatically highlight key actions and climactic moments in the competition and achieves smooth transitions between different perspectives. With the development of computer vision, motion recognition, and video analysis technologies, automated video generation methods based on motion detection, motion intensity assessment, and scene analysis are gradually emerging. However, existing methods often focus on a single perspective or lack intelligent judgment of main and auxiliary shots and transition strategies, failing to meet the audience's dual needs for overall competition information and the athlete's experience. Summary of the Invention

[0003] Therefore, it is necessary to provide an intelligent narrative video generation method that integrates first-person and third-person perspectives to solve at least one of the aforementioned technical problems.

[0004] To achieve the above objectives, an intelligent narrative video generation method that integrates first-person and third-person perspectives includes the following steps: Step S1: During the game, perform event capture processing on the global screen from the third-person perspective; when a high-frequency action is detected, determine the moment as the narrative trigger point and output the trigger identifier; Step S2: Centered on the moment of the trigger mark, perform time alignment on the third-person view and the first-person view to form a synchronized segment; in the synchronized segment, perform scene structure analysis on the third-person segment and main action analysis on the first-person segment, and use the two types of analysis results as segment attributes; Step S3: Based on the fragment attributes, determine the main and auxiliary shots of the synchronized fragments and output the shot-preserving fragments; perform link arrangement on the shot-preserving fragments to construct a fragment sequence with narrative fluidity; Step S4: Generate transition patterns based on rhythm differences in the segment sequence, and insert matching transition effects at the boundaries of the segment sequence to form a narrative shot sequence; after the narrative shot sequence is generated, splice the segments one by one into the target video, and output a narrative match highlight reel that blends first-person and third-person perspectives.

[0005] Preferably, step S2 includes: Based on the time point corresponding to the trigger marker, timeline alignment processing is performed on the first-person and third-person shot sequences to form a synchronized segment containing dual-view content. Analyze the scene structure of the third-person shots in the synchronized clips, and use the results of the scene structure analysis as the analysis result; Analyze the main action of the first-person shot in the synchronized segment, and use it as the result of the main action analysis; The scene structure analysis results and the main action analysis results are determined as fragment attributes.

[0006] Preferably, based on the time point corresponding to the trigger marker, timeline alignment processing is performed on the first-person and third-person shot sequences to form a synchronized segment containing dual-view content, including: The starting point and sequence span of the trigger flag are determined based on the trigger flag's time, and the frame sequences of the windows before and after that time are captured from the first-person and third-person perspectives, respectively. Perform time difference analysis on the frame sequences of first-person and third-person shots, determine the time axis offset between the frame sequences, and record the time deviation parameters; Based on the time deviation parameter, the frame time of the first-person and third-person shots is compensated and calibrated to form a unified time scale; Under a unified time scale, establish the correspondence between first-person and third-person frame views, and output synchronized segments containing dual-view content.

[0007] Preferably, time difference analysis is performed on the frame sequences of first-person and third-person shots to determine the time axis offset between the frame sequences, and the time deviation parameters are recorded, including: Based on the time point of the trigger identifier, extract a continuous set of frames of equal length from the first-person view frame sequence and the third-person view frame sequence respectively. In a continuous set of frames, the continuous changes in the images are compared frame by frame to determine the segments of change in the same image. For each set of change segments, read its time information in its respective frame sequence and calculate the time difference between the two. The average of all time differences is used as the overall offset between the two time axes, and the result is output as a time deviation parameter.

[0008] Preferably, the scene structure of the third-person shots in the synchronized segments is analyzed, and the scene structure analysis results include: In the synchronized segment, scene segmentation is performed on the third-person frame sequence to distinguish between the foreground moving area and the background static area, and the spatial proportion of each area is recorded to form a scene area map; Target detection is performed on the scene area map to identify all moving subjects, extract the center coordinates of the bounding box of each subject and the pixel ratio, and form a moving subject distribution map; Perform differential processing on the motion subject distribution maps of adjacent frames to calculate the change in the center displacement of the bounding box and the change in the pixel ratio; When the center displacement or pixel ratio change of any frame exceeds the threshold, the frame is marked as a view coverage change event. The distribution map of the moving subjects and the set of timestamps of the events covering the change in viewpoint are combined to form the scene structure analysis result.

[0009] Preferably, differential processing is performed on the motion subject distribution maps of adjacent frames, and the calculation of the change in the center displacement of the bounding box and the change in pixel proportion includes: Using consecutive third-person frames within a synchronized segment as the order, select the motion subject distribution map of the current frame and the previous frame to form adjacent distribution map pairs; In adjacent distribution map pairs, center coordinate matching is performed on the bounding boxes of the same moving subject, and the inter-frame displacement distance of the center coordinates is calculated; The area of ​​the bounding boxes of the same moving subject is statistically analyzed, and the absolute difference in the pixel ratio is calculated to obtain the change in pixel ratio. Record the inter-frame displacement distance and pixel ratio change at the corresponding inter-frame positions.

[0010] Preferably, the main action of the first-person shot in the synchronized segment is analyzed, and the results of the main action analysis include: Using the starting frame of the synchronized segment as a reference, target center point trajectory tracking is performed on the first-person frame sequence, and the displacement distance of the target center point in adjacent frames is calculated to form a viewpoint swing amplitude sequence; Within the same frame sequence, hand position recognition is performed on the subject of the action, hand position coordinates are extracted, the maximum displacement distance between adjacent coordinates is calculated, and a motion detail intensity sequence is formed. The viewpoint swing amplitude sequence and the motion detail intensity sequence are combined to form the motion analysis result.

[0011] Preferably, within the same frame sequence, hand position recognition is performed on the subject of the action, hand position coordinates are extracted, and the maximum displacement distance between adjacent coordinates is calculated to form an action detail intensity sequence, including: Using the starting frame of the same frame sequence as a reference, hand position recognition is performed on the subject of the action, the hand position coordinates are extracted, and recorded into the coordinate sequence; Perform displacement calculations on each adjacent coordinate pair in the coordinate sequence, and calculate the distance between the current hand position coordinate and the adjacent hand position coordinates; Select the maximum displacement distance from each adjacent coordinate pair and record it as the maximum displacement distance between adjacent coordinates; Arrange the maximum displacement distance of all adjacent coordinates in frame order to form a motion detail intensity sequence.

[0012] Preferably, step S3 includes: The scene complexity is calculated based on the scene structure analysis results in the fragment attributes, and the action intensity is calculated based on the main action analysis results in the fragment attributes. The scene complexity and action intensity are weighted and summed to obtain a comprehensive attribute score. Synchronous segments with a comprehensive attribute score higher than a preset threshold are marked as main shots, while those with a lower score are marked as auxiliary shots, thus forming the shot retention segments. The remaining footage segments are sorted in ascending order by trigger timestamp to obtain the initial link sequence, and the time interval between adjacent segments in the sequence is calculated; Adjacent segments with a time interval less than a preset threshold are merged into a continuous segment group, and all continuous segment groups are connected sequentially to construct a segment sequence with narrative fluidity.

[0013] Preferably, the retained footage segments are arranged in ascending order of their identified timestamps to obtain an initial link sequence, and the time interval between adjacent segments in the sequence is calculated, including: The retained clips of each shot are sorted in ascending order using the trigger timestamp corresponding to the retained clip of each shot as the sorting key. The sorted shot clips are then connected sequentially to form an initial link sequence; Select the current shot retention segment and the next shot retention segment from the initial link sequence one by one, and extract their respective trigger identifier timestamps; Calculate the difference between the trigger timestamp of the next shot's retained segment and the trigger timestamp of the current shot's retained segment to obtain the time interval between adjacent segments; Record the time intervals of all adjacent segments in sequence to form a time interval sequence.

[0014] This invention provides an intelligent narrative video generation method for fusing first-person and third-person perspectives. By capturing events in the third-person global view during a competition, it identifies high-frequency actions in real time and determines narrative trigger points, ensuring the generated video focuses on the segments with the most significant action changes during the competition. Based on trigger markers, it aligns the first-person and third-person perspectives in time, forming synchronized segments and achieving a high degree of temporal consistency between different perspectives, fully presenting the action process and scene information. Furthermore, by analyzing the scene structure of the third-person segments and the main action of the first-person segments, it obtains the distribution of moving subjects, action details, and the amplitude of the camera's perspective swing, providing a quantitative basis for shot selection. By comprehensively analyzing the scene complexity and action intensity in the segment attributes, the system determines the main and auxiliary shots and arranges the retained shot segments in the order of trigger time to construct a segment sequence with narrative fluidity, making the video coherent and rhythmic. Transition modes are automatically generated based on the rhythmic differences in action intensity in the segment sequence, and quick cuts or gradual blending transition effects are inserted at the boundaries to achieve smooth transitions between different perspectives, forming a structurally complete and narratively smooth shot sequence. The narrative shot sequence is spliced ​​into the target video to achieve an organic integration of first-person and third-person perspectives, so that the generated match highlights not only show the overall situation of the match, but also highlight the dynamic details of the main sports, improving the level of intelligent video processing and viewing experience. Attached Figure Description

[0015] Figure 1 A flowchart illustrating the steps of an intelligent narrative video generation method that integrates first-person and third-person perspectives; Figure 2 This is a schematic diagram of the original screen from a third-person global perspective. Figure 3 This is a first-person perspective view illustration; Figure 4 A block diagram of an intelligent narrative video generation apparatus according to an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a computer device suitable for implementing the embodiments of this application.

[0016] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0017] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0018] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0019] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0020] To achieve the above objectives, please refer to Figures 1 to 3 A method for generating intelligent narrative videos that integrates first-person and third-person perspectives includes the following steps: Step S1: During the game, perform event capture processing on the global screen from the third-person perspective; when a high-frequency action is detected, determine the moment as the narrative trigger point and output the trigger identifier; Step S2: Centered on the moment of the trigger mark, perform time alignment on the third-person view and the first-person view to form a synchronized segment; in the synchronized segment, perform scene structure analysis on the third-person segment and main action analysis on the first-person segment, and use the two types of analysis results as segment attributes; Step S3: Based on the fragment attributes, determine the main and auxiliary shots of the synchronized fragments and output the shot-preserving fragments; perform link arrangement on the shot-preserving fragments to construct a fragment sequence with narrative fluidity; Step S4: Generate transition patterns based on rhythm differences in the segment sequence, and insert matching transition effects at the boundaries of the segment sequence to form a narrative shot sequence; after the narrative shot sequence is generated, splice the segments one by one into the target video, and output a narrative match highlight reel that blends first-person and third-person perspectives.

[0021] In one embodiment, two imaging systems are deployed at the competition venue: one is a fixed-position camera covering the entire venue, with a resolution of 4K and a frame rate of 60fps; the other is a tracking camera mounted on a pan-tilt unit, with a focal length that can switch between 85mm and 135mm, and the frame rate is also set to 60fps. The two cameras are synchronized before the competition, with a time deviation of less than ±2ms. After the competition begins, the fixed-position camera footage is continuously input to the event detection unit. The event detection unit performs inter-frame motion calculation on the footage. The calculation method is to take the average amplitude of the motion vector field of each frame. If the average amplitude is higher than a set threshold A1 (e.g., 1.8 pixels / frame) for three consecutive frames, the timestamp T0 of that frame is recorded as a narrative trigger point and a corresponding trigger identifier is generated.

[0022] After the trigger flag is generated, time windows centered at T0 are extracted from both the fixed-camera sequence and the PTZ tracking sequence, with a window length set to 2 seconds. Frame-level time alignment is performed on both types of footage after input. By calculating the cross-correlation peak position of the brightness change sequence of each frame within the window, the time offset Δt between the two sequences is obtained. Frame shift compensation is then applied to the PTZ tracking sequence using Δt, thus forming a synchronized segment with consistent time. After the synchronized segment is established, scene structure analysis is performed on the fixed-camera segment. The analysis process is as follows: first, a background mask is generated using a full-field background modeling algorithm; then, the shape, center position, and spatial distribution ratio of the connected components of the moving subject are extracted outside the masked area to obtain a sequence of scene structure parameters for consecutive frames. Subject motion analysis is performed on the PTZ tracking segment, using three indicators: center frame jitter, local texture change, and viewpoint sway amplitude. These are calculated frame-by-frame and summarized into a motion feature sequence. These two types of results together constitute the segment attributes, providing a basis for subsequent shot judgment.

[0023] Subsequently, the segment attributes are evaluated to determine whether it is a primary or secondary shot. Scene structure parameters generate scene complexity C1 based on the rate of change of area proportion, and action feature sequences generate action intensity C2 based on the angle of view swing amplitude. The two are then summed with a weight of 1:1 to obtain a comprehensive attribute score S. If S ≥ threshold B1 (e.g., 0.65), the synchronous segment is marked as a primary shot, and the remaining synchronous segments are marked as secondary shots. If a synchronous segment is marked as a primary shot, in one embodiment, if C1 ≥ C2, the third-person view in the synchronous segment is selected as the shot to be retained; otherwise, the first-person view in the synchronous segment is selected as the shot to be retained. In another embodiment, a threshold B2 is set (e.g., 0.7). If both C1 and C2 are greater than or equal to B2, the system performs at least one perspective switch between the first-person view and the third-person view within the synchronous segment using a transition strategy. The system may first select to retain the third-person view to explain the overall scene relationship, and then transition to the first-person view to highlight the key subject's actions, or use the reverse order according to a preset narrative strategy. If a synchronized segment is marked as a secondary shot, in one embodiment, if C1 ≥ C2, the third-person view within that synchronized segment is selected as the retained shot segment; otherwise, the first-person view within that synchronized segment is selected as the retained shot segment. In another embodiment, in addition to determining the retained shot segment by comparing the values ​​of C1 and C2, a threshold B3 (e.g., 0.5) is set. If both C1 and C2 are less than or equal to B3, the system can choose to reduce the output duration of the retained shot segment. All retained shot segments are arranged in ascending order according to their trigger timestamps to obtain a segment link. If the time interval between adjacent segments in the link is less than the threshold B4 (e.g., 0.4s), the two segments are merged into a continuous segment group. The continuous segment groups are sequentially connected to form a segment sequence with time extension.

[0024] After the segment sequence is established, the rhythm difference within the sequence is analyzed. The rhythm difference is calculated by the difference in S-values ​​of consecutive segments. When the difference is greater than a threshold, different transition modes such as fade-in, fast cut, or push-pull are selected according to a preset correspondence. After the transition mode is implanted at the segment boundary, a narrative shot sequence is output, and the segments in the sequence are spliced ​​together in order to generate the target video, thus forming a narrative-style competition highlight that integrates first-person and third-person perspectives.

[0025] Please refer to [link / reference needed] for further information. Figure 3 The gimbal is used to track the focus of the competition (such as the athletes running ahead) to obtain stable and immersive close-up shots. The outdoor sports scene in the background corresponds to a third-person global view shot from a fixed position.

[0026] Preferably, step S2 includes: Based on the time point corresponding to the trigger marker, timeline alignment processing is performed on the first-person and third-person shot sequences to form a synchronized segment containing dual-view content. Analyze the scene structure of the third-person shots in the synchronized clips, and use the results of the scene structure analysis as the analysis result; Analyze the main action of the first-person shot in the synchronized segment, and use it as the result of the main action analysis; The scene structure analysis results and the main action analysis results are determined as fragment attributes.

[0027] In one embodiment, using the time point corresponding to the trigger identifier as a reference, timeline alignment processing is performed on the first-person and third-person shot sequences to form a synchronized segment containing dual-view content. During the timeline alignment process, the alignment reference is determined based on the time point of the trigger identifier. Frames adjacent to the alignment reference in the first-person and third-person shot sequences are corrected frame by frame according to the acquisition sequence. After correction, frames within the same time interval are combined into a synchronized segment. The scene structure of the third-person shot in the synchronized segment is analyzed as the scene structure analysis result. During the scene structure analysis process, the scene structure of the synchronized segment is used as the basis for further analysis. Each frame of third-person view is used as input to extract structural information that constitutes the scene structure, such as the visible area, occluded area, and field of view boundary. This extracted structural information is then arranged in frame order as the scene structure analysis result. The main action of the first-person view in the synchronized segment is analyzed as the main action analysis result. During the main action analysis, each frame of the first-person view in the synchronized segment is used as input to extract action elements such as the action amplitude, direction, and rhythm of the main action. These action elements are then arranged in frame order as the main action analysis result. The scene structure analysis result and the main action analysis result are then determined as segment attributes. The segment attributes are defined as a set of attributes composed of the scene structure analysis result and the main action analysis result, and this attribute set is mapped to the time interval of the synchronized segment to form the segment attributes of the synchronized segment.

[0028] Preferably, based on the time point corresponding to the trigger marker, timeline alignment processing is performed on the first-person and third-person shot sequences to form a synchronized segment containing dual-view content, including: The starting point and sequence span of the trigger flag are determined based on the trigger flag's time, and the frame sequences of the windows before and after that time are captured from the first-person and third-person perspectives, respectively. Perform time difference analysis on the frame sequences of first-person and third-person shots, determine the time axis offset between the frame sequences, and record the time deviation parameters; Based on the time deviation parameter, the frame time of the first-person and third-person shots is compensated and calibrated to form a unified time scale; Under a unified time scale, establish the correspondence between first-person and third-person frame views, and output synchronized segments containing dual-view content.

[0029] In one embodiment, the time start point and sequence span of the trigger identifier are determined based on the trigger identifier's time, and frame sequences of the windows before and after that time are captured from the first-person and third-person perspectives, respectively. During the capture process, a fixed number of frames are extended forward and backward from the time start point to generate two types of frame sequences. Time difference analysis is performed on the frame sequences of the first-person and third-person perspectives to determine the time axis offset between the frame sequences and record the time deviation parameters. The time difference analysis is based on the acquisition time of each frame, comparing the acquisition time frame by frame and calculating the offset value, and then organizing the offset values ​​into time values ​​according to the sequence order. The time deviation parameter is used to compensate and calibrate the frame time of the first-person and third-person shots to form a unified time scale. The compensation calibration uses the time deviation parameter as the correction amount to perform a single correction on the timestamps of the two types of frames, so that the corrected timestamps fall into the same time scale. Under the unified time scale, the correspondence between the first-person and third-person shot frames is established, and a synchronized segment containing dual-view content is output. The correspondence uses the adjacency of the timestamps as the matching condition, and the frames with the closest time are paired one by one, and the paired frames are combined as a synchronized segment.

[0030] In another embodiment, the starting point and sequence span of the trigger identifier are determined based on the trigger identifier's time, and frame sequences of the windows before and after that moment are extracted from the first-person and third-person perspectives, respectively. During extraction, the trigger identifier is used as the center point, the window length is fixed, and two types of window frame sequences are generated. Time difference analysis is performed on the frame sequences of the first-person and third-person perspectives to determine the time axis offset between the frame sequences and record the time deviation parameters. The time difference analysis uses the start and end times of the window frame sequence as references to calculate the overall offset of the frame sequence and records the offset as the time deviation parameters. Based on the time deviation parameters, the frame time of the first-person and third-person perspectives is compensated and calibrated to form a unified time scale. The compensation and calibration uses the offset as the adjustment basis to perform unified time adjustment on the two types of frame sequences, so that the adjusted frame sequences fall into a consistent time interval. Under the unified time scale, a correspondence between the first-person and third-person perspective frames is established, and a synchronized segment containing dual-view content is output. The establishment of the correspondence uses frames within the same time interval as the matching standard, and frames with overlapping time intervals are paired to form synchronized segments.

[0031] Preferably, time difference analysis is performed on the frame sequences of first-person and third-person shots to determine the time axis offset between the frame sequences, and the time deviation parameters are recorded, including: Based on the time point of the trigger identifier, extract a continuous set of frames of equal length from the first-person view frame sequence and the third-person view frame sequence respectively. In a continuous set of frames, the continuous changes in the images are compared frame by frame to determine the segments of change in the same image. For each set of change segments, read its time information in its respective frame sequence and calculate the time difference between the two. The average of all time differences is used as the overall offset between the two time axes, and the result is output as a time deviation parameter.

[0032] In one embodiment, taking the time point corresponding to the trigger identifier as the center, video frame sequences of 2 seconds before and after are extracted from the first-person perspective video (1080p, 30fps) and the third-person perspective video (1080p, 25fps), respectively, to obtain a first-person continuous frame set of 60 frames and a third-person continuous frame set of 50 frames. The two types of frame sequences are compared frame by frame: the bounding box of the moving subject and the background contour in each frame are extracted, and the inter-frame difference index of adjacent frames (such as the absolute value of pixel difference or contour overlap rate) is calculated; a segment of frames with continuous differences below the threshold (0.1) is determined as a change segment of the same scene; for each group of change segments of the same scene, the start timestamp of the segment in the first-person frame sequence and the third-person frame sequence is read; the difference of the start timestamp of the change segment is calculated as the time difference value of the segment (unit: seconds); the time difference values ​​of all change segments are averaged to obtain the overall time offset of the two frame sequences, which is used as a reference for subsequent synchronization compensation and recorded as a time deviation parameter.

[0033] In another embodiment, taking the time point corresponding to the trigger identifier as the center, 1.5-second video frame sequences are extracted from the first-person perspective video (1920×1080, 30fps) and the third-person perspective video (1920×1080, 30fps) respectively, resulting in two consecutive frame sets of 45 frames each. Inter-frame brightness change analysis is performed on the frame sets: the mean square error (MSE) between each frame's grayscale image and the previous frame is calculated, and frames with consecutive MSE values ​​below the threshold (50 grayscale units) are marked as the same change segment; the time information of each group of change segments in the first-person and third-person frame sequences is read, and the timestamps corresponding to the start and end frames are extracted; the time difference value of each group of change segments (the average of the difference between the start and end timestamps) is calculated to obtain the time offset of that change segment; the time offsets of all change segments are averaged, and the overall time offset is output and recorded as a time deviation parameter for subsequent frame time compensation.

[0034] Preferably, the scene structure of the third-person shots in the synchronized segments is analyzed, and the scene structure analysis results include: In the synchronized segment, scene segmentation is performed on the third-person frame sequence to distinguish between the foreground moving area and the background static area, and the spatial proportion of each area is recorded to form a scene area map; Target detection is performed on the scene area map to identify all moving subjects, extract the center coordinates of the bounding box of each subject and the pixel ratio, and form a moving subject distribution map; Perform differential processing on the motion subject distribution maps of adjacent frames to calculate the change in the center displacement of the bounding box and the change in the pixel ratio; When the center displacement or pixel ratio change of any frame exceeds the threshold, the frame is marked as a view coverage change event. The distribution map of the moving subjects and the set of timestamps of the events covering the change in viewpoint are combined to form the scene structure analysis result.

[0035] In one embodiment, a global frame sequence from a third-person perspective is extracted from the synchronized segment. Each frame has a resolution of 1920×1080 and a sampling frequency of 30 frames per second. The entire synchronized segment contains approximately 90 frames. Scene segmentation is performed on the frame sequence. Optical flow and background modeling algorithms are used to distinguish between foreground moving regions and background static regions. The pixel percentage of the foreground and background regions in each frame is calculated to form a scene region map for each frame. On the generated scene region map, the YOLOv5 object detection model is used to identify all moving subjects. Bounding boxes are extracted for each moving subject, and the center coordinates of the bounding boxes are calculated. At the same time, the pixel percentage covered by the bounding boxes is statistically analyzed to generate a moving subject distribution map. Subsequently, the moving subject distribution maps of adjacent frames are differentially processed. The inter-frame displacement distance is calculated by matching the center coordinates of the bounding boxes of the same moving subjects, and the absolute difference in the pixel percentage of adjacent frames is calculated to obtain the change in pixel percentage. When the center displacement distance of the bounding box or the change in pixel percentage of any frame exceeds a preset threshold, the frame is marked as a viewpoint coverage change event, and the timestamp of the corresponding frame is recorded. The distribution map of the moving subjects and the set of timestamps of the events covering the change in perspective are combined to form the scene structure analysis result.

[0036] In another embodiment, a third-person perspective video frame sequence is selected from the synchronized segment, with each frame having a resolution of 1280×720 and a sampling frequency of 25 frames per second. The entire synchronized segment contains approximately 75 frames. Foreground-background separation processing is performed on the frame sequence. A frame difference method based on background modeling is used to identify moving foreground regions and static background regions, and the spatial proportion of each region in each frame is calculated to form a scene region map. Object detection is performed on the scene region map. A pre-trained SSD model is used to identify moving subjects, extracting the bounding box center coordinates of each subject and calculating the bounding box pixel proportion, thereby generating a moving subject distribution map. Then, a difference operation is performed on the moving subject distribution maps of consecutive frames. The inter-frame displacement distance is calculated for the bounding box center coordinates of the same moving subject, and the pixel proportion change is calculated by absolute difference. When the center displacement or pixel proportion change of a frame exceeds a threshold, the frame is recorded as a viewpoint coverage change event, and its timestamp information is stored. The moving subject distribution maps and the set of timestamps for all frames are combined to form a complete scene structure analysis result.

[0037] Please refer to [link / reference needed] for further information. Figure 2 It is a third-person perspective (a global view shot from a fixed camera position), presenting the complete spatial structure of the stadium and the distribution of all participants. It serves as the starting point and foundation, using the detection of high-frequency actions such as "dribbling attack" as the starting point of the story, and analyzing information such as player distribution and field structure.

[0038] Preferably, differential processing is performed on the motion subject distribution maps of adjacent frames, and the calculation of the change in the center displacement of the bounding box and the change in pixel proportion includes: Using consecutive third-person frames within a synchronized segment as the order, select the motion subject distribution map of the current frame and the previous frame to form adjacent distribution map pairs; In adjacent distribution map pairs, center coordinate matching is performed on the bounding boxes of the same moving subject, and the inter-frame displacement distance of the center coordinates is calculated; The area of ​​the bounding boxes of the same moving subject is statistically analyzed, and the absolute difference in the pixel ratio is calculated to obtain the change in pixel ratio. Record the inter-frame displacement distance and pixel ratio change at the corresponding inter-frame positions.

[0039] In one embodiment, within a synchronized segment, consecutive third-person frame images are arranged chronologically, and each frame forms an adjacent distribution map pair with its preceding frame. Within each adjacent distribution map pair, based on the bounding box identifiers of the moving subject, the center coordinates of the bounding boxes of the same subject are matched, the displacement distance between the center coordinates of the current frame and the center coordinates of the previous frame is calculated, and this displacement value is recorded. Subsequently, the bounding box areas of the same moving subject are statistically analyzed, and the absolute difference between the pixel percentage of the current frame and the pixel percentage of the previous frame is calculated to obtain the change in pixel percentage. The calculated inter-frame center displacement distance and the change in pixel percentage are stored in their respective inter-frame positions, forming a complete differential data sequence.

[0040] In another embodiment, within the synchronized segment, consecutive third-person frame sequences are processed sequentially in chronological order, with each frame forming an adjacent distribution map pair with its preceding frame. For each adjacent distribution map pair, the unique ID of the moving subject is used to match the center coordinates of the bounding boxes of the same subject, the displacement distance of the center coordinates between the two frames is calculated, and this displacement information is saved. Simultaneously, the pixel area of ​​the bounding box of the same subject is statistically analyzed, and the absolute change in pixel percentage is calculated to obtain the change in pixel percentage. The displacement distance and the change in pixel percentage for each frame are recorded sequentially to form complete inter-frame difference data for subsequent scene structure analysis.

[0041] Preferably, the main action of the first-person shot in the synchronized segment is analyzed, and the results of the main action analysis include: Using the starting frame of the synchronized segment as a reference, target center point trajectory tracking is performed on the first-person frame sequence, and the displacement distance of the target center point in adjacent frames is calculated to form a viewpoint swing amplitude sequence; Within the same frame sequence, hand position recognition is performed on the subject of the action, hand position coordinates are extracted, the maximum displacement distance between adjacent coordinates is calculated, and a motion detail intensity sequence is formed. The viewpoint swing amplitude sequence and the motion detail intensity sequence are combined to form the motion analysis result.

[0042] In one embodiment, within a synchronized segment, using the starting frame as a reference, the target center point trajectory is tracked across the first-person view frame sequence. The change in the target center point's position is calculated frame by frame to obtain the displacement distance between adjacent frames. These displacement values ​​are recorded sequentially to form a complete viewpoint sway amplitude sequence. Simultaneously, within the same frame sequence, the hand position of the subject is identified, the hand coordinates for each frame are extracted, the displacement distance is calculated for the coordinates of adjacent frames, and the maximum displacement value for each frame is selected to form a motion detail intensity sequence. Combining the viewpoint sway amplitude sequence and the motion detail intensity sequence yields a complete subject motion analysis result.

[0043] In another embodiment, within a synchronized segment, starting from the initial frame, target center point tracking is performed on the first-person view frame sequence. By recording the target center point coordinates in each frame and calculating the displacement distance of the target in adjacent frames, a viewpoint sway amplitude sequence is obtained. Subsequently, in the same frame sequence, the hand position of the subject is detected, the hand coordinates in each frame are extracted, and the distance values ​​of the hand coordinates in adjacent frames are calculated. The maximum value is selected to form a motion detail intensity sequence. The viewpoint sway amplitude sequence and the motion detail intensity sequence are combined in chronological order to generate a complete subject motion analysis result.

[0044] Preferably, within the same frame sequence, hand position recognition is performed on the subject of the action, hand position coordinates are extracted, and the maximum displacement distance between adjacent coordinates is calculated to form an action detail intensity sequence, including: Using the starting frame of the same frame sequence as a reference, hand position recognition is performed on the subject of the action, the hand position coordinates are extracted, and recorded into the coordinate sequence; Perform displacement calculations on each adjacent coordinate pair in the coordinate sequence, and calculate the distance between the current hand position coordinate and the adjacent hand position coordinates; Select the maximum displacement distance from each adjacent coordinate pair and record it as the maximum displacement distance between adjacent coordinates; Arrange the maximum displacement distance of all adjacent coordinates in frame order to form a motion detail intensity sequence.

[0045] In one embodiment, within the same frame sequence, using the starting frame as a reference, a target detection model is applied to identify the hand position of the subject. This model can employ a deep convolutional neural network architecture, such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), or Faster R-CNN (Faster Region-based Convolutional Neural Network). After model processing, the detection box of the subject's hand in the frame is output, typically represented by four parameters (x, y, width, height). Here, x and y represent the horizontal and vertical coordinates of the detection box's center point, respectively, together constituting the coordinates of the subject's hand center point; width represents the width of the detection box, and height represents its height. The hand center point coordinates of each frame are extracted and sequentially recorded in the coordinate sequence. Displacement calculations are performed on each pair of adjacent frames' hand center point coordinates in the coordinate sequence to obtain the distance between the current frame and the next frame. The maximum distance is selected from each frame and recorded as the maximum displacement value for that frame. Arrange the maximum displacement values ​​of all adjacent frames in chronological order to form a complete motion detail intensity sequence.

[0046] In another embodiment, within the same frame sequence, starting from the first frame, a target detection model is used to detect the position of the subject's hand. This model can employ a deep convolutional neural network architecture, such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), or Faster R-CNN (Faster Region-based Convolutional Neural Network). After model processing, the detection box for the subject's hand in each frame is output, typically represented by four parameters (x, y, width, height). Here, x and y represent the horizontal and vertical coordinates of the detection box's center point, respectively, together constituting the coordinates of the subject's hand's center point; width represents the width of the detection box, and height represents its height. The hand's center point coordinates are extracted sequentially for each frame, constructing a coordinate sequence. Then, the distance between the hand's center point coordinates of each adjacent frame in the coordinate sequence is calculated, and the maximum displacement value in each pair of coordinates is selected as the motion intensity for that frame. The maximum displacement values ​​of all frames are arranged sequentially to generate a complete motion detail intensity sequence.

[0047] Preferably, step S3 includes: The scene complexity is calculated based on the scene structure analysis results in the fragment attributes, and the action intensity is calculated based on the main action analysis results in the fragment attributes. The scene complexity and action intensity are weighted and summed to obtain a comprehensive attribute score. Synchronous segments with a comprehensive attribute score higher than a preset threshold are marked as main shots, while those with a lower score are marked as auxiliary shots, thus forming the shot retention segments. The remaining footage segments are sorted in ascending order by trigger timestamp to obtain the initial link sequence, and the time interval between adjacent segments in the sequence is calculated; Adjacent segments with a time interval less than a preset threshold are merged into a continuous segment group, and all continuous segment groups are connected sequentially to construct a segment sequence with narrative fluidity.

[0048] In one embodiment, based on the scene structure analysis results in the segment attributes of the synchronized segments, the number of moving subjects and their spatial coverage ratio are counted, the scene change between frames is calculated, and the scene complexity C1 is accumulated. Simultaneously, based on the viewpoint swing amplitude sequence and action detail intensity sequence from the subject action analysis results, the action intensity C2 is calculated for each synchronized segment. The action intensity C2 and the scene complexity C1 are weighted and summed according to a pre-set weight to obtain a comprehensive attribute score S. Synchronized segments with a comprehensive attribute score S higher than a threshold B1 (e.g., 0.65) are marked as main shots, and those lower than the threshold B1 are marked as auxiliary shots. If a synchronized segment is marked as the main shot, in one embodiment, if C1≥C2, then the third-person view in the synchronized segment is selected as the shot to be retained; otherwise, the first-person view in the synchronized segment is selected as the shot to be retained. In another embodiment, a threshold B2 (e.g., 0.7) is set. If both C1 and C2 are greater than or equal to B2, then the system performs at least one perspective switch between the first-person view and the third-person view within the synchronized segment using a transition strategy. The system may first select to retain the third-person view to explain the overall scene relationship, and then transition to the first-person view to highlight the key subject's actions, or adopt the reverse order according to a preset narrative strategy. If a synchronized segment is marked as a secondary shot, in one embodiment, if C1 ≥ C2, the third-person shot in that synchronized segment is selected as the retained shot segment; otherwise, the first-person shot in that synchronized segment is selected as the retained shot segment. In another embodiment, in addition to determining the retained shot segment by comparing the sizes of C1 and C2, a threshold B3 (e.g., 0.5) is set. If both C1 and C2 are less than or equal to B3, the system can choose to reduce the output duration of the retained shot segment. Subsequently, the retained shot segments are sorted in ascending order by the trigger identifier timestamp to obtain the initial link sequence, and the time interval between adjacent segments is calculated for each pair. Segments with a time interval less than the set threshold B4 (e.g., 0.4s) are merged into continuous segment groups, and all continuous segment groups are connected sequentially to construct a complete segment sequence with narrative fluidity.

[0049] In another embodiment, based on the segment attributes of the synchronized segments, the proportion of the foreground moving area to the background static area is first calculated according to the scene structure analysis results, and the inter-frame coverage change is calculated to obtain the scene complexity C1 of each synchronized segment. At the same time, the action intensity C2 is calculated based on the perspective swing amplitude sequence and action detail intensity sequence of the main action analysis results. The scene complexity C1 and action intensity C2 are summed according to the weighted rules to obtain the comprehensive attribute score S, and the synchronized segment is determined to be a main shot or a secondary shot using a threshold B1 (e.g., 0.65). If a synchronized segment is marked as the main shot, in one embodiment, if C1≥C2, then the third-person view in the synchronized segment is selected as the shot to be retained; otherwise, the first-person view in the synchronized segment is selected as the shot to be retained. In another embodiment, a threshold B2 (e.g., 0.7) is set. If both C1 and C2 are greater than or equal to B2, then the system performs at least one perspective switch between the first-person view and the third-person view within the synchronized segment using a transition strategy. The system may first select to retain the third-person view to explain the overall scene relationship, and then transition to the first-person view to highlight the key subject's actions, or adopt the reverse order according to a preset narrative strategy. If a synchronized segment is marked as a secondary shot, in one embodiment, if C1 ≥ C2, the third-person shot in that synchronized segment is selected as the retained shot segment; otherwise, the first-person shot in that synchronized segment is selected as the retained shot segment. In another embodiment, in addition to determining the retained shot segment by comparing the sizes of C1 and C2, a threshold B3 (e.g., 0.5) is set. If both C1 and C2 are less than or equal to B3, the system can choose to reduce the output duration of the retained shot segment. The retained shot segments are arranged in ascending order according to the trigger identifier timestamp to form an initial link sequence. The time interval between each pair of adjacent segments is calculated. Segments with an interval less than the threshold B4 (e.g., 0.4s) are merged into a continuous segment group. Then, all continuous segment groups are connected sequentially to obtain the final narrative flow segment sequence.

[0050] Preferably, the retained shot segments are arranged in ascending order according to their identified timestamps to obtain an initial link sequence, and the time interval between adjacent segments in the sequence is calculated, including: The retained clips of each shot are sorted in ascending order using the trigger timestamp corresponding to the retained clip of each shot as the sorting key. The sorted shot clips are then connected sequentially to form an initial link sequence; Select the current shot retention segment and the next shot retention segment from the initial link sequence one by one, and extract their respective trigger identifier timestamps; Calculate the difference between the trigger timestamp of the next shot's retained segment and the trigger timestamp of the current shot's retained segment to obtain the time interval between adjacent segments; Record the time intervals of all adjacent segments in sequence to form a time interval sequence.

[0051] In one embodiment, based on the trigger timestamp corresponding to each retained shot fragment, all retained shot fragments are sorted in ascending order to obtain an ordered shot sequence. The sorted retained shot fragments are then sequentially connected to form an initial link sequence. Pairs of the current retained shot fragment and the next retained shot fragment in the initial link sequence are selected, their respective trigger timestamps are read, and the difference between the timestamp of the next fragment and the timestamp of the current fragment is calculated. This difference is used as the time interval between adjacent fragments. This operation is repeated for all fragment pairs in the initial link sequence, and all calculated time intervals are recorded sequentially to form a complete time interval sequence, providing a data foundation for subsequent fragment merging or link optimization. Specifically, the timestamp numbers can be directly used as the sorting key, or the timestamp numbers can be simplified, mapped, or otherwise processed using labels that do not affect their relative size before sorting.

[0052] Most importantly, using the starting frame of the synchronized segment as a reference, target center point trajectory tracking is performed on the first-person frame sequence, and the displacement distance of the target center point in adjacent frames is calculated to form a sequence of viewpoint swing amplitudes, including: Based on the starting frame of the synchronization segment, the current frame is selected frame by frame, the algorithm is processed on the first-person frame sequence, the pixel-level movement trajectory of the first-person target center point is tracked, the target center point coordinates of the current frame are generated, and recorded to the target center point coordinate sequence. Perform matching calculations on each adjacent coordinate pair in the target center point coordinate sequence, calculate the Euclidean distance between the target center point coordinates of the current frame and the target center point coordinates of the previous frame, use it as the displacement distance of the target center point in the adjacent frames, and record it in the displacement distance sequence; Peak filtering is performed on the displacement distance sequence, and displacement distances greater than a preset displacement threshold in the displacement distance sequence are marked as valid swing events, and the duration of valid swing events is calculated. The displacement distances of the target center points in all adjacent frames are arranged in frame order and the duration of effective sway events is weighted to form a sequence of viewpoint sway amplitudes.

[0053] In one embodiment, using the starting frame of the synchronized segment as a reference, the current frame is selected frame by frame from the first-person frame sequence. An object detection algorithm is then applied to the current frame, such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), or Faster R-CNN (Faster Region-based Convolutional Neural Network). After processing, the object detection box of the first-person target in the current frame is output, typically represented by four parameters (x, y, width, height). Here, x and y represent the horizontal and vertical coordinates of the target detection box's center point, respectively, together forming the coordinates of the first-person target's center point; width represents the width of the target detection box, and height represents its height. Based on the frame-by-frame processing of the first-person frame sequence, the pixel-level movement trajectory of the first-person target's center point is tracked, and the target center point coordinates for each frame are recorded in the target center point coordinate sequence. Subsequently, the Euclidean distance is calculated for each adjacent coordinate pair in the target center point coordinate sequence to obtain the displacement distance of the target center point in each pair of adjacent frames, and these displacement distances are recorded sequentially in the displacement distance sequence. Peak filtering is performed on the displacement distance sequence, and displacement distances greater than a preset threshold are marked as valid swing events, and the duration of each valid swing event is counted. The displacement distances of all adjacent frames are arranged in frame order, and combined with the duration of valid swing events as weights to generate a complete viewpoint swing amplitude sequence.

[0054] In another embodiment, starting with the first frame of the synchronized segment, the first-person frame sequence is processed frame by frame, executing target detection algorithms such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), and Faster R-CNN (Faster Region-based Convolutional Neural Network) frame by frame. After processing, the target detection box of the first-person target in each frame is output, typically represented by four parameters (x, y, width, height). Here, x and y represent the horizontal and vertical coordinates of the target detection box's center point, respectively, together constituting the coordinates of the first-person target's center point; width represents the width of the target detection box, and height represents the height of the target detection box. Based on the frame-by-frame processing of the first-person frame sequence, the trajectory of the first-person target's center point is tracked, and the coordinates of the target's center point in each frame are recorded in the sequence. The Euclidean distance is calculated for each pair of consecutive frames in the target's center point coordinate sequence, serving as the displacement of the target's center point in adjacent frames, and the result is stored in the displacement distance sequence. Peak filtering is applied to the displacement distance sequence to identify displacements exceeding a preset threshold as valid swings, and the duration of these swings is counted. The displacement distance sequence is then arranged chronologically, and weights are assigned based on the duration of valid swing events to generate a viewpoint swing amplitude sequence for subsequent motion analysis or segment attribute calculation.

[0055] Of particular importance, step S4 includes: Select the current segment and the next segment in the segment sequence one by one, and extract the motion intensity value from their respective segment attributes; Calculate the absolute difference between the intensity value of the next segment and the intensity value of the current segment to obtain the rhythm drop value; Compare the rhythm drop value with the preset drop threshold; Boundaries with a rhythm difference value greater than a preset difference threshold are marked as high-rhythm transition modes, and boundaries with a rhythm difference value not greater than a preset difference threshold are marked as low-rhythm transition modes. Quick cut transitions are inserted at the boundaries of high-paced transition modes, and gradual blending transitions are inserted at the boundaries of low-paced transition modes to form a narrative shot sequence. Frame-level stitching is performed on all segments in the narrative shot sequence in sequence order to generate a continuous video stream; Encode continuous video streams into a standard video format and output narrative match highlights that blend first-person and third-person perspectives.

[0056] In one embodiment, based on a sequence of segments, the current segment and the next segment are selected in pairs. The motion intensity value is read from the respective segment attributes, and the absolute difference between the motion intensity of the next segment and the current segment is calculated to obtain the rhythm difference value. All rhythm difference values ​​are compared with a preset difference threshold. Boundaries greater than the threshold are marked as high-rhythm transition modes, and boundaries not greater than the threshold are marked as low-rhythm transition modes. A fast-cut transition effect is inserted at the boundaries of high-rhythm transition modes, and a gradual blending transition effect is inserted at the boundaries of low-rhythm transition modes, thus forming a complete narrative shot sequence. Subsequently, the segments in the narrative shot sequence are spliced ​​frame-by-frame in sequence order, and all frames are continuously synthesized into a video stream. The video stream is then encoded to generate a standard video file, outputting a narrative-style match highlight reel that blends first-person and third-person perspectives.

[0057] In another embodiment, starting with the first segment, the current segment and the next segment are selected in pairs, and the motion intensity values ​​in their respective segment attributes are read. The absolute value of the difference in motion intensity between the two segments is calculated as the rhythm drop. Based on the comparison result of the drop value with a preset threshold, the boundary is divided into a high-rhythm transition mode or a low-rhythm transition mode. A fast cut transition effect is inserted at the boundary of the high-rhythm transition mode, and a fade-in transition effect is inserted at the boundary of the low-rhythm transition mode, thereby generating a narrative shot sequence. Subsequently, the frames of each segment are sequentially spliced ​​according to the shot sequence to form a continuous video stream, and the video stream is encoded into a standard format file to output a complete match highlight that includes a fusion of first-person and third-person perspectives.

[0058] Of particular importance is the insertion of rapid cut transitions at the boundaries of high-paced transition modes and gradual blending transitions at the boundaries of low-paced transition modes, forming a narrative shot sequence including: The transition mode markers of each boundary are read, taking the boundaries of adjacent segments in the segment sequence as the processing objects. When the boundary is marked as a high-rhythm transition mode, the switch is performed directly at the boundary, and a dynamic impact effect frame lasting 0.1-0.3 seconds is superimposed after the switch frame; When the boundary is marked as a low-rhythm transition mode, a cross-fade-out transition lasting 0.8-1.5 seconds is generated at the boundary, and during the transition, a slow blur is applied to the first-person view and a synchronous transparency gradient is applied to the third-person view. All segments after the insertion of quick cut transitions or fade-in transitions are reconnected in their original sequence order to form a complete narrative shot sequence with matching transition effects.

[0059] In one embodiment, the boundary of each pair of adjacent segments in the segment sequence is used as the processing object, and the transition mode marker corresponding to the boundary is read. When the boundary marker is a high-rhythm transition mode, the camera switches directly at the boundary frame, and a dynamic impact effect is superimposed on the next 5-9 frames after the switch to make the transition more visually impactful. When the boundary marker is a low-rhythm transition mode, a cross-fade-in / fade-out transition effect lasting 24-36 frames is generated at the boundary. At the same time, slow blur processing is applied to the first-person view and synchronous transparency gradient processing is applied to the third-person view to maintain visual continuity. After the transition effect is implanted, all processed segments are reconnected in the original sequence order to form a complete narrative shot sequence.

[0060] In another embodiment, the boundaries of each adjacent segment in the segment sequence are selected, and the transition mode markers of the boundaries are read. For boundaries marked as high-rhythm transition modes, a quick cut is performed directly at the boundary frame, and motion impact effect frames are superimposed for 0.12-0.28 seconds after the switch to highlight the rhythm change. For boundaries marked as low-rhythm transition modes, a 0.8-1.2 second fade-in / fade-out cross transition is generated at the boundary frame. During the transition, a slight motion blur is applied to the first-person view, and a linear gradient of transparency is applied to the third-person view to ensure a smooth transition. After all boundary transition processing is completed, the segments are spliced ​​together in sequence to form a complete narrative shot sequence with matching transition effects.

[0061] This application also provides an intelligent narrative video generation device, comprising: The event triggering module is suitable for performing event capture and processing on the global screen from a third-person perspective during the game; when a high-frequency action is detected, the moment is determined as the narrative trigger point and a trigger identifier is output. The synchronization parsing module is suitable for performing time alignment between third-person and first-person perspective shots, centered on the moment of the trigger marker, to form a synchronized segment. In the synchronized segment, the scene structure of the third-person segment is analyzed, and the main action of the first-person segment is analyzed. The two types of analysis results are used as segment attributes. The judgment and arrangement module is suitable for performing main and auxiliary shot determination on synchronous segments based on segment attributes and outputting shot-retained segments; performing link arrangement on shot-retained segments to construct a segment sequence with narrative fluidity; The synthesis output module is suitable for generating transition patterns based on the rhythm differences in the segment sequence and inserting matching transition effects at the boundaries of the segment sequence to form a narrative shot sequence. After the narrative shot sequence is generated, the segments are spliced ​​together in sequence to form the target video, and a narrative-style match highlight reel that blends first-person and third-person perspectives is output.

[0062] Figure 4This is a block diagram of an intelligent narrative video generation apparatus 400 according to an embodiment of this application. Figure 4 As shown, the intelligent narrative video generation device 400 includes an event triggering module 401, a synchronous parsing module 402, a judgment and arrangement module 403, and a synthesis output module 404.

[0063] The event triggering module 401 is adapted to perform event capture processing on the global screen from a third-person perspective during the game. When a high-frequency action is detected, the moment is identified as a narrative trigger point, and a trigger identifier is output. In one embodiment, the module can achieve real-time or offline event recognition based on a pre-trained deep learning model; in another embodiment, the module can also receive and process structured event data provided by other detection systems or interfaces. The event category can be predefined or dynamically configured according to the application scenario. The event triggering module 401 is adapted to output a trigger identifier and send the trigger identifier to the synchronization parsing module 402. Optionally, the event triggering module 401 may not have an active information sending function, and instead, the synchronization parsing module 402 actively obtains the trigger identifier from the event triggering module 401.

[0064] The synchronization parsing module 402 is adapted to perform time alignment between the third-person and first-person perspective views, centered on the moment of the trigger marker, to form a synchronized segment. Within the synchronized segment, the third-person segment undergoes scene structure analysis, and the first-person segment undergoes main action analysis, with the two analysis results serving as segment attributes. The synchronization parsing module 402 can be implemented as an executable program or code in a smart terminal or server. The synchronization parsing module 402 is adapted to form synchronized segments and parse segment attributes, and then send the formed segment attributes to the judgment and arrangement module 403. Optionally, the synchronization parsing module 402 may not have an active information sending function; instead, the judgment and arrangement module 403 may actively obtain segment attributes from the synchronization parsing module 402.

[0065] The decision-making and arrangement module 403 is adapted to perform main and auxiliary shot determination on synchronized segments based on segment attributes and output the shot-retained segments; it performs link arrangement on the shot-retained segments to construct a segment sequence with narrative fluidity. The decision-making and arrangement module 403 can be implemented as an executable program or code in a smart terminal or server. The decision-making and arrangement module 403 is adapted to construct a segment sequence with narrative fluidity and send the constructed segment sequence with narrative fluidity to the synthesis output module 404. Optionally, the decision-making and arrangement module 403 may not have an active information sending function, and instead, the synthesis output module 404 actively obtains the segment sequence with narrative fluidity from the synchronous decision-making and arrangement module 403.

[0066] The compositing output module 404 is adapted to generate transition patterns based on rhythmic differences in the segment sequence and insert matching transition effects at the boundaries of the segment sequence to form a narrative shot sequence. After the narrative shot sequence is generated, the segments are sequentially spliced ​​together to form the target video, outputting a narrative-style match highlight reel that blends first-person and third-person perspectives. The compositing output module 404 can be implemented as an executable program or code in a smart terminal or server.

[0067] The specific methods for each module of this device have been described in detail in the aforementioned corresponding embodiments, and will not be repeated here.

[0068] This application also provides a computing device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the intelligent narrative video generation method for fusing first-person and third-person perspectives as described in this invention.

[0069] Figure 5 A schematic diagram of a computer device suitable for implementing embodiments of this application is shown. The computer device can be implemented as a terminal device or a server.

[0070] like Figure 5 As shown, the terminal device or server includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the terminal device or server. The CPU 501, ROM 502, and RAM 503 are interconnected via bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0071] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0072] Specifically, according to embodiments of this application, the above method flow steps can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program carried on a machine-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined in the system of this application.

[0073] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, or any suitable combination thereof.

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0075] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be located in a processor. The names of these units or modules do not, in certain circumstances, constitute a limitation on the unit or module itself.

[0076] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium stores one or more programs that are used by one or more processors to execute the methods described in this application.

[0077] In another aspect, embodiments of this application also provide a computer program product that, when executed by a processor, implements the methods of any of the above embodiments.

[0078] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for generating intelligent narrative videos that integrates first-person and third-person perspectives, characterized in that, The first-person perspective is the view obtained by tracking the focus of the competition using a gimbal, and the third-person perspective is the overall view of the competition captured by a fixed camera. The method includes the following steps: Step S1: During the game, perform event capture processing on the global screen from the third-person perspective; when a high-frequency action is detected, determine the moment as the narrative trigger point and output the trigger identifier; Step S2: Centered on the moment of the trigger mark, perform time alignment on the third-person view and the first-person view to form a synchronized segment; in the synchronized segment, perform scene structure analysis on the third-person segment and main action analysis on the first-person segment, and use the two types of analysis results as segment attributes; Step S3: Based on the fragment attributes, determine the main and auxiliary shots of the synchronized fragments and output the shot-preserving fragments; perform link arrangement on the shot-preserving fragments to construct a fragment sequence with narrative fluidity; Step S4: Generate transition patterns based on rhythm differences in the segment sequence, and insert matching transition effects at the boundaries of the segment sequence to form a narrative shot sequence; after the narrative shot sequence is generated, splice the segments one by one into the target video, and output a narrative match highlight reel that blends first-person and third-person perspectives.

2. The intelligent narrative video generation method for fusing first-person and third-person perspectives according to claim 1, characterized in that, Step S2 includes: Based on the time point corresponding to the trigger marker, timeline alignment processing is performed on the first-person and third-person shot sequences to form a synchronized segment containing dual-view content. Analyze the scene structure of the third-person shots in the synchronized clips, and use the results of the scene structure analysis as the analysis result; Analyze the main action of the first-person shot in the synchronized segment, and use it as the result of the main action analysis; The scene structure analysis results and the main action analysis results are determined as fragment attributes.

3. The intelligent narrative video generation method for fusing first-person and third-person perspectives according to claim 2, characterized in that, Based on the time point corresponding to the trigger marker, timeline alignment is performed on the first-person and third-person shot sequences to form synchronized segments containing dual-view content, including: The starting point and sequence span of the trigger flag are determined based on the trigger flag's time, and the frame sequences of the windows before and after that time are captured from the first-person and third-person perspectives, respectively. Perform time difference analysis on the frame sequences of first-person and third-person shots, determine the time axis offset between the frame sequences, and record the time deviation parameters; Based on the time deviation parameter, the frame time of the first-person and third-person shots is compensated and calibrated to form a unified time scale; Under a unified time scale, establish the correspondence between first-person and third-person frame views, and output synchronized segments containing dual-view content.

4. The intelligent narrative video generation method for fusing first-person and third-person perspectives according to claim 3, characterized in that, Perform time difference analysis on the frame sequences of first-person and third-person shots to determine the time axis offset between the frame sequences and record the time deviation parameters, including: Based on the time point of the trigger identifier, extract a continuous set of frames of equal length from the first-person view frame sequence and the third-person view frame sequence respectively. In a continuous set of frames, the continuous changes in the images are compared frame by frame to determine the segments of change in the same image. For each set of change segments, read its time information in its respective frame sequence and calculate the time difference between the two. The average of all time differences is used as the overall offset between the two time axes, and the result is output as a time deviation parameter.

5. The intelligent narrative video generation method for fusing first-person and third-person perspectives according to claim 2, characterized in that, The scene structure of the third-person shots in the synchronized clips is analyzed, and the results of the scene structure analysis include: In the synchronized segment, scene segmentation is performed on the third-person frame sequence to distinguish between the foreground moving area and the background static area, and the spatial proportion of each area is recorded to form a scene area map; Target detection is performed on the scene area map to identify all moving subjects, extract the center coordinates of the bounding box of each subject and the pixel ratio, and form a moving subject distribution map; Perform differential processing on the motion subject distribution maps of adjacent frames to calculate the change in the center displacement of the bounding box and the change in the pixel ratio; When the center displacement or pixel ratio change of any frame exceeds the threshold, the frame is marked as a view coverage change event. The distribution map of the moving subjects and the set of timestamps of the events covering the change in viewpoint are combined to form the scene structure analysis result.

6. The intelligent narrative video generation method for fusing first-person and third-person perspectives according to claim 5, characterized in that, Perform differential processing on the motion subject distribution maps of adjacent frames, and calculate the change in bounding box center displacement and pixel proportion, including: Using consecutive third-person frames within a synchronized segment as the order, select the motion subject distribution map of the current frame and the previous frame to form adjacent distribution map pairs; In adjacent distribution map pairs, center coordinate matching is performed on the bounding boxes of the same moving subject, and the inter-frame displacement distance of the center coordinates is calculated; The area of ​​the bounding boxes of the same moving subject is statistically analyzed, and the absolute difference in the pixel ratio is calculated to obtain the change in pixel ratio. Record the inter-frame displacement distance and pixel ratio change at the corresponding inter-frame positions.

7. The intelligent narrative video generation method for fusing first-person and third-person perspectives according to claim 2, characterized in that, The main action in the first-person perspective of the synchronized clip is analyzed, and the results of the main action analysis include: Using the starting frame of the synchronized segment as a reference, target center point trajectory tracking is performed on the first-person frame sequence, and the displacement distance of the target center point in adjacent frames is calculated to form a viewpoint swing amplitude sequence; Within the same frame sequence, hand position recognition is performed on the subject of the action, hand position coordinates are extracted, the maximum displacement distance between adjacent coordinates is calculated, and a motion detail intensity sequence is formed. The viewpoint swing amplitude sequence and the motion detail intensity sequence are combined to form the motion analysis result.

8. The intelligent narrative video generation method for fusing first-person and third-person perspectives according to claim 7, characterized in that, Within the same frame sequence, hand position recognition is performed on the subject of the action, hand position coordinates are extracted, and the maximum displacement distance between adjacent coordinates is calculated to form a motion detail intensity sequence, including: Using the starting frame of the same frame sequence as a reference, hand position recognition is performed on the subject of the action, the hand position coordinates are extracted, and recorded into the coordinate sequence; Perform displacement calculations on each adjacent coordinate pair in the coordinate sequence, and calculate the distance between the current hand position coordinate and the adjacent hand position coordinates; Select the maximum displacement distance from each adjacent coordinate pair and record it as the maximum displacement distance between adjacent coordinates; Arrange the maximum displacement distance of all adjacent coordinates in frame order to form a motion detail intensity sequence.

9. The intelligent narrative video generation method for fusing first-person and third-person perspectives according to claim 1, characterized in that, Step S3 includes: The scene complexity is calculated based on the scene structure analysis results in the fragment attributes, and the action intensity is calculated based on the main action analysis results in the fragment attributes. The scene complexity and action intensity are weighted and summed to obtain a comprehensive attribute score. Synchronous segments with a comprehensive attribute score higher than a preset threshold are marked as main shots, while those with a lower score are marked as auxiliary shots, thus forming the shot retention segments. The remaining footage segments are sorted in ascending order by trigger timestamp to obtain the initial link sequence, and the time interval between adjacent segments in the sequence is calculated; Adjacent segments with a time interval less than a preset threshold are merged into a continuous segment group, and all continuous segment groups are connected sequentially to construct a segment sequence with narrative fluidity.

10. The intelligent narrative video generation method for fusing first-person and third-person perspectives according to claim 9, characterized in that, The remaining footage clips are sorted in ascending order by their timestamps to obtain an initial link sequence. The time intervals between adjacent clips in the sequence are then calculated, including: The retained clips of each shot are sorted in ascending order using the trigger timestamp corresponding to the retained clip of each shot as the sorting key. The sorted shot clips are then connected sequentially to form an initial link sequence; Select the current shot retention segment and the next shot retention segment from the initial link sequence one by one, and extract their respective trigger identifier timestamps; Calculate the difference between the trigger timestamp of the next shot's retained segment and the trigger timestamp of the current shot's retained segment to obtain the time interval between adjacent segments; Record the time intervals of all adjacent segments in sequence to form a time interval sequence.

Images