Event short video generation method and system based on trajectory tracking and edge pre-clipping

Abstract

The application relates to the technical field of video processing, and particularly discloses a race short video generation method and system based on track tracking and edge pre-editing, which comprises the following steps: an edge computing node receives a race original video stream collected by a local camera, performs adaptive target tracking based on a target shielding degree, generates a fragmented effective video slice containing multi-dimensional metadata, and uploads the fragmented effective video slice to a cloud end; the multi-dimensional metadata comprises a target identity and a spatial coordinate. The scheme effectively alleviates the contradiction between a large amount of data concurrent transmission and picture frame missing detection in a high-density race scene, and ensures seamless coherence of exclusive short videos of each contestant on a time axis, physical consistency of spatial perspective and integrity of content plots under the condition of limited physical bandwidth hardware constraints.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a method and system for generating short sports videos based on trajectory tracking and edge pre-editing. Background Technology

[0002] In large-scale sporting events (such as marathons and cycling races), generating personalized short videos for participants typically relies on camera arrays deployed along the route to capture footage. Most existing video generation technologies employ a centralized processing architecture, where all raw ultra-high-definition video streams are directly uploaded to the cloud, where cloud servers centrally perform target feature extraction, identity recognition, and image cropping; or they rely solely on single-dimensional sensors (such as radio frequency timing chips) for coarse-grained, trigger-based recording and segmentation.

[0003] However, in real-world applications, such as the starting line, aid stations, and sharp bends—areas with high-density crowds—severe physical occlusion frequently occurs between competitors. To ensure continuous target tracking and prevent loss of target identity in such environments, the system needs to perform extensive cross-camera multi-view feature comparisons and joint analysis. This requires uploading large amounts of high-concurrency raw multi-angle video data in real time, which directly leads to edge transmission network bandwidth exceeding its transmission capacity limit and cloud computing power overload.

[0004] If, in order to compromise and adapt to limited communication bandwidth resources, the system adopts a strategy of reducing video quality or implementing simple frame dropping on the terminal side, it will completely lose the target's identity coordinates when encountering dense crowds, resulting in a large number of missed video detections, image breaks, or character confusion. Ultimately, this will lead to incomplete generated exclusive short video content, which cannot meet the requirements of video continuity and the uniqueness of the protagonist in actual business scenarios. Summary of the Invention

[0005] This invention aims to at least partially address one of the technical problems in related technologies. Therefore, the objective of this invention is to propose a method and system for generating short sports videos based on trajectory tracking and edge pre-editing, in order to improve the consistency and integrity of the video over time.

[0006] To achieve the above objectives, a first aspect of the present invention proposes a method for generating short sports videos based on trajectory tracking and edge pre-editing, comprising:

[0007] Edge computing nodes receive raw video streams of the event captured by local cameras, perform adaptive target tracking based on the degree of target occlusion, generate fragmented effective video slices containing multi-dimensional metadata, and upload them to the cloud; the multi-dimensional metadata includes target identification and spatial coordinates;

[0008] The cloud server receives the fragmented valid video slices and aligns them spatiotemporally with the synchronously acquired event timing data and positioning trajectory data, and generates the target global reference trajectory through a dynamic weighted fusion algorithm;

[0009] The cloud server performs a missed detection judgment on the fragmented valid video slices based on the target global reference trajectory. When a missed detection interval is determined, cross-camera candidate slices are retrieved and the target segment is extracted. The target segment is then subjected to viewpoint correction and parameter consistency processing to generate a complete slice.

[0010] The cloud server receives user on-demand requests, aligns the fragmented valid video slices with the completed slices in a spatiotemporal logic, and assembles them based on a semantic mapping model to generate the final short video of the event.

[0011] To achieve the above objectives, a second aspect of the present invention proposes a short video generation system for sports events based on trajectory tracking and edge pre-editing, comprising:

[0012] The edge preprocessing unit, deployed on the edge computing node, is configured to receive the raw video stream of the event captured by the local camera and perform adaptive target tracking based on the degree of target occlusion to generate fragmented effective video slices containing multidimensional metadata and upload them to the cloud server; the multidimensional metadata includes target identification and spatial coordinates;

[0013] The cloud server includes:

[0014] The global trajectory fusion module is configured to receive the fragmented valid video slices, align them spatiotemporally with the synchronously acquired event timing data and positioning trajectory data, and generate the target global reference trajectory through a dynamic weighted fusion algorithm.

[0015] The omission detection and completion module is configured to perform omission detection judgment on the fragmented valid video slices based on the target global reference trajectory. When an omission detection interval is determined, cross-camera candidate slices are retrieved and the target segment is extracted. The target segment is then subjected to viewpoint correction and parameter consistency processing to generate a completed slice.

[0016] The intelligent assembly module is configured to receive user on-demand requests, perform spatiotemporal logical alignment between the fragmented valid video slices and the completed slices, and assemble the final short video of the event based on a semantic mapping model.

[0017] To achieve the above objectives, a third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the above-described method for generating short sports videos based on trajectory tracking and edge pre-editing.

[0018] This invention addresses the inherent conflict between network transmission computing power bottlenecks and tracking continuity in complex occlusion scenarios, proposing a decoupled collaborative processing solution. During actual event execution, edge computing nodes no longer blindly transmit the entire video stream. Instead, they perform adaptive tracking by calculating the physical occlusion level of the target, only extracting and uploading fragmented, valid video slices containing the target. This significantly reduces uplink bandwidth pressure and avoids network congestion at the physical channel source.

[0019] Meanwhile, after receiving the slices, the cloud server breaks through the limitations of a single visual dimension and dynamically weights and fuses the visual spatial coordinates with real-time event timing data and positioning trajectory data at the spatiotemporal level to generate a global reference trajectory with extremely high anti-interference capabilities. Furthermore, when faced with the inevitable single-camera missed detection caused by the high density of people on the actual track, this solution cleverly uses this global reference trajectory as an absolute coordinate ruler to accurately locate the missed detection interval, and extracts candidate segments across camera positions to perform strict perspective projection correction and parameter unification, and finally completes logical assembly by combining with a semantic mapping network.

[0020] This solution effectively alleviates the contradiction between the concurrent transmission of large amounts of data and the failure to detect broken frames in high-density competition scenarios. Under the constraints of limited physical bandwidth hardware, it ensures the seamless continuity of each contestant's exclusive short video on the timeline, the physical consistency of spatial perspective, and the integrity of the content and plot. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the method for generating short sports videos based on trajectory tracking and edge pre-editing provided by the present invention.

[0022] Figure 2 This invention provides a method for generating short race videos based on trajectory tracking and edge pre-editing, which includes a three-dimensional response surface map of track personnel density based on GIS data.

[0023] Figure 3 This is a time-series distribution diagram of target occlusion degree measurement and multi-mode tracking state switching in the short video generation method for sports events based on trajectory tracking and edge pre-editing provided by the present invention;

[0024] Figure 4 This is a Kalman filter error convergence curve of the dynamic weighted fusion of multi-source heterogeneous data in the event short video generation method based on trajectory tracking and edge pre-editing provided by the present invention;

[0025] Figure 5 This invention provides a Pareto front optimization graph for multidimensional resource allocation based on edge node collaborative tracking in the short video generation method for sports events based on trajectory tracking and edge pre-editing.

[0026] Figure 6 This is a three-dimensional spatial depth topology mapping map of multiple targets relative to the optical center in the event short video generation method based on trajectory tracking and edge pre-editing provided by the present invention;

[0027] Figure 7 This is an image showing the effect of homography projection and pixel-level occlusion removal process under foreground occlusion conflict in the short video generation method for sports events based on trajectory tracking and edge pre-editing provided by the present invention.

[0028] Figure 8 This is a schematic diagram illustrating the implementation of the event short video generation system based on trajectory tracking and edge pre-editing provided by the present invention;

[0029] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0030] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0031] The following description, with reference to the accompanying drawings, describes an embodiment of the present invention of a sports short video generation system, method, and electronic device based on trajectory tracking and edge pre-editing.

[0032] Example 1:

[0033] like Figure 1 As shown, this embodiment provides a method for generating short sports videos based on trajectory tracking and edge pre-editing. Considering practical application scenarios such as marathons and road cycling events, the method described in this embodiment is mainly executed collaboratively by multiple edge computing nodes distributed along the race track and a cloud server located in a central data center.

[0034] Specifically, before the official start of a major event, the system first enters the pre-event infrastructure deployment and initialization phase. Before the edge computing nodes receive the raw video stream of the event captured by local cameras, the process also includes: acquiring the spatial road network structure of the event venue and dividing the venue into multi-level physical density zones based on historical personnel density indicators.

[0035] In this embodiment, the spatial road network structure refers to vector map data generated by high-precision geographic information system (GIS) scanning, which includes track width, slope, curve curvature, and coordinate points. The historical personnel density index is calculated based on statistical data from previous events, representing the average number of athletes simultaneously present per unit area.

[0036] For example, the physical density area is divided into three levels: when the number of people per unit area is greater than or equal to the preset high density threshold, it is classified as a level 1 physical density area, such as the starting area, sharp bend area, and aid station; when the number of people per unit area is in the medium range, it is classified as a level 2 physical density area; when the number of people per unit area is less than the preset low density threshold, it is classified as a level 3 physical density area, such as an open straight road.

[0037] like Figure 2 A three-dimensional response surface map of the density of participants on the track, based on geographic information system data, is presented. This map objectively reflects the distribution of participants in the physical space of the track, providing quantitative data support for the system to execute edge node computing power allocation and adaptive tracking strategies.

[0038] The two horizontal axes at the bottom of the graph represent the longitudinal and transverse coordinates of the track, respectively, both measured in meters, and are used to define the physical spatial grid of the track. The vertical axis represents the crowd density, measured in people per square meter, and is used to indicate the crowd density at each spatial coordinate point. The density level legend at the top of the graph corresponds to the density value range under different color gradients.

[0039] The three-dimensional surface in the figure maps the spatial distribution characteristics of personnel density through changes in height and color. The transition from cool to warm colors and the vertically upward bulging peaks of the surface both indicate a significant increase in personnel density.

[0040] Specifically, based on the data in the figure, within the spatial range of 0 to 150 meters in the longitudinal coordinates of the track, the curved surface exhibits the highest main peak shape, with the population density value at the peak reaching 7 or higher. This graphic feature corresponds to the extremely high density area of ​​the race starting area described in the implementation plan. Based on this, the system will allocate high-priority edge computing power and network bandwidth resources in the corresponding physical area.

[0041] Near the longitudinal coordinate of 450 meters and the lateral coordinate of 2 meters on the track, the surface shows another secondary ridge curve. The aggregation density value at this point fluctuates around 5, corresponding to high-density areas such as supply stations. The system will activate the multi-node collaborative tracking mode command in advance based on the waveform characteristics of this surface.

[0042] Within the stable range of 200 to 300 meters along the longitudinal coordinates of the track, the surface height decreases significantly and the shape becomes gentler. The corresponding cluster density value remains stable below 2, indicating a normal density zone. In this area, the system only needs to allocate basic computing resources and maintain a normal target tracking mode.

[0043] By analyzing the data extraction curves and surface waveform transformation patterns in the response surface graph, the system can establish a global spatial understanding of the site's operational status, thereby dynamically adjusting the cloud-edge computing power scheduling resources in each area to ensure the continuous operation of the event's short video preprocessing and target tracking processes.

[0044] Based on the above division results, a corresponding number of edge computing nodes and network bandwidth resources are allocated to different levels of physical density regions.

[0045] Specifically, for the first-level physical density area, the system deploys a high-density camera array and configures it with edge computing nodes equipped with high-performance neural network processing units (NPUs). At the same time, the switch routing policy reserves a dedicated high-bandwidth channel for the uplink physical link of this area to cope with the demand for a large number of concurrent visual data processing.

[0046] For areas with a physical density of 3, the device deployment density is appropriately reduced. Based on this, a global pulse protocol is used to synchronize the clocks of all edge computing nodes and cloud servers, establishing a unified absolute timestamp benchmark across the entire link.

[0047] For example, the global pulse protocol can adopt the IEEE 1588 Precision Time Protocol (PTP). The cloud master clock sends synchronization messages with hardware timestamps to each edge node. The edge nodes calculate the round-trip delay of network transmission and perform sub-millisecond phase and frequency compensation on their local system clocks. The purpose of establishing a unified absolute timestamp reference is to ensure that camera images and sensor signals from different physical locations can be aligned based on a strictly unified timeline when subsequently transmitted to the cloud, preventing data fusion misalignment caused by device clock drift.

[0048] It is important to note that during the event, the system faces real-time dynamic changes in the physical environment. Therefore, the method also includes a dynamic resource scheduling step: continuously monitoring the real-time rate of change in crowd density at the event venue.

[0049] Specifically, edge computing nodes utilize a lightweight crowd counting network to count the total number of people in the current monitoring frame at preset time intervals, such as every second, and calculate the derivative of the number of people in adjacent periods, i.e., the real-time crowd density change rate. When the real-time crowd density change rate in a specific area exceeds a preset mutation threshold, a dynamic computing power scheduling command is triggered. The preset mutation threshold refers to the safe upper limit of the crowd gathering speed allowed by the system. The dynamic computing power scheduling command forces the cloud server to prioritize the allocation of target tracking and trajectory fusion computing power to areas where crowd gathering mutations occur, and reduces the execution priority of background computing tasks in non-mutation areas.

[0050] For example, after receiving the instruction, the cloud server will reallocate the computing resources pool through containerized orchestration tools (such as Kubernetes), suspending or degrading the CPU cores and GPU memory originally used to perform background computing tasks such as historical video transcoding and log compression, and urgently allocating the released computing resources to the processing queue where the mutation area is located, thereby ensuring low-latency response of core tracing services in high-concurrency scenarios.

[0051] During the event's execution phase, the system officially enters the visual acquisition and tracking process. Specifically, edge computing nodes receive the raw video stream of the event captured by local cameras, perform adaptive target tracking based on the degree of target occlusion, generate fragmented effective video slices containing multi-dimensional metadata, and upload them to the cloud; the multi-dimensional metadata includes the target's identity identifier and spatial coordinates. A fragmented effective video slice refers to a non-continuous video segment containing only the athlete's movements, after removing redundant background footage where no target is present from the continuous raw video stream. The target identity identifier (ID) is a unique digital signature assigned to each tracked participant by the system.

[0052] Optionally, the adaptive target tracking based on the target occlusion degree includes: extracting target region frames using a preset target detection algorithm.

[0053] For example, the preset target detection algorithm can use a single-stage convolutional neural network from the YOLO (You Only Look Once) series. This algorithm extracts feature maps from frame-by-frame images of the original video stream and outputs the bounding box coordinates and classification confidence of all athletes in the image, thereby defining the target region frame.

[0054] Then, the occlusion level value for each target is calculated using the following formula:

[0055] ;

[0056] in, Indicates the first The degree of occlusion of each target; Indicates the first The total area of ​​the detection box for each target; Indicates the first The detection bounding box of the target and the target in the same frame The formula calculates the physical intersection area of ​​the detection boxes of each target. Its purpose is to objectively and quantitatively characterize the severity of occlusion of the current target by other entities in the physical optical dimension by calculating the ratio of the overlap area between the current target and all other targets in the image to its total area.

[0057] After obtaining the occlusion level value, the occlusion level value is compared with a first occlusion threshold and a second occlusion threshold, wherein the first occlusion threshold is strictly less than the second occlusion threshold. For example, based on engineering practice and model testing verification, the first occlusion threshold can be set to 0.3 and the second occlusion threshold to 0.6.

[0058] like Figure 3 This diagram illustrates the temporal distribution of target occlusion measurement and multi-mode tracking state switching. The horizontal axis represents the video frame sequence, in frames, indicating the temporal evolution of the video stream captured by the camera. The vertical axis represents the target occlusion degree value, in dimensionless units, objectively quantifying the severity of physical occlusion of the target.

[0059] The legend includes a continuous occlusion measurement curve and two horizontal threshold baselines, specifically a first occlusion threshold set to 0.3 and a second occlusion threshold set to 0.6. In terms of color layout, the two threshold lines divide the vertical axis space into three background areas from bottom to top: light green, light yellow, and light red.

[0060] According to the waveform transformation process, it can be observed that in the initial 0 to 30 frame interval, the occlusion measurement curve is below the first occlusion threshold of 0.3. At this time, the target is in an unoccluded or slightly occluded state. The light green background area corresponding to the position of the curve represents that the system is executing the normal target tracking mode.

[0061] As the target continues to move, the curve shows an upward trend and crosses the baseline of 0.3 around frame 30, entering the light yellow area. Within this range of 0.3 to 0.6, the system automatically switches to local feature recognition mode based on the comparison results, relying on extracting local, unoccluded features to maintain identity identification.

[0062] Then, around frame 45, the curve continued to climb and broke through the second occlusion threshold of 0.6, entering the light red high occlusion area, and reaching a peak value of about 0.85 in frame 60.

[0063] Within the heavily occluded range where this value is greater than 0.6, the high-position feature of the curve triggers the system to switch to multi-node collaborative tracking mode, sending synchronization requests to adjacent nodes to compensate for the visual blind spots of the current camera position.

[0064] Subsequently, as the target leaves the congested area, the curve smoothly descends and successively crosses the two threshold baselines. The system's tracking status is also adaptively downgraded according to the corresponding threshold logic, demonstrating the dynamic tracking response process of the system under different degrees of occlusion.

[0065] Based on the comparison results, perform adaptive mode switching:

[0066] When the occlusion level is less than the first occlusion threshold, the system executes the conventional target tracking mode. Specifically, in this mode, the target is unobstructed or slightly occluded, and the system directly adopts a conventional multi-target tracking model (such as the SORT algorithm) based on Kalman filtering and Hungarian matching algorithms, using the intersection-over-union (IoU) ratio of the detection boxes for inter-frame data association. When the occlusion level is greater than or equal to the first occlusion threshold and less than the second occlusion threshold, the system switches to the local feature re-identification mode.

[0067] Within this range, the target experiences moderate occlusion, and relying solely on bounding box intersection and union (BLU) is prone to causing frequent ID switching. When entering the local feature re-identification mode, the following steps are performed: extracting the target's three local features, which include the bib number sequence, the color histogram features of the main color tone of the clothing, and the head shape features.

[0068] For example, the edge computing node calls the Optical Character Recognition (OCR) network to extract the sequence of numbers on the bib number on the front or back of the athlete; at the same time, it extracts the hue and saturation histogram of the athlete's clothing pixel set in the HSV color space as color histogram features; and uses the edge detection operator to extract the contour feature vector of the helmet or hat worn as head shape features.

[0069] Furthermore, based on the aforementioned triple local features and historical trajectory sequences, similarity matching is performed to recover the identity of the occluded target. Even if the target's main torso is obscured, as long as the head or some clothing features are visible, the system can still use a cosine similarity vector distance algorithm to match the current incomplete features with the historical feature database before the occlusion, thereby retrieving the correct identity. Simultaneously, Kalman filtering is used to predict the target's hidden motion trajectory during re-identification, generating corresponding fragmented effective video slices. That is, using the Kalman filter's state transition matrix, based on the target's velocity components before occlusion, the hidden position within the several frames where occlusion occurred is calculated, ensuring the smoothness of the slice coordinate sequence.

[0070] When the occlusion level value is greater than or equal to the second occlusion threshold, the system switches to multi-node collaborative tracking mode to maintain the consistency of the target's identity. Within this range, the target experiences severe occlusion or even complete disappearance from the current camera position. When entering the multi-node collaborative tracking mode, the following steps are executed:

[0071] The current edge computing node sends a feature synchronization request to a physically adjacent collaborative edge computing node. A physically adjacent collaborative edge computing node refers to another node on the spatial road network topology map that has an effective line-of-sight intersection with the camera under the current node's jurisdiction or is connected to it via the road network. The current edge computing node receives target observation data and adjacent trajectory segments from the collaborative edge computing nodes. Upon receiving the request, the collaborative node searches its own cache for the observation data of the lost target and sends it back. The current edge computing node's target observation data and the adjacent trajectory segments are then weighted and fused.

[0072] By fusing coordinate system and probability data from different camera positions, blind spot information from the current viewpoint is supplemented. If no target observation data is matched by any of the collaborative edge computing nodes, the target's average moving speed and track terrain parameters from the race prior knowledge base are used to fit and extrapolate the target's blind spot trajectory. The race prior knowledge base stores the athlete's past pacing habits, and combined with slope parameters from the GIS map, a uniform acceleration or uniform velocity kinematic model is established to mathematically extrapolate the athlete's position within the blind spot.

[0073] After the video slices and tracking data are uploaded at the edge, the process enters the cloud processing stage. Specifically, the cloud server receives the fragmented valid video slices and performs spatiotemporal alignment with the synchronously acquired race timing data and positioning trajectory data. A target global reference trajectory is generated through a dynamic weighted fusion algorithm. Race timing data typically comes from RFID carpets laid on the track surface and timing chips on the athletes' shoes; positioning trajectory data comes from accompanying vehicles or GPS / BeiDou satellite positioning terminals worn by athletes. Spatiotemporal alignment refers to aligning the heterogeneous data from three different physical principles—visual, RFID, and satellite—using a unified absolute timestamp reference for time axis alignment and mapping them to the same latitude and longitude reference plane through a coordinate system transformation matrix.

[0074] Optionally, generating the target global baseline trajectory through a dynamic weighted fusion algorithm includes: calculating the total fusion weight of the multi-source data, using the following formula:

[0075] ;

[0076] in, Indicates the total weight of the fusion; , , These represent the preset baseline weighting coefficients for visual data, timing data, and location data, respectively. This represents the tracking confidence of the fragmented valid video slice; This indicates the hardware confidence level of the timing data for the event; This represents the signal strength confidence level of the positioning trajectory data. The preset baseline weighting coefficient is a basic proportional value allocated by the system based on the theoretical reliability of different sensors in a general environment; while each confidence level is a value dynamically evaluated for the current actual data quality, such as in the event of signal interference or equipment malfunction. Significantly reduced.

[0077] The purpose of this formula is to establish a data quality evaluation system resistant to interference, preventing trajectory tracking interruptions due to the failure of a single sensor. Subsequently, a dynamic Kalman filter model incorporating the fused total weights is used to smooth the spatial coordinates of multi-source targets at the same absolute timestamp, outputting the target's global reference trajectory. When updating the covariance matrix and Kalman gain, the dynamic Kalman filter model uses the fused total weights as an adjustment factor for the measurement noise covariance, automatically tilting the system towards data sources with higher confidence and better quality, thus outputting a high-precision global reference trajectory.

[0078] Subsequently, the system enters the missed detection and image recovery phase. Specifically, the cloud server performs missed detection judgment on the fragmented valid video slices based on the target global reference trajectory. When a missed detection interval is determined, cross-camera candidate slices are retrieved and the target segment is extracted. The target segment undergoes perspective correction and parameter consistency processing to generate a complete slice. The missed detection interval refers to the time period in which the system failed to extract valid footage containing the contestant.

[0079] like Figure 4 The graph shows the Kalman filter error convergence curve for dynamic weighted fusion of multi-source heterogeneous data. The horizontal axis represents the number of filtering iterations (in seconds), and the vertical axis represents the root mean square error of the positioning error (in meters).

[0080] The graph contains three curves with different colors and waveform transformation characteristics: a red single visual tracking error curve, a blue single satellite positioning error curve, and a green dynamic weighted fusion error curve.

[0081] Combining the numerical changes in the figure, it can be seen that the red single visual tracking error curve shows an upward drifting trend as the number of iterations increases, and the error value eventually rises to between 5 and 6 meters. This objectively corresponds to the cumulative deviation of visual tracking caused by the frequent physical occlusion of the target in the scene of dense crowds at the event.

[0082] The blue single satellite positioning error curve shows a continuous high-frequency oscillation waveform within a value range of about 3 meters, reflecting that the single satellite positioning terminal is easily affected by signal interference in the complex environment of the track, resulting in positioning jumps.

[0083] In contrast, the green dynamic weighted fusion error curve exhibits a steep downward waveform in the early stages of filtering iterations, rapidly converging by the 20th iteration, and then maintaining a stable linear shape at a low error value of approximately 0.5 meters. This rapid decline and low-level stability of the curve waveform indicates that after receiving multi-dimensional data, the cloud server effectively adjusts the dynamic Kalman filter model by calculating the total fusion weights derived from the confidence levels of visual tracking, timing hardware, and positioning signal strength.

[0084] This mechanism enables the system to automatically shift to a data source with higher data quality when the confidence of a single data source decreases. It performs smoothing filtering on the spatial coordinates of multi-source targets under the same absolute timestamp, thereby generating a global reference trajectory of the target with high anti-interference capability and a positioning error of less than 1 meter. This provides a reliable spatial coordinate scale for subsequent accurate execution of missed detection judgment and cross-camera candidate slice perspective correction.

[0085] Optionally, the step of performing a missed detection determination on the fragmented valid video slices based on the target global reference trajectory determines the existence of the missed detection interval when any of the following conditions are met:

[0086] Condition 1: Within a continuous time period exceeding a preset duration threshold, no fragmented valid video slices containing a specific target identifier are received. The preset duration threshold can be set according to the minimum continuity requirement of short videos (e.g., 3 seconds). If an edge node fails to report a slice of the contestant for an extended period, it is determined that the target has been lost at the visual level.

[0087] Condition 2: The absolute distance deviation between the target spatial coordinates in the fragmented valid video slice and the target global reference trajectory at the same absolute timestamp exceeds a preset distance threshold. To quantify this deviation, the formula for calculating the absolute distance deviation is:

[0088] ;

[0089] in, Indicates absolute distance deviation; and These represent the horizontal and vertical values ​​of the local planar coordinate system after the Gauss-Kruger projection transformation of the target coordinates in the fragmented effective video slices; and These represent the lateral and longitudinal values ​​of the target's global baseline trajectory projected onto a plane of the same latitude, respectively. The preset distance threshold represents the maximum allowable positional tolerance boundary of the system (e.g., 5 meters).

[0090] The function of this formula is to perform spatial logic verification. If the visual tracking target coordinates of the edge node deviate from the global reference trajectory coordinates fused by multi-sensor cross-verification, and the distance difference exceeds a reasonable range, it indicates that the visual tracking of the edge node has likely caused false detection or tracked the wrong approximate object. At this time, the visual slice should be discarded and the missed detection judgment mechanism should be triggered.

[0091] Condition 3: The event timing data indicates that the target has passed the preset physical timing point, but no fragmented valid video slices corresponding to the passing time have been received. This is a strong verification of the visual results using the hardware hard trigger signal of the radio frequency sensor, and is a high-priority criterion for missed detection.

[0092] After determining that there is a missed detection range, the intelligent completion mechanism for the missed detection screen is executed.

[0093] Optionally, the step of retrieving cross-camera candidate slices and extracting target segments, and performing viewpoint correction and parameter consistency processing on the target segments to generate complete slices, includes: retrieving cross-camera candidate slices that meet preset resolution and frame rate parameters from the global video cache library based on the start and end timestamps and spatial coordinate range of the missed detection interval. The cloud server has a spatial database index table containing redundant video streams from all cameras on the track. By inputting the timestamp and physical coordinate range of the missed detection event, it can quickly retrieve footage from other secondary cameras that could capture that area at that time.

[0094] Next, using local target features, frame-level target localization is performed in the cross-camera candidate slices, and target segments with the same duration as the missed detection interval are extracted. Using the aforementioned local features such as the bib number or clothing histogram, the target is locked in the secondary camera's view, and segments that strictly correspond in time are cropped out.

[0095] Furthermore, since the shooting angles and lighting conditions of the cross-camera footage differ significantly from the original missed footage, direct splicing would cause drastic image jumps. Therefore, a pre-generated homography transformation matrix is ​​used to perform spatial perspective projection correction on the target segment.

[0096] The homography matrix is ​​a mathematical parameter matrix that describes the projective geometric relationship between the imaging planes of different cameras, based on the camera's intrinsic parameters (such as focal length and optical center point) and extrinsic parameters (such as rotation matrix and translation vector). By calling this matrix, the target image captured by the side camera can be rotated, scaled, and perspective-distorted at the pixel level, so that it is directly projected and transformed to a viewpoint space close to that of the original camera.

[0097] Subsequently, the image parameters of the target segment after projection correction are adjusted to be consistent with the fragmented effective video slice parameters corresponding to the original camera that generated the missed detection interval, and a smooth transition frame is inserted at the beginning and end to generate the completed slice; the image parameters include brightness and contrast.

[0098] For example, the formula for calculating the adjustment of image parameters is:

[0099] ;

[0100] in, This indicates the output pixel brightness value after adjusting the image parameters; This represents the input pixel brightness value of the target segment after projection correction; This represents the contrast gain coefficient, the value of which is determined by calculating the ratio of the brightness variance of the original camera position slice to the brightness variance of the cross-camera position slice. This represents the brightness compensation bias constant, the value of which is determined by calculating the difference between the average brightness values ​​of the two image segments.

[0101] The purpose of this linear mapping formula is to eliminate the differences in optical parameters caused by different exposure controls of multiple cameras at the image processing level, so that the completed slices can be seamlessly integrated into the fragmented video slice sequence of the original camera position in terms of visual perception. The smooth transition frame is a gradual picture generated by the image alpha blending technology to prevent visual discontinuity or abrupt changes in picture when switching frames.

[0102] Finally, the system enters the automated video synthesis stage. Specifically, the cloud server receives user on-demand requests, aligns the fragmented valid video slices with the completed slices in a spatiotemporal logic, and assembles them based on a semantic mapping model to generate the final short video of the event.

[0103] Optionally, the assembly process includes: parsing the user's on-demand request into a set of structured tags through a multi-level semantic mapping model, wherein the set of structured tags includes target identity requirements, video style requirements, and total duration constraints. For example, the semantic mapping model employs a natural language processing (NLP) deep learning model based on a joint architecture of Long Short-Term Memory (LSTM) network and Conditional Random Field (CRF).

[0104] For example, when a user inputs a natural language request such as "Generate a 30-second short video of contestant number 001 in a dynamic style during the sprint phase," the model recognizes the text feature sequence and extracts entities, outputting a formatted set of structured tags (e.g., {"Target_ID":"001","Style":"Dynamic","Duration":30s,"Phase":"Sprint"}). Based on the video style requirement, the system extracts corresponding transition effect configuration parameters and audio rhythm configuration parameters from the event template library. For example, for the "Dynamic" style, the system retrieves parameters for rapidly switching transition masking effects and audio parameters for a high-intensity drum rhythm track.

[0105] Based on this, the fragmented effective video slices and the completed slices are sequentially spliced ​​together according to their absolute timestamps. Under the constraint of a unified absolute timestamp benchmark, the originally discontinuous fragmented slices and the completed slices generated through homography matrix correction are precisely arranged sequentially on the video timeline according to their absolute timestamps, filling in the time gaps caused by occlusion. Furthermore, redundant images without targets are removed based on the structured tag set, and the transition effect configuration parameters and corresponding audio are superimposed to generate the short video of the event that meets the total duration constraint. During audio superposition, the system also uses audio spectrum analysis technology to detect the position of the drum peaks, forcing the transition effect's screen switching points to be frame-synchronized with the drum peaks, thereby completing the non-linear editing and rendering export of professional short videos with broadcast-level quality.

[0106] Based on the specific implementation steps described above, it can be found that when processing videos of large crowds at major sporting events, existing technologies often suffer from insufficient computing power of edge computing devices and network bandwidth congestion, resulting in the inability to upload large amounts of cross-camera video streams in real time. Consequently, when faced with severe physical obstruction of athletes, technical problems such as large-scale target loss, missed detection of images, and fragmented video plots may occur.

[0107] In contrast, the embodiments of this invention not only reduce invalid video backgrounds at the source, significantly alleviating bandwidth load, but also creatively couple three major mechanisms: multi-node collaborative edge tracking, multi-sensor dynamic weighted fusion, and cross-camera perspective projection correction. Under the constraints of limited physical bandwidth hardware, even in extreme visual occlusion situations, the system can rely on strong verification of multi-source data and mathematical projection compensation of cross-camera images to effectively overcome the technical defects of frame loss and image artifacts in the generated personalized short videos, significantly improving the robustness of target tracking and the final quality of personalized sports short videos.

[0108] Example 2:

[0109] In practical sports event short video generation systems, a large number of edge computing nodes and camera devices are typically densely deployed along the race track. When large areas are obscured by crowds, if the current node indiscriminately broadcasts feature synchronization requests to all neighboring nodes, it will trigger a large number of concurrent communication messages. This not only consumes valuable edge LAN bandwidth but also causes the central processing unit (CPU) or neural network processing unit (NPU) of unrelated nodes to suffer from computing power overload due to processing invalid matching requests.

[0110] To address the aforementioned issues, this embodiment introduces a multi-dimensional dimensionality reduction optimization mechanism based on visual spatial geometric topology and dynamic load of physical computing resources.

[0111] Specifically, before the current edge computing node sends a feature synchronization request to a physically adjacent collaborative edge computing node, this embodiment also includes a collaborative target dynamic optimization step. The core purpose of this step is to perform mathematical calculations and logical filtering locally before sending cross-node data interaction requests to determine the set of target nodes with the greatest collaborative tracking potential, thereby minimizing the overall network communication overhead and computing power consumption of the system while ensuring the continuity of target tracking.

[0112] Optionally, when performing the collaborative target dynamic optimization step, the system first obtains a pre-initialized visual overlap topology graph, which records the physically overlapping polygonal regions of the camera views of any two edge computing nodes. In engineering implementation, the visual overlap topology graph is a weighted undirected graph data structure built based on graph theory.

[0113] During the pre-race deployment phase, the system uses a joint calibration algorithm to obtain the intrinsic parameter matrices (including focal length, principal point coordinates, etc.) and extrinsic parameter matrices (including rotation and translation vectors relative to the world coordinate system) of all cameras. Using these parameters, the system projects the three-dimensional view frustum of each camera onto a two-dimensional plane on the track surface, forming a two-dimensional polygonal outline representing the camera's field of view.

[0114] Subsequently, the system employs polygon clipping algorithms, such as the Sutherland-Hodgman algorithm, to perform pairwise intersection operations on the 2D polygon contours of cameras belonging to different edge computing nodes. If the visible ranges of two cameras intersect in physical space, a connecting edge is established for these two edge computing nodes in the visual overlap topology map. The vertex coordinate sequence and area values ​​of the obtained intersection region, i.e., the physically overlapping polygon region, are stored as attribute data in this connecting edge. This visual overlap topology map is serialized into a structured data file, distributed during the system initialization phase, and resides in the local memory of all edge computing nodes for low-latency, fast querying during tracking.

[0115] Specifically, after acquiring the visual overlap topology map, the system filters candidate edge computing nodes with intersecting line-of-sight paths based on the predicted motion vector of the occluded target within the visual overlap topology map. The predicted motion vector is a spatial vector containing velocity magnitude and direction of motion, calculated using a first-order kinematic model or a Kalman filter's state transition matrix, based on the target's historical spatial coordinate sequence over several consecutive frames prior to occlusion. The current edge computing node extracts the direction information of this predicted motion vector and, starting from the spatial coordinates of the target's last effective detection, emits a ray along the direction of the predicted motion vector on a two-dimensional plane.

[0116] Subsequently, the system traverses all adjacent nodes connected to the current node in the visual overlap topology graph, performing a geometric intersection test between the ray and the polygon. If the ray passes through the camera-visible 2D polygon outline of an adjacent node, indicating a line-of-sight intersection path, it suggests that the occluded target is highly likely to enter the field of view of that adjacent node based on its current movement trend. The system extracts all adjacent nodes that meet this intersection test condition, forming a candidate edge calculation node set. This process effectively eliminates invalid edge calculation nodes that are physically close but move in the opposite direction to the target.

[0117] It is important to note that, in order to further evaluate the potential contribution of candidate edge computing nodes in actual cooperative tracking, it is necessary to quantify the spatial uncertainty range of the occluded target over a future period. Therefore, the system calculates the predicted blind zone area of ​​the occluded target using the following formula:

[0118] ;

[0119] in, This indicates the predicted blind zone area of ​​the occluded target; This represents the scalar average speed of the occluded target during a preset time period before the occlusion occurred; This represents the preset prediction time window constant; This indicates the physical width of the area where the track is obstructed.

[0120] The practical function of this formula is to calculate the possible longitudinal movement distance by multiplying the target's historical average speed by the expected blind zone duration. This, combined with the track's lateral physical boundaries, constructs a rectangular or polygonal predicted activity area, thus representing the theoretical dispersion range of the target after it leaves the current node's field of vision. The larger this area value, the higher the uncertainty of the target's position.

[0121] For example, after selecting candidate edge computing nodes and obtaining the predicted blind zone area, the system calculates the collaborative utility evaluation value of each candidate edge computing node. The calculation formula is as follows:

[0122] ;

[0123] in, This represents the synergistic utility assessment value; This represents the area of ​​the physically overlapping polygon region of the candidate edge computing nodes; This indicates the predicted blind zone area of ​​the occluded target; This indicates the real-time computing load rate of candidate edge computing nodes; This indicates the real-time communication latency between the current edge computing node and the candidate edge computing nodes; , , These correspond to preset visual overlap weight coefficients, computing load weight coefficients, and communication delay weight coefficients, respectively.

[0124] In the formula, This constitutes a dimensionless coverage metric, representing the proportion of the area of ​​the physically overlapping polygon region of the candidate edge computing node that can cover the blind zone area of ​​the occluded target prediction. A higher coverage rate indicates a greater probability that the candidate node will capture the lost target; therefore, this term is used as a positive gain term in the formula. This is the percentage of computing resources used, collected in real time and periodically broadcast by the operating system kernel of the candidate edge computing node, such as by reading the / proc / stat interface of the Linux system. If this value is too high, it indicates that the node is processing a large number of concurrent tracing tasks, and the resource redundancy allocated to newly added collaborative tasks is insufficient. Directly allocating tasks will lead to frame drops; therefore, this item is used as a negative penalty in the formula. This is the round-trip latency calculated by sending timestamped Internet Control Message Protocol (ICMP) probes or application-layer heartbeat packets from the current edge computing node to the candidate edge computing node. A longer latency means that feature synchronization is not timely, which can easily lead to lag in fusion decisions; therefore, this item is also used as a negative penalty. , , It is an empirical normalization coefficient pre-calibrated by the system administrator based on the actual hardware configuration of the edge network topology, used to balance the influence of the three dimensions of spatial probability, computing power status and network conditions on the final decision.

[0125] like Figure 5This paper presents a Pareto front optimization graph for multi-dimensional resource allocation in collaborative tracking of edge nodes. The graph constructs a three-dimensional coordinate system, where the horizontal axis represents visual coverage as a percentage, the horizontal axis represents computing load as a percentage, and the vertical axis represents communication latency in milliseconds. These three coordinate axes objectively correspond to the three core evaluation dimensions considered in the dynamic optimization step of the collaborative objective in the specific implementation: geometric spatial probability, underlying hardware resource status, and network link quality.

[0126] In the data point distribution of the graph, there are two different colored clusters of scattered points. The gray clusters represent the set of regular candidate nodes, which are widely and randomly scattered throughout the three-dimensional coordinate space. This indicates that when relying solely on physical distance for flooding broadcast requests, the system will include a large number of invalid edge nodes with excessively high computing power load or excessive communication latency.

[0127] The red scatter cluster represents the Pareto optimal cooperative node set selected through multidimensional evaluation of the system. These data points converge in space to form a clear smooth front surface.

[0128] Observing the specific numerical distribution in the figure, it can be seen that the optimal collaborative nodes, represented by red, are closely clustered in the boundary region where the visual coverage value is greater than 70, the computing load value is less than 40, and the communication latency value is less than 30. This significant spatial clustering and leading-edge distribution pattern intuitively demonstrates that the system can adaptively filter out candidate nodes that, although they have some overlap in the field of view, lack computing resources or are congested in the network, before sending feature synchronization requests.

[0129] By extracting a small number of nodes at the forefront of the Pareto front surface for data interaction, the system effectively reduces redundant communication messages in the edge local area network while ensuring cross-camera tracking recall, thus achieving a reasonable allocation of computational load and transmission bandwidth for event video preprocessing.

[0130] Specifically, after completing the above calculations, the current edge computing node identifies candidate edge computing nodes whose collaborative utility evaluation value is greater than a preset utility threshold as physically adjacent collaborative edge computing nodes, and sorts them in descending order of the collaborative utility evaluation value, selecting the top-ranked node. Each node sends the feature synchronization request, wherein This is a preset positive integer. The preset utility threshold sets a minimum system acceptance threshold. Any candidate node whose collaborative utility evaluation value is lower than this threshold usually means that its vision coverage is extremely low, or its current computing power is severely overloaded, or its network link is severely congested. Such nodes will be directly removed from the collaborative tracing sequence.

[0131] For nodes screened by the threshold, the system sorts them in descending order by the collaborative utility evaluation value to form a priority queue. A preset positive integer represents the maximum number of concurrent Remote Procedure Call (RPC) connections allowed to be established by the current edge computing node system, and its specific value can be set according to the remaining bandwidth of the current node's own Network Interface Controller (NIC), for example, set to 2 or 3. The current edge computing node only sends feature synchronization request packets to the top optimal nodes in the priority queue. Necessary parameters such as the local feature vector of the occluded target, timestamp, and predicted spatial coordinates are encapsulated in the feature synchronization request packet, to trigger the retrieval and fusion logic at the receiving end.

[0132] A comparative analysis is carried out in combination with the video processing solutions in the prior art. Conventional multi-camera cooperative tracking systems generally adopt a Flooding Broadcast strategy based on physical distance when facing target loss. This strategy assumes that all nodes close in spatial position have equal processing capabilities and tracking fields of view, which leads to an exponential increase in useless signaling interactions in a complex event environment with dense crowds. This will not only instantly exhaust the throughput bandwidth of the edge aggregation switch, but also cause some already fully loaded nodes to experience process blocking due to processing additional cross-node retrieval requests, further leading to widespread missed frame detection and network congestion.

[0133] However, the technical solution described in this embodiment extends the multi-node cooperative tracking mode in the first embodiment to the level of collaborative target dynamic optimization, and changes the original data request routing logic. By constructing the underlying visual overlap topology map in advance, the system has the prior knowledge of global field-of-view correlation in the geometric space dimension; through ray intersection test and calculation of the predicted blind area, the system deeply integrates the target kinematic features with the static environment topology;

[0134] Most critically, by introducing a ternary evaluation formula that includes visual coverage, computing power load and communication delay, the system realizes a multi-dimensional resource utilization evaluation before sending data requests. This dynamic optimization mechanism that comprehensively considers the characteristics of computer vision algorithms, the underlying hardware resource status of the system, and the network protocol layer delay ensures that the feature synchronization request is always routed to a few core nodes with "the widest field of view, the most abundant computing power, and the smoothest network".

[0135] In practical engineering applications, this solution significantly reduces the number of invalid communication messages within the edge computing cluster while maintaining or even improving the target cross-camera tracking recall rate. This effectively maintains the system's resource stability and tracking continuity in high-density event scenarios, thus providing a solid and reliable data supply foundation for ultimately generating high-quality, seamless, customized event short videos in the cloud.

[0136] Example 3:

[0137] In real-world sports event scenarios, athletes and obstacles around the track are all three-dimensional spatial entities with specific heights and volumes. When a two-dimensional homography transformation matrix is ​​used to project the target segment captured by the secondary camera back onto the imaging plane of the original camera, the homography transformation, mathematically a two-dimensional planar perspective transformation, assumes that all pixels are on the same flat reference plane. This means it cannot directly reflect the relative depth relationship between different athletes in three-dimensional space.

[0138] In the finish line area of ​​a marathon or the peloton sprint of a cycling race, the positions of the runners are staggered. If the missing target segments after projection correction are directly superimposed on the original camera position, it will produce depth conflict defects such as overlapping and disordered pixels of foreground and background targets and inverted perspective.

[0139] To eliminate such visual anomalies, this embodiment introduces a pixel-level occlusion removal step based on three-dimensional spatial topological relationships after the target segment is corrected by calling the homography transformation matrix generated by pre-joint calibration.

[0140] Specifically, the first stage of the pixel-level occlusion culling step based on three-dimensional spatial topology is to obtain the physical optical center coordinates of the original camera and, in conjunction with the target's global reference trajectory, calculate the theoretical spatial depth distance of the missing target relative to the physical optical center. Obtaining the physical optical center coordinates of the original camera is achieved by retrieving the camera extrinsic parameter matrix stored in the pre-competition joint calibration. Assuming the translation vector and rotation matrix of the original camera in the three-dimensional world coordinate system are known, the absolute three-dimensional spatial coordinates of the camera lens's optical center in the three-dimensional world coordinate system can be calculated through alignment and inversion operations.

[0141] Simultaneously, the system retrieves the target global reference trajectory generated through multi-source data weighted fusion in Example 1. This target global reference trajectory contains the high-precision three-dimensional world coordinates of the missing target at the current absolute timestamp. Next, the cloud server uses a three-dimensional Euclidean distance calculation function to vector subtract the three-dimensional world coordinates of the missing target from the physical optical center coordinates of the original camera, and calculates its modulus, thereby obtaining the theoretical spatial depth distance of the missing target relative to the physical optical center, denoted as . This value represents the theoretical physical distance from the original camera lens at the current timestamp, assuming no target is missed.

[0142] Specifically, the second stage of this step involves extracting the actual spatial depth distance and two-dimensional contour mask of other valid tracking targets captured by the original camera within the missed detection interval. Within the missed detection interval, although a specific competitor may have been visually missed at the original camera position, the original camera is still normally capturing other competitors on the track. These competitors who are not lost and are in normal tracking status are defined as other valid tracking targets. The system continuously tracks these other valid tracking targets through the original camera's computing node, synchronously acquiring their three-dimensional spatial world coordinates at the current absolute timestamp, and using the same Euclidean distance calculation method as described above, calculates the line distance of each other valid tracking target relative to the physical optical center of the original camera, which is taken as the actual spatial depth distance of each other valid tracking target, denoted as . , where k is a positive integer, representing the index of the kth other valid tracking target in the current original camera's video frame.

[0143] Meanwhile, to obtain the precise pixel edges of these contestants on the two-dimensional plane of the original camera image, the cloud server runs a pre-defined instance segmentation network. This network performs pixel-by-pixel binary classification prediction on the original camera video frames, thereby outputting a two-dimensional contour mask for each other valid tracking target. This two-dimensional contour mask is a binary matrix with the same resolution as the video frame, where pixels belonging to the foreground region of the specific valid tracking target are assigned a first flag, while background regions are assigned a second flag.

[0144] For example, after obtaining the aforementioned spatial geometry and image edge parameters, the system performs the third stage, which involves constructing a spatial depth difference comparison function. The calculation formula is as follows:

[0145] ;

[0146] in, This represents the relative spatial depth difference between the kth other valid tracked target and the missing target in the camera's Z-axis viewpoint; Indicates the first The actual spatial depth distance of other effective tracking targets; This represents the theoretical spatial depth distance of the missing target.

[0147] It is important to note that this spatial depth difference comparison function has clear physical perspective guidance significance in practical operations. The cloud server traverses all other valid tracking targets within the aforementioned missed detection interval, and when a target is detected that meets the criteria... And the corresponding first When the 2D contour mask of another valid tracked target and the projected corrected target fragment have a physical intersection region in the 2D pixel coordinate system, a foreground occlusion conflict is determined to have occurred. From the perspective of physical space topology, when When it is less than zero, it means that the first The distance of another effective tracking target relative to the physical optical center of the original camera position is less than the theoretical physical distance of the missing target. That is, the effective tracking target is in front of the missing target in three-dimensional space and belongs to the side closer to the camera lens.

[0148] In this spatial state, if the images of the two objects overlap in the original camera position's two-dimensional pixel coordinate system, it indicates that the foreground object will obscure the missing target behind it from the viewpoint. The system performs a matrix bitwise AND operation between the two-dimensional contour mask matrix of other effectively tracked targets and the non-zero pixel region of the projected target segment. If the result contains a connected component with a pixel count greater than a preset pixel threshold, it proves that a physical intersection region does exist on the two-dimensional plane. The simultaneous satisfaction of these two conditions confirms the occurrence of foreground occlusion conflict, indicating that directly covering the image will lead to perspective logic errors.

[0149] like Figure 6 This diagram presents a three-dimensional spatial depth topology mapping of multiple targets relative to the optical center. The diagram constructs a three-dimensional spatial coordinate system with the camera as the origin. The two horizontal axes at the bottom represent the camera's horizontal and vertical coordinates, respectively, while the vertical axis represents the spatial depth distance. All three axes are measured in meters.

[0150] The black dots in the diagram are located at the origin 0 of the coordinate system, representing the optical center of the camera in the physical world. The black dashed lines extending outward from this optical center represent the projection path of the camera's line of sight.

[0151] Data nodes of different colors are distributed in a three-dimensional coordinate space. The red dots are located at a spatial depth of 15 meters, representing the theoretical location of the missing target that was missed in the current video frame.

[0152] Along the same line of sight projection pointing to the red dot, there exists a blue dot located at a spatial depth distance of 10 meters. This blue dot represents the foreground occlusion target. By comparing the data in the spatial coordinate system, it can be found that the spatial depth distance of the blue dot is less than that of the red dot, and both are located on the same visual projection line segment. This relative relationship of geometric data objectively indicates that in physical three-dimensional space, the foreground target closer to the camera obstructs the optical path of the missing target behind it.

[0153] The graphic distribution logic reveals that when performing multi-camera video clip extraction and fusion, if the depth level is ignored and the image is directly overlaid on a two-dimensional plane, a perspective error of inverted foreground and background will occur.

[0154] The image also shows green dots located at a spatial depth of 18 meters and off the main line-of-sight projection path, which represent normal background targets that do not spatially overlap with the missing target.

[0155] The system extracts the relative depth distances in such 3D spatial topology maps, performs cross-analysis and comparison of the spatial order of multiple targets in 3D spatial dimensions, and provides a clear spatial logic verification basis for determining whether to trigger the underlying pixel removal operation for foreground occlusion conflicts.

[0156] Specifically, after accurately determining foreground occlusion conflicts, the system enters the fourth stage, which involves generating a dynamic occlusion mask based on the physical intersection region. This dynamic occlusion mask is then used to perform pixel culling on the projected target segment. The generation process of the dynamic occlusion mask involves marking the pixel coordinates of the conflicting physical intersection region as active states in the mask matrix. To perform fine-grained processing at the lowest level of image synthesis, the system constructs culling and synthesis output functions, calculated as follows:

[0157] ;

[0158] in, Represents two-dimensional pixel coordinates The final output pixel value at that location; This represents the set of target indices indicating foreground occlusion conflicts. Indicates the first The two-dimensional contour mask of another effective tracking target in coordinates The binarization weight at point , when the point belongs to the . When the point is in the foreground region of another valid tracking target, its value is 1; when the point is in the background region, its value is 0. This indicates the target segment after projection correction in coordinates The original pixel value at that location.

[0159] The function of this formula is to examine and filter each coordinate point in the projected and corrected target segment pixel by pixel. If a certain coordinate point... If the shape falls within the outline mask of one or more other players in the foreground, then the binarization weight at that point... A value of 1 causes the coefficient term within the curly braces to become 0, thus affecting the final output pixel value at that position. The pixels were forcibly reset to zero. Through this matrix operation, the system precisely stripped away the target pixels in the background that should have been obscured by the foreground athlete's body.

[0160] Conversely, if the pixel is not occluded by any foreground element, then binarization is performed. The target pixels that are not occluded are fully preserved. Through this operation, the system retains the valid pixels of the target that are not occluded as input to the subsequent parameter consistency processing flow.

[0161] like Figure 7 This image illustrates the homography projection and pixel-level occlusion culling process under foreground occlusion conflict. The image constructs a three-dimensional pixel matrix coordinate system. The two horizontal axes at the bottom represent the pixel x-coordinate and pixel y-coordinate, respectively, with the unit of measurement being pixels, used to define the two-dimensional spatial matrix range of the processed image. The vertical axis represents the pixel grayscale value, measured in orders of magnitude, with values ​​ranging from 0 to 255, used to quantify the brightness information of the corresponding pixel location. The grayscale level legend at the top of the image corresponds to the grayscale mapping standard for different pixel brightness levels.

[0162] The 3D surface in the figure reflects the final output pixel matrix shape after pixel culling. In the main part of the surface, the pixel grayscale values ​​exhibit a raised waveform distribution that smoothly increases from the periphery to the center. The pixel grayscale value at the highest point in the central region reaches about 250 levels. This continuous grayscale surface represents the set of effective target pixels after being captured from across camera positions and corrected for spatial perspective projection using a homography transformation matrix.

[0163] In the corresponding area at a horizontal coordinate of 60 pixels and a vertical coordinate of 40 pixels, a vertically downward columnar indentation appears on the smooth surface. All pixel grayscale values ​​within this indentation are accurately set to zero, resulting in a deep black bottom plane.

[0164] This unique surface spatial transformation visually demonstrates the process by which the system performs bitwise multiplication on the underlying pixel matrix using the two-dimensional contour mask of the foreground target when it detects foreground occlusion conflicts.

[0165] The overlapping pixels of the obstructing objects that were originally in front of the viewpoint are converted to zero values. The deep black bottom zero-value plane and the light gray raised effective pixel area form a clear digital boundary. The remaining unobstructed raised pixel group will serve as the underlying visual material for subsequent parameter consistency processing, thereby avoiding perspective logic errors at the spatial depth level during the multi-layer video stitching stage.

[0166] Specifically, after the pixel-level culling operation described above, overlapping pixels belonging to the foreground contestant in the projected target segment are removed, eliminating the visual anomaly of depth inversion. At this point, the remaining unoccluded effective target pixels are used as clean material and formally input into the subsequent parameter consistency processing flow. The parameter consistency processing flow, as described in Example 1, includes grayscale histogram normalization adjustment of the optical characteristics such as brightness, contrast, and color temperature of these retained pixels. Since the occluded area of ​​the foreground contestant has been zeroed out at the bottom layer, when the completed slice is finally fused with the original camera's normal video stream using the alpha channel image, the real image of the foreground contestant in the original camera's view will naturally fill in these culled spaces, thereby reproducing the correct 3D perspective visual effect of the missing target passing behind the foreground contestant in the final generated video.

[0167] Analysis of existing technologies reveals that traditional sports editing or multi-camera video stitching techniques often limit cross-camera video fusion to hard cropping of two-dimensional images or simple layer overlay. When faced with the complex spatial interweaving and high-frequency occlusion of athletes in high-density competitive sports such as marathons and cycling races, existing solutions, lacking the deduction and verification of three-dimensional spatial topology depth, are highly likely to produce depth conflicts during video completion and assembly stages, such as image artifacts, layer inversion, and pixel edge perspective logic errors. This damages the professionalism and viewing experience of personalized short videos.

[0168] This embodiment further utilizes the target's global baseline trajectory as a three-dimensional spatial metric, incorporating the original camera's optical center coordinates for Euclidean geometric depth estimation. This elevates traditional two-dimensional image stream alignment to a level of three-dimensional spatial relative depth verification. Through cross-comparison of spatial depth difference comparison functions and two-dimensional contour masks, the system can adaptively identify and eliminate foreground visual conflict pixels caused by homography projection with micro-pixel-level precision. The overall technical solution is interconnected, from source diversion of data acquisition to macro-calibration of spatiotemporal trajectories, and then to micro-pixel reconstruction of supplementary materials. Without relying on additional depth sensors, it effectively solves the problems of perspective logic errors and logical discontinuities in the intelligent assembly process of multi-view short videos, significantly improving the output quality and visual coherence of the automatic editing system for sports short videos.

[0169] Example 4:

[0170] like Figure 8 As shown, this embodiment provides a short video generation system for sports events based on trajectory tracking and edge pre-editing. This system serves as the physical hardware carrier and engineering implementation of the methods described in Embodiments 1 to 3, and is specifically deployed in high-density, high-concurrency outdoor large-scale sports event network environments.

[0171] Specifically, the event short video generation system based on trajectory tracking and edge pre-editing described in this embodiment includes an edge preprocessing unit and a cloud server. At the physical track site, the system is deployed using a distributed edge-and-center architecture. The edge preprocessing unit is deployed on edge computing nodes and configured to receive the raw event video stream captured by local cameras, and perform adaptive target tracking based on the degree of target occlusion to generate fragmented, effective video slices containing multi-dimensional metadata, which are then uploaded to the cloud server. The multi-dimensional metadata includes target identification and spatial coordinates.

[0172] In actual operation, the local cameras consist of multiple ultra-high-definition network cameras deployed in a high-density overlapping pattern along the track. They are connected to local edge computing nodes via gigabit Ethernet physical links, continuously transmitting ultra-high-definition, high-frame-rate raw video streams of the race using Real-Time Streaming Protocol (RTSP) or Secure and Reliable Transport Protocol (SRT). The edge computing nodes employ embedded edge AI computing chips or industrial computers with industrial-grade dustproof and heat dissipation structures, integrating dedicated neural network processing units (NPUs) and hardware video decoding modules. The edge preprocessing unit deployed on these nodes, upon receiving the video stream, invokes the underlying hardware computing power to execute a preset target detection algorithm and calculates the physical overlap ratio of the detection boxes for each target in real time, thereby obtaining the target occlusion degree value.

[0173] Based on the threshold range to which this value falls, the edge preprocessing unit automatically switches between working modes such as regular tracking, re-identification, and cross-node collaborative tracking to maintain the stability of the target identity. The generated fragmented effective video slices and the bound multidimensional metadata are uploaded to the remote central cloud server at high frequency and concurrently through the 5G wireless private network modem or local fiber optic transceiver configured on the edge computing node.

[0174] The cloud server is deployed in a centralized data center or a distributed cloud computing resource pool, and internally consists of a computing cluster composed of multiple high-performance CPU servers, GPU servers, and high-speed distributed storage arrays. The cloud server includes: a global trajectory fusion module, a missed detection and completion module, and an intelligent assembly module.

[0175] Specifically, the global trajectory fusion module is configured to receive the fragmented valid video slices, align them spatiotemporally with synchronously acquired event timing data and positioning trajectory data, and generate a target global baseline trajectory through a dynamic weighted fusion algorithm. At the physical device connection level, the global trajectory fusion module includes a high-throughput network communication interface and is connected to a message queue server (such as a Kafka cluster) to smoothly receive fragmented valid video slices uploaded from a large number of edge computing nodes.

[0176] Meanwhile, the data input of the global trajectory fusion module is connected in real time via a network API interface to the RFID timing mat receiving system deployed at key points on the track (such as the starting line and finish line), as well as the raw positioning trajectory data transmitted back to the positioning data server by the mobile satellite positioning tags (such as GPS / BeiDou positioning terminals) worn by the participants. The global trajectory fusion module utilizes the reference clock source of the cloud server to perform millisecond-level time axis calibration on the three types of heterogeneous data using a unified absolute timestamp. It then converts the visual spatial coordinates into geographic coordinates based on the calibrated camera spatial extrinsic matrix, and finally runs a dynamically weighted Kalman filter algorithm to output the target global reference trajectory with positioning accuracy within a preset tolerance range.

[0177] The omission detection and completion module is configured to determine omissions in the fragmented valid video slices based on the target global reference trajectory. When an omission interval is determined, it retrieves cross-camera candidate slices and extracts the target segment. It then performs viewpoint correction and parameter consistency processing on the target segment to generate a completed slice. In terms of hardware architecture, the omission detection and completion module establishes a high-bandwidth internal bus connection with the high-speed solid-state drive (SSD) distributed video cache library and spatial geometry database built into the cloud server. The omission detection and completion module monitors the matching status between the reference trajectory output by the global trajectory fusion module and the currently received valid video slices in real time. Once a slice with a specific target identifier is found to have a time interruption or spatial coordinate deviation, an omission event is determined, and the spatiotemporal boundaries of the omission interval are defined.

[0178] Subsequently, the omission detection and completion module sends a structured query command to the spatial geometry database to retrieve cross-camera candidate slices captured by other adjacent secondary cameras and temporarily stored in the video cache within the corresponding timestamp and coordinate range. After extracting the corresponding segments, the omission detection and completion module calls the parallel computing core of the cloud GPU cluster to perform matrix multiplication operations based on the homography transformation matrix to perform perspective correction on the target segments at the spatial perspective level. At the same time, it uses the image processing pipeline of the media processing chip to perform pixel-level grayscale histogram normalization and reverse pixel removal operations to eliminate conflicts between multi-camera lighting differences and foreground 3D occlusion. Finally, it generates a complete slice with consistent parameters in the memory block and writes it to the storage array.

[0179] The intelligent assembly module is configured to receive user on-demand requests, perform spatiotemporal logical alignment between the fragmented valid video slices and the completed slices, and assemble the final short video of the competition based on a semantic mapping model. At the application interface level, the intelligent assembly module is configured with a WebAPI gateway for clients (such as mobile applications on smartphones or web browser portals). When a user issues an on-demand command containing a specific natural language description on the client, the user's on-demand request is intercepted by the gateway and forwarded to the intelligent assembly module. The intelligent assembly module runs a dedicated natural language processing deep learning inference network to parse the video into a structured set of tags containing target identity, video style, and total duration constraints.

[0180] Subsequently, the intelligent assembly module calls the non-linear video editing engine in the cloud to retrieve the original fragmented valid video slices of the contestant within the corresponding time period from the distributed storage array, as well as the completed slices generated by the omission detection and completion module. The module splices the beginning and end of the video according to the unified absolute timestamp benchmark across the entire link, automatically removes redundant images, overlays the competition template, transition effects and background audio tracks, and finally renders and exports the final competition short video, which is accurately distributed to user terminals through the content delivery network (CDN) nodes.

[0181] The system topology of this embodiment has clear practical advantages. Traditional sports video editing systems usually rely on centralized uploading of the full ultra-high-definition video stream or single capture without multi-dimensional verification. When facing the high-density and high-complexity physical obstruction environment of large-scale sports events, they are prone to large-scale image omissions and identity switching errors due to network uplink bandwidth paralysis, computing power overload or visual blind spots.

[0182] The system designed in this embodiment significantly reduces the communication load of the wireless network by directly stripping redundant backgrounds and performing adaptive multi-mode tracking at the edge preprocessing unit. The cloud server combines multi-source sensor devices with cross-camera image reconstruction through hardware function decoupling and data collaboration of the global trajectory fusion module, the missing detection completion module, and the intelligent assembly module. This system can effectively eliminate the defects of frame loss and perspective logic errors in the automatic video completion and stitching process without upgrading the physical communication link or relying on the existing hardware constraints of depth sensors. It provides solid hardware and software collaborative support for realizing broadcast-grade, seamless, personalized sports short video automated assembly and highly reliable distribution.

[0183] Example 5:

[0184] Corresponding to the above embodiments, the present invention also proposes an electronic device.

[0185] like Figure 9The diagram shows a structural schematic of an electronic device according to the present invention. The electronic device 100 includes a processor 101 and a memory 103. The processor 101 and the memory 103 are connected, for example, via a bus 102. Optionally, the electronic device 100 may further include a transceiver 104. It should be noted that in practical applications, the transceiver 104 is not limited to one unit, and the structure of this electronic device 100 does not constitute a limitation on the embodiments of the present invention.

[0186] Processor 101 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 101 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0187] Bus 102 may include a pathway for transmitting information between the aforementioned components. Bus 102 may be a PCI bus or an EISA bus, etc. Bus 102 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0188] The memory 103 stores a computer program corresponding to the method for generating short sports videos based on trajectory tracking and edge pre-editing according to the above embodiments of the present invention. This computer program is controlled and executed by the processor 101. The processor 101 executes the computer program stored in the memory 103 to implement the content shown in the aforementioned method embodiments.

[0189] Among them, electronic devices 100 include, but are not limited to: mobile terminals such as laptops and PADs (tablet computers) and fixed terminals such as desktop computers. Figure 9 The electronic device 100 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0190] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for generating short sports videos based on trajectory tracking and edge pre-editing, characterized in that, include: Edge computing nodes receive raw video streams of the event captured by local cameras, perform adaptive target tracking based on the degree of target occlusion, generate fragmented effective video slices containing multi-dimensional metadata, and upload them to the cloud; The multidimensional metadata includes the target identity identifier and spatial coordinates; The fragmented effective video slices refer to the non-continuous video segments containing the athlete's own movements that are retained after removing redundant background images without the target from the continuous original video stream of the event. The adaptive target tracking based on the target occlusion degree includes: extracting target region frames using a preset target detection algorithm; calculating the occlusion degree value of each target, using the following formula: ;in, Indicates the first The degree of occlusion of each target. Indicates the first The total area of ​​the detection box for each target. Indicates the first The detection bounding box of the target and the target in the same frame The physical intersection area of ​​the detection boxes of each target; comparing the occlusion degree value with a first occlusion threshold and a second occlusion threshold, wherein the first occlusion threshold is strictly less than the second occlusion threshold; when the occlusion degree value is less than the first occlusion threshold, executing the normal target tracking mode; when the occlusion degree value is greater than or equal to the first occlusion threshold and less than the second occlusion threshold, switching to the local feature recognition mode; when the occlusion degree value is greater than or equal to the second occlusion threshold, switching to the multi-node collaborative tracking mode; thereby maintaining the consistency of target identity identification; The cloud server receives the fragmented valid video slices and aligns them spatiotemporally with the synchronously acquired event timing data and positioning trajectory data, and generates the target global reference trajectory through a dynamic weighted fusion algorithm; The cloud server performs a missed detection judgment on the fragmented valid video slices based on the target global reference trajectory. When a missed detection interval is determined, cross-camera candidate slices are retrieved and the target segment is extracted. The target segment is then subjected to viewpoint correction and parameter consistency processing to generate a complete slice. The cloud server receives user on-demand requests, aligns the fragmented valid video slices with the completed slices in a spatiotemporal logic, and assembles the final short video of the event based on a semantic mapping model. This includes: parsing the user on-demand request into a structured tag set using a multi-level semantic mapping model, where the structured tag set includes target identity requirements, video style requirements, and total duration constraints; extracting corresponding transition effect configuration parameters and audio rhythm configuration parameters from the event template library based on the video style requirements; splicing the fragmented valid video slices and the completed slices sequentially according to absolute timestamps, removing redundant frames without a target based on the structured tag set, and overlaying the transition effect configuration parameters and corresponding audio to generate the short video of the event that meets the total duration constraints. The semantic mapping model adopts a natural language processing deep learning model based on a joint architecture of long short-term memory network and conditional random field.

2. The method according to claim 1, characterized in that, When entering the local feature recognition mode, the following steps are performed: Extract the target's triple local features, which include the number sequence of the bib number, the color histogram feature of the main color tone of the clothing, and the head shape feature; Based on the aforementioned triple local features and historical trajectory sequences, similarity matching is performed to restore the identity of the occluded target; By combining Kalman filtering to predict the hidden motion trajectory of the target during re-identification, corresponding fragmented effective video slices are generated.

3. The method according to claim 1, characterized in that, When entering the multi-node collaborative tracking mode, the following steps are performed: The current edge computing node sends a feature synchronization request to the physically adjacent collaborative edge computing nodes; Receive target observation data and adjacent trajectory segments fed back by the collaborative edge computing node; The target observation data of the current edge computing node is weighted and fused with the adjacent trajectory segments; If none of the collaborative edge computing nodes match the target observation data, the target's average moving speed and track terrain parameters from the prior knowledge base of the event are called to fit and extrapolate the target's blind spot trajectory.

4. The method according to claim 1, characterized in that, The generation of the target global baseline trajectory through the dynamic weighted fusion algorithm includes: The formula for calculating the total weight of multi-source data fusion is as follows: ； in, Indicates the total weight of the fusion; , , These represent the preset baseline weighting coefficients for visual data, timing data, and location data, respectively. This represents the tracking confidence level of the fragmented valid video slice. This indicates the hardware confidence level of the timing data for the event. This indicates the signal strength confidence level of the positioning trajectory data; Using a dynamic Kalman filter model that includes the total fusion weight, the spatial coordinates of multi-source targets at the same absolute timestamp are smoothed and filtered to output the global baseline trajectory of the targets.

5. The method according to claim 1, characterized in that, The method for determining missed detections of the fragmented valid video slices based on the target global reference trajectory is defined as follows: A missed detection interval is determined to exist when any of the following conditions are met: No fragmented valid video slices containing specific target identifiers were received within a continuous time period exceeding a preset duration threshold; The absolute distance deviation between the target spatial coordinates in the fragmented effective video slice and the target global reference trajectory under the same absolute timestamp exceeds a preset distance threshold. The event timing data indicates that the target has passed the preset physical timing point, but no fragmented valid video slices corresponding to the passing time have been received.

6. The method according to claim 1, characterized in that, The process of retrieving cross-camera candidate slices and extracting target segments, and performing viewpoint correction and parameter consistency processing on the target segments to generate complete slices, includes: Based on the start and end timestamps and spatial coordinate range of the missed detection interval, cross-camera candidate slices that meet the preset resolution and frame rate parameters are retrieved from the global video cache library. Frame-level target localization is performed in the cross-camera candidate slice using local target features, and target segments with the same duration as the missed detection interval are extracted. The target segment is subjected to spatial perspective projection correction by invoking the homography transformation matrix generated in advance through joint calibration. The image parameters of the target segment after projection correction are adjusted to be consistent with the fragmented effective video slice parameters corresponding to the original camera that generated the missed detection interval, and a smooth transition frame is inserted at the beginning and end to generate the complete slice; the image parameters include brightness and contrast.

7. The method according to claim 1, characterized in that, Before the edge computing node receives the raw video stream of the event captured by the local camera, it also includes: Obtain the spatial road network structure of the event venue and divide the venue into multi-level physical density zones based on historical personnel density indicators; Allocate a corresponding number of edge computing nodes and network bandwidth resources to regions with different levels of physical density. The global pulse protocol is used to synchronize the clocks of all edge computing nodes and cloud servers, establishing a unified absolute timestamp benchmark across the entire link.

8. A short video generation system for sports events based on trajectory tracking and edge pre-editing, characterized in that, include: The edge preprocessing unit, deployed on the edge computing node, is configured to receive the raw video stream of the event captured by the local camera and perform adaptive target tracking based on the degree of target occlusion to generate fragmented effective video slices containing multidimensional metadata and upload them to the cloud server. The multidimensional metadata includes the target identity identifier and spatial coordinates; The fragmented effective video slices refer to the non-continuous video segments containing the athlete's own movements that are retained after removing redundant background images without the target from the continuous original video stream of the event. The adaptive target tracking based on the target occlusion degree includes: extracting target region frames using a preset target detection algorithm; calculating the occlusion degree value of each target, using the following formula: ;in, Indicates the first The degree of occlusion of each target. Indicates the first The total area of ​​the detection box for each target. Indicates the first The detection bounding box of the target and the target in the same frame The physical intersection area of ​​the detection boxes of each target; comparing the occlusion degree value with a first occlusion threshold and a second occlusion threshold, wherein the first occlusion threshold is strictly less than the second occlusion threshold; when the occlusion degree value is less than the first occlusion threshold, executing the normal target tracking mode; when the occlusion degree value is greater than or equal to the first occlusion threshold and less than the second occlusion threshold, switching to the local feature recognition mode; when the occlusion degree value is greater than or equal to the second occlusion threshold, switching to the multi-node collaborative tracking mode; thereby maintaining the consistency of target identity identification; The cloud server includes: The global trajectory fusion module is configured to receive the fragmented valid video slices, align them spatiotemporally with the synchronously acquired event timing data and positioning trajectory data, and generate the target global reference trajectory through a dynamic weighted fusion algorithm. The omission detection and completion module is configured to perform omission detection judgment on the fragmented valid video slices based on the target global reference trajectory. When an omission detection interval is determined, cross-camera candidate slices are retrieved and the target segment is extracted. The target segment is then subjected to viewpoint correction and parameter consistency processing to generate a completed slice. The intelligent assembly module is configured to receive user on-demand requests, perform spatiotemporal logical alignment of the fragmented valid video slices and the completed slices, and assemble the final short video of the event based on a semantic mapping model. This includes: parsing the user on-demand request into a structured tag set through a multi-level semantic mapping model, the structured tag set containing target identity requirements, video style requirements, and total duration constraints; extracting corresponding transition effect configuration parameters and audio rhythm configuration parameters from the event template library based on the video style requirements; splicing the fragmented valid video slices and the completed slices sequentially according to absolute timestamps, removing redundant frames without a target based on the structured tag set, and overlaying the transition effect configuration parameters and corresponding audio to generate the short video of the event that meets the total duration constraints. The semantic mapping model adopts a natural language processing deep learning model based on a joint architecture of long short-term memory network and conditional random field.

Images