A method for extracting volleyball motion trajectory based on video data

Abstract

The present invention relates to the field of computer vision technology, and in particular to a method for extracting volleyball motion trajectories based on video data. The method comprises the following steps: acquiring multi-perspective video data through a camera array; performing perspective image weight analysis and dynamic perspective selection based on the multi-perspective video data to obtain perspective image registration data and perspective subset data; performing panoramic image fusion based on the perspective image registration data and perspective subset data to obtain panoramic video data; performing foreground and background adaptive threshold extraction based on the panoramic video data to obtain adaptive threshold map data. The present invention can solve the problem of difficulty in tracking volleyball motion trajectories in complex scenarios and the problem of trajectory extraction differences caused by multiple perspectives through innovative strategies of multi-perspective fusion, adaptive segmentation, multi-target joint analysis, and trajectory optimization, thereby improving the accuracy and stability of volleyball motion trajectory extraction.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for extracting volleyball motion trajectories based on video data. Background Art

[0002] Extracting volleyball motion trajectories based on video data is an important technical requirement in the fields of sports event analysis, athlete training assistance, etc. With the popularity of volleyball and the rise of sports data analysis, the demand for analyzing volleyball trajectories during games is growing. This can be used for visual presentation of events and provide decision-making basis for coaches and athletes. With the development of technologies such as computer vision and artificial intelligence, this direction has made great progress in recent years, but it still faces many challenges.

[0003] Methods based on traditional image processing: such as background subtraction and frame difference method, which extract moving targets by analyzing video frames. Such methods have high computational efficiency, but are easily disturbed in complex scenes and have limited extraction effects. Methods based on target detection and tracking: such as SiamFC and ATOM, which first use the target detection model to locate the volleyball and the athlete, and then use the tracking algorithm to obtain the target trajectory. Such methods can better cope with complex backgrounds, but are prone to losing targets in situations such as occlusion and rapid movement.

[0004] Therefore, in complex game scenes, due to the presence of many occlusions, fast movement, lighting changes and other factors, the accurate detection and continuous tracking of volleyballs and athletes face great challenges; missed and wrong detection of targets will directly affect the accuracy of trajectory extraction, making target detection and tracking in complex scenes difficult. Volleyball games usually have multiple camera angles, which will lead to differences in trajectory extraction methods under different camera perspectives, making it difficult to achieve a unified trajectory extraction algorithm; but the perspective and position of each camera are different, which limits the perspective. Summary of the invention

[0005] Based on this, it is necessary to provide a volleyball motion trajectory extraction method based on video data to solve at least one of the above technical problems.

[0006] To achieve the above purpose, a volleyball motion trajectory extraction method based on video data includes the following steps:

[0007] Step S1: acquiring multi-view video data through a camera array; performing view image weight analysis and dynamic view selection based on the multi-view video data to obtain view image registration data and view subset data; performing panoramic image fusion based on the view image registration data and view subset data to obtain panoramic video data;

[0008] Step S2: performing foreground and background adaptive threshold extraction according to the panoramic video data to obtain adaptive threshold map data; performing multi-view mask processing according to the adaptive threshold map data to obtain background mask data;

[0009] Step S3: performing foreground feature analysis based on the background mask data and the panoramic video data to obtain foreground feature enhanced representation data; performing multi-target frame joint learning on the foreground feature enhanced representation data to obtain joint multi-task learning data; performing high-confidence target frame detection on the joint multi-task learning data to obtain high-confidence target frame data;

[0010] Step S4: using the high-confidence target frame data to perform comprehensive target semantic analysis to obtain comprehensive target semantic data; performing hierarchical semantic trajectory matching according to different time scales based on the comprehensive target semantic data to obtain cross-view trajectory matching data; performing hierarchical semantic trajectory filtering on the cross-view trajectory matching data to obtain semantic trajectory fragment data;

[0011] Step S5: performing volleyball motion trajectory smoothing processing according to the semantic trajectory segment data to obtain smooth trajectory segment data; performing global volleyball motion trajectory optimization according to the smooth trajectory segment data to obtain global consistent trajectory data; performing adaptive trajectory iterative optimization on the global consistent trajectory data to obtain global optimal motion trajectory data for the volleyball;

[0012] Step S6: reconstruct the motion trajectory in three dimensions according to the global optimal motion trajectory data of the volleyball to obtain the motion trajectory data of the volleyball; and automatically annotate key events of multi-target interaction according to the motion trajectory data of the volleyball to obtain key event annotated data.

[0013] The present invention uses a multi-view camera to obtain more comprehensive game scene information, and integrates images from multiple perspectives into a complete and clear panoramic video through dynamic perspective selection and panoramic image fusion technology, providing a rich data foundation for subsequent motion trajectory extraction. Through adaptive threshold extraction and multi-view mask processing, the moving foreground target (such as volleyball and players) is effectively separated from the background to obtain a clear foreground target area, providing accurate data for subsequent feature analysis and trajectory extraction. By performing feature analysis on the foreground target and joint learning of multiple target frames, multiple moving targets in the scene can be accurately identified and located, and reliable target areas can be screened out through high-confidence target frame detection technology, providing high-quality data for subsequent semantic analysis and trajectory extraction. Through comprehensive characterization of target semantic analysis, the category and attributes of the target can be identified, and through hierarchical semantic trajectory matching and filtering technology, the target trajectories under different perspectives are associated and integrated to obtain cross-perspective semantic trajectory fragments, providing a reliable foundation for subsequent trajectory smoothing and optimization. By smoothing, globally optimizing, and iteratively optimizing semantic trajectory segments, we can eliminate noise and jitter in the trajectory and ensure the consistency of the trajectory in time and space, ultimately obtaining a smooth, complete, and accurate global optimal motion trajectory for the volleyball. By converting the two-dimensional motion trajectory of the volleyball into a motion trajectory in three-dimensional space, and automatically identifying and annotating key events in the volleyball motion process, such as serving, receiving, passing, spiking, blocking, etc., it provides intuitive and accurate data support.

[0014] Therefore, the present invention provides a volleyball motion trajectory extraction method based on video data, which can solve the problem of difficulty in tracking volleyball motion trajectories in complex scenes and the problem of trajectory extraction differences caused by multiple perspectives through innovative strategies of multi-perspective fusion, adaptive segmentation, multi-target joint analysis, and trajectory optimization, thereby improving the accuracy and stability of volleyball motion trajectory extraction.

[0015] Preferably, step S1 comprises the following steps:

[0016] Step S11: acquiring multi-view video data through a camera array; extracting view features from the multi-view video data to obtain view feature descriptor data;

[0017] Step S12: performing view feature matching on the view feature description sub-data to obtain matching feature pair data; performing view mapping matrix estimation based on the matching feature pair data to obtain a view mapping matrix;

[0018] Step S13: performing adaptive weight fusion on the data according to the view feature description sub-data and the matching feature to obtain view fusion weight data;

[0019] Step S14: performing view image registration using the view mapping matrix to obtain view image registration data; performing dynamic view selection according to the view feature description sub-data to obtain view subset data;

[0020] Step S15: performing multi-resolution pyramid fusion on the view image registration data using the view subset data to obtain multi-resolution pyramid fusion data; performing edge-aware fusion optimization on the multi-resolution pyramid fusion data to obtain edge-aware fusion data;

[0021] Step S16: performing panoramic image stitching according to the edge-aware fusion data and the perspective fusion weight data to obtain preliminary panoramic image data; performing panoramic image enhancement on the preliminary panoramic image data to obtain panoramic video data.

[0022] The present invention obtains multi-view video data through a camera array, and can fully capture the information of volleyball sports scenes; extracts view features, and the obtained view feature descriptor data provides an important basis for subsequent view matching and fusion. View feature matching is performed, and the obtained matching feature pair data can be used to estimate the mapping relationship between different view angles. The view mapping matrix is ​​estimated, and the obtained view mapping matrix lays the foundation for realizing the registration of images with different view angles. Adaptive weight fusion is performed, and the obtained view fusion weight data can reflect the contribution of different view images to the final panoramic image, which is helpful to generate a higher quality panoramic image. View image registration is performed using the view mapping matrix, and images with different view angles can be aligned to the same coordinate system to prepare for subsequent image fusion. Dynamic view selection is performed, and the obtained view subset data can reduce redundant information and improve calculation efficiency. Multi-resolution pyramid fusion is performed, and image information can be fused at different scales to improve fusion quality. Edge-aware fusion optimization is performed, and the obtained edge-aware fusion data can retain the edge details of the image, making the fusion result clearer. Panoramic image stitching is performed to obtain preliminary panoramic image data. Panoramic image enhancement processing can improve the visual quality of panoramic images and ultimately obtain clear and coherent panoramic video data, providing a high-quality data foundation for subsequent volleyball motion trajectory extraction.

[0023] Preferably, step S2 comprises the following steps:

[0024] Step S21: dividing the panoramic video data into blocks according to the spatiotemporal dimension to obtain video block data; extracting appearance and motion multi-scale features from the video block data to obtain multi-scale feature pyramid data;

[0025] Step S22: performing adaptive weight learning according to the multi-scale feature pyramid data to obtain adaptive feature representation data; performing online incremental clustering on the adaptive feature representation data to obtain cluster center data;

[0026] Step S23: Gaussian mixture background modeling is performed based on the cluster center data and background distribution parameters are generated to obtain background distribution parameters; foreground and background adaptive threshold decisions are performed based on the background distribution parameters to obtain adaptive threshold map data;

[0027] Step S24: performing graph cut optimization segmentation according to the adaptive threshold graph data to obtain graph cut optimization data; performing spatiotemporal consistency constraint processing on the graph cut optimization data to obtain consistency enhancement mask data;

[0028] Step S25: performing morphological processing on the consistency enhanced mask data to obtain morphologically optimized mask data; performing multi-view mask fusion based on the morphologically optimized mask data to obtain background mask data.

[0029] The present invention can divide the panoramic video data into smaller blocks that are easier to process by dividing them into blocks according to the spatiotemporal dimensions. The multi-scale feature pyramid data obtained by extracting the appearance and motion multi-scale features of the video block data can comprehensively characterize the appearance and motion features of the video block at different scales, and provide rich feature information for subsequent adaptive weight learning and background modeling. Adaptive weight learning can automatically adjust the weights of different features to obtain more discriminative adaptive feature representation data. Online incremental clustering of the adaptive feature representation data can update the cluster center data in real time to adapt to the dynamic changes of the video content and provide a dynamically updated reference for background modeling. Gaussian mixture background modeling can more accurately describe the distribution characteristics of complex backgrounds and obtain background distribution parameters. Foreground and background adaptive threshold decision can be performed, and the threshold can be automatically adjusted according to the statistical characteristics of the background to obtain more accurate adaptive threshold graph data, effectively distinguishing the foreground and background. According to the adaptive threshold graph data, graph cut optimization segmentation can be performed, and more refined foreground and background segmentation results, i.e., graph cut optimization data, can be obtained by global optimization. Applying spatiotemporal consistency constraint processing to the graph cut optimization data can improve the spatiotemporal continuity of the foreground and background segmentation, obtain consistency-enhanced mask data, and reduce the noise and fracture of the segmentation results. Morphological processing of the consistency-enhanced mask data can further optimize the shape and edge of the mask to obtain morphologically optimized mask data. Multi-view mask fusion based on the morphologically optimized mask data can comprehensively utilize the segmentation results of different viewpoints to obtain more robust background mask data, laying the foundation for subsequent foreground feature analysis.

[0030] Preferably, step S3 includes the following steps:

[0031] Step S31: extracting foreground candidate regions from the panoramic video data using background mask data to obtain foreground candidate region data;

[0032] Step S32: constructing a foreground multi-scale feature pyramid according to the foreground candidate region data to obtain foreground feature pyramid data;

[0033] Step S33: performing foreground feature enhancement on the foreground feature pyramid data according to the foreground candidate region data to obtain foreground feature enhanced representation data;

[0034] Step S34: performing multi-object box joint learning on the foreground feature enhanced representation data to obtain joint multi-task learning data;

[0035] Step S35: Adaptively filter the joint multi-task learning data with confidence to obtain an initial high-confidence target frame set; and fuse the multi-view detection results based on the initial high-confidence target frame set to obtain high-confidence target frame data.

[0036] The present invention can effectively screen out potential foreground target areas in a video by extracting foreground candidate regions, reduce the amount of calculation and interference of subsequent processing, and lay a good foundation for subsequent feature extraction and target detection. By constructing a multi-scale feature pyramid, the feature information of the foreground target at different scales can be fully captured, the robustness and discrimination of the feature representation can be enhanced, and the accuracy of subsequent target detection and recognition can be improved. By enhancing the foreground features, the significant features of the foreground target can be further highlighted, the background interference can be suppressed, the discrimination between the foreground target and the background can be enhanced, and a more discriminative feature representation can be provided for subsequent target frame regression and classification. By performing joint learning of multiple target frames, the position regression and category classification tasks of the target frame can be optimized at the same time. Compared with a single target detection task, joint learning can better utilize the complementary information between different tasks, improve detection accuracy and generalization ability. By performing adaptive confidence filtering on the joint multi-task learning data, the redundant and erroneous detection results with low confidence can be effectively removed. Combined with the fusion of multi-view detection results, the accuracy and robustness of the target frame positioning can be further improved, and a high-confidence final detection result can be obtained.

[0037] Preferably, step S34 includes the following steps:

[0038] Step S341: generating and classifying joint target candidates for the foreground feature enhanced representation data to obtain the player candidate target frame data and the volleyball candidate target frame data;

[0039] Step S342: performing cross-category feature alignment on the player candidate target frame data and the volleyball candidate target frame data to obtain cross-category feature alignment data;

[0040] Step S343: performing graph convolution target relationship reasoning according to the player candidate target frame data and the volleyball candidate target frame data to obtain relationship enhanced target representation data;

[0041] Step S344: performing multi-scale context fusion on the foreground feature enhancement representation data to obtain context enhancement feature data;

[0042] Step S345: performing joint multi-task learning based on the cross-category feature alignment data, the relationship-enhanced target representation data, and the context-enhanced feature data to obtain joint multi-task learning data.

[0043] The present invention can simultaneously obtain the candidate target frames of athletes and volleyballs by performing joint target candidate generation and classification, and preliminarily determine their category information, provide a basis for subsequent cross-category feature alignment and relationship reasoning, and help improve the accuracy of target detection and recognition. By performing cross-category feature alignment on the athlete candidate target frame data and the volleyball candidate target frame data, the features of different categories of targets can be mapped to a shared feature space, reducing the feature differences between categories, enhancing the consistency and robustness of feature representation, and facilitating subsequent target relationship reasoning and multi-task learning. By performing graph convolution target relationship reasoning based on the athlete candidate target frame data and the volleyball candidate target frame data, the spatial relationship and interaction information between different targets can be modeled, and the target representation with enhanced relationship can be obtained, so as to better characterize the semantic association between targets and improve the accuracy of target recognition and behavior understanding. By performing multi-scale context fusion on the foreground feature enhanced representation data, the context information of different scales can be combined to capture the association between the target and the surrounding environment, enhance the semantic richness and robustness of the target representation, and help improve the performance of target detection and recognition. By performing joint multi-task learning based on cross-category feature alignment data, relationship-enhanced target representation data, and context-enhanced feature data, we can fully utilize the complementary information and constraint relationships between different tasks, while optimizing tasks such as target box regression, target classification, and relationship reasoning, thereby improving the generalization ability and robustness of the model and obtaining more accurate and comprehensive target detection and recognition results.

[0044] Preferably, step S4 includes the following steps:

[0045] Step S41: extracting target appearance features using high-confidence target frame data to obtain target visual feature data; analyzing target position changes in consecutive frames using high-confidence target frame data to obtain target position change data; calculating target motion features based on the target position change data to obtain target motion feature data;

[0046] Step S42: performing multi-level semantic feature fusion on the target visual feature data and the target motion feature data to obtain semantic data that fully represents the target;

[0047] Step S43: construct hierarchical trajectories according to different time scales based on the comprehensive representation target semantic data to obtain long-term and short-term motion hierarchical trajectory data; perform trajectory similarity measurement learning based on the long-term and short-term motion hierarchical trajectory data to obtain a trajectory similarity measurement function;

[0048] Step S44: using the trajectory similarity measurement function to perform cross-view trajectory matching on the long-term and short-term motion layered trajectory data to obtain cross-view trajectory matching data;

[0049] Step S45: performing hierarchical semantic trajectory filtering on the cross-view trajectory matching data to obtain semantic trajectory segment data.

[0050] The present invention can obtain the visual features and motion feature data of the target by using high-confidence target frame data to extract the target appearance features and analyze the target position changes in continuous frames, comprehensively characterize the appearance and motion attributes of the target, provide a rich information basis for subsequent semantic feature fusion and trajectory construction, and help improve the accuracy of trajectory matching and semantic understanding. By performing multi-level semantic feature fusion on the target visual feature data and the target motion feature data, the appearance and motion information of the target can be effectively integrated to obtain a more comprehensive and abstract target semantic representation, capture the features of the target at different semantic levels, and improve the robustness of target recognition and behavior understanding. By constructing hierarchical trajectories according to different time scales based on the comprehensive characterization of the target semantic data, the long-term and short-term motion trajectories of the target can be obtained at the same time, the motion mode and change trend of the target at different time scales can be captured, and the similarity measurement function between trajectories can be established through trajectory similarity measurement learning, so as to provide an effective metric standard for subsequent cross-view trajectory matching. By using the trajectory similarity measurement function to perform cross-view trajectory matching on the long-term and short-term motion hierarchical trajectory data, the target trajectories under different perspectives can be associated and corresponded, the influence of perspective changes can be overcome, and consistent cross-view trajectory matching results can be obtained. By performing hierarchical semantic trajectory filtering on cross-view trajectory matching data, we can remove incorrect matches and discontinuous trajectory segments according to the semantic consistency and continuity of the trajectory, obtain semantically coherent and complete trajectory segment data, improve the quality and reliability of the trajectory, and provide high-quality input for subsequent trajectory smoothing and global optimization.

[0051] Preferably, step S45 includes the following steps:

[0052] Step S451: performing spatiotemporal consistency constraint matching processing on the cross-view trajectory matching data to obtain spatiotemporal consistency constraint matching data;

[0053] Step S452: Generate semantic trajectory segments according to the spatiotemporal consistency constraint matching data to obtain initial semantic trajectory segment data;

[0054] Step S453: performing scene complexity estimation based on the high-confidence target frame data to obtain scene complexity data; performing target feature description processing based on the target visual feature data and the target motion feature data to obtain target feature description data;

[0055] Step S454: using the scene complexity data and the target feature description data to perform adaptive trajectory segment filtering on the initial semantic trajectory segment data to obtain semantic trajectory segment data.

[0056] The present invention can further optimize and refine the trajectory matching results by performing spatiotemporal consistency constraint matching processing on cross-view trajectory matching data, utilize the time and space correspondence of the target trajectory under different perspectives, improve the accuracy and robustness of matching, and remove wrong matching and inconsistent trajectory segments. By generating semantic trajectory segments according to spatiotemporal consistency constraint matching data, the matched trajectory segments can be combined and spliced ​​according to semantic continuity and consistency to form initial semantic trajectory segments, capturing the continuous movement and behavior of the target over a period of time. By estimating the scene complexity according to the high-confidence target frame data, the number, density and interaction of the targets in the scene can be analyzed to evaluate the complexity of the scene, and the target feature description processing can be performed according to the target visual feature data and the target motion feature data to characterize the appearance and motion characteristics of different targets. By using the scene complexity data and the target feature description data to perform adaptive trajectory segment filtering on the initial semantic trajectory segment data, the filtering threshold and strategy can be dynamically adjusted according to the scene complexity and target characteristics, the semantic trajectory segments can be adaptively screened and optimized, unreliable and redundant segments can be removed, the quality and accuracy of the trajectory segments can be improved, and trajectory segment data with better semantic consistency and continuity can be obtained.

[0057] Preferably, step S5 includes the following steps:

[0058] Step S51: extracting volleyball motion trajectory segment data according to the semantic trajectory segment data to obtain volleyball motion trajectory segment data;

[0059] Step S52: constructing a volleyball motion trajectory segment graph according to the volleyball motion trajectory segment data to obtain a volleyball motion trajectory segment graph; performing embedding representation learning on the volleyball motion trajectory segment graph to obtain volleyball motion trajectory segment embedding data;

[0060] Step S53: Adaptively smoothing the volleyball motion trajectory segment data according to the volleyball motion trajectory segment embedding data to obtain smoothed trajectory segment data;

[0061] Step S54: Optimizing the global volleyball motion trajectory according to the smooth trajectory segment data to obtain global consistent trajectory data;

[0062] Step S55: performing iterative optimization and solving on the global consistency trajectory data to obtain iterative optimization trajectory data; performing adaptive convergence judgment on the iterative optimization trajectory data to obtain the volleyball global optimal motion trajectory data.

[0063] The present invention accurately identifies and extracts the motion trajectory segments of volleyball from the semantic trajectory segment data containing multiple motion trajectories. This effectively eliminates the interference of other irrelevant targets. The extracted volleyball motion trajectory segments are constructed into a graph structure and converted into a low-dimensional vector representation through embedded representation learning. This graph structure can effectively capture the spatiotemporal relationship between trajectory segments, and the embedded representation facilitates the subsequent machine learning algorithm to process, thereby providing more effective information for trajectory smoothing and optimization. The volleyball motion trajectory segments are adaptively smoothed. This can effectively eliminate the noise and jitter in the trajectory, making it smoother and continuous, thereby more accurately reflecting the true motion trajectory of the volleyball. The smoothed trajectory segments are optimized globally to eliminate the discontinuity between segments and ensure the consistency of the trajectory in time and space. This makes the final trajectory more complete and accurate, and can better reflect the entire motion process of the volleyball. The globally consistent trajectory data is further refined through an iterative optimization algorithm, and the best optimization result is determined through an adaptive convergence judgment mechanism. This can maximize the accuracy and reliability of the trajectory while ensuring the smoothness and consistency of the trajectory, and finally obtain the global optimal motion trajectory of the volleyball.

[0064] Preferably, step S54 includes the following steps:

[0065] Step S541: performing trajectory integrity constraints on the smooth trajectory segment data to obtain integrity constrained trajectory data; performing physical rationality constraints on the smooth trajectory segment data to obtain physically rational trajectory data;

[0066] Step S542: performing multi-target interaction modeling according to the smooth trajectory segment data and the semantic trajectory segment data to obtain multi-target interaction enhanced trajectory data;

[0067] Step S543: performing adaptive trajectory completion according to the multi-target interactive enhanced trajectory data to obtain volleyball motion completion trajectory data;

[0068] Step S544: performing global consistency optimization based on the smooth trajectory segment data, the integrity constraint trajectory data, the physically reasonable trajectory data and the volleyball motion completion trajectory data to obtain globally consistent trajectory data.

[0069] The present invention further optimizes the smooth trajectory segment data by applying trajectory integrity constraints and physical rationality constraints. The trajectory integrity constraint ensures that the trajectory is not missing or broken in time and space, while the physical rationality constraint ensures that the trajectory conforms to the laws of physical motion, such as restrictions on speed, acceleration, etc. This makes the optimized trajectory more in line with the actual situation and improves the reliability of the trajectory. The smooth trajectory segment data and semantic trajectory segment data are used to model the interaction between the volleyball and other targets (such as players). This can capture the details of the volleyball being affected by other targets during the movement, so that the trajectory is more realistic and accurate, and is helpful for subsequent trajectory completion and optimization. By utilizing multi-target interaction to enhance the trajectory data, the possible trajectory missing is adaptively completed. This can effectively fill the gaps in the trajectory, make it more complete, and provide a more reliable data basis for subsequent global optimization. The smooth trajectory segment data, integrity constraint trajectory data, physical rationality trajectory data, and volleyball motion completion trajectory data are comprehensively considered to perform global consistency optimization. This can ensure that the final trajectory reaches the best state in all aspects, including smoothness, integrity, physical rationality, and accuracy, thereby providing high-quality data support for subsequent three-dimensional reconstruction and key event annotation.

[0070] The present invention uses a multi-view camera to obtain more comprehensive game scene information, and integrates images from multiple perspectives into a complete and clear panoramic video through dynamic perspective selection and panoramic image fusion technology, providing a rich data foundation for subsequent motion trajectory extraction. Through adaptive threshold extraction and multi-view mask processing, the moving foreground target (such as volleyball and players) is effectively separated from the background to obtain a clear foreground target area, providing accurate data for subsequent feature analysis and trajectory extraction. By performing feature analysis on the foreground target and joint learning of multiple target frames, multiple moving targets in the scene can be accurately identified and located, and reliable target areas can be screened out through high-confidence target frame detection technology, providing high-quality data for subsequent semantic analysis and trajectory extraction. Through comprehensive characterization of target semantic analysis, the category and attributes of the target can be identified, and through hierarchical semantic trajectory matching and filtering technology, the target trajectories under different perspectives are associated and integrated to obtain cross-perspective semantic trajectory fragments, providing a reliable foundation for subsequent trajectory smoothing and optimization. By smoothing, globally optimizing, and iteratively optimizing semantic trajectory segments, we can eliminate noise and jitter in the trajectory and ensure the consistency of the trajectory in time and space, ultimately obtaining a smooth, complete, and accurate global optimal motion trajectory for the volleyball. By converting the two-dimensional motion trajectory of the volleyball into a motion trajectory in three-dimensional space, and automatically identifying and annotating key events in the volleyball motion process, such as serving, receiving, passing, spiking, blocking, etc., it provides intuitive and accurate data support.

[0071] Therefore, the present invention provides a volleyball motion trajectory extraction method based on video data, which can solve the problem of difficulty in tracking volleyball motion trajectories in complex scenes and the problem of trajectory extraction differences caused by multiple perspectives through innovative strategies of multi-perspective fusion, adaptive segmentation, multi-target joint analysis, and trajectory optimization, thereby improving the accuracy and stability of volleyball motion trajectory extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A schematic diagram of the steps of a method for extracting volleyball motion trajectory based on video data;

[0073] Figure 2 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0074] Figure 3 for Figure 1 Detailed implementation steps of step S4 in FIG.

[0075] Figure 4 for Figure 1 Detailed implementation steps of step S5 are shown in the flowchart.

[0076] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0077] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0078] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0079] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0080] To achieve this, please refer to Figures 1 to 4 , a volleyball motion trajectory extraction method based on video data, comprising the following steps:

[0081] Step S1: acquiring multi-view video data through a camera array; performing view image weight analysis and dynamic view selection based on the multi-view video data to obtain view image registration data and view subset data; performing panoramic image fusion based on the view image registration data and view subset data to obtain panoramic video data;

[0082] Step S2: performing foreground and background adaptive threshold extraction according to the panoramic video data to obtain adaptive threshold map data; performing multi-view mask processing according to the adaptive threshold map data to obtain background mask data;

[0083] Step S3: performing foreground feature analysis based on the background mask data and the panoramic video data to obtain foreground feature enhanced representation data; performing multi-target frame joint learning on the foreground feature enhanced representation data to obtain joint multi-task learning data; performing high-confidence target frame detection on the joint multi-task learning data to obtain high-confidence target frame data;

[0084] Step S4: using the high-confidence target frame data to perform comprehensive target semantic analysis to obtain comprehensive target semantic data; performing hierarchical semantic trajectory matching according to different time scales based on the comprehensive target semantic data to obtain cross-view trajectory matching data; performing hierarchical semantic trajectory filtering on the cross-view trajectory matching data to obtain semantic trajectory fragment data;

[0085] Step S5: performing volleyball motion trajectory smoothing processing according to the semantic trajectory segment data to obtain smooth trajectory segment data; performing global volleyball motion trajectory optimization according to the smooth trajectory segment data to obtain global consistent trajectory data; performing adaptive trajectory iterative optimization on the global consistent trajectory data to obtain global optimal motion trajectory data for the volleyball;

[0086] Step S6: reconstruct the motion trajectory in three dimensions according to the global optimal motion trajectory data of the volleyball to obtain the motion trajectory data of the volleyball; and automatically annotate key events of multi-target interaction according to the motion trajectory data of the volleyball to obtain key event annotated data.

[0087] The present invention uses a multi-view camera to obtain more comprehensive game scene information, and integrates images from multiple perspectives into a complete and clear panoramic video through dynamic perspective selection and panoramic image fusion technology, providing a rich data foundation for subsequent motion trajectory extraction. Through adaptive threshold extraction and multi-view mask processing, the moving foreground target (such as volleyball and players) is effectively separated from the background to obtain a clear foreground target area, providing accurate data for subsequent feature analysis and trajectory extraction. By performing feature analysis on the foreground target and joint learning of multiple target frames, multiple moving targets in the scene can be accurately identified and located, and reliable target areas can be screened out through high-confidence target frame detection technology, providing high-quality data for subsequent semantic analysis and trajectory extraction. Through comprehensive characterization of target semantic analysis, the category and attributes of the target can be identified, and through hierarchical semantic trajectory matching and filtering technology, the target trajectories under different perspectives are associated and integrated to obtain cross-perspective semantic trajectory fragments, providing a reliable foundation for subsequent trajectory smoothing and optimization. By smoothing, globally optimizing, and iteratively optimizing semantic trajectory segments, we can eliminate noise and jitter in the trajectory and ensure the consistency of the trajectory in time and space, ultimately obtaining a smooth, complete, and accurate global optimal motion trajectory for the volleyball. By converting the two-dimensional motion trajectory of the volleyball into a motion trajectory in three-dimensional space, and automatically identifying and annotating key events in the volleyball motion process, such as serving, receiving, passing, spiking, blocking, etc., it provides intuitive and accurate data support.

[0088] Therefore, the present invention provides a volleyball motion trajectory extraction method based on video data, which can solve the problem of difficulty in tracking volleyball motion trajectories in complex scenes and the problem of trajectory extraction differences caused by multiple perspectives through innovative strategies of multi-perspective fusion, adaptive segmentation, multi-target joint analysis, and trajectory optimization, thereby improving the accuracy and stability of volleyball motion trajectory extraction.

[0089] In the embodiment of the present invention, reference Figure 1 The above is a schematic diagram of the steps of the volleyball motion trajectory extraction method based on video data of the present invention. In this example, the volleyball motion trajectory extraction method based on video data includes the following steps:

[0090] Step S1: acquiring multi-view video data through a camera array; performing view image weight analysis and dynamic view selection based on the multi-view video data to obtain view image registration data and view subset data; performing panoramic image fusion based on the view image registration data and view subset data to obtain panoramic video data;

[0091] In the embodiment of the present invention, first, multiple high-definition cameras arranged around the volleyball court are used to synchronously collect multi-view volleyball game video data. Then, the collected multi-view video data is analyzed, the image quality of each view and the occlusion of the target are evaluated, and the best view combination is dynamically selected. Finally, image registration is performed according to the selected view subset, and the registered images are fused into panoramic video data to provide a more comprehensive field of view for subsequent analysis.

[0092] Step S2: performing foreground and background adaptive threshold extraction according to the panoramic video data to obtain adaptive threshold map data; performing multi-view mask processing according to the adaptive threshold map data to obtain background mask data;

[0093] In the embodiment of the present invention, based on the acquired panoramic video data, an adaptive threshold segmentation algorithm is used to extract the foreground target and generate an adaptive threshold map. Subsequently, the adaptive threshold map is masked using multi-view video information to remove background interference and obtain accurate background mask data so that the foreground target can be analyzed more accurately in the subsequent steps.

[0094] Step S3: performing foreground feature analysis based on the background mask data and the panoramic video data to obtain foreground feature enhanced representation data; performing multi-target frame joint learning on the foreground feature enhanced representation data to obtain joint multi-task learning data; performing high-confidence target frame detection on the joint multi-task learning data to obtain high-confidence target frame data;

[0095] In the embodiment of the present invention, the background mask data is used to extract the features of the foreground target from the panoramic video data, and the enhanced representation is performed to better describe the feature information of the target. Then, a multi-target box joint learning method is used to simultaneously perform target detection and feature extraction to obtain joint multi-task learning data containing target position and feature information. Finally, by setting a confidence threshold, high-confidence target box data is screened out to provide reliable target positioning information for subsequent semantic analysis and trajectory tracking.

[0096] Step S4: using the high-confidence target frame data to perform comprehensive target semantic analysis to obtain comprehensive target semantic data; performing hierarchical semantic trajectory matching according to different time scales based on the comprehensive target semantic data to obtain cross-view trajectory matching data; performing hierarchical semantic trajectory filtering on the cross-view trajectory matching data to obtain semantic trajectory fragment data;

[0097] In the embodiment of the present invention, based on the high-confidence target frame data, a deep learning model is used to perform a comprehensive semantic analysis of the target to extract the semantic features of the target, such as the identity and action of the player. Then, based on the extracted semantic features, hierarchical semantic trajectory matching is performed at different time scales to associate the trajectory segments of the same target under different perspectives to obtain cross-perspective trajectory matching data. Finally, by setting a semantic consistency threshold, the incorrectly matched trajectory segments are filtered out to obtain reliable semantic trajectory segment data.

[0098] Step S5: performing volleyball motion trajectory smoothing processing according to the semantic trajectory segment data to obtain smooth trajectory segment data; performing global volleyball motion trajectory optimization according to the smooth trajectory segment data to obtain global consistent trajectory data; performing adaptive trajectory iterative optimization on the global consistent trajectory data to obtain global optimal motion trajectory data for the volleyball;

[0099] In the embodiment of the present invention, a smoothing algorithm is used to process the obtained semantic trajectory segment data, eliminate noise and jitter, and obtain smooth trajectory segment data. Then, global trajectory optimization is performed based on the smoothed trajectory segment to ensure the continuity and consistency of the trajectories between different segments, and obtain globally consistent trajectory data. Finally, the global trajectory is further optimized using an adaptive iterative optimization algorithm to obtain the global optimal motion trajectory data of the volleyball, providing accurate trajectory information for subsequent three-dimensional reconstruction and event annotation.

[0100] Step S6: reconstructing the motion trajectory in three dimensions according to the global optimal motion trajectory data of the volleyball to obtain the motion trajectory data of the volleyball; and automatically labeling the key events of multi-target interaction according to the motion trajectory data of the volleyball to obtain the key event labeling data;

[0101] In the embodiment of the present invention, 3D reconstruction is performed based on the global optimal motion trajectory data of the volleyball, combined with multi-view geometric information, to obtain the motion trajectory of the volleyball in 3D space. Finally, based on the reconstructed 3D trajectory, combined with volleyball game rules and domain knowledge, key events such as serving, spiking, blocking, etc. are automatically identified and annotated, providing valuable information for game analysis and athlete training.

[0102] Preferably, step S1 comprises the following steps:

[0103] Step S11: acquiring multi-view video data through a camera array; extracting view features from the multi-view video data to obtain view feature descriptor data;

[0104] Step S12: performing view feature matching on the view feature description sub-data to obtain matching feature pair data; performing view mapping matrix estimation based on the matching feature pair data to obtain a view mapping matrix;

[0105] Step S13: performing adaptive weight fusion on the data according to the view feature description sub-data and the matching feature to obtain view fusion weight data;

[0106] Step S14: performing view image registration using the view mapping matrix to obtain view image registration data; performing dynamic view selection according to the view feature description sub-data to obtain view subset data;

[0107] Step S15: performing multi-resolution pyramid fusion on the view image registration data using the view subset data to obtain multi-resolution pyramid fusion data; performing edge-aware fusion optimization on the multi-resolution pyramid fusion data to obtain edge-aware fusion data;

[0108] Step S16: performing panoramic image stitching according to the edge-aware fusion data and the perspective fusion weight data to obtain preliminary panoramic image data; performing panoramic image enhancement on the preliminary panoramic image data to obtain panoramic video data.

[0109] The present invention obtains multi-view video data through a camera array, and can fully capture the information of volleyball sports scenes; extracts view features, and the obtained view feature descriptor data provides an important basis for subsequent view matching and fusion. View feature matching is performed, and the obtained matching feature pair data can be used to estimate the mapping relationship between different view angles. The view mapping matrix is ​​estimated, and the obtained view mapping matrix lays the foundation for realizing the registration of images with different view angles. Adaptive weight fusion is performed, and the obtained view fusion weight data can reflect the contribution of different view images to the final panoramic image, which is helpful to generate a higher quality panoramic image. View image registration is performed using the view mapping matrix, and images with different view angles can be aligned to the same coordinate system to prepare for subsequent image fusion. Dynamic view selection is performed, and the obtained view subset data can reduce redundant information and improve calculation efficiency. Multi-resolution pyramid fusion is performed, and image information can be fused at different scales to improve fusion quality. Edge-aware fusion optimization is performed, and the obtained edge-aware fusion data can retain the edge details of the image, making the fusion result clearer. Panoramic image stitching is performed to obtain preliminary panoramic image data. Panoramic image enhancement processing can improve the visual quality of panoramic images and ultimately obtain clear and coherent panoramic video data, providing a high-quality data foundation for subsequent volleyball motion trajectory extraction.

[0110] In an embodiment of the present invention, multiple cameras are deployed around a volleyball competition venue to capture the game screen from different angles at the same time and obtain multi-view video data. Feature extraction is performed on the video frame of each view to obtain feature descriptor data describing the local features of the image. A feature matching algorithm, such as FLANN or BFMatcher, is used to find matching feature pairs between feature descriptor data of different viewpoints to obtain matching feature pair data. Then, based on the matching feature pair data, RANSAC or other robust estimation methods are used to estimate the geometric transformation relationship between different viewpoints to obtain a view mapping matrix for subsequent view image registration. The weight of each view is calculated based on information such as the number and distance of matching feature pairs, as well as factors such as the reliability of feature descriptors, to obtain view fusion weight data. Views with higher weights will play a greater role in subsequent panoramic image fusion. Using the view mapping matrix, images of different viewpoints are geometrically transformed to align them in space to obtain view image registration data. Then, based on information such as the distribution and number of view feature descriptors, as well as factors such as the degree of overlap between viewpoints, the most representative and complementary view subset is selected. The view subset data is constructed into a multi-resolution pyramid, and image fusion is performed at each layer of the pyramid, such as using Laplacian pyramid fusion or gradient domain fusion methods to obtain multi-resolution pyramid fusion data. Then, edge-aware optimization is performed on the fused image, such as using guided filtering or weighted least squares filtering methods to enhance the edge details of the image and obtain edge-aware fusion data. Using the view fusion weight data, the edge-aware fusion data is weighted averaged or other fusion strategies are performed to obtain preliminary panoramic image data. Then, the preliminary panoramic image is enhanced, such as color correction, contrast enhancement, image inpainting, etc., to obtain the final panoramic video data.

[0111] Preferably, step S2 comprises the following steps:

[0112] Step S21: dividing the panoramic video data into blocks according to the spatiotemporal dimension to obtain video block data; extracting appearance and motion multi-scale features from the video block data to obtain multi-scale feature pyramid data;

[0113] Step S22: performing adaptive weight learning according to the multi-scale feature pyramid data to obtain adaptive feature representation data; performing online incremental clustering on the adaptive feature representation data to obtain cluster center data;

[0114] Step S23: Gaussian mixture background modeling is performed based on the cluster center data and background distribution parameters are generated to obtain background distribution parameters; foreground and background adaptive threshold decisions are performed based on the background distribution parameters to obtain adaptive threshold map data;

[0115] Step S24: performing graph cut optimization segmentation according to the adaptive threshold graph data to obtain graph cut optimization data; performing spatiotemporal consistency constraint processing on the graph cut optimization data to obtain consistency enhancement mask data;

[0116] Step S25: performing morphological processing on the consistency enhanced mask data to obtain morphologically optimized mask data; performing multi-view mask fusion based on the morphologically optimized mask data to obtain background mask data.

[0117] The present invention can divide the panoramic video data into smaller blocks that are easier to process by dividing them into blocks according to the spatiotemporal dimensions. The multi-scale feature pyramid data obtained by extracting the appearance and motion multi-scale features of the video block data can comprehensively characterize the appearance and motion features of the video block at different scales, and provide rich feature information for subsequent adaptive weight learning and background modeling. Adaptive weight learning can automatically adjust the weights of different features to obtain more discriminative adaptive feature representation data. Online incremental clustering of the adaptive feature representation data can update the cluster center data in real time to adapt to the dynamic changes of the video content and provide a dynamically updated reference for background modeling. Gaussian mixture background modeling can more accurately describe the distribution characteristics of complex backgrounds and obtain background distribution parameters. Foreground and background adaptive threshold decision can be performed, and the threshold can be automatically adjusted according to the statistical characteristics of the background to obtain more accurate adaptive threshold graph data, effectively distinguishing the foreground and background. According to the adaptive threshold graph data, graph cut optimization segmentation can be performed, and more refined foreground and background segmentation results, i.e., graph cut optimization data, can be obtained by global optimization. Applying spatiotemporal consistency constraint processing to the graph cut optimization data can improve the spatiotemporal continuity of the foreground and background segmentation, obtain consistency-enhanced mask data, and reduce the noise and fracture of the segmentation results. Morphological processing of the consistency-enhanced mask data can further optimize the shape and edge of the mask to obtain morphologically optimized mask data. Multi-view mask fusion based on the morphologically optimized mask data can comprehensively utilize the segmentation results of different viewpoints to obtain more robust background mask data, laying the foundation for subsequent foreground feature analysis.

[0118] In an embodiment of the present invention, the panoramic video data is divided according to the time and space dimensions, for example, the video frame is divided into a plurality of image blocks, and the continuous image blocks are composed of a video block sequence. Appearance features (such as color, texture) and motion features (such as optical flow) are extracted for each video block, and a multi-scale feature pyramid is constructed to capture feature information at different scales. The weights of different features in the multi-scale feature pyramid are learned using a machine learning algorithm, such as a neural network or a support vector machine, to obtain adaptive feature representation data. Then, the adaptive feature representation data is clustered using an online incremental clustering algorithm to obtain cluster center data representing different foreground and background patterns. The feature distribution of each cluster center is modeled using a Gaussian mixture model, and background distribution parameters such as mean, covariance, etc. are generated. Then, according to the background distribution parameters and the features of the current video block, the probability that each pixel belongs to the foreground is calculated, and an adaptive threshold is set to classify the pixel as the foreground or the background to obtain adaptive threshold map data. The adaptive threshold map data is optimized and segmented using a graph cut algorithm to obtain a more refined foreground target area and obtain graph cut optimized data. Then, considering the spatiotemporal continuity of the video sequence, the graph cut optimization data is processed with spatiotemporal consistency constraints, such as using conditional random fields or Markov random fields, to obtain consistency-enhanced mask data. Morphological operations, such as opening and closing operations, are used to process the consistency-enhanced mask data, eliminate noise and holes, and smooth the target boundaries to obtain morphologically optimized mask data. Finally, the morphologically optimized mask data from different perspectives are fused, such as using weighted average or voting mechanisms, to obtain the final background mask data.

[0119] Preferably, step S3 includes the following steps:

[0120] Step S31: extracting foreground candidate regions from the panoramic video data using background mask data to obtain foreground candidate region data;

[0121] Step S32: constructing a foreground multi-scale feature pyramid according to the foreground candidate region data to obtain foreground feature pyramid data;

[0122] Step S33: performing foreground feature enhancement on the foreground feature pyramid data according to the foreground candidate region data to obtain foreground feature enhanced representation data;

[0123] Step S34: performing multi-object box joint learning on the foreground feature enhanced representation data to obtain joint multi-task learning data;

[0124] Step S35: Adaptively filter the joint multi-task learning data with confidence to obtain an initial high-confidence target frame set; and fuse the multi-view detection results based on the initial high-confidence target frame set to obtain high-confidence target frame data.

[0125] The present invention can effectively screen out potential foreground target areas in a video by extracting foreground candidate regions, reduce the amount of calculation and interference of subsequent processing, and lay a good foundation for subsequent feature extraction and target detection. By constructing a multi-scale feature pyramid, the feature information of the foreground target at different scales can be fully captured, the robustness and discrimination of the feature representation can be enhanced, and the accuracy of subsequent target detection and recognition can be improved. By enhancing the foreground features, the significant features of the foreground target can be further highlighted, the background interference can be suppressed, the discrimination between the foreground target and the background can be enhanced, and a more discriminative feature representation can be provided for subsequent target frame regression and classification. By performing joint learning of multiple target frames, the position regression and category classification tasks of the target frame can be optimized at the same time. Compared with a single target detection task, joint learning can better utilize the complementary information between different tasks, improve detection accuracy and generalization ability. By performing adaptive confidence filtering on the joint multi-task learning data, the redundant and erroneous detection results with low confidence can be effectively removed. Combined with the fusion of multi-view detection results, the accuracy and robustness of the target frame positioning can be further improved, and a high-confidence final detection result can be obtained.

[0126] As an example of the present invention, refer to Figure 2 As shown, in this example, step S3 includes:

[0127] Step S31: extracting foreground candidate regions from the panoramic video data using background mask data to obtain foreground candidate region data;

[0128] In the embodiment of the present invention, the background mask data is used to extract the foreground region in the panoramic video data as a potential target region to obtain the foreground candidate region data. This can be achieved by a simple image mask operation, such as inverting the background mask data and then multiplying it pixel by pixel with the panoramic video data.

[0129] Step S32: constructing a foreground multi-scale feature pyramid according to the foreground candidate region data to obtain foreground feature pyramid data;

[0130] In the embodiment of the present invention, for each foreground candidate region, a method similar to that in step S21 is used to extract multi-scale appearance and motion features, and a foreground feature pyramid is constructed, which can help the model capture feature information of the target at different scales and improve the robustness of target detection.

[0131] Step S33: performing foreground feature enhancement on the foreground feature pyramid data according to the foreground candidate region data to obtain foreground feature enhanced representation data;

[0132] In an embodiment of the present invention, the foreground feature pyramid data is enhanced using the information of the foreground candidate region, for example, by using methods such as region of interest pooling or region of interest alignment to map target features of different sizes to feature vectors of fixed size for subsequent target detection model processing.

[0133] Step S34: performing multi-object box joint learning on the foreground feature enhanced representation data to obtain joint multi-task learning data;

[0134] In the embodiment of the present invention, a multi-target detection model is used to jointly learn the foreground feature enhancement representation data and simultaneously predict the category and bounding box of the target. This can be achieved by multi-task learning, for example, using a shared feature extraction network and then using different branches to predict the target category and bounding box respectively.

[0135] Step S35: Adaptively filter the joint multi-task learning data with confidence to obtain an initial high-confidence target frame set; fuse the multi-view detection results based on the initial high-confidence target frame set to obtain high-confidence target frame data;

[0136] In the embodiment of the present invention, according to the prediction results of the target detection model, such as the confidence of the target category and the overlap degree of the bounding box, an adaptive threshold is set to filter out the detection results with low confidence, and obtain an initial high-confidence target frame set. Then, the initial high-confidence target frame sets of different perspectives are fused, for example, using methods such as non-maximum suppression or weighted averaging, to obtain the final high-confidence target frame data.

[0137] Preferably, step S34 includes the following steps:

[0138] Step S341: generating and classifying joint target candidates for the foreground feature enhanced representation data to obtain the player candidate target frame data and the volleyball candidate target frame data;

[0139] Step S342: performing cross-category feature alignment on the player candidate target frame data and the volleyball candidate target frame data to obtain cross-category feature alignment data;

[0140] Step S343: performing graph convolution target relationship reasoning according to the player candidate target frame data and the volleyball candidate target frame data to obtain relationship enhanced target representation data;

[0141] Step S344: performing multi-scale context fusion on the foreground feature enhancement representation data to obtain context enhancement feature data;

[0142] Step S345: performing joint multi-task learning based on the cross-category feature alignment data, the relationship-enhanced target representation data, and the context-enhanced feature data to obtain joint multi-task learning data.

[0143] The present invention can simultaneously obtain the candidate target frames of athletes and volleyballs by performing joint target candidate generation and classification, and preliminarily determine their category information, provide a basis for subsequent cross-category feature alignment and relationship reasoning, and help improve the accuracy of target detection and recognition. By performing cross-category feature alignment on the athlete candidate target frame data and the volleyball candidate target frame data, the features of different categories of targets can be mapped to a shared feature space, reducing the feature differences between categories, enhancing the consistency and robustness of feature representation, and facilitating subsequent target relationship reasoning and multi-task learning. By performing graph convolution target relationship reasoning based on the athlete candidate target frame data and the volleyball candidate target frame data, the spatial relationship and interaction information between different targets can be modeled, and the target representation with enhanced relationship can be obtained, so as to better characterize the semantic association between targets and improve the accuracy of target recognition and behavior understanding. By performing multi-scale context fusion on the foreground feature enhanced representation data, the context information of different scales can be combined to capture the association between the target and the surrounding environment, enhance the semantic richness and robustness of the target representation, and help improve the performance of target detection and recognition. By performing joint multi-task learning based on cross-category feature alignment data, relationship-enhanced target representation data, and context-enhanced feature data, we can fully utilize the complementary information and constraint relationships between different tasks, while optimizing tasks such as target box regression, target classification, and relationship reasoning, thereby improving the generalization ability and robustness of the model and obtaining more accurate and comprehensive target detection and recognition results.

[0144] In an embodiment of the present invention, a target detection model, such as a region proposal network (RPN) in Faster R-CNN, is used to analyze the foreground feature enhancement representation data, generate candidate target frames, and preliminarily classify the candidate target frames to distinguish between athletes and volleyballs, and obtain athlete candidate target frame data and volleyball candidate target frame data. Using feature alignment technology, such as a spatial attention mechanism or a feature transformation network, the features of the athlete candidate target frame data and the volleyball candidate target frame data are aligned so that the target features of different categories have similar semantic information, which facilitates subsequent relationship reasoning and multi-task learning. The athlete candidate target frame data and the volleyball candidate target frame data are constructed into a graph structure, in which nodes represent targets and edges represent the relationship between targets, such as spatial distance, relative position, etc. Then, a graph convolution network is used to reason about the graph structure, learn the interactive relationship between targets, and integrate it into the target representation to obtain relationship-enhanced target representation data. Using methods such as a feature pyramid network or a dilated convolution, the foreground feature enhancement representation data is multi-scale context fused, and feature information of different scales is integrated to obtain context-enhanced feature data. This can help the model better understand the environmental information around the target and improve the accuracy of target detection. The cross-category feature alignment data, relation-enhanced target representation data, and context-enhanced feature data are input into the multi-task learning model to simultaneously learn target classification and bounding box regression to obtain joint multi-task learning data. This can be achieved through a shared feature extraction network and different branches, such as using one branch for target classification and another branch for bounding box regression.

[0145] Preferably, step S4 includes the following steps:

[0146] Step S41: extracting target appearance features using high-confidence target frame data to obtain target visual feature data; analyzing target position changes in consecutive frames using high-confidence target frame data to obtain target position change data; calculating target motion features based on the target position change data to obtain target motion feature data;

[0147] Step S42: performing multi-level semantic feature fusion on the target visual feature data and the target motion feature data to obtain semantic data that fully represents the target;

[0148] Step S43: construct hierarchical trajectories according to different time scales based on the comprehensive representation target semantic data to obtain long-term and short-term motion hierarchical trajectory data; perform trajectory similarity measurement learning based on the long-term and short-term motion hierarchical trajectory data to obtain a trajectory similarity measurement function;

[0149] Step S44: using the trajectory similarity measurement function to perform cross-view trajectory matching on the long-term and short-term motion layered trajectory data to obtain cross-view trajectory matching data;

[0150] Step S45: performing hierarchical semantic trajectory filtering on the cross-view trajectory matching data to obtain semantic trajectory segment data.

[0151] The present invention can obtain the visual features and motion feature data of the target by using high-confidence target frame data to extract the target appearance features and analyze the target position changes in continuous frames, comprehensively characterize the appearance and motion attributes of the target, provide a rich information basis for subsequent semantic feature fusion and trajectory construction, and help improve the accuracy of trajectory matching and semantic understanding. By performing multi-level semantic feature fusion on the target visual feature data and the target motion feature data, the appearance and motion information of the target can be effectively integrated to obtain a more comprehensive and abstract target semantic representation, capture the features of the target at different semantic levels, and improve the robustness of target recognition and behavior understanding. By constructing hierarchical trajectories according to different time scales based on the comprehensive characterization of the target semantic data, the long-term and short-term motion trajectories of the target can be obtained at the same time, the motion mode and change trend of the target at different time scales can be captured, and the similarity measurement function between trajectories can be established through trajectory similarity measurement learning, so as to provide an effective metric standard for subsequent cross-view trajectory matching. By using the trajectory similarity measurement function to perform cross-view trajectory matching on the long-term and short-term motion hierarchical trajectory data, the target trajectories under different perspectives can be associated and corresponded, the influence of perspective changes can be overcome, and consistent cross-view trajectory matching results can be obtained. By performing hierarchical semantic trajectory filtering on cross-view trajectory matching data, we can remove incorrect matches and discontinuous trajectory segments according to the semantic consistency and continuity of the trajectory, obtain semantically coherent and complete trajectory segment data, improve the quality and reliability of the trajectory, and provide high-quality input for subsequent trajectory smoothing and global optimization.

[0152] As an example of the present invention, refer to Figure 3 As shown, in this example, step S4 includes:

[0153] Step S41: extracting target appearance features using high-confidence target frame data to obtain target visual feature data; analyzing target position changes in consecutive frames using high-confidence target frame data to obtain target position change data; calculating target motion features based on the target position change data to obtain target motion feature data;

[0154] In an embodiment of the present invention, for each high-confidence target frame, the image area within the frame is extracted, and a deep learning model or a traditional image feature extraction method, such as a color histogram, texture features, etc., is used to extract the appearance features of the target to obtain the target visual feature data. At the same time, the position changes of the target frame in consecutive frames, such as the center point coordinates, the bounding box size, etc., are analyzed to obtain the target position change data. Then, based on the target position change data, the motion features of the target, such as speed, acceleration, direction, etc., are calculated to obtain the target motion feature data.

[0155] Step S42: performing multi-level semantic feature fusion on the target visual feature data and the target motion feature data to obtain semantic data that fully represents the target;

[0156] In the embodiment of the present invention, the target visual feature data and the target motion feature data are fused to obtain data that fully represents the target semantics, so as to more fully understand the motion state and appearance features of the target.

[0157] Step S43: construct hierarchical trajectories according to different time scales based on the comprehensive representation target semantic data to obtain long-term and short-term motion hierarchical trajectory data; perform trajectory similarity measurement learning based on the long-term and short-term motion hierarchical trajectory data to obtain a trajectory similarity measurement function;

[0158] In the embodiment of the present invention, according to the semantic features of the target, hierarchical trajectory data is constructed according to different time scales, such as short-term motion trajectory and long-term motion trajectory, to obtain long-term and short-term motion hierarchical trajectory data. Then, a trajectory similarity measurement function is learned using methods such as Siamese network or triplet loss to measure the similarity between different trajectories.

[0159] Step S44: using the trajectory similarity measurement function to perform cross-view trajectory matching on the long-term and short-term motion layered trajectory data to obtain cross-view trajectory matching data;

[0160] In the embodiment of the present invention, the trajectory similarity measurement function learned in step S43 is used to match the long-term and short-term motion layered trajectory data under different perspectives, find trajectory pairs with high similarity, and obtain cross-perspective trajectory matching data.

[0161] Step S45: performing hierarchical semantic trajectory filtering on the cross-view trajectory matching data to obtain semantic trajectory segment data;

[0162] In an embodiment of the present invention, the cross-view trajectory matching data is filtered according to the semantic features of the target and the trajectory matching results, for example, trajectory pairs with low matching confidence are removed, or trajectories inconsistent with the semantic features of the target are removed to obtain the final semantic trajectory segment data.

[0163] Preferably, step S45 includes the following steps:

[0164] Step S451: performing spatiotemporal consistency constraint matching processing on the cross-view trajectory matching data to obtain spatiotemporal consistency constraint matching data;

[0165] Step S452: Generate semantic trajectory segments according to the spatiotemporal consistency constraint matching data to obtain initial semantic trajectory segment data;

[0166] Step S453: performing scene complexity estimation based on the high-confidence target frame data to obtain scene complexity data; performing target feature description processing based on the target visual feature data and the target motion feature data to obtain target feature description data;

[0167] Step S454: using the scene complexity data and the target feature description data to perform adaptive trajectory segment filtering on the initial semantic trajectory segment data to obtain semantic trajectory segment data.

[0168] The present invention can further optimize and refine the trajectory matching results by performing spatiotemporal consistency constraint matching processing on cross-view trajectory matching data, utilize the time and space correspondence of the target trajectory under different perspectives, improve the accuracy and robustness of matching, and remove wrong matching and inconsistent trajectory segments. By generating semantic trajectory segments according to spatiotemporal consistency constraint matching data, the matched trajectory segments can be combined and spliced ​​according to semantic continuity and consistency to form initial semantic trajectory segments, capturing the continuous movement and behavior of the target over a period of time. By estimating the scene complexity according to the high-confidence target frame data, the number, density and interaction of the targets in the scene can be analyzed to evaluate the complexity of the scene, and the target feature description processing can be performed according to the target visual feature data and the target motion feature data to characterize the appearance and motion characteristics of different targets. By using the scene complexity data and the target feature description data to perform adaptive trajectory segment filtering on the initial semantic trajectory segment data, the filtering threshold and strategy can be dynamically adjusted according to the scene complexity and target characteristics, the semantic trajectory segments can be adaptively screened and optimized, unreliable and redundant segments can be removed, the quality and accuracy of the trajectory segments can be improved, and trajectory segment data with better semantic consistency and continuity can be obtained.

[0169] In the embodiment of the present invention, considering the continuity and physical laws of the target motion, the cross-view trajectory matching data is constrained in time and space consistency, such as limiting the range of variation of the velocity and acceleration of the trajectory, and ensuring that the matched trajectories under different perspectives are consistent in time and space, so as to obtain time and space consistency constraint matching data. Based on the time and space consistency constraint matching data, the matched trajectory segments under different perspectives are merged, and the initial semantic trajectory segment data is generated according to the semantic features of the target, such as the target category, motion state, etc. The high-confidence target frame data, such as the number of targets, the distance between targets, the complexity of the target motion, etc., are analyzed to estimate the complexity of the scene and obtain scene complexity data. At the same time, according to the target visual feature data and the target motion feature data, the characteristics of the target, such as the size, shape, color, motion speed, motion direction, etc. of the target are described to obtain the target feature description data. According to the scene complexity data and the target feature description data, an adaptive filtering threshold is set to filter the initial semantic trajectory segment data, such as removing the trajectory segments with low confidence in the complex area of ​​the scene, or removing the trajectory segments inconsistent with the target feature description, so as to obtain the final semantic trajectory segment data.

[0170] Preferably, step S5 includes the following steps:

[0171] Step S51: extracting volleyball motion trajectory segment data according to the semantic trajectory segment data to obtain volleyball motion trajectory segment data;

[0172] Step S52: constructing a volleyball motion trajectory segment graph according to the volleyball motion trajectory segment data to obtain a volleyball motion trajectory segment graph; performing embedding representation learning on the volleyball motion trajectory segment graph to obtain volleyball motion trajectory segment embedding data;

[0173] Step S53: Adaptively smoothing the volleyball motion trajectory segment data according to the volleyball motion trajectory segment embedding data to obtain smoothed trajectory segment data;

[0174] Step S54: Optimizing the global volleyball motion trajectory according to the smooth trajectory segment data to obtain global consistent trajectory data;

[0175] Step S55: performing iterative optimization and solving on the global consistency trajectory data to obtain iterative optimization trajectory data; performing adaptive convergence judgment on the iterative optimization trajectory data to obtain the volleyball global optimal motion trajectory data.

[0176] The present invention accurately identifies and extracts the motion trajectory segments of volleyball from the semantic trajectory segment data containing multiple motion trajectories. This effectively eliminates the interference of other irrelevant targets. The extracted volleyball motion trajectory segments are constructed into a graph structure and converted into a low-dimensional vector representation through embedded representation learning. This graph structure can effectively capture the spatiotemporal relationship between trajectory segments, and the embedded representation facilitates the subsequent machine learning algorithm to process, thereby providing more effective information for trajectory smoothing and optimization. The volleyball motion trajectory segments are adaptively smoothed. This can effectively eliminate the noise and jitter in the trajectory, making it smoother and continuous, thereby more accurately reflecting the true motion trajectory of the volleyball. The smoothed trajectory segments are optimized globally to eliminate the discontinuity between segments and ensure the consistency of the trajectory in time and space. This makes the final trajectory more complete and accurate, and can better reflect the entire motion process of the volleyball. The globally consistent trajectory data is further refined through an iterative optimization algorithm, and the best optimization result is determined through an adaptive convergence judgment mechanism. This can maximize the accuracy and reliability of the trajectory while ensuring the smoothness and consistency of the trajectory, and finally obtain the global optimal motion trajectory of the volleyball.

[0177] As an example of the present invention, refer to Figure 4 As shown, in this example, step S5 includes:

[0178] Step S51: extracting volleyball motion trajectory segment data according to the semantic trajectory segment data to obtain volleyball motion trajectory segment data;

[0179] In the embodiment of the present invention, from the semantic trajectory segment data containing multiple targets, the trajectory segments with the target category of volleyball are screened out to obtain the volleyball motion trajectory segment data. This can be achieved by simple category judgment, such as checking the semantic label of the trajectory segment.

[0180] Step S52: constructing a volleyball motion trajectory segment graph according to the volleyball motion trajectory segment data to obtain a volleyball motion trajectory segment graph; performing embedding representation learning on the volleyball motion trajectory segment graph to obtain volleyball motion trajectory segment embedding data;

[0181] In the embodiment of the present invention, the volleyball motion trajectory fragment data is constructed into a graph structure, in which nodes represent trajectory fragments and edges represent the spatiotemporal relationship between fragments, such as temporal sequence or spatial distance. Then, a graph embedding algorithm, such as DeepWalk, Node2Vec or GraphSAGE, is used to map the nodes in the graph structure to a low-dimensional vector space to obtain the volleyball motion trajectory fragment embedded data.

[0182] Step S53: Adaptively smoothing the volleyball motion trajectory segment data according to the volleyball motion trajectory segment embedding data to obtain smoothed trajectory segment data;

[0183] In an embodiment of the present invention, a trajectory smoothing algorithm based on embedded representation, such as a graph convolutional network or a recurrent neural network, is used to smooth the volleyball motion trajectory segment data, eliminate noise and jitter in the trajectory, and obtain smooth trajectory segment data.

[0184] Step S54: Optimizing the global volleyball motion trajectory according to the smooth trajectory segment data to obtain global consistent trajectory data;

[0185] In an embodiment of the present invention, a global optimization algorithm, such as a dynamic programming or a graph cut algorithm, is used to optimize the smooth trajectory segment data, ensure the consistency of the trajectory in time and space, and eliminate the discontinuity between segments to obtain globally consistent trajectory data.

[0186] Step S55: performing iterative optimization on the global consistent trajectory data to obtain iteratively optimized trajectory data; performing adaptive convergence judgment on the iteratively optimized trajectory data to obtain the global optimal motion trajectory data of the volleyball;

[0187] In an embodiment of the present invention, an iterative optimization algorithm, such as a gradient descent method or a Newton method, is used to further optimize the global consistency trajectory data, and whether the optimization process converges is determined based on a preset convergence condition or a change in the objective function value to obtain the final volleyball global optimal motion trajectory data.

[0188] Preferably, step S54 includes the following steps:

[0189] Step S541: performing trajectory integrity constraints on the smooth trajectory segment data to obtain integrity constrained trajectory data; performing physical rationality constraints on the smooth trajectory segment data to obtain physically rational trajectory data;

[0190] Step S542: performing multi-target interaction modeling according to the smooth trajectory segment data and the semantic trajectory segment data to obtain multi-target interaction enhanced trajectory data;

[0191] Step S543: performing adaptive trajectory completion according to the multi-target interactive enhanced trajectory data to obtain volleyball motion completion trajectory data;

[0192] Step S544: performing global consistency optimization based on the smooth trajectory segment data, the integrity constraint trajectory data, the physically reasonable trajectory data and the volleyball motion completion trajectory data to obtain globally consistent trajectory data.

[0193] The present invention further optimizes the smooth trajectory segment data by applying trajectory integrity constraints and physical rationality constraints. The trajectory integrity constraint ensures that the trajectory is not missing or broken in time and space, while the physical rationality constraint ensures that the trajectory conforms to the laws of physical motion, such as restrictions on speed, acceleration, etc. This makes the optimized trajectory more in line with the actual situation and improves the reliability of the trajectory. The smooth trajectory segment data and semantic trajectory segment data are used to model the interaction between the volleyball and other targets (such as players). This can capture the details of the volleyball being affected by other targets during the movement, so that the trajectory is more realistic and accurate, and is helpful for subsequent trajectory completion and optimization. By utilizing multi-target interaction to enhance the trajectory data, the possible trajectory missing is adaptively completed. This can effectively fill the gaps in the trajectory, make it more complete, and provide a more reliable data basis for subsequent global optimization. The smooth trajectory segment data, integrity constraint trajectory data, physical rationality trajectory data, and volleyball motion completion trajectory data are comprehensively considered to perform global consistency optimization. This can ensure that the final trajectory reaches the best state in all aspects, including smoothness, integrity, physical rationality, and accuracy, thereby providing high-quality data support for subsequent three-dimensional reconstruction and key event annotation.

[0194] In an embodiment of the present invention, trajectory integrity constraints are applied to the smooth trajectory segment data, such as limiting the distance or time interval between trajectory segments, ensuring that the trajectory is not missing or broken in time and space, and obtaining integrity constrained trajectory data. At the same time, physical rationality constraints are applied to the smooth trajectory segment data, such as limiting the range of variation of the trajectory's speed, acceleration, etc., to ensure that the trajectory conforms to the laws of physical motion and obtain physically reasonable trajectory data. Using the smooth trajectory segment data and the semantic trajectory segment data, an interaction model including volleyball and other targets (such as players) is constructed, such as using methods such as graph neural networks or attention mechanisms to learn the interaction relationship between targets and integrate it into the trajectory representation to obtain multi-target interaction enhanced trajectory data. Based on the multi-target interaction enhanced trajectory data, a trajectory prediction model, such as a recurrent neural network or a Kalman filter, is used to predict the missing parts of the trajectory, and the trajectory is completed based on the prediction results to obtain volleyball motion completion trajectory data. Taking into account the smooth trajectory fragment data, integrity constrained trajectory data, physically reasonable trajectory data and volleyball motion completion trajectory data, a global optimization objective function is constructed, such as minimizing the energy of the trajectory or maximizing the smoothness of the trajectory. An optimization algorithm, such as the gradient descent method or the simulated annealing algorithm, is used to solve the optimal solution of the objective function and obtain globally consistent trajectory data.

[0195] Preferably, step S6 includes the following steps:

[0196] Step S61: performing trajectory data preprocessing on the volleyball global optimal motion trajectory data to obtain volleyball trajectory preprocessing data; performing multi-dimensional trajectory feature extraction on the volleyball trajectory preprocessing data to obtain a volleyball multi-dimensional trajectory feature set;

[0197] Step S62: performing trajectory clustering on the volleyball trajectory preprocessing data according to the volleyball multi-dimensional trajectory feature set to obtain trajectory clustering data;

[0198] Step S63: performing three-dimensional reconstruction of the volleyball motion trajectory according to the volleyball multi-dimensional trajectory feature set, the trajectory clustering data and the volleyball trajectory preprocessing data to obtain the volleyball motion trajectory data;

[0199] Step S64: Automatically annotate key events of multi-target interaction according to the semantic trajectory segment data and the volleyball motion trajectory data to obtain key event annotation data.

[0200] The present invention pre-processes the volleyball global optimal motion trajectory data, such as removing noise, smoothing the trajectory, etc., to improve the quality and consistency of the data. Then, multi-dimensional trajectory features, such as position, speed, acceleration, direction, etc., are extracted from the pre-processed data. These features can comprehensively describe the motion state of the volleyball and provide rich information for subsequent trajectory clustering and three-dimensional reconstruction. Cluster analysis is performed on the volleyball trajectory pre-processed data, which can classify trajectory segments with similar motion features into one category, thereby identifying different stages or modes of volleyball motion, such as serving, passing, spiking, etc., and providing important reference information for subsequent three-dimensional reconstruction and key event annotation. By comprehensively utilizing the volleyball multi-dimensional trajectory feature set, trajectory clustering data, and volleyball trajectory pre-processed data, three-dimensional reconstruction of the volleyball motion trajectory is performed. This can convert the two-dimensional trajectory information into a motion trajectory in three-dimensional space, thereby more intuitively displaying the motion process of the volleyball and providing more accurate spatial information for subsequent key event annotation. By combining semantic trajectory fragment data and volleyball motion trajectory data, key events in the volleyball motion process, such as serving, receiving, passing, spiking, blocking, etc., can be automatically identified and annotated. This can help people better understand the game process and analyze the players' technical movements.

[0201] In an embodiment of the present invention, the volleyball global optimal motion trajectory data is subjected to preprocessing operations such as smoothing and filtering to eliminate noise and jitter, thereby obtaining volleyball trajectory preprocessing data. Then, multi-dimensional features, such as position, velocity, acceleration, direction, curvature, etc., are extracted from the preprocessed trajectory data to obtain a volleyball multi-dimensional trajectory feature set. Clustering algorithms, such as K-means, DBSCAN or hierarchical clustering, are used to perform cluster analysis on the volleyball trajectory preprocessing data according to the volleyball multi-dimensional trajectory feature set, and trajectory segments with similar motion characteristics are classified into one category to obtain trajectory clustering data. The volleyball multi-dimensional trajectory feature set, trajectory clustering data and volleyball trajectory preprocessing data are comprehensively utilized, and a three-dimensional reconstruction algorithm, such as a reconstruction method based on multi-view geometry or a reconstruction method based on deep learning, is used to convert the two-dimensional trajectory information into a motion trajectory in three-dimensional space to obtain volleyball motion trajectory data. Combining the semantic trajectory fragment data and volleyball motion trajectory data, using key event detection algorithms, such as rule-based methods or machine learning-based methods, automatically identify and annotate key events in the volleyball motion process, such as serving, receiving, passing, spiking, blocking, etc., to obtain key event annotation data.

[0202] The present invention uses a multi-view camera to obtain more comprehensive game scene information, and integrates images from multiple perspectives into a complete and clear panoramic video through dynamic perspective selection and panoramic image fusion technology, providing a rich data foundation for subsequent motion trajectory extraction. Through adaptive threshold extraction and multi-view mask processing, the moving foreground target (such as volleyball and players) is effectively separated from the background to obtain a clear foreground target area, providing accurate data for subsequent feature analysis and trajectory extraction. By performing feature analysis on the foreground target and joint learning of multiple target frames, multiple moving targets in the scene can be accurately identified and located, and reliable target areas can be screened out through high-confidence target frame detection technology, providing high-quality data for subsequent semantic analysis and trajectory extraction. Through comprehensive characterization of target semantic analysis, the category and attributes of the target can be identified, and through hierarchical semantic trajectory matching and filtering technology, the target trajectories under different perspectives are associated and integrated to obtain cross-perspective semantic trajectory fragments, providing a reliable foundation for subsequent trajectory smoothing and optimization. By smoothing, globally optimizing, and iteratively optimizing semantic trajectory segments, we can eliminate noise and jitter in the trajectory and ensure the consistency of the trajectory in time and space, ultimately obtaining a smooth, complete, and accurate global optimal motion trajectory for the volleyball. By converting the two-dimensional motion trajectory of the volleyball into a motion trajectory in three-dimensional space, and automatically identifying and annotating key events in the volleyball motion process, such as serving, receiving, passing, spiking, blocking, etc., it provides intuitive and accurate data support.

[0203] Therefore, the present invention provides a volleyball motion trajectory extraction method based on video data, which can solve the problem of difficulty in tracking volleyball motion trajectories in complex scenes and the problem of trajectory extraction differences caused by multiple perspectives through innovative strategies of multi-perspective fusion, adaptive segmentation, multi-target joint analysis, and trajectory optimization, thereby improving the accuracy and stability of volleyball motion trajectory extraction.

[0204] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0205] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A volleyball motion trajectory extraction method based on video data, characterized in that: The following steps are involved: Step S1: Acquire multi-view video data through a camera array; Performing perspective image weight analysis and dynamic perspective selection based on multi-perspective video data to obtain perspective image registration data and perspective subset data; The panoramic image is fused according to the view image registration data and the view subset data to obtain the panoramic video data, wherein step S1 includes the following steps: Step S11: acquiring multi-view video data through a camera array; extracting view features from the multi-view video data to obtain view feature descriptor data; Step S12: performing view feature matching on the view feature description sub-data to obtain matching feature pair data; performing view mapping matrix estimation based on the matching feature pair data to obtain a view mapping matrix; Step S13: performing adaptive weight fusion on the data according to the view feature description sub-data and the matching feature to obtain view fusion weight data; Step S14: Performing perspective image registration using a perspective mapping matrix, geometrically transforming images of different perspectives so as to align them in space, and obtaining perspective image registration data; performing dynamic perspective selection based on the perspective feature description subdata, selecting the most representative and complementary perspective subset, and obtaining perspective subset data; Step S15: performing multi-resolution pyramid fusion on the view image registration data using the view subset data to obtain multi-resolution pyramid fusion data; performing edge-aware fusion optimization on the multi-resolution pyramid fusion data to obtain edge-aware fusion data; Step S16: performing panoramic image stitching according to the edge-sensing fusion data and the perspective fusion weight data to obtain preliminary panoramic image data; performing panoramic image enhancement on the preliminary panoramic image data to obtain panoramic video data; Step S2: performing foreground and background adaptive threshold extraction according to the panoramic video data to obtain adaptive threshold map data; performing multi-view mask processing according to the adaptive threshold map data to obtain background mask data; Step S3: performing foreground feature analysis based on the background mask data and the panoramic video data to obtain foreground feature enhanced representation data; performing multi-target frame joint learning on the foreground feature enhanced representation data to obtain joint multi-task learning data; performing high-confidence target frame detection on the joint multi-task learning data to obtain high-confidence target frame data; Step S4: using the high-confidence target frame data to perform comprehensive target semantic analysis to obtain comprehensive target semantic data; performing hierarchical semantic trajectory matching according to different time scales based on the comprehensive target semantic data to obtain cross-view trajectory matching data; performing hierarchical semantic trajectory filtering on the cross-view trajectory matching data to obtain semantic trajectory fragment data; Step S5: performing volleyball motion trajectory smoothing processing according to the semantic trajectory segment data to obtain smooth trajectory segment data; performing global volleyball motion trajectory optimization according to the smooth trajectory segment data to obtain global consistent trajectory data; performing adaptive trajectory iterative optimization on the global consistent trajectory data to obtain iteratively optimized trajectory data; performing adaptive convergence judgment on the iteratively optimized trajectory data to obtain global optimal motion trajectory data of the volleyball; Step S6: reconstruct the motion trajectory in three dimensions according to the global optimal motion trajectory data of the volleyball to obtain the motion trajectory data of the volleyball; and automatically annotate key events of multi-target interaction according to the motion trajectory data of the volleyball to obtain key event annotated data.

2. The volleyball motion trajectory extraction method based on video data according to claim 1 is characterized in that: Step S2 includes the following steps: Step S21: dividing the panoramic video data into blocks according to the spatiotemporal dimension to obtain video block data; extracting appearance and motion multi-scale features from the video block data to obtain multi-scale feature pyramid data; Step S22: performing adaptive weight learning according to the multi-scale feature pyramid data to obtain adaptive feature representation data; performing online incremental clustering on the adaptive feature representation data to obtain cluster center data; Step S23: Gaussian mixture background modeling is performed based on the cluster center data and background distribution parameters are generated to obtain background distribution parameters; foreground and background adaptive threshold decision is performed based on the background distribution parameters to obtain adaptive threshold map data; Step S24: performing graph cut optimization segmentation according to the adaptive threshold graph data to obtain graph cut optimization data; performing spatiotemporal consistency constraint processing on the graph cut optimization data to obtain consistency enhancement mask data; Step S25: performing morphological processing on the consistency enhanced mask data to obtain morphologically optimized mask data; performing multi-view mask fusion based on the morphologically optimized mask data to obtain background mask data.

3. The volleyball motion trajectory extraction method based on video data according to claim 1 is characterized in that: Step S3 includes the following steps: Step S31: extracting foreground candidate regions from the panoramic video data using background mask data to obtain foreground candidate region data; Step S32: constructing a foreground multi-scale feature pyramid according to the foreground candidate region data to obtain foreground feature pyramid data; Step S33: performing foreground feature enhancement on the foreground feature pyramid data according to the foreground candidate region data to obtain foreground feature enhanced representation data; Step S34: performing multi-object box joint learning on the foreground feature enhanced representation data to obtain joint multi-task learning data; Step S35: Adaptively filter the joint multi-task learning data with confidence to obtain an initial high-confidence target frame set; and fuse the multi-view detection results based on the initial high-confidence target frame set to obtain high-confidence target frame data.

4. The volleyball motion trajectory extraction method based on video data according to claim 3 is characterized in that: Step S34 includes the following steps: Step S341: generating and classifying joint target candidates for the foreground feature enhanced representation data to obtain the player candidate target frame data and the volleyball candidate target frame data; Step S342: performing cross-category feature alignment on the player candidate target frame data and the volleyball candidate target frame data to obtain cross-category feature alignment data; Step S343: performing graph convolution target relationship reasoning according to the player candidate target frame data and the volleyball candidate target frame data to obtain relationship enhanced target representation data; Step S344: performing multi-scale context fusion on the foreground feature enhancement representation data to obtain context enhancement feature data; Step S345: performing joint multi-task learning based on the cross-category feature alignment data, the relationship-enhanced target representation data, and the context-enhanced feature data to obtain joint multi-task learning data.

5. The volleyball motion trajectory extraction method based on video data according to claim 1 is characterized in that: Step S4 includes the following steps: Step S41: extracting target appearance features using high-confidence target frame data to obtain target visual feature data; Utilize high-confidence target frame data to analyze the target position change of consecutive frames to obtain target position change data; calculate target motion features based on the target position change data to obtain target motion feature data; Step S42: performing multi-level semantic feature fusion on the target visual feature data and the target motion feature data to obtain semantic data that fully represents the target; Step S43: construct hierarchical trajectories according to different time scales based on the comprehensive representation of the target semantic data to obtain long-term and short-term motion hierarchical trajectory data; Based on the long-term and short-term motion hierarchical trajectory data, trajectory similarity measurement learning is performed to obtain the trajectory similarity measurement function; Step S44: using the trajectory similarity measurement function to perform cross-view trajectory matching on the long-term and short-term motion layered trajectory data to obtain cross-view trajectory matching data; Step S45: performing hierarchical semantic trajectory filtering on the cross-view trajectory matching data to obtain semantic trajectory segment data.

6. The volleyball motion trajectory extraction method based on video data according to claim 5 is characterized in that: Step S45 includes the following steps: Step S451: performing spatiotemporal consistency constraint matching processing on the cross-view trajectory matching data to obtain spatiotemporal consistency constraint matching data; Step S452: Generate semantic trajectory segments according to the spatiotemporal consistency constraint matching data to obtain initial semantic trajectory segment data; Step S453: Estimating scene complexity based on the high-confidence target frame data to obtain scene complexity data; Performing target feature description processing according to the target visual feature data and the target motion feature data to obtain target feature description data; Step S454: using the scene complexity data and the target feature description data to perform adaptive trajectory segment filtering on the initial semantic trajectory segment data to obtain semantic trajectory segment data.

7. The volleyball motion trajectory extraction method based on video data according to claim 1 is characterized in that: Step S5 The following steps are included Step S51: extracting volleyball motion trajectory segment data according to the semantic trajectory segment data to obtain volleyball motion trajectory segment data; Step S52: constructing a volleyball motion trajectory segment graph according to the volleyball motion trajectory segment data to obtain a volleyball motion trajectory segment graph; performing embedding representation learning on the volleyball motion trajectory segment graph to obtain volleyball motion trajectory segment embedding data; Step S53: Adaptively smoothing the volleyball motion trajectory segment data according to the volleyball motion trajectory segment embedding data to obtain smoothed trajectory segment data; Step S54: Optimizing the global volleyball motion trajectory according to the smooth trajectory segment data to obtain global consistent trajectory data; Step S55: performing iterative optimization on the global consistent trajectory data to obtain iteratively optimized trajectory data; According to the preset convergence conditions or the change of the objective function value, the iterative optimization trajectory data is adaptively judged to determine whether the optimization process converges and obtain the global optimal motion trajectory data of the volleyball.

8. The volleyball motion trajectory extraction method based on video data according to claim 7 is characterized in that: Step S54 includes the following steps: Step S541: performing trajectory integrity constraints on the smooth trajectory segment data to obtain integrity constrained trajectory data; performing physical rationality constraints on the smooth trajectory segment data to obtain physically rational trajectory data; Step S542: performing multi-target interaction modeling according to the smooth trajectory segment data and the semantic trajectory segment data to obtain multi-target interaction enhanced trajectory data; Step S543: performing adaptive trajectory completion according to the multi-target interactive enhanced trajectory data to obtain volleyball motion completion trajectory data; Step S544: performing global consistency optimization based on the smooth trajectory segment data, the integrity constraint trajectory data, the physically reasonable trajectory data and the volleyball motion completion trajectory data to obtain globally consistent trajectory data.

9. The volleyball motion trajectory extraction method based on video data according to claim 1 is characterized in that: Step S6 includes the following steps: Step S61: performing trajectory data preprocessing on the volleyball global optimal motion trajectory data to obtain volleyball trajectory preprocessing data; performing multi-dimensional trajectory feature extraction on the volleyball trajectory preprocessing data to obtain a volleyball multi-dimensional trajectory feature set; Step S62: performing trajectory clustering on the volleyball trajectory preprocessing data according to the volleyball multi-dimensional trajectory feature set to obtain trajectory clustering data; Step S63: performing three-dimensional reconstruction of the volleyball motion trajectory according to the volleyball multi-dimensional trajectory feature set, the trajectory clustering data and the volleyball trajectory preprocessing data to obtain the volleyball motion trajectory data; Step S64: Automatically annotate key events of multi-target interaction according to the semantic trajectory segment data and the volleyball motion trajectory data to obtain key event annotation data.

Images