A method, system and medium for fast retrieval of video of sports event referee enforcement

Abstract

The application provides a sports event referee law enforcement video quick retrieval method, system and medium, and belongs to the technical field of sports event video intelligent processing and referee assistance. The method is characterized in that: a plurality of camera and sound pickup collection units are arranged, a global clock is unified, and a three-dimensional grid coordinate mapping model of the site is constructed; high-frame-rate video streams are collected, moving targets are identified, player numbers are extracted, occlusion missing information is completed, and a target trajectory file is obtained; a plurality of audio data are collected to independently identify whistles, and a law enforcement time point list is generated in combination with wireless referee whistle blowing data; a preset time length is traced back according to the retrieval condition input by a user, and a matched trajectory file is extracted; the coordinates of multi-target three-dimensional trajectories are compared by threshold value, and a plurality of stored videos are positioned synchronously; and for a whistle-free dispute scene, a playback frame position is obtained through trajectory intersection node identification. The application realizes second-level positioning and efficient retrieval of a dispute scene, and greatly improves referee law enforcement review efficiency and penalty accuracy.

Description

Technical Field

[0001] This application relates to the field of intelligent processing of sports event videos and referee assistance technology. Specifically, it relates to a method, system, and medium for rapid retrieval of sports event referee enforcement videos. Background Technology

[0002] In existing technologies, sports referee assistance systems mainly rely on single video playback or manual recording methods. For example, a sports referee assistance system (CN201510340361.5) acquires match footage and player location information through cameras and a positioning subsystem, and generates player movement trajectories based on query commands. Although this system can query player movement, it has the following shortcomings: First, it relies on only a single positioning subsystem and lacks a multi-view data completion mechanism, making it prone to losing trajectory information when players are obscured or moving at high speeds; second, it does not involve whistle recognition and automatic extraction of whistle timing, and cannot automatically associate the ruling event with video clips; third, the retrieval depends on manually input time periods and player identifiers, lacking intelligent matching capabilities based on multi-target spatial interaction, making it difficult to automatically locate controversial rulings. Furthermore, existing sports event video search methods (CN201210046448) achieve event location by establishing an index synchronously with the live broadcast script and video. However, this method relies on real-time input of external text scripts and cannot automatically identify the interaction between referee whistles and athlete trajectories during the event. It also has poor adaptability when there is no live broadcast script or the script is incomplete. Another method for retrieving highlights from large-scale events (CN201010126992) retrieves highlights based on the absolute time recorded by the match reporters. However, the match reporters' work relies on manual operation, which suffers from strong subjectivity and delayed response, making it difficult to meet the needs of rapid and accurate review of controversial calls in high-paced matches. In summary, existing technologies have not yet achieved a fully automated retrieval system integrating multiple video streams, multiple audio streams, wireless whistle signals, and three-dimensional trajectory data, especially in the automatic identification and replay location of controversial scenarios without whistles (such as concealed fouls and minor offsides), where there is a technological gap. Summary of the Invention

[0003] The purpose of this application is to provide a method, system, and medium for rapid retrieval of sports event refereeing videos. It aims to solve the technical problems of low retrieval efficiency in existing refereeing scenarios and difficulty in automatically locating disputed scenarios without whistles. The method first deploys multiple sets of cameras and audio acquisition units and unifies the global clock to collect field dimensions and camera setup parameters, constructing a three-dimensional grid coordinate mapping model of the field and establishing a mapping relationship between the image and the three-dimensional coordinates of the field. Each sub-processing unit acquires video streams at a frame rate of no less than 60 frames per second, identifies moving targets and extracts jersey numbers, uses multi-view data to fill in occlusion-related lost information, and combines the three-dimensional mapping model to generate a continuous target trajectory file carrying a global timestamp. Each sub-processing unit independently performs whistle recognition on multiple audio streams, and the main scheduling unit collects no fewer than two valid signals for cross-validation, while simultaneously acquiring wireless referee whistle data, summarizing and generating annotations. The system provides a list of enforcement time points, including whistle types and global timestamps. After the user inputs the subject identifier and search criteria for the penalty action, the system uses each whistle timestamp as a baseline, shifting forward by a preset time to define the search interval and extract matching target trajectories. Based on preset coordinate threshold parameters in the penalty rules, the system compares the coordinate differences of multiple target 3D trajectories to determine the spatial interaction state. After completing scene matching, it synchronously retrieves the stored recordings from all camera units. For scenarios without whistle disputes, the system uses filtering conditions such as player number, time period, and field area to define the trajectory traversal range, identifies trajectory intersection nodes where the coordinate differences of multiple targets fall within the threshold, and directly obtains the video playback frame positions. This invention organically integrates technologies such as 3D mesh pre-reconstruction of the field, multi-view occlusion completion, multi-channel audio cross-verification, wireless whistle assistance, multi-target trajectory threshold comparison, and trajectory intersection node identification. It achieves a complete closed loop from whistle event-driven retrieval to automatic discovery of non-whistle scenarios. It can assist the referee team in accurately locating multi-view video segments corresponding to controversial decisions within seconds, significantly improving VAR review efficiency. At the same time, it reveals hidden fouls or offside behaviors that are difficult to capture through conventional video replay through trajectory intersection analysis, further ensuring the fairness and transparency of the game's rulings.

[0004] This application provides a method for rapid retrieval of sports event refereeing videos, including the following steps: Multiple sets of camera acquisition units, audio acquisition units, sub-processing units and main scheduling units are deployed and a unified global clock is used to collect site dimensions and camera setup parameters, construct a three-dimensional grid coordinate mapping model of the site, and obtain the image-site three-dimensional coordinate mapping relationship; High frame rate video streams are acquired, moving targets on the field are identified and player numbers are extracted, centroid pixel data of target images are acquired, occlusion and missing information are filled in based on multi-view target data, and target trajectory files are obtained by combining the field three-dimensional grid coordinate mapping model. Collect multiple audio data and independently identify whistle sounds, verify the valid identification signals of no less than two channels, collect whistle data transmitted by wireless referee whistle, and obtain a list of law enforcement time points labeled with whistle types based on the two types of whistle data; Collect the user-input entity identifier and judgment behavior retrieval conditions, and backtrack the preset time period point by point according to the law enforcement time point list to extract the motion target trajectory file matching the retrieval conditions within the corresponding time period; According to the competition's judging rules, coordinate threshold comparisons are performed on the three-dimensional trajectories of multiple targets to complete the matching of the competition's interactive scenarios. Based on the preceding moments corresponding to the successfully matched enforcement time points, multiple camera units are simultaneously located and their videos are stored. By selecting the subject, time period, and location as criteria, limiting the trajectory traversal range, and identifying the intersection nodes of trajectory coordinates, the video playback frame positions corresponding to the no-whistle dispute scene are obtained.

[0005] In the rapid retrieval method for sports event refereeing videos described in this application, the method involves deploying multiple sets of camera acquisition units, audio acquisition units, sub-processing units, and a main scheduling unit, all with a unified global clock. This process collects data on the venue dimensions and camera setup parameters, constructs a three-dimensional mesh coordinate mapping model of the venue, and obtains the mapping relationship between the image and the three-dimensional coordinates of the venue. Specifically: Multiple sets of video and audio acquisition units are deployed around the site to collect local clock data from all hardware units and generate a global clock reference through unified calibration. Collect field size data such as length and width, markings, and goal dimensions; collect camera unit installation height and horizontal angle installation parameter data. Based on the collected site dimensions and camera setup parameters, the site's 3D mesh pre-reconstruction is completed offline, generating a site coordinate reference mesh; Based on the pre-reconstructed mesh, the image pixel coordinates are associated with the site's three-dimensional world coordinates, thus obtaining a fixed mapping relationship between the image and the site's three-dimensional coordinates.

[0006] In the fast retrieval method for sports event refereeing videos described in this application, the steps of acquiring high frame rate video streams, identifying moving targets on the field and extracting player numbers, acquiring centroid pixel data of target images, supplementing occlusion and missing information based on multi-view target data, and combining the field's three-dimensional mesh coordinate mapping model to obtain the target trajectory file are as follows: Each sub-processing unit acquires a video stream of the field with a frame rate of no less than 60 frames per second, acquires image data of players, referees, and ball targets, identifies target color features, and extracts jersey number character data; Collect image centroid pixel coordinate data of moving targets from various shooting angles, and record the frame time sequence marker data lost due to target occlusion in a single viewpoint; Based on the target pixel data collected from the opposite side of the site, the occluded and missing target data is completed, and the pixel coordinates are transformed by combining the site's three-dimensional grid coordinate mapping model. By binding the converted 3D spatial coordinates to the corresponding frame global timestamp, a continuous target trajectory file storing the 3D coordinates and timestamps is obtained.

[0007] In the rapid retrieval method for sports event refereeing videos described in this application, the steps of collecting multiple audio data and independently identifying whistles, verifying at least two valid identification signals, collecting whistle data transmitted via wireless referee whistle, and obtaining a list of enforcement time points labeled with whistle types based on the two types of whistle data are as follows: Each sub-processing unit collects multi-channel audio data from the site after noise reduction and filtering, extracts audio spectrum feature data, and independently performs whistle feature matching and recognition. The main scheduling unit collects valid whistle recognition data from no less than two sub-processing units and completes cross-validation of the validity of the audio recognition signal. Collect wireless data on the duration, interval, and number of whistles output by the wireless referee whistle, and summarize the two types of whistle data: audio whistle sound and wireless whistle sound. Based on the standard whistle rules for the competition, two types of whistle data are matched to obtain a list of enforcement time points labeled with whistle type and global timestamp.

[0008] In the rapid retrieval method for sports event referee enforcement videos described in this application, the step of collecting the subject identifier and penalty behavior retrieval conditions input by the user, and extracting the trajectory file of the moving target matching the retrieval conditions within the corresponding time period by traversing back a preset duration point by point according to the enforcement time point list, specifically involves: The main scheduling unit collects the team and player identification data input by the user, and collects the retrieval conditions data for fouls, offsides, and goals. Collect preset backtracking duration ΔT parameter data, and define the retrieval interval based on the forward offset ΔT of each whistle-blowing timestamp in the law enforcement time point list; Filter the trajectory file storage range based on the start and end global timestamps of the search interval, and extract all moving target trajectory files that match the main body identifier within the interval; The extracted target trajectory files are stored in a temporary cache space for subsequent multi-target 3D trajectory coordinate threshold comparison calculations.

[0009] In the rapid retrieval method for sports event referee enforcement video described in this application, the step of performing coordinate threshold comparison of multi-target three-dimensional trajectories according to the event's judging rules, completing the matching of the field interaction scene, and simultaneously locating and storing videos from multiple camera units based on the preceding time corresponding to the successfully matched enforcement time point, specifically involves: Collect the preset coordinate threshold parameter data of the corresponding penalty rules of the event, and read the trajectory file of the X / Y / Z three-dimensional coordinates of multiple targets stored in the cache; Based on the coordinate threshold parameter, a comparison calculation is performed on the coordinate difference of multiple target trajectories, and the spatial interaction status of the targets is determined based on the comparison difference. Based on the interaction status determination result, the field interaction scene matching is completed, and the replay start time is obtained by subtracting the backtracking time from the whistle timestamp of the matched whistle. Based on the start time of the replay, all the video recordings stored in the camera units are retrieved synchronously to obtain the replay video of the event after multi-channel synchronous positioning.

[0010] In the rapid retrieval method for sports event refereeing videos described in this application, the filtering conditions for the collecting subject, time period, and venue area, limiting the trajectory traversal range, identifying trajectory coordinate intersection nodes, and obtaining the video playback frame position corresponding to the non-whistle dispute scene are specifically as follows: The main scheduling unit collects player ID movement data, first and second half match time data, and midfield and penalty area filtering data. The start and end time intervals for trajectory file reading and the spatial traversal range of the site are defined based on the three types of screening conditions collected. Based on the defined range, traverse all target 3D trajectory data within the interval and identify trajectory intersection nodes where the coordinate difference of multiple targets falls within the threshold. Based on the global frame timestamp bound to the intersection node, the video playback frame position corresponding to the dispute scene without whistle is obtained.

[0011] Secondly, this application provides a rapid retrieval system for sports event referee videos, including: The camera acquisition unit captures high frame rate video streams from the stadium. The sound pickup and acquisition unit collects multiple audio data streams from the venue. The sub-processing unit completes moving target tracking and audio whistle recognition; The main scheduling unit unifies the global clock, constructs a three-dimensional grid coordinate mapping model of the site, summarizes two types of whistle data to generate a list of law enforcement time points, and matches trajectories and locates multiple stored videos based on search conditions.

[0012] In the sports event referee enforcement video rapid retrieval system described in this application, the system further includes a memory and a processor. The memory includes a sports event referee enforcement video rapid retrieval method program. When the sports event referee enforcement video rapid retrieval method program is executed by the processor, it performs the following steps: Multiple sets of camera acquisition units, audio acquisition units, sub-processing units and main scheduling units are deployed and a unified global clock is used to collect site dimensions and camera setup parameters, construct a three-dimensional grid coordinate mapping model of the site, and obtain the image-site three-dimensional coordinate mapping relationship; High frame rate video streams are acquired, moving targets on the field are identified and player numbers are extracted, centroid pixel data of target images are acquired, occlusion and missing information are filled in based on multi-view target data, and target trajectory files are obtained by combining the field three-dimensional grid coordinate mapping model. Collect multiple audio data and independently identify whistle sounds, verify the valid identification signals of no less than two channels, collect whistle data transmitted by wireless referee whistle, and obtain a list of law enforcement time points labeled with whistle types based on the two types of whistle data; Collect the user-input entity identifier and judgment behavior retrieval conditions, and backtrack the preset time period point by point according to the law enforcement time point list to extract the motion target trajectory file matching the retrieval conditions within the corresponding time period; According to the competition's judging rules, coordinate threshold comparisons are performed on the three-dimensional trajectories of multiple targets to complete the matching of the competition's interactive scenarios. Based on the preceding moments corresponding to the successfully matched enforcement time points, multiple camera units are simultaneously located and their videos are stored. By selecting the subject, time period, and location as criteria, limiting the trajectory traversal range, and identifying the intersection nodes of trajectory coordinates, the video playback frame positions corresponding to the no-whistle dispute scene are obtained.

[0013] Thirdly, this application also provides a computer-readable storage medium including a program for a fast retrieval method for sports event referee enforcement videos. When executed by a processor, the program implements the steps of the fast retrieval method for sports event referee enforcement videos as described in any of the preceding claims.

[0014] As can be seen from the above, the embodiments of this application provide a method, system, and medium for rapid retrieval of sports event referee enforcement videos. This method involves deploying multiple sets of camera acquisition units and audio acquisition units around the perimeter of the field, unifying the global clock, collecting the field dimensions and camera setup parameters, and then offline constructing a three-dimensional grid coordinate mapping model of the field to establish a fixed mapping relationship between image pixel coordinates and the three-dimensional world coordinates of the field. Furthermore, each sub-processing unit acquires the field video stream at a frame rate of no less than 60 frames per second, identifies moving targets such as players, referees, and balls, extracts jersey numbers, collects the centroid pixel coordinates of the targets from various perspectives, uses data from the opposite side of the field to supplement information lost due to occlusion, and combines the three-dimensional mapping model to obtain a continuous target trajectory file carrying a global timestamp. Based on this, each sub-processing unit independently extracts spectral features and performs whistle matching and recognition on the multiple audio streams after noise reduction. The main scheduling unit collects valid whistle recognition signals from at least two sub-processing units for cross-verification. Simultaneously, it collects data on the duration, interval, and number of whistles transmitted by the wireless referee whistle, summarizing the audio whistle and wireless whistle signals. After matching according to the standard whistle rules of the competition, it generates a list of enforcement time points labeled with whistle type and global timestamp. When the user inputs main identifiers such as team and player numbers, as well as search conditions such as fouls, offsides, and goals, the system uses each whistle timestamp as a baseline and shifts forward by a preset backtracking time ΔT to define the search interval. It then extracts all target trajectories matching the main identifiers within the interval from the trajectory file and stores them in a temporary cache. Subsequently, based on the pre-set coordinate threshold parameters of the corresponding ruling rules for the match, the system performs coordinate difference comparison calculations on the 3D trajectory data of multiple targets in the cache to determine the spatial interaction status between targets (such as physical contact, goal line crossing, offside positional relationship, etc.). After completing the matching of the on-field interaction scene, the system subtracts the replay duration from the whistle timestamp of the successfully matched target to obtain the replay start time, thereby synchronously retrieving the stored recordings of all camera units to generate multi-channel synchronously positioned match replay videos. In addition, for disputed scenarios without whistles, the system can collect player numbers, first / second half time periods, midfield / penalty area filtering conditions, etc., define the reading time interval and spatial traversal range of trajectory files, fully traverse the 3D trajectory data of targets within the range, identify trajectory intersection nodes where the coordinate difference of multiple targets falls within the preset threshold, and directly obtain the video replay frame position based on the global frame timestamp bound to this node, thereby achieving automatic positioning of hidden disputed scenarios.This invention organically integrates technologies such as 3D mesh pre-reconstruction of the field, multi-view occlusion completion, multi-channel audio cross-verification, wireless whistle assistance, multi-target trajectory threshold comparison, and trajectory intersection node identification. It achieves a complete closed loop from whistle event-driven retrieval to automatic discovery of non-whistle scenarios. This not only assists the referee team in accurately locating multi-view video segments corresponding to controversial decisions within seconds, significantly improving the efficiency of VAR (Video Assistant Referee) review, but also reveals hidden fouls or offside behaviors that are difficult to capture through conventional video replay through trajectory intersection analysis, further ensuring the fairness and transparency of the game's rulings.

[0015] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a method for rapid retrieval of sports event referee enforcement videos provided in this application embodiment; Figure 2 A flowchart illustrating a method for rapidly retrieving video recordings of sports event referees, provided in this application embodiment, shows how to obtain the video playback frame position corresponding to a dispute scene without a whistle. Figure 3 This is a structural block diagram of a rapid retrieval system for sports event referee enforcement video provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0019] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first" and "second" are used only to distinguish descriptions and should not be construed as indicating or implying relative importance. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0020] Please refer to Figure 1 , Figure 1 This is a flowchart of a method for rapid retrieval of sports event referee enforcement video recordings in some embodiments of this application.

[0021] The first aspect of this invention discloses a method for rapid retrieval of sports event refereeing videos for use in terminal devices, such as computers and mobile phones. This method for rapid retrieval of sports event refereeing videos includes the following steps: S101. Deploy multiple sets of camera acquisition units, audio acquisition units, sub-processing units and main scheduling units, and unify the global clock to collect site dimensions and camera setup parameters, construct a three-dimensional grid coordinate mapping model of the site, and obtain the image-site three-dimensional coordinate mapping relationship. S102. Acquire high frame rate video stream, identify moving targets on the field and extract player numbers, acquire centroid pixel data of target images, fill in occlusion and missing information based on multi-view target data, and obtain target trajectory file by combining the field three-dimensional grid coordinate mapping model. S103. Collect multiple audio data and independently identify whistle sounds, verify the valid identification signals of no less than two channels, collect the whistle data transmitted by the wireless referee whistle, and obtain a list of law enforcement time points labeled with whistle types based on the two types of whistle data. S104. Collect the main identifier and judgment behavior retrieval conditions input by the user, and backtrack the preset time period point by point according to the law enforcement time point list to extract the motion target trajectory file that matches the retrieval conditions within the corresponding time period; S105. Based on the competition's judging rules, perform coordinate threshold comparison on the three-dimensional trajectories of multiple targets, complete the matching of the competition's interactive scenarios, and simultaneously locate and store videos from multiple camera units based on the preceding moments corresponding to the successfully matched enforcement time points. S106. The subject of collection, time period, and site area are selected as the filtering conditions to limit the trajectory traversal range, identify the intersection nodes of trajectory coordinates, and obtain the video playback frame position corresponding to the scene without whistle disputes.

[0022] The process involves deploying multiple camera acquisition units, audio acquisition units, sub-processing units, and a main scheduling unit within the competition venue, and unifying the global clock. This allows for the synchronous acquisition of venue dimensions and camera setup parameters, constructing a 3D mesh coordinate mapping model of the venue, and obtaining a precise mapping relationship between the images and the 3D coordinates of the venue. For example, deploying 6 to 8 high-definition camera devices in a standard basketball court, and using PTP clock synchronization to control the time error of each device to within 1ms, can eliminate time deviations in multi-device acquisition, laying a solid foundation for subsequent spatiotemporal alignment throughout the entire process. Then, high frame rate video streams are acquired, moving targets within the court are identified, player numbers are extracted, and centroid pixel data of the target images is acquired. Based on multi-view target data, occlusion and missing information are supplemented, combined with the aforementioned... The 3D mesh coordinate mapping model of the field generates a complete target trajectory file. For example, in a football match, for short-term occlusion caused by players quickly moving through the field, trajectory breakpoints can be filled in by using the image information from adjacent camera positions to obtain a continuous and complete player movement trajectory, avoiding trajectory gaps caused by image occlusion. Subsequently, multiple audio data are collected and whistle sounds are independently identified. At least two valid identification signals are verified, and whistle data transmitted by the wireless referee whistle are collected simultaneously. Combining the two types of whistle data, a list of enforcement time points labeled with whistle types is generated. For example, in amateur basketball games, multiple sets of audio pickup devices are deployed, and multi-path verification filters out misidentification caused by the noise of the audience, accurately distinguishing different whistle types such as foul whistles and timeout whistles, avoiding invalid whistles. The subsequent retrieval process addresses audio interference. Next, the system collects the user-inputted subject identifier and retrieval criteria for the judged action. Based on the list of enforcement time points, it traces back a preset duration point by point, extracting the trajectory files of moving targets matching the retrieval criteria within the corresponding time period. For example, when a user searches for a foul-related scene involving a specific player, the system can automatically trace back the corresponding trajectory segment 2 seconds before the whistle, quickly locking onto the target's associated motion data and significantly narrowing the scope of subsequent searches. Then, based on the game's judging rules, it performs coordinate threshold comparisons on the three-dimensional trajectories of multiple targets, completing the matching of the game's interactive scenes. Based on the preceding moment corresponding to the successfully matched enforcement time point, it synchronously locates the video stored in multiple camera units. For example, in a blocking foul scene in a basketball game, it compares the defensive player's... The system interacts with the trajectory coordinate threshold of offensive players to determine the most relevant data, allowing direct and synchronous retrieval of video footage from all cameras at the corresponding moment, eliminating the need for manual dragging and searching. Finally, it collects subject, time period, and court area filtering conditions to limit the trajectory traversal range, identify trajectory coordinate intersection nodes, and obtain the video replay frame positions corresponding to controversial scenarios without whistles. For example, in a controversial scenario where players appear to collide but the referee doesn't whistle, by defining a limited area under the basket and filtering the trajectory, the corresponding replay frame position can be quickly located. This provides complete video support for game review and verification of controversial calls. The overall solution eliminates the need for manual sifting of massive amounts of game footage, significantly improving the retrieval efficiency of referee-related videos and adapting to the review needs of various mainstream ball games.

[0023] According to an embodiment of the present invention, the deployment of multiple sets of camera acquisition units, audio acquisition units, sub-processing units, and a main scheduling unit, along with a unified global clock, to acquire site dimensions and camera setup parameters, construct a three-dimensional grid coordinate mapping model of the site, and obtain the mapping relationship between the image and the three-dimensional coordinates of the site, specifically: Multiple sets of video and audio acquisition units are deployed around the site to collect local clock data from all hardware units and generate a global clock reference through unified calibration. Collect field size data such as length and width, markings, and goal dimensions; collect camera unit installation height and horizontal angle installation parameter data. Based on the collected site dimensions and camera setup parameters, the site's 3D mesh pre-reconstruction is completed offline, generating a site coordinate reference mesh; Based on the pre-reconstructed mesh, the image pixel coordinates are associated with the site's three-dimensional world coordinates, thus obtaining a fixed mapping relationship between the image and the site's three-dimensional coordinates.

[0024] In actual implementation, multiple sets of video and audio acquisition units are evenly deployed around the perimeter of the competition venue. Simultaneously, all sub-processing units and the main scheduling unit are integrated into a clock calibration system. Local clock data from all hardware units is collected for unified calibration, generating a global clock reference with errors controlled within milliseconds. For example, a 4K high-definition camera unit is deployed 20 meters outside the sidelines and end lines of a standard 11-a-side football field, with an audio acquisition unit deployed alongside each camera unit. All devices are connected to the main scheduling unit via a wired network for clock synchronization. This method completely avoids subsequent trajectory misalignment and audio / video asynchrony issues caused by inconsistent time bases between different acquisition devices. It then comprehensively collects all field dimensions data, including length, width, marking positions, and goal dimensions. Simultaneously, it records the installation parameters of each camera unit, such as its height, horizontal angle, and tilt angle. For example, for a standard indoor basketball court, 3D laser scanning can obtain the precise dimensions and coordinates of the three-point line, restricted area, and basket backboard. The installation height and horizontal orientation parameters of each camera unit installed under the eaves of the venue are recorded simultaneously to ensure... The collected field and equipment parameters perfectly match the actual scene. Subsequently, based on the collected field size data and camera setup parameters, the field's 3D mesh pre-reconstruction is completed offline before the official start of the event, generating a field coordinate reference mesh covering the competition area and the buffer zone around the sidelines. For example, a standard volleyball court is divided into uniform 3D mesh units with a side length of 0.5 meters. Each mesh unit corresponds to a unique 3D spatial coordinate, providing a unified spatial reference for subsequent pixel coordinate transformation. Finally, based on the pre-reconstructed field coordinate reference mesh, the image pixel coordinates in the footage captured by each camera unit are associated and bound one by one with the corresponding 3D world coordinates in the field, obtaining a stable fixed mapping relationship between the image and the field's 3D coordinates. There is no need to repeatedly perform coordinate calibration during the event. This step, through pre-positioning of hardware, parameter collection, and model construction, provides a unified spatiotemporal reference for the subsequent target trajectory generation, whistle timing alignment, and precise video recording positioning throughout the entire process. It fundamentally avoids the problems of multi-camera image coordinate transformation deviation and spatiotemporal misalignment of data from different acquisition devices, greatly improving the stability and accuracy of subsequent full-process retrieval.

[0025] According to an embodiment of the present invention, the steps of acquiring high frame rate video streams, identifying moving targets on the field and extracting player numbers, acquiring centroid pixel data of target images, supplementing occlusion and missing information based on multi-view target data, and combining the field's three-dimensional mesh coordinate mapping model to obtain a target trajectory file are as follows: Each sub-processing unit acquires a video stream of the field with a frame rate of no less than 60 frames per second, acquires image data of players, referees, and ball targets, identifies target color features, and extracts jersey number character data; Collect image centroid pixel coordinate data of moving targets from various shooting angles, and record the frame time sequence marker data lost due to target occlusion in a single viewpoint; Based on the target pixel data collected from the opposite side of the site, the occluded and missing target data is completed, and the pixel coordinates are transformed by combining the site's three-dimensional grid coordinate mapping model. By binding the converted 3D spatial coordinates to the corresponding frame global timestamp, a continuous target trajectory file storing the 3D coordinates and timestamps is obtained.

[0026] The process involves implementing the system based on the pre-built 3D mesh coordinate mapping relationship of the field. Each sub-processing unit collects high frame rate video streams of the field at a rate of no less than 60 frames per second, simultaneously acquiring image data of all on-field targets, including players, referees, and balls. Target color feature matching identifies the affiliation of different teams, and character recognition algorithms extract numerical character data from jerseys. For example, in basketball, a frame rate of 60 frames per second can fully capture the details of players' rapid changes of direction, and can reliably identify the corresponding numbers even if the jersey numbers are partially obscured, avoiding target identification errors. Subsequently, the system simultaneously collects the centroid pixel coordinate data of all moving targets from various shooting angles, and marks the frames where targets are lost due to occlusion in a single viewpoint with unique frame time-series markers. For example, in a football match's penalty area with intense competition, if the view from one side of the camera is completely obscured by multiple players, the system will automatically mark that... The missing target data in a frame does not generate invalid coordinate data. Then, target pixel data from the opposite camera position is retrieved to fill in the missing target data obscured from the current viewpoint. Combined with the previously constructed 3D grid coordinate mapping model, the completed pixel coordinates are accurately converted into 3D spatial coordinates within the court. For example, in volleyball, when a net player's jump completely obscures the camera view of an attacking player, the attacking player's position data can be completed from the camera view on the other side of the court, preventing trajectory breaks. Finally, the converted 3D spatial coordinates are bound to the global timestamp of the corresponding frame, generating a continuous target trajectory file that simultaneously stores the 3D coordinates and the global timestamp. This step completely solves the problem of target occlusion trajectory loss in intense competitive scenarios by using multi-camera data complementarity. The resulting full continuous trajectory data provides reliable data support for subsequent judgment scene matching and accurate video retrieval.

[0027] According to an embodiment of the present invention, the process of collecting multiple audio data and independently identifying whistles, verifying at least two valid identification signals, collecting whistle data transmitted by the wireless referee whistle, and obtaining a list of enforcement time points labeled with whistle types based on the two types of whistle data, specifically includes: Each sub-processing unit collects multi-channel audio data from the site after noise reduction and filtering, extracts audio spectrum feature data, and independently performs whistle feature matching and recognition. The main scheduling unit collects valid whistle recognition data from no less than two sub-processing units and completes cross-validation of the validity of the audio recognition signal. Collect wireless data on the duration, interval, and number of whistles output by the wireless referee whistle, and summarize the two types of whistle data: audio whistle sound and wireless whistle sound. Based on the standard whistle rules for the competition, two types of whistle data are matched to obtain a list of enforcement time points labeled with whistle type and global timestamp.

[0028] In this process, the entire data alignment is carried out based on the global clock reference that was completed in the previous step. In the actual event operation scenario, each sub-processing unit collects multi-channel audio data of the venue after adaptive noise reduction and filtering, filtering out background interference sounds such as the shouts of the audience, the stadium broadcasts, and the friction sounds of players running. The spectral feature data of specific frequency bands in the audio is extracted, and each sub-processing unit independently carries out whistle feature matching and recognition. For example, in the scenario of a full-capacity spectator of a large football event, even if the intensity of the background noise is high, the sub-processing units corresponding to each sound acquisition unit can accurately separate the exclusive features of the referee's whistle through targeted spectral feature filtering, without being interfered with by the noisy background sound of the venue. Next, the main scheduling unit collects valid whistle recognition data from at least two sub-processing units. It then performs cross-validation on the recognition results from different sub-processing units to determine the validity of the audio recognition signals. Only signals whose time deviation between multiple recognition results is within the allowable range are considered valid whistle signals. For example, during halftime in a basketball game, high-frequency sound effects similar to whistles may occur. Through cross-validation of multiple recognition results, invalid signals misidentified by a single device can be directly filtered out, significantly reducing the probability of misjudgment at the recognition stage. Subsequently, the wireless referee whistle outputs data on whistle duration, whistle interval, and number of whistles are simultaneously collected. The whistle data obtained from audio recognition and the whistle data transmitted by the wireless referee whistle are fully aggregated to form a unified dataset of the two types of whistle data. For example, when a referee presses the whistle button on their wireless whistle, it simultaneously transmits the whistle trigger status data back to the main scheduling unit in real time. This data is completely unaffected by ambient noise and can serve as a reliable supplementary verification basis for the audio recognition data. Finally, feature matching was performed on the two types of whistle data according to the standard whistle rules of the corresponding competition. Different combinations of whistle duration and whistle interval were mapped to the corresponding whistle types. For example, a short single whistle corresponds to a foul, three consecutive short whistles correspond to a goal, and a long whistle corresponds to the end of the first half. Finally, a list of enforcement time points that simultaneously marks the whistle type and global timestamp was generated. This step completely solved the industry pain points of single audio recognition solutions being easily interfered with by on-site noise and having a high misidentification rate through dual verification of two types of data: audio recognition and direct hardware transmission. The resulting list of enforcement time points perfectly matches the time nodes of the referee's actual enforcement actions, providing accurate and reliable time anchors for subsequent video playback retrieval. The entire process does not require manual verification of whistle times one by one and is fully adaptable to various high-noise large-scale competition scenarios.

[0029] According to an embodiment of the present invention, the step of collecting the subject identifier and judgment behavior retrieval conditions input by the user, and extracting the trajectory file of the moving target matching the retrieval conditions within the corresponding time period by tracing back a preset time period point by point according to the law enforcement time point list, specifically involves: The main scheduling unit collects the team and player identification data input by the user, and collects the retrieval conditions data for fouls, offsides, and goals. Collect preset backtracking duration ΔT parameter data, and define the retrieval interval based on the forward offset ΔT of each whistle-blowing timestamp in the law enforcement time point list; Filter the trajectory file storage range based on the start and end global timestamps of the search interval, and extract all moving target trajectory files that match the main body identifier within the interval; The extracted target trajectory files are stored in a temporary cache space for subsequent multi-target 3D trajectory coordinate threshold comparison calculations.

[0030] As a crucial execution step for narrowing the search scope and targeting relevant data, the system utilizes a pre-generated list of fully labeled enforcement time points for targeted data filtering. In actual match replay scenarios, the main scheduling unit collects user-inputted key identifiers such as team affiliation and player number, while simultaneously collecting user-selected search criteria for penalties such as fouls, offsides, and goals. For example, if a match replay specialist needs to search for all foul-related scenarios involving player number 7 in a particular match, they only need to input the corresponding player number and foul search criteria into the system interface, without manually entering complex time range parameters, to complete the initial input of the search requirements. The system then collects the user-pre-configured or system-default replay duration ΔT parameter data. Based on each whistle timestamp in the enforcement time point list, it shifts forward by ΔT to define a specific search interval for each whistle. For example, for contact fouls in basketball, the replay duration can be set to 2 seconds; for offside penalties in football, the replay duration can be set to 3 seconds, ensuring that the defined search interval completely covers the entire process of the penalty action and does not miss any key action segments. Subsequently, based on the global timestamps of the defined search interval, the system filters out the corresponding time range from the fully stored target trajectory files, extracting all motion target trajectory files within that interval that match the user-input subject identifier. For example, when a user specifies to search for all offensive-related scenarios of a certain team, the system automatically filters out irrelevant trajectory data of opposing team players, retaining only the complete motion trajectories of the target team's players within the search interval, significantly reducing the amount of data that needs to be processed in subsequent calculations. Finally, the extracted target trajectory files are uniformly stored in a high-speed temporary cache space for direct use in subsequent multi-target 3D trajectory coordinate threshold comparison calculations. This step, through targeted backtracking and conditional filtering, compresses the massive trajectory data originally covering the entire match into a small dataset highly relevant to the search requirements, avoiding the problems of excessive resource consumption and low computational efficiency caused by processing the entire match data in subsequent calculations, thus laying a solid data foundation for rapid matching of subsequent on-field interaction scenarios.

[0031] According to an embodiment of the present invention, the step of performing coordinate threshold comparison on the three-dimensional trajectories of multiple targets according to the competition's judging rules, completing the matching of the competition scene interaction, and simultaneously locating and storing videos from multiple camera units based on the preceding time point corresponding to the successfully matched enforcement time point, specifically includes: Collect the preset coordinate threshold parameter data of the corresponding penalty rules of the event, and read the trajectory file of the X / Y / Z three-dimensional coordinates of multiple targets stored in the cache; Based on the coordinate threshold parameter, a comparison calculation is performed on the coordinate difference of multiple target trajectories, and the spatial interaction status of the targets is determined based on the comparison difference. Based on the interaction status determination result, the field interaction scene matching is completed, and the replay start time is obtained by subtracting the backtracking time from the whistle timestamp of the matched whistle. Based on the start time of the replay, all the video recordings stored in the camera units are retrieved synchronously to obtain the replay video of the event after multi-channel synchronous positioning.

[0032] The core execution component, enabling intelligent matching of penalty scenarios and precise positioning of multi-channel video recordings, relies on directional filtering trajectory data in the preceding cache for computation. In actual match review scenarios, it collects pre-configured penalty rule coordinate threshold parameters for the current match and reads multi-target X / Y / Z 3D coordinate trajectory files stored in the temporary cache. For example, in an indoor volleyball match involving a net hit, the system pre-configures spatial coordinate threshold boundaries on both sides of the net and directly retrieves the 3D trajectory data of the player's hand and the volleyball from the cache for computation, without needing to load the full trajectory information for the entire match. Then, based on the preset coordinate threshold parameters, it performs frame-by-frame comparison of the real-time coordinate differences between the multi-target trajectories. Based on the comparison results, it determines the spatial interaction state between different moving targets. For example, in a basketball match involving a blocking foul, the system can compare the 3D coordinate differences between the offensive and defensive players to determine whether they have made physical contact within a reasonable spatial range, accurately identifying interaction states that meet the penalty characteristics and avoiding misjudging distant, non-contact movement trajectories as foul scenarios. Subsequently, based on the judgment result of the target space interaction state, the system accurately matches the corresponding field interaction scene. Then, it subtracts the preset backtracking time from the matched whistle timestamp to directly obtain the start time of the corresponding replay segment. For example, after matching an offside scene in a football match, the system automatically backtracks the preset time from the whistle time to lock the complete start node of the offside action, eliminating the need for manual repeated dragging of the timeline to find the start position of the action. Finally, based on the calculated replay start time, the system synchronously retrieves the corresponding time period recordings stored in all camera units to obtain multi-channel fully synchronized replay videos. This step achieves intelligent recognition of the ruling scene through the coordinate threshold comparison of three-dimensional trajectories. It eliminates the need for manual frame-by-frame screening of massive recording segments and can output the synchronized replay content of all camera positions at once, greatly improving the efficiency of match officiating review and disputed ruling verification, and fully adapting to the fast replay needs of various mainstream ball games.

[0033] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a rapid retrieval method for sports event refereeing videos in some embodiments of this application, specifically a method for obtaining the video playback frame location corresponding to a disputed scene without a whistle. According to embodiments of the present invention, the filtering conditions for the acquisition subject, time period, and venue area, limiting the trajectory traversal range, identifying trajectory coordinate intersection nodes, and obtaining the video playback frame location corresponding to the disputed scene without a whistle are as follows: The main scheduling unit collects player ID movement data, first and second half match time data, and midfield and penalty area filtering data. The start and end time intervals for trajectory file reading and the spatial traversal range of the site are defined based on the three types of screening conditions collected. Based on the defined range, traverse all target 3D trajectory data within the interval and identify trajectory intersection nodes where the coordinate difference of multiple targets falls within the threshold. Based on the global frame timestamp bound to the intersection node, the video playback frame position corresponding to the dispute scene without whistle is obtained.

[0034] As a supplementary execution step to cover the retrieval needs of scenarios without whistle-related disputes, the system relies on full and continuous target trajectory data to accurately locate non-whistle-related scenarios. In extended verification scenarios during actual game replays, the main scheduling unit collects player ID-type sports subject data input by the user, game time data such as the first half, second half, and overtime, and court area filtering data such as the halftime area, restricted area, and under-basket restricted area. For example, when a game technical specialist needs to verify a suspected confrontation scene in the restricted area during the second half of a game, they only need to select the corresponding player, the second half time period, and the restricted area to quickly input the retrieval requirements without additional association with whistle-related data. Subsequently, based on the three filtering conditions of subject, time period, and court area, the system defines the start and end time intervals for trajectory file reading and the corresponding court space traversal range, completely filtering out irrelevant trajectory data that is not in the selected time period or the specified area. For example, when the user selects the under-basket area as the search range, the system will directly exclude trajectory content outside the three-point line area, significantly reducing the data range that needs to be traversed subsequently and avoiding unnecessary computation that consumes system resources.

[0035] The system then traverses all 3D trajectory data of targets within the defined spatiotemporal range, identifying trajectory intersection nodes where the coordinate differences between multiple targets fall within a preset threshold. For example, in a football match, if two players' trajectories intersect at close range in the penalty area, suggesting physical contact but the referee does not blow the whistle, the system can accurately capture these whistle-free trajectory intersection nodes, ensuring no potential controversial moments are missed. Finally, based on the global frame timestamps bound to the identified intersection nodes, the system directly locates the corresponding multi-channel video replay frames of the whistle-free controversial scene. This step breaks the limitation of traditional retrieval methods that rely solely on whistle timestamps to find recordings. It eliminates the need to depend on the time anchor points generated by the referee's whistle, quickly locating all suspected confrontations and violations not covered by the whistle during the match. This provides comprehensive supplementary support for the entire process of match review, technical analysis, and retrospective analysis of controversial decisions, adapting to the needs of various refined match operations and technical statistics.

[0036] According to an embodiment of the present invention, the method further includes a multimodal data fusion and intelligent auxiliary judgment suggestion generation step, specifically: Collect multiple video frames, audio clips, trajectory files, and competition rule database data corresponding to the list of law enforcement time points, and construct multimodal feature vectors; Based on the multimodal feature vectors, a pre-trained deep learning model is used for fusion reasoning to obtain the recognition results of target interaction behavior, contact intensity level, and the rationality of action timing. Based on the semantic matching between the fusion reasoning results and the event rule base, an auxiliary penalty suggestion report is obtained, which includes the penalty type, the applicable clause, the confidence score, and the suggested replay perspective. The auxiliary judgment suggestion report is pushed to the referee terminal and VAR terminal, and after the referee confirms it, it is stored in the event enforcement knowledge base, which is used for subsequent similar scenario retrieval and model optimization.

[0037] Among them, the multimodal intelligent auxiliary judgment execution link, which is an extension of the rapid retrieval, forms a complete technical closed loop with all the aforementioned retrieval steps. In actual VAR game officiating scenarios, it collects multiple synchronous video frames corresponding to the list of enforcement time points, audio clips of the corresponding time periods, full-volume associated target trajectory files, and locally stored game rule database data. It performs feature normalization processing on different types of heterogeneous data to construct a unified multimodal feature vector covering visual, audio, spatial trajectory, and rule text. For example, in the confrontation foul scenario of a basketball game, the system will simultaneously integrate all camera footage at the moment of the foul, audio clips before and after the whistle, the three-dimensional motion trajectory of both players, and the corresponding FIBA ​​foul rule text, fusing the core features of multiple types of data into a unified feature vector to avoid judgment bias caused by incomplete information from a single modality of data. The constructed multimodal feature vectors are then input into the pre-trained deep learning model for fusion inference. The model relies on extensive historical match data for feature cross-validation, yielding identification results for the specific type of target interaction behavior, the intensity of contact between the parties, and the reasonableness of the timing of the actions. For example, in a football match where a player falls in the penalty area, the model can accurately distinguish between a normal fall and a dive through multimodal data cross-validation, determine the severity of contact, and verify the logical consistency of the player's actions and the referee's whistle, avoiding misjudgments based on a single frame. Subsequently, the identification results obtained from the fusion inference are semantically matched with corresponding clauses in the match rule base, automatically generating an auxiliary ruling suggestion report containing the suggested ruling type, the corresponding rule clause, the result confidence score, and recommended priority replay angles. For example, in a suspected offside scenario, the suggestion report clearly marks the coordinates of the offside player's touch, the corresponding offside rule clause, and the confidence score of the ruling. It also prioritizes side-view cameras that clearly show the relationship between attacking and defending players and the baseline position as the primary replay angle, providing VAR referees with intuitive reference. Finally, the generated auxiliary judgment suggestion report is pushed to the referee's terminal and the VAR replay room terminal in real time. After the referee confirms the final judgment, the full amount of multimodal data of the scene and the final judgment result are simultaneously stored in the event's officiating knowledge base. This forms labeled storage data that can be used for rapid retrieval of similar scenes and iterative optimization of deep learning models. This step, based on the rapid location of relevant officiating videos, further opens up the intelligent analysis link of multimodal data. It can provide referees with structured auxiliary judgment references without the need for manual verification of rules and multi-camera footage. At the same time, the model recognition accuracy is continuously optimized through the continuous accumulation of officiating results, forming a positive cycle of becoming more accurate with use, which greatly improves the fairness and efficiency of event officiating.

[0038] Please refer to Figure 3 , Figure 3This is a structural block diagram of a rapid retrieval system for sports event referee enforcement video provided in an embodiment of this application.

[0039] This invention also discloses a rapid retrieval system for sports event referee videos, comprising: Camera acquisition unit 301 acquires high frame rate video streams from the stadium; The sound pickup and acquisition unit 302 collects multiple audio data from the venue. Sub-processing unit 303 completes moving target tracking and audio whistle recognition; The main scheduling unit 304 unifies the global clock, constructs a three-dimensional grid coordinate mapping model of the site, summarizes two types of whistle data to generate a list of law enforcement time points, and matches trajectories and locates multiple stored videos based on search conditions.

[0040] According to an embodiment of the present invention, the sports event referee enforcement video rapid retrieval system further includes a memory and a processor. The memory includes a sports event referee enforcement video rapid retrieval method program. When the sports event referee enforcement video rapid retrieval method program is executed by the processor, it implements the steps of the sports event referee enforcement video rapid retrieval method as described in any one of the embodiments.

[0041] A third aspect of the present invention provides a computer-readable storage medium comprising a program for a method of quickly retrieving video recordings of referees in sports events. When executed by a processor, the program implements the steps of the method for quickly retrieving video recordings of referees in sports events as described in any of the preceding claims.

[0042] This invention discloses a method, system, and medium for rapid retrieval of sports event refereeing videos. It involves first deploying multiple sets of camera acquisition units, audio acquisition units, sub-processing units, and a main scheduling unit around the perimeter of the field, and completing a unified hardware global clock calibration. The system collects parameters such as the length and width of the field, markings, goal dimensions, and the camera unit mounting height and horizontal angle. An offline pre-reconstruction generates a three-dimensional coordinate reference grid for the field, establishing a fixed mapping relationship between image pixel coordinates and the field's three-dimensional world coordinates. This serves as a unified spatiotemporal reference throughout the process. Subsequently, each sub-processing unit acquires a high frame rate video stream of the field at a frame rate of no less than 60 frames per second, identifying the color features and jersey numbers of players, referees, and ball targets on the field, and collecting moving targets from various perspectives. The image centroid pixel coordinates are used to mark the temporal positions of frames lost due to single-view occlusion. Occlusion information is supplemented using target pixel data from the opposite camera position. A pre-constructed 3D mesh mapping model is used to convert pixel coordinates to 3D spatial coordinates. The corresponding frame's global timestamp is bound to generate a continuous, uninterrupted target trajectory file. Simultaneously, each sub-processing unit collects multiple channels of noise-reduced and filtered audio data from the field, independently extracting spectral features to complete whistle matching and recognition. The main scheduling unit performs cross-verification of the audio whistles using at least two valid recognition signals. The duration, interval, and number of whistles transmitted via wireless referee whistle are simultaneously collected. The two types of whistle data are matched according to the competition's standard whistle rules to generate enforcement data labeled with whistle type and global timestamp. The system first compiles a timeline list. Then, the main scheduling unit collects user-input team and player IDs, along with search criteria such as fouls, offsides, and goals. Combining this with a preset backtracking duration ΔT, it defines a dedicated search interval forward from each whistle time stamp. It then filters and extracts the trajectories of all moving targets matching the user IDs within this interval and stores them in a high-speed temporary cache. The system reads the multi-target 3D trajectory files from the cache and compares them with preset coordinate thresholds corresponding to the match's rulings. By comparing the coordinate differences of the multi-target trajectories, it determines the target space interaction state. After completing the match interaction scene matching, it subtracts the backtracking duration from the matched whistle time stamp to obtain the playback start time. Simultaneously, it retrieves the stored recordings from all camera units to obtain multi-channel synchronously positioned match playback videos. Finally, it can also collect data for use with other functions. The system uses three filtering criteria—the subject of the sport, the time period of the match, and the venue area—to define the temporal and spatial range of the trajectory traversal. It identifies the intersection nodes of trajectories where the coordinate differences of multiple targets fall within a threshold. Based on the global frame timestamps bound to the intersection nodes, it obtains the video replay frame positions corresponding to controversial scenes without whistles. The entire solution relies on multi-camera data complementarity under a unified spatiotemporal benchmark, dual-source whistle verification, and three-dimensional trajectory orientation filtering. It completely solves the pain points of traditional manual frame-by-frame video retrieval, such as low efficiency, easy omissions, and inaccurate positioning. It can quickly and accurately locate the replay segments corresponding to the referee's whistle, and also cover the retrieval needs of potential controversial scenes without whistles. It is fully adaptable to the scenarios of officiating review, VAR-assisted judgment, and event technical analysis for various mainstream ball games.

[0043] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0044] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0045] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0046] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory, random access memory, magnetic disks, or optical disks.

[0047] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for rapid retrieval of sports event referee enforcement videos, characterized in that, Includes the following steps: Multiple sets of camera acquisition units, audio acquisition units, sub-processing units and main scheduling units are deployed and a unified global clock is used to collect site dimensions and camera setup parameters, construct a three-dimensional grid coordinate mapping model of the site, and obtain the image-site three-dimensional coordinate mapping relationship; High frame rate video streams are acquired, moving targets on the field are identified and player numbers are extracted, centroid pixel data of target images are acquired, occlusion and missing information are filled in based on multi-view target data, and target trajectory files are obtained by combining the field three-dimensional grid coordinate mapping model. Collect multiple audio data and independently identify whistle sounds, verify the valid identification signals of no less than two channels, collect whistle data transmitted by wireless referee whistle, and obtain a list of law enforcement time points labeled with whistle types based on the two types of whistle data; Collect the user-input entity identifier and judgment behavior retrieval conditions, and backtrack the preset time period point by point according to the law enforcement time point list to extract the motion target trajectory file matching the retrieval conditions within the corresponding time period; According to the competition's judging rules, coordinate threshold comparisons are performed on the three-dimensional trajectories of multiple targets to complete the matching of the competition's interactive scenarios. Based on the preceding moments corresponding to the successfully matched enforcement time points, multiple camera units are simultaneously located and video is stored. By selecting the subject, time period, and location as criteria, limiting the trajectory traversal range, and identifying the intersection nodes of trajectory coordinates, the video playback frame positions corresponding to the no-whistle dispute scene are obtained.

2. The method for rapid retrieval of sports event refereeing videos according to claim 1, characterized in that, The system deploys multiple sets of camera acquisition units, audio acquisition units, sub-processing units, and a main scheduling unit, all operating under a unified global clock. It acquires site dimensions and camera setup parameters, constructs a three-dimensional grid coordinate mapping model of the site, and obtains the image-site three-dimensional coordinate mapping relationship. Specifically: Multiple sets of video and audio acquisition units are deployed around the site to collect local clock data from all hardware units and generate a global clock reference through unified calibration. Collect field size data such as length and width, markings, and goal dimensions; collect camera unit installation height and horizontal angle installation parameter data. Based on the collected site dimensions and camera setup parameters, the site's 3D mesh pre-reconstruction is completed offline, generating a site coordinate reference mesh; Based on the pre-reconstructed mesh, the image pixel coordinates are associated with the site's three-dimensional world coordinates, thus obtaining a fixed mapping relationship between the image and the site's three-dimensional coordinates.

3. The method for rapid retrieval of sports event refereeing videos according to claim 1, characterized in that, The process involves acquiring high frame rate video streams, identifying moving targets on the field and extracting player numbers, acquiring centroid pixel data of target images, supplementing occlusion and missing information based on multi-view target data, and combining this with the field's three-dimensional mesh coordinate mapping model to obtain the target trajectory file. Specifically: Each sub-processing unit acquires a video stream of the field with a frame rate of no less than 60 frames per second, acquires image data of players, referees, and ball targets, identifies target color features, and extracts jersey number character data; Collect image centroid pixel coordinate data of moving targets from various shooting angles, and record the frame time sequence marker data lost due to target occlusion in a single viewpoint; Based on the target pixel data collected from the opposite side of the site, the occluded and missing target data is completed, and the pixel coordinates are transformed by combining the site's three-dimensional grid coordinate mapping model. By binding the converted 3D spatial coordinates to the corresponding frame global timestamp, a continuous target trajectory file storing the 3D coordinates and timestamps is obtained.

4. The method for rapid retrieval of sports event refereeing videos according to claim 1, characterized in that, The process involves collecting multiple audio data streams and independently identifying whistle sounds, verifying at least two valid identification signals, collecting whistle data transmitted via wireless referee whistle, and obtaining a list of enforcement time points labeled with whistle types based on the two types of whistle data. Specifically: Each sub-processing unit collects multi-channel audio data from the site after noise reduction and filtering, extracts audio spectrum feature data, and independently performs whistle feature matching and recognition. The main scheduling unit collects valid whistle recognition data from no less than two sub-processing units and completes cross-validation of the validity of the audio recognition signal. Collect wireless data on the duration, interval, and number of whistles output by the wireless referee whistle, and summarize the two types of whistle data: audio whistle sound and wireless whistle sound. Based on the standard whistle rules for the competition, two types of whistle data are matched to obtain a list of enforcement time points labeled with whistle type and global timestamp.

5. The method for rapid retrieval of sports event refereeing videos according to claim 1, characterized in that, The process involves collecting the user-input entity identifier and penalty behavior retrieval conditions, and then, based on the enforcement time point list, backtracking through each point for a preset duration to extract the motion target trajectory file matching the retrieval conditions within the corresponding time period. The main scheduling unit collects the team and player identification data input by the user, and collects the retrieval conditions data for fouls, offsides, and goals. Collect preset backtracking duration ΔT parameter data, and define the retrieval interval based on the forward offset ΔT of each whistle-blowing timestamp in the law enforcement time point list; Filter the trajectory file storage range based on the start and end global timestamps of the search interval, and extract all moving target trajectory files that match the main body identifier within the interval; The extracted target trajectory files are stored in a temporary cache space for subsequent multi-target 3D trajectory coordinate threshold comparison calculations.

6. The method for rapid retrieval of sports event refereeing videos according to claim 1, characterized in that, The process involves comparing the coordinate thresholds of multiple target 3D trajectories according to the competition's judging rules, matching the interactive scene on the field, and simultaneously locating and storing videos from multiple camera units based on the preceding time point corresponding to the successfully matched enforcement time point. Specifically: Collect the preset coordinate threshold parameter data of the corresponding penalty rules of the event, and read the trajectory file of the X / Y / Z three-dimensional coordinates of multiple targets stored in the cache; Based on the coordinate threshold parameter, a comparison calculation is performed on the coordinate difference of multiple target trajectories, and the spatial interaction status of the targets is determined based on the comparison difference. Based on the interaction status determination result, the field interaction scene matching is completed, and the replay start time is obtained by subtracting the backtracking time from the whistle timestamp of the matched whistle. Based on the start time of the replay, all the video recordings stored in the camera units are retrieved synchronously to obtain the replay video of the event after multi-channel synchronous positioning.

7. The method for rapid retrieval of sports event refereeing videos according to claim 1, characterized in that, The selection criteria, including the subject of data collection, time period, and location area, limit the trajectory traversal range, identify the intersection nodes of trajectory coordinates, and obtain the video playback frame positions corresponding to the no-whistle dispute scene. Specifically: The main scheduling unit collects player ID movement data, first and second half match time data, and midfield and penalty area filtering data. The start and end time intervals for trajectory file reading and the spatial traversal range of the site are defined based on the three types of screening conditions collected. Based on the defined range, traverse all target 3D trajectory data within the interval and identify trajectory intersection nodes where the coordinate difference of multiple targets falls within the threshold. Based on the global frame timestamp bound to the intersection node, the video playback frame position corresponding to the dispute scene without whistle is obtained.

8. A rapid retrieval system for sports event referee enforcement video recordings, characterized in that, include: The camera acquisition unit captures high frame rate video streams from the stadium. The sound pickup and acquisition unit collects multiple audio data streams from the venue. The sub-processing unit completes moving target tracking and audio whistle recognition; Main scheduling unit; A unified global clock was used to construct a three-dimensional grid coordinate mapping model of the site, and a list of law enforcement time points was generated by summarizing two types of whistle data. Trajectories were matched and multiple stored videos were located based on search criteria.

9. The rapid retrieval system for sports event referee video recordings according to claim 8, characterized in that, The system also includes a memory and a processor. The memory contains a program for a rapid retrieval method for sports event refereeing videos. When the processor executes the program for a rapid retrieval method for sports event refereeing videos, it performs the following steps: Multiple sets of camera acquisition units, audio acquisition units, sub-processing units and main scheduling units are deployed and a unified global clock is used to collect site dimensions and camera setup parameters, construct a three-dimensional grid coordinate mapping model of the site, and obtain the image-site three-dimensional coordinate mapping relationship; High frame rate video streams are acquired, moving targets on the field are identified and player numbers are extracted, centroid pixel data of target images are acquired, occlusion and missing information are filled in based on multi-view target data, and target trajectory files are obtained by combining the field three-dimensional grid coordinate mapping model. Collect multiple audio data and independently identify whistle sounds, verify the valid identification signals of no less than two channels, collect whistle data transmitted by wireless referee whistle, and obtain a list of law enforcement time points labeled with whistle types based on the two types of whistle data; Collect the user-input entity identifier and judgment behavior retrieval conditions, and backtrack the preset time period point by point according to the law enforcement time point list to extract the motion target trajectory file matching the retrieval conditions within the corresponding time period; According to the competition's judging rules, coordinate threshold comparisons are performed on the three-dimensional trajectories of multiple targets to complete the matching of the competition's interactive scenarios. Based on the preceding moments corresponding to the successfully matched enforcement time points, multiple camera units are simultaneously located and video is stored. By selecting the subject, time period, and location as criteria, limiting the trajectory traversal range, and identifying the intersection nodes of trajectory coordinates, the video playback frame positions corresponding to the no-whistle dispute scene are obtained.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a program for a rapid retrieval method for sports event referee enforcement videos. When the program is executed by a processor, it implements the steps of the rapid retrieval method for sports event referee enforcement videos as described in any one of claims 1 to 7.

Images