Stage multi-camera cooperative tracking method, device and equipment and storage medium

Abstract

The invention discloses a stage multi-camera cooperative tracking method, device and equipment and a storage medium. The method comprises the steps of obtaining stage three-dimensional model data, performing gridding processing through a particle filtering algorithm, presetting a boundary buffer distance, and forming an initial coverage network; and predicting a performer trajectory state vector through Kalman filtering, and performing priority division and camera allocation on an overlapping region to obtain an adjusted overlay network. Monitoring position data, triggering a signal during cross-region boundary and transmitting a fusion data packet, updating a weight by a target camera based on particle filtering to realize data fusion to obtain a first trajectory deduction result, backtracking and optimizing to obtain a second trajectory deduction result when an error exceeds a threshold value, activating multi-target separation when a speed exceeds a threshold value, and performing multi-target separation; and a continuous tracking sequence is obtained through a Kalman filtering smoothing matrix. According to the method, the problems of stage multi-camera cross-region connection fault, multi-target confusion and insufficient trajectory precision are solved, the tracking precision and continuity are improved, and seamless cooperative tracking is realized.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, device, and storage medium for collaborative tracking of multiple cameras on a stage. Background Technology

[0002] Multi-camera collaborative tracking technology is a core tool in modern stage performance for enhancing visual presentation, live recording, and post-production effects. By enabling multiple devices to work together, it allows for real-time positioning and dynamic capture of performers, accurately recording every moment of the performance on stage. This technology not only directly impacts the audience's viewing experience but also plays a crucial supporting role in the dissemination and secondary creation of performance content, making it an indispensable and vital component of the stage technology system.

[0003] Existing multi-camera tracking methods for stages have significant technical limitations, generally lacking efficient spatial coordination and overall collaboration mechanisms. When faced with situations such as rapid performer movement, complex stage scenes, or multi-target interactions, the tracking process is prone to problems such as interruptions and deviations. The mode of relying solely on a single camera working independently is difficult to meet the seamless connection requirements under multi-area and multi-angle coverage, and cannot adapt to dynamic and ever-changing stage performance scenarios.

[0004] The core challenges of this technology lie in two aspects: constructing a stage space coverage network and facilitating cross-device information exchange. Firstly, the division of stage space areas must balance seamless monitoring with rational resource allocation; improper division can easily lead to blind spots or wasted resources. Secondly, the lack of an effective real-time information sharing mechanism between cameras means that when performers move across areas, subsequent cameras cannot acquire crucial data such as trajectory and speed in a timely manner, resulting in tracking delays and image jitter. Therefore, constructing a seamless camera coverage network and achieving real-time information sharing among multiple devices to achieve continuous and stable performance tracking has become a critical issue that urgently needs to be addressed in this field. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and storage medium for collaborative tracking of multiple cameras on a stage, which specifically addresses the problems of cross-regional connection gaps, multi-target confusion, and insufficient trajectory accuracy of multiple cameras on a stage, thereby improving tracking accuracy and continuity and achieving seamless collaborative tracking.

[0006] A first aspect of the present invention provides a stage multi-camera collaborative tracking method, comprising:

[0007] Initiate multi-camera collaborative tracking on stage, extract the boundary lines of adjacent areas of the camera coverage network and the corresponding boundary buffer distances. If the distance between the performer's current position and the boundary line of the adjacent area is less than the boundary buffer distance, transmit a fused data packet containing the trajectory state vector, current position and historical motion trajectory sequence to the target camera, and trigger an interactive start signal.

[0008] The target camera responds to the interactive start signal and receives the fusion data packet, updates the particle resampling mechanism weights of the preset particle filter algorithm, and performs fusion processing to obtain the first trajectory projection result of the performer.

[0009] Construct the trajectory covariance matrix corresponding to the first trajectory inference result and calculate the continuity error. If the continuity error exceeds the preset error threshold, backtrack and call the fusion data packet to readjust the particle resampling mechanism weights, and perform a second fusion process to obtain the second trajectory inference result.

[0010] Based on the second trajectory deduction result, the real-time velocity information of the performers is obtained, and multi-target separation processing is performed to determine the independent trajectory state vector corresponding to each performer's trajectory.

[0011] The traversal time of the performer crossing the boundary of the area is calculated based on the independent trajectory state vector. The filtering parameters of the preset Kalman filter algorithm are adjusted in combination with the traversal time to optimize the smoothing strategy of the trajectory covariance matrix. Then, the trajectory covariance matrix is ​​smoothed to obtain a continuous tracking sequence.

[0012] A second aspect of the present invention provides a stage multi-camera collaborative tracking device, comprising:

[0013] The boundary triggering and transmission module is configured to: initiate multi-camera collaborative tracking on stage, extract the boundary lines of adjacent areas of the camera coverage network and the corresponding boundary buffer distances; if the distance between the performer's current position and the boundary line of the adjacent area is less than the boundary buffer distance, transmit a fused data packet containing the trajectory state vector, current position and historical motion trajectory sequence to the target camera, and trigger an interactive start signal.

[0014] The first trajectory deduction module is configured to: respond to the interactive start signal and receive the fusion data packet through the target camera, update the particle resampling mechanism weight of the preset particle filter algorithm, and perform fusion processing to obtain the first trajectory deduction result of the performer;

[0015] The second trajectory deduction module is configured to: construct the trajectory covariance matrix corresponding to the first trajectory deduction result and calculate the continuity error; if the continuity error exceeds a preset error threshold, backtrack and call the fusion data packet to readjust the particle resampling mechanism weights, and perform secondary fusion processing to obtain the second trajectory deduction result.

[0016] The independent trajectory configuration module is configured to: obtain the real-time velocity information of the performers based on the second trajectory deduction result, and perform multi-target separation processing to determine the independent trajectory state vector corresponding to each performer's trajectory;

[0017] Tracking sequence generation module: Calculates the crossing time of the performer across the boundary of the area based on the independent trajectory state vector, adjusts the filtering parameters of the preset Kalman filter algorithm based on the crossing time, optimizes the smoothing strategy of the trajectory covariance matrix, and then smooths the trajectory covariance matrix to obtain a continuous tracking sequence.

[0018] A third aspect of the present invention provides an electronic device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the stage multi-camera collaborative tracking method described in the first aspect above.

[0019] A fourth aspect of the present invention provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform a stage multi-camera collaborative tracking method as described in the first aspect.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] (1) To achieve precise division of the stage area and initial coverage network construction, relying on the meshing processing of the particle filter algorithm and the preset boundary buffer distance, the defects of the traditional single camera in capturing the dynamic trajectory of the performer are compensated, and a stable and reliable regional foundation is laid for subsequent tracking.

[0022] (2) Based on the Kalman filter algorithm, the trajectory state vector prediction and camera priority allocation in the overlapping area are optimized to improve the camera resource scheduling strategy, effectively improve the problem of unsmooth and distorted angle switching, and enhance the accuracy and rationality of camera adaptation during the tracking process.

[0023] (3) Establish a data interaction mechanism triggered by the regional boundary. By determining the location distance, the data packet transmission and interaction signal are triggered to ensure the real-time performance and integrity of the data when tracking across regions, solve the pain point of poor frequency adaptation, and strengthen the linkage of multi-camera collaboration.

[0024] (4) By using the particle resampling mechanism of the particle filter algorithm to update the weights, the trajectory of the performer under the fusion of multi-camera data is deduced, the accuracy of trajectory capture in complex scenes is improved, and the ability to continuously perceive the complex motion trajectory of multiple performers is further enhanced.

[0025] (5) Construct an error backtracking adjustment mechanism. Through the calculation of continuous error and covariance matrix and threshold determination, the trajectory extrapolation results are dynamically corrected to ensure the low error characteristics of the tracking results and take into account the continuity and stability of the detection.

[0026] (6) Activate multi-target separation processing for real-time speed over-limit scenarios, determine independent trajectory state vectors, improve the adaptability of the method to complex motion scenarios with multiple performers, and avoid perception bias caused by multi-target interference.

[0027] (7) Based on the Kalman filter algorithm, the trajectory covariance matrix is ​​smoothed to generate an optimized continuous tracking sequence, which completely solves the angle switching distortion problem and achieves a comprehensive and coordinated improvement in detection continuity, adaptability and stability. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating a stage multi-camera collaborative tracking method provided in an embodiment of the present invention;

[0029] Figure 2 This is a flowchart illustrating the method for obtaining camera coverage network provided in an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of the structure of a stage multi-camera collaborative tracking device provided in an embodiment of the present invention;

[0031] Figure 4 This is another structural schematic diagram of a stage multi-camera collaborative tracking device provided in an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of the structure of a device provided in an embodiment of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Example 1

[0035] Reference Figure 1 This invention provides a stage multi-camera collaborative tracking method, including steps S11 to S15.

[0036] S11: Start stage multi-camera collaborative tracking, extract the boundary lines of adjacent areas of the camera coverage network and the corresponding boundary buffer distances. If the distance between the performer's current position and the boundary line of the adjacent area is less than the boundary buffer distance, transmit a fused data packet containing the trajectory state vector, current position and historical motion trajectory sequence to the target camera, and trigger the interaction start signal.

[0037] S12: The target camera responds to the interactive start signal and receives the fusion data packet, updates the particle resampling mechanism weights of the preset particle filter algorithm, and performs fusion processing to obtain the first trajectory deduction result of the performer.

[0038] S13: Construct the trajectory covariance matrix corresponding to the first trajectory deduction result and calculate the continuity error. If the continuity error exceeds the preset error threshold, backtrack and call the fusion data packet to readjust the particle resampling mechanism weights, and perform secondary fusion processing to obtain the second trajectory deduction result.

[0039] S14: Obtain the real-time velocity information of the performers based on the second trajectory deduction result, and perform multi-target separation processing to determine the independent trajectory state vector corresponding to each performer's trajectory;

[0040] S15: Calculate the crossing time of the performer across the boundary of the area based on the independent trajectory state vector, adjust the filtering parameters of the preset Kalman filter algorithm in combination with the crossing time, optimize the smoothing strategy of the trajectory covariance matrix, and then smooth the trajectory covariance matrix to obtain a continuous tracking sequence.

[0041] This embodiment uses a large-scale variety show stage as the application scenario. The stage's physical dimensions are 20m × 15m. A central 8m × 8m core performance area, two 6m × 8m interactive areas on either side, and a 20m × 4m prop area at the rear are designated. Twelve high-definition network cameras (model: IPC-9300, 120° field of view, 4K resolution, 30fps) are deployed on the stage's trusses and ceiling. Seamless collaborative tracking by multiple cameras is achieved using computer vision technology. The main process steps are as follows:

[0042] In step S11, stage multi-camera collaborative tracking is initiated, and the boundary lines of adjacent areas of the camera coverage network and the corresponding boundary buffer distances are extracted. If the distance between the performer's current position and the boundary line of the adjacent area is less than the boundary buffer distance, a fused data packet containing the trajectory state vector, current position, and historical motion trajectory sequence is transmitted to the target camera, and an interactive start signal is triggered.

[0043] In one implementation, this embodiment continues the aforementioned application scenario for large-scale variety show stages. Previously, the construction of the camera coverage network (75 grids G1-G75) and camera allocation were completed through a pre-processing step. Four high-definition cameras (C1-C4) were allocated to grids G15-G20 in the core performance area. All 12 IPC-9300 high-definition network cameras deployed on the stage were connected to the computer vision collaborative tracking device. After starting multi-camera collaborative tracking on the stage, the visual image data of the performers was acquired at a frame rate of 30fps through camera C3 corresponding to grid area G18 in the camera coverage network, relying on the real-time capture module of the computer vision device. After image coordinate transformation and feature point matching processing, the performer's current position data was output in real time (e.g., at t=120s, the position coordinates are X=11.8m, Y=8.2m).

[0044] Simultaneously, from the spatial database of the camera coverage network, the boundary line information of the adjacent region G19 corresponding to the G18 grid (boundary line coordinates are X=12m, Y∈[7-9m]), and the boundary buffer distance of 0.8m corresponding to the core performance area grid, are extracted to provide a computer vision spatial reference for cross-region determination. Using the Euclidean distance calculation method commonly used in computer vision, the straight-line distance between the performer's current position coordinates and the boundary line of the adjacent region is calculated based on the coordinate parameters: using the formula d=|X Performer - X Boundary Line| (since the boundary line is a straight line perpendicular to the X-axis), substituting the data, we get d=|11.8-12|=0.2m. This distance is then compared with the preset 0.8m boundary buffer distance, and it is determined that d < 0.8m, satisfying the cross-region data transmission trigger condition.

[0045] Based on the camera allocation mapping table stored in the camera coverage network, the target camera pre-assigned to the adjacent area G19 is identified as C4 (high-priority area dual-camera redundant coverage configuration). The computer vision device automatically marks C4 as a device to be responded to, completing the rapid localization of the target camera. The system integrates the current performer's trajectory state vector ([11.8,8.2,0.5,0.3], containing position and velocity parameters), real-time position data, and the historical motion trajectory sequence of the past 10 seconds (300 sets of time-coordinate feature points, optimized by computer vision trajectory fitting). Following the computer vision data interaction specifications, a standardized JSON format fusion data package is generated, containing three core fields: "trajectory vector," "real-time position," and "historical trajectory," with an overall size of approximately 50KB.

[0046] The fused data packets are transmitted to the target camera C4 via Gigabit Ethernet. Data integrity is ensured through a CRC32 checksum mechanism of the computer vision device during transmission, with an actual transmission latency of ≤40ms (matching the camera's 30fps frame rate, covering the entire data reception and preprocessing process without timing conflicts), meeting real-time collaboration requirements. A synchronous trigger signal for the computer vision device is activated. This signal is encapsulated using the TCP / IP protocol, with the instruction code 0x0001, notifying the target camera C4 to initiate a tracking state. Upon receiving the signal, C4 immediately activates its own computer vision preprocessing module, initiating data reception buffering and feature point preloading processes. This prepares the device for seamless tracking of the performer upon entering the grid, ensuring uninterrupted visual capture during cross-region switching.

[0047] In step S12, the target camera responds to the interactive start signal and receives the fusion data packet, and updates the particle resampling mechanism weights of the preset particle filter algorithm, and the fusion process obtains the first trajectory projection result of the performer.

[0048] In one implementation, this embodiment continues the application scenario of a large-scale variety show stage. The physical dimensions of the stage are 20m × 15m. The camera coverage network has been constructed through a pre-process, including 75 grid areas (numbered G1-G75). Twelve high-definition network cameras (model: IPC-9300, 120° shooting angle, 4K resolution, 30fps) are deployed on the stage. The core relies on computer vision target tracking and multi-source data fusion technology to achieve accurate estimation of the performer's trajectory. When the performer is about to cross the boundary between grids G18 and G19, camera C3, which was previously responsible for grid G18, has triggered an interaction start signal and transmitted a standardized fusion data packet. Target camera C4, as the pre-assigned device for grid G19, responds first to the interaction start signal, completes hardware initialization and software readiness within 45ms, and enters the tracking ready state.

[0049] Subsequently, the standardized fusion data packet transmitted by camera C3 is received via gigabit Ethernet. The fusion data packet is encapsulated in JSON format and contains the trajectory state vector, the current position, and the historical motion trajectory sequence of the past 10 seconds (a total of 300 sets of time-position coordinates). After receiving, the integrity of the fusion data packet is verified by the CRC32 check algorithm commonly used in the field of computer vision to ensure that there is no data loss or transmission error. After the verification is passed, the historical motion trajectory sequence and trajectory state vector are accurately extracted from the fusion data packet (e.g., the vector at t=360.2s is [8.85,7.02,0.3,0.45]).

[0050] Combining the performer's current position data captured in real-time at 30fps by the target camera C4 (e.g., the coordinates captured at t=360.3s are [8.88, 7.05]), a particle filter algorithm widely used in target tracking in computer vision is loaded. This algorithm presets the number of particles to 600 and the resampling threshold to 0.25. The particle resampling mechanism weight is the core of the particle filter algorithm, used to measure the matching degree between each particle (sample point simulating the performer's motion state) and the actual motion trajectory. The higher the weight, the closer the motion state corresponding to that particle is to the real situation. Based on the analysis of the extracted historical motion trajectory sequence, the performer's motion pattern is analyzed (e.g., the core performance area is mostly uniform linear motion with small fluctuations). Combining the positional deviation between the current position data captured by the C4 camera and the historical data, the particle resampling mechanism weight is dynamically updated: the weight of effective data with high consistency with the historical trajectory trend is adjusted to 0.88, and the weight of interference data with instantaneous fluctuations (e.g., instantaneous coordinate shifts caused by light reflections) is adjusted to 0.12. By allocating weights, the contribution of effective data to trajectory calculation is strengthened, and the influence of interference data is weakened.

[0051] Based on the updated particle filter algorithm, multi-source data fusion technology in computer vision is used to fuse the fused data packet transmitted by camera C3 with the current position data collected by camera C4 itself. The weight allocation is based on the following: the historical trajectory data of the preceding camera C3 has higher reference value and is assigned a weight of 0.45; the data of the current camera C4 has stronger timeliness and is assigned a weight of 0.55. The difference in accuracy and timeliness between the two types of data sources is balanced by a weighted average method, generating fused trajectory data containing 180 sets of fused time-position coordinates, effectively integrating the complementary advantages of cross-device data.

[0052] Based on the fused trajectory data, the least squares method, commonly used in computer vision, is employed to fit and calculate the performer's trajectory. A trajectory multinomial model is constructed, with time t as the independent variable and position coordinates (X, Y) as the dependent variable. By minimizing the sum of squared errors between the fitted value and the actual fused data, the following model is obtained. , The trajectory equation was derived, ultimately yielding the performer's first trajectory projection result after multi-camera data fusion. Verification showed that the positional deviation of this first trajectory projection result was ≤0.09m, fully meeting the accuracy requirements of multi-camera collaborative tracking on stage, providing reliable basic data support for subsequent trajectory error verification and optimization.

[0053] In step S13, the trajectory covariance matrix corresponding to the first trajectory inference result is constructed and the continuity error is calculated. If the continuity error exceeds the preset error threshold, the fusion data packet is back-called to readjust the particle resampling mechanism weights, and a second fusion process is performed to obtain the second trajectory inference result.

[0054] In one implementation, this embodiment continues the computer vision multi-camera collaborative tracking scenario of a large-scale variety show stage. The stage's physical dimensions are 20m × 15m. The camera coverage network has been constructed through a pre-process, dividing the stage into 75 grid areas (G1-G75) and deploying 12 high-definition network cameras. Currently, the first trajectory projection result of the performers has been obtained through multi-camera data fusion (the trajectory is expressed as a polynomial function). , The trajectory deviation is ≤0.1m.

[0055] Based on the first trajectory inference result after fusion of multi-camera data from computer vision, the coordinate data of trajectory points in 30 consecutive frames (corresponding to 1 second, with a camera frame rate of 30fps) are first extracted. The deviation value of trajectory points between adjacent time points is calculated: taking t=200s to t=201s as an example, the coordinates of the trajectory point at t=200.00s are (19.2m, 14.0m), and the coordinates at t=200.033s are (19.25m, 14.08m), with deviation values ​​ΔX=0.05m and ΔY=0.08m. After calculating the deviation values ​​of trajectory points in all adjacent frames, the continuity error is determined to be 0.12m using the root mean square error formula. Simultaneously, based on the statistical analysis method of trajectory data in computer vision, the trajectory covariance matrix corresponding to the first trajectory inference result is constructed (the trajectory covariance matrix is ​​a 2×2 matrix [[σx] 2 ,σxy],[σxy,σy 2 ]], where σx 2 σy 2 σxy represents the variance of the trajectory data in the X and Y directions, reflecting the degree of dispersion, and σxy represents the covariance, reflecting the correlation between the X and Y direction data. The matrix is ​​[[0.025,0.012],[0.012,0.031]], which provides a basis for the reliability of the data at the computer vision level for subsequent weight adjustment.

[0056] Considering the accuracy requirements of live stage recording (allowable deviation ≤ 0.1m), a continuity error threshold of 0.1m is set to ensure that the tracking effect meets the post-production requirements. The calculated continuity error (0.12m) is compared with this threshold. If the continuity error exceeds the preset threshold, the backtracking mechanism is triggered. The standardized fusion data packet previously received by the target camera is called (corresponding to the fusion data packet transmitted in step S11, which contains trajectory state vector, current position, and historical motion trajectory sequence of the past 10 seconds, including 300 sets of coordinate data and vector parameters). The historical motion trajectory sequence, trajectory state vector (such as [19.2,14.0,0.5,0.3]) and current position data in the fusion data packet are extracted using computer vision data parsing technology to ensure the integrity and usability of the backtracking data.

[0057] The particle filter algorithm, commonly used in computer vision for target tracking optimization, is loaded. Based on preset parameters (initial particle count 600, resampling threshold 0.25), and combined with historical trajectory data, trajectory state vectors, and current position data extracted from backtracking, and referring to the trajectory covariance matrix corresponding to the first trajectory derivation result reflecting the dispersion of trajectory data (0.025 in the X direction and 0.031 in the Y direction), the particle resampling mechanism weights are readjusted: the weight of effective trajectory data in the X direction is increased from 0.85 to 0.9, and the weight of effective trajectory data in the Y direction is increased from 0.88 to 0.92, while the particle weights in areas with high dispersion (such as reducing the weight of some abnormal trajectory points in the Y direction to 0.1) are simultaneously reduced. This results in a readjusted particle filter algorithm that is more adapted to the distribution characteristics of the current trajectory data.

[0058] Based on a readjusted particle filter algorithm, the historical data extracted from the backtracking is re-fused with the original multi-camera fusion data (current position data collected by cameras C3 and C4). A weighted fusion strategy from computer vision is used to balance the weight ratio of different data sources (backtracking data weight 0.5, original fused data weight 0.5). Then, the least squares method is used to complete the trajectory fitting calculation, generating a second trajectory inference result with optimized accuracy. Finally, a second trajectory inference result with low error is obtained, and its trajectory equation is: , The continuity error was reduced to 0.07m and the trajectory deviation was ≤0.05m, meeting the high-precision requirements of multi-camera collaborative tracking in computer vision.

[0059] In step S14, the real-time velocity information of the performers is obtained based on the second trajectory deduction result, and multi-target separation processing is performed to determine the independent trajectory state vector corresponding to each performer's trajectory.

[0060] In one embodiment, this example continues the application scenario of a large-scale variety show stage. The physical size of the stage is 20m×15m. The initial coverage network construction, the adjusted coverage network optimization, the cross-regional fusion data packet transmission, and the optimization of the first trajectory inference results have been completed through the steps described above. The current computer vision multi-camera collaborative tracking device has obtained the second trajectory inference results with low error (trajectory continuity error ≤0.08m). The stage contains 3 performers (1 core singer and 2 backup dancers), and 12 deployed high-definition network cameras continuously collect data. Based on the target velocity estimation method in computer vision technology, and using the second trajectory derivation results as a basis, the trajectory position data of performers at consecutive time points are extracted. The trajectory position coordinates of adjacent time points t1=300.0s and t2=300.033s (corresponding to a camera frame rate of 30fps, and a time interval Δt=1 / 30s≈0.033s) are selected: the core singer's coordinates at t1 are (12.5m, 8.3m), and at t2 are (12.65m, 8.42m); the backup dancer A's coordinates at t1 are (11.2m, 7.6m), and at t2 are (11.31m, 7.72m); the backup dancer B's coordinates at t1 are (13.8m, 9.0m), and at t2 are (13.92m, 9.12m). By calculating the ratio of the difference in trajectory position between adjacent moments to the time interval, the real-time speed information of each performer is obtained: the position difference of the core singer is ΔX=0.15m, ΔY=0.12m, and the combined speed v≈5.82m / s (a reasonable range for rapid stage movement, approximately 16.2km / h); the combined speed of backup dancer A is approximately 3.8m / s, and the combined speed of backup dancer B is approximately 3.6m / s. This speed threshold was derived from statistical analysis of stage performance data from 100 variety shows, concerts, and other events, covering different performer types such as singers and dancers. In fast-moving scenes (such as the climax of a dance), the maximum speed of performers does not exceed 5.5m / s, so a threshold of 5m / s is set to accurately trigger multi-target separation. Comparing the calculated real-time speed information of each performer with the preset speed threshold, the core singer's speed exceeds 5m / s, and backup dancers A and B are close to the threshold, indicating a risk of overlapping and confusing multi-target trajectories, requiring the initiation of multi-target separation processing.Then, the multi-target separation processing based on the particle filter algorithm commonly used in the field of computer vision multi-target tracking is activated. Relying on the particle filter algorithm readjusted in step S15 (effective data weight in the X direction is 0.9, and in the Y direction it is 0.92), based on the complexity of the multi-target scene (the trajectory overlap rate of 3 performers is 35%), 100 more particles are added to the initial 600 particles so that the particles cover all overlapping trajectory areas, improving the splitting accuracy. Experimental verification shows that 700 particles can reduce the overlapping trajectory splitting error by more than 25%. The resampling threshold is set to 0.2, and the observation noise covariance is adjusted to 0.015. The overlapping trajectories are then split and identified. First, using real-time image frames captured by a camera, the bounding boxes of three performers were located using YOLO object detection technology in computer vision (confidence threshold 0.85). The overlapping trajectory area was determined to be concentrated in the G18-G20 grid of the core performance area (coordinate range X∈[8-12m], Y∈[6-10m]), with a pixel coverage of 35%. Then, a particle filtering algorithm was used to assign an independent particle set to each performer (250 for the core singer and 225 for each backup dancer). K-Means clustering was performed based on the position, velocity, and direction of motion of the particles, with the number of clusters set to 3. The initial cluster center was set based on the center coordinates of the bounding box obtained from object detection. The clustering iteration termination condition was "the change in the sum of squared errors within the cluster ≤ 0.001". The particles in the overlapping area were divided into 3 independent clusters, each corresponding to the trajectory particles of one performer, thus achieving the initial splitting of the overlapping trajectory. Combining SIFT feature matching technology from computer vision, the historical motion trajectory features of each performer were extracted: the core singer's past trajectories were mostly concentrated in the core performance area, with smooth motion trajectories and low turning frequency, with an average turning angle ≤30°; the historical trajectory of backup dancer A mostly moved around the left side of the core singer, with a moderate turning frequency; the trajectory of backup dancer B mostly moved around the right side of the core singer, with a larger turning angle (average ≥45°). Simultaneously, based on real-time image frames, the real-time position distribution of the three performers was extracted (core singer located in the center of the overlapping area (12.6m, 8.4m), backup dancer A located on the left (11.5m, 7.9m), backup dancer B located on the right (13.7m, 8.9m)) and differences in motion direction (core singer facing northeast, backup dancer A facing southeast, backup dancer B facing northwest). Through feature fusion and difference comparison, the motion trajectories of different performers were further distinguished, eliminating interference caused by trajectory intersections.For each performer's independent trajectory, a corresponding trajectory state vector was constructed based on the state vector construction method in computer vision (vector dimension [X, Y, Vx, Vy, α], where X and Y are the real-time position coordinates, Vx and Vy are the velocity components in the X and Y directions, and α is the motion direction angle, increasing clockwise from 0° east of the stage). The resulting independent trajectory state vectors for the core singer are [12.6, 8.4, 5.2, 2.1, 45°], for backup dancer A are [11.5, 7.9, 3.4, 1.8, 135°], and for backup dancer B are [13.7, 8.9, 3.2, 2.0, 315°]. These independent trajectory state vectors provide accurate data support for subsequent cross-regional time calculations and trajectory smoothing.

[0061] In step S15, the crossing time of the performer across the boundary of the area is calculated based on the independent trajectory state vector. The filtering parameters of the preset Kalman filter algorithm are adjusted in combination with the crossing time to optimize the smoothing strategy of the trajectory covariance matrix. Then, the trajectory covariance matrix is ​​smoothed to obtain a continuous tracking sequence.

[0062] In one embodiment, this example continues the application scenario of a large-scale variety show stage. The physical dimensions of the stage are 20m × 15m. The camera coverage network has been constructed through the pre-process (dividing into 75 grid areas G1-G75), and multi-target separation processing has been completed through the previous steps. The computer vision multi-camera collaborative tracking device has acquired the independent trajectory state vectors (vector dimensions are [X, Y, V)) corresponding to the three performers (core singer, backup dancer A, backup dancer B). x V y The 12 high-definition network cameras deployed continuously collect real-time data.

[0063] Based on state vector analysis technology in computer vision, key parameters are extracted from the independent trajectory state vectors of each performer: the current position of the core singer is (14.2m, 8.6m), the movement speed is 4.5m / s, and the movement direction angle is 50° (northeast direction); the current position of backup dancer A is (12.5m, 7.8m), the movement speed is 3.5m / s, and the movement direction angle is 60°; the current position of backup dancer B is (15.8m, 9.3m), the movement speed is 3.2m / s, and the movement direction angle is 40°.

[0064] Based on the boundary positions of each performer's current grid area within the camera coverage network—the core singer is located in grid G20 (coordinate range X∈[12-14m], Y∈[8-10m]), and the boundary position of its adjacent grid G21 is X=14m (Y∈[8-10m]); backup dancer A is located in grid G19, and the boundary position of its adjacent grid G20 is X=12m (Y∈[7-9m]); backup dancer B is located in grid G21, and the boundary position of its adjacent grid G22 is X=16m (Y∈[9-11m]), using the spatial distance measurement method in computer vision (Euclidean distance formula), the straight-line distances of each performer to their corresponding boundaries are calculated: the straight-line distance d≈0.45m for the core singer (the midpoint coordinates of the boundary are (14m, 9m)), the straight-line distance for backup dancer A is approximately 0.52m, and the straight-line distance for backup dancer B is approximately 0.28m.

[0065] Based on the measured distance and the movement speed of each performer, the crossing time across the boundary of the area was calculated: the core singer's crossing time t = 0.45 / 4.5 = 0.1s, backup dancer A's crossing time was approximately 0.149s, and backup dancer B's crossing time was approximately 0.088s. Simultaneously, based on a computer vision-based direction determination algorithm, combining the movement direction angle and boundary orientation (e.g., G20 grid X = 14m is an eastward boundary), it was determined whether each performer's movement direction was towards the boundary: the core singer's movement direction angle of 50° included an eastward component, backup dancer A's direction angle of 60° included an eastward component, and backup dancer B's direction angle of 40° included an eastward component; all were determined to be towards the boundary, and the crossing times of these three were retained as valid crossing times, while invalid data without boundary orientation were discarded.

[0066] The preset Kalman filter algorithm is invoked, and the filter parameters are adjusted in a targeted manner based on the effective travel time mentioned above. The filter parameters include the process noise covariance and the observation noise covariance: the state transition matrix F of the Kalman filter is adjusted to [[1,0,Δt,0],[0,1,0,Δt],[0,0,1,0],[0,0,0,1]] (Δt is set to 0.033s, matching the camera's 30fps frame rate), the process noise covariance Q is optimized to diag ([0.008,0.008,0.0008,0.0008]), and the observation noise covariance R is adjusted to 0.015. Compared with the initial Q value (diag ([0.01,0.01,0.001,0.001])), the noise interference during cross-region switching is reduced, and the trajectory smoothness is improved.

[0067] The smoothing strategy for optimizing the trajectory covariance matrix based on the adjusted filtering parameters is as follows: a low-intensity smoothing strategy is adopted for short effective crossing times (≤0.1s) to reduce trajectory lag; and a high-intensity smoothing strategy is adopted for long effective crossing times (>0.1s) to enhance noise suppression. Following the optimized smoothing strategy, the covariance matrices of the independent trajectories of each performer (the covariance matrix for the core singer is [[0.012,0.006],[0.006,0.018]], for backup dancer A it is [[0.01,0.005],[0.005,0.015]], and for backup dancer B it is [[0.009,0.004],[0.004,0.013]]) are smoothed. A matrix iterative update algorithm is used to suppress the discrete fluctuations in the trajectory data. After smoothing, the covariance matrix of the core singer is optimized to [[0.007,0.003],[0.003,0.011]], and the matrix dispersion of backup dancers A and B is also significantly reduced, improving the stability of the trajectory data by more than 40%.

[0068] Finally, the smoothed trajectory data (containing 600 sets of time-position coordinates for each of the three performers) and their corresponding independent trajectory state vector information were integrated. Using computer vision trajectory sequence integration technology, the data was structured and arranged according to timestamp order to generate a seamless, low-bias, optimized continuous tracking sequence. This sequence is stored in CSV format and includes key parameters such as timestamps, performer identifiers, position coordinates, speed, and orientation angles. Verification showed that the tracking sequence maintains 99.8% continuity, with a trajectory deviation of ≤0.05m, enabling seamless cross-regional collaborative tracking by multiple cameras and meeting the high-precision requirements of live stage broadcasts, recordings, and post-production.

[0069] Reference Figure 2 Before initiating multi-camera collaborative tracking on the stage, this method also includes steps S21 to S22.

[0070] S21: Obtain the 3D model data of the stage space, perform meshing processing and set boundary buffer distances to obtain the area division results.

[0071] S22: Based on the region division results, the trajectory state vector of the performer is predicted by a preset Kalman filter algorithm, and the overlapping coverage area of ​​the trajectory state vector is divided into priority levels and camera configurations to obtain a camera coverage network.

[0072] In step S21, the three-dimensional model data of the stage space is acquired, and meshing is performed and boundary buffer distances are set to obtain the region division results.

[0073] In one embodiment, this embodiment is based on computer vision technology and takes a large-scale variety show stage as the application scenario to provide basic support for multi-camera collaborative tracking. The physical size of the stage is 20m×15m, with a core performance area of ​​8m×8m in the center, interactive areas of 6m×8m on both sides, and a prop area of ​​20m×4m at the rear. A total of 12 high-definition network cameras (model: IPC-9300, shooting angle 120°, resolution 4K, frame rate 30fps) are deployed on the trusses around the stage and on the top.

[0074] The RIEGL VZ-6000 laser scanner was used to scan the entire stage space. The scanner has a scanning accuracy of ±1mm and a scanning frequency of 1 million dots / second. The scanning range covers the stage floor, platform, all corners and the surrounding 0.5m area, ensuring no blind spots in visual capture. The 1.5 million 3D coordinate point cloud data generated by scanning were preprocessed using the PCL library (Point Cloud Library, an open-source point cloud processing library with core functions such as data denoising, registration, and simplification), which is commonly used in the field of computer vision. The specific process is as follows: denoising is performed using the Statistical OutlierRemoval filter (with 50 neighboring points and a standard deviation threshold of 1.0), point cloud registration is completed using the Iterative ClosestPoint algorithm, and simplification is performed using the Voxel Grid filter (voxel size 0.01m×0.01m×0.01m). Redundant points and abnormal interference points are removed, and finally, complete 3D stage model data is obtained and stored in PLY format, providing a high-quality data foundation for subsequent spatial analysis in computer vision technology.

[0075] This paper utilizes a particle filter algorithm, widely used in target tracking in computer vision. The particle filter algorithm is a probabilistic filtering method that simulates the target state distribution using a large number of particles. In this scheme, the number of particles is set to 600, the resampling threshold to 0.25, the state transition matrix to [[1,0,0],[0,1,0],[0,0,1]], and the observation noise covariance to 0.01. Considering the physical dimensions of the stage (20m × 15m) and the deployment range of 12 cameras, a grid granularity of 2m × 2m is set. The algorithm is used to spatially discretize the stage's 3D model data, uniformly dividing the entire stage space into 75 grid regions. Each grid region is assigned an independent number (G1-G75), achieving accurate segmentation of the stage space using computer vision.

[0076] Based on the gridding process, a unique identifier is assigned to each grid region. Simultaneously, using computer vision-based spatial positioning technology, the corresponding actual stage location coordinates and coverage area are linked. For example, grid G18 corresponds to coordinates X∈[8-10m], Y∈[6-8m], and covers an area of ​​4m². 2 This clarifies that the grid belongs to the core performance area; the grid G5 corresponds to the coordinate range X∈[0-2m], Y∈[6-8m], which belongs to the interactive area, forming a complete "grid number-actual location-coverage range" mapping table, providing a clear spatial reference for the subsequent camera allocation and trajectory tracking of the computer vision device.

[0077] Based on the camera's 120° shooting angle, the performer's maximum movement range (3m / s), and the cross-area tracking response requirements (response latency ≤40ms), appropriate buffer distances are set for the grid boundaries of different areas: the core performance area (G12-G25) has a grid boundary buffer distance of 0.8m to ensure that the computer vision device can still stably capture the performer when he moves quickly in the core area; the interactive area (G1-G11, G26-G40) has a grid boundary buffer distance of 0.6m to balance tracking accuracy and resource consumption; and the prop area (G41-G75) has a grid boundary buffer distance of 0.4m to adapt to scenarios with lower personnel activity frequency in the prop area and avoid resource waste.

[0078] The identification codes, location coordinates, coverage areas, and corresponding boundary buffer distance parameters of 75 grid areas are integrated to form a complete area division result. This result is stored in JSON format and includes basic grid information, boundary coordinate data, and buffer distance parameters. It serves as the basic data support for subsequent trajectory state vector prediction and camera configuration, ensuring that there are no blind spots in the stage space coverage and no parameter conflicts.

[0079] In step S22, based on the region division results, the performer's trajectory state vector is predicted using a preset Kalman filter algorithm, and the overlapping coverage area of ​​the trajectory state vector is divided into priority levels and camera configurations to obtain a camera coverage network.

[0080] In one embodiment, this embodiment continues the aforementioned application scenario of a large-scale variety show stage. The stage has completed the area division (75 grid areas G1-G75, core performance area G12-G25, interactive area G1-G11 and G26-G40, prop area G41-G75) through step S21. The basic deployment of 12 high-definition network cameras deployed on the trusses around the stage and on the top has been completed.

[0081] Based on the area division results obtained in step S21, the historical movement trajectory data of three core performers from previous similar programs of the gala are first imported. This includes 1200 sets of time-location coordinates for each performer in different performance segments (e.g., the trajectory data of a performer during the climax of a song: coordinates X=9.5m, Y=7.2m at t=180s, coordinates X=10.1m, Y=7.5m at t=181s). At the same time, the current position data of the performers is captured in real time by the cameras corresponding to each grid area driven by computer vision technology (e.g., during the live broadcast, at t=360s, performer A is located in grid G18 with coordinates X=8.8m, Y=6.9m; performer B is located in grid G16 with coordinates X=9.2m, Y=7.8m).

[0082] Load the preset Kalman filter algorithm, which is one of the core algorithms for target trajectory prediction in the field of computer vision. It achieves target state estimation through state prediction and observation update, and sets its state vector as [X, Y, V]. x V y [X, Y are the performer's planar coordinates, Vx and Vy are the movement velocities in the X and Y directions, respectively], the state transition matrix F=[[1,0,Δt,0],[0,1,0,Δt],[0,0,1,0],[0,0,0,1]] (Δt=1 / 30s, consistent with the camera's 30fps frame rate, ensuring temporal synchronization between computer vision data acquisition and trajectory prediction), process noise covariance Q=diag([0.01,0.01,0.001,0.001]), observation noise covariance R=0.02 (Q corresponds to small fluctuations during the movement, and R corresponds to the observation error of the camera). This algorithm integrates the changing trends of historical motion trajectory data with current position information to predict the performer's trajectory within the next 8 seconds, generating a corresponding trajectory state vector (the trajectory state vector is a set of core parameters describing the performer's motion state; in this scheme, the dimension is [X, Y, V...). x V y [This technology can completely capture the position and velocity elements of performers. For example, the trajectory state vector of performer A at t=360.2s is [8.85,7.02,0.3,0.45], and the trajectory state vector of performer B at t=360.2s is [9.28,7.86,0.4,0.2]. This accurately reflects the positional change trend of the two people over a period of time, providing a predictive basis for multi-camera collaboration of computer vision devices.

[0083] Relying on computer vision-based region recognition technology, the system traverses all grid areas covered by trajectory state vectors, identifying regions where multiple trajectory vectors overlap: the G15-G19 grid area in the core performance area (covered by the trajectory vectors of performers A, B, and backup dancer C, with a total of 3 overlapping trajectory vectors), and the G9-G11 grid area in the interaction area (covered by the trajectory vectors of backup dancers C and D, with a total of 2 overlapping trajectory vectors). The system clarifies the specific range of each overlapping area and the number of trajectories involved, forming a detailed list of overlapping areas. This provides a precise regional positioning basis at the computer vision level for subsequent resource allocation.

[0084] Based on the probability of performers appearing within overlapping areas, trajectory concentration (overlap of trajectory points), and stage performance area attributes, combined with computer vision feature analysis technology, overlapping areas are divided into three priority levels: high, medium, and low. Specifically, for core performance areas, regions with a high probability of performers appearing and high trajectory concentration are designated as high priority (according to computer vision data statistics, the probability of performers appearing in this area reaches 95%, and the trajectory concentration is 75%. When multiple performers collaborate, the trajectories are reasonably dispersed; the 75% concentration ensures the tracking focus in the core area while also conforming to the actual scenario). For non-core performance areas, regions with a moderate probability of performers appearing and moderate trajectory concentration are designated as medium priority (the probability of performers appearing is 68%, and the trajectory concentration is 62%, belonging to a secondary scenario of performer-audience interaction). The remaining overlapping areas are designated as low priority. A complete level classification result is output, ensuring that the priority classification is based on objective computer vision data.

[0085] Based on the classification results and the camera's shooting angle range and performance parameters, combined with the resource scheduling logic of the computer vision device, a targeted camera allocation scheme was formulated: Cameras C1-C4 correspond to the core performance area G12-G25; cameras C5-C8 correspond to the interactive area G1-G11 / G26-G40; and cameras C9-C12 correspond to the prop area G41-G75. Specifically, the high-priority grid area G15-G19 is allocated 4 high-definition cameras (C1-C4) with superior performance: 2 for real-time capture of the core performer's movement details, and 2 for auxiliary coverage of the area's edges. Computer vision algorithms are used to achieve multi-view data complementarity, ensuring no blind spots. The medium-priority grid area G9-G11 is allocated 2 high-definition cameras (C5-C6) to meet the tracking needs of interactive scenes. The remaining cameras in the low-priority areas are allocated according to the principle of "1 camera covering 2-3 adjacent grids," ensuring the basic effect of computer vision tracking while avoiding waste of hardware resources. If multiple performers simultaneously cross different areas, a dynamic camera scheduling mechanism will be activated to prioritize camera resources in high-priority areas. Low-priority areas can temporarily borrow cameras from adjacent areas, and the original allocation scheme will be restored after the scene ends.

[0086] Finally, by integrating the trajectory state vector prediction results, overlapping area priority division results, and camera allocation scheme, the computer vision device updates the camera numbers, shooting parameters, and collaborative working rules corresponding to each grid area, forming a camera coverage network adapted to the motion trajectories of all performers. This network is stored in JSON format and contains the grid-associated camera information, priority parameters, and data interaction protocols required for computer vision tracking. It can respond in real time to changes in the performer's motion state, providing stable computer vision collaborative support for subsequent cross-regional tracking and fusion data packet transmission.

[0087] In summary, this invention first constructs a three-dimensional meshed area of ​​the stage and a priority-adapted camera coverage network, then achieves real-time data interaction through a cross-regional boundary triggering mechanism. Combined with a particle filter algorithm, it completes trajectory extrapolation and error backtracking optimization, activates multi-target separation processing, and relies on Kalman filtering to dynamically adjust parameters and smooth the trajectory. This effectively solves the core pain points of traditional multi-camera stage tracking—controlling cross-regional data transmission latency within 40ms, completely eliminating connection gaps; reducing multi-target separation errors by more than 25%, avoiding trajectory confusion; improving trajectory data stability by 40%, and ultimately achieving a tracking sequence continuity of 99.8% and a trajectory deviation ≤0.05m, significantly improving tracking accuracy and continuity. This method is perfectly suited for complex scenarios such as stage performers' rapid movement (up to 5.82m / s) and multi-target interaction, providing high-quality trajectory data support for stage live streaming, recording, and post-production, fully meeting the high standards of visual presentation and motion capture required by modern stage performances.

[0088] Example 2

[0089] refer to Figure 3The second embodiment of the invention provides a stage multi-camera collaborative tracking device, including a boundary triggering and transmission module 31, a first trajectory deduction module 32, a second trajectory deduction module 33, an independent trajectory configuration module 34, and a tracking sequence generation module 35. The boundary triggering and transmission module 31 is configured to: initiate stage multi-camera collaborative tracking, extract the boundary lines of adjacent areas of the camera coverage network and their corresponding boundary buffer distances; if the distance between the performer's current position and the boundary line of the adjacent area is less than the boundary buffer distance, transmit a fused data packet containing the trajectory state vector, current position, and historical motion trajectory sequence to the target camera, and trigger an interactive start signal. The first trajectory deduction module 32 is configured to: respond to the interactive start signal and receive the fused data packet through the target camera, update the particle resampling mechanism weights of a preset particle filter algorithm, and perform fusion processing to obtain the first trajectory deduction result of the performer. The second trajectory deduction module 33 is configured to: construct the trajectory covariance matrix corresponding to the first trajectory deduction result and calculate the continuity error; if the continuity error exceeds a preset error threshold, backtrack and call the fusion data packet to readjust the particle resampling mechanism weights, and perform secondary fusion processing to obtain the second trajectory deduction result. The independent trajectory configuration module 34 is configured to: obtain the performer's real-time velocity information based on the second trajectory deduction result, perform multi-target separation processing, and determine the independent trajectory state vector corresponding to each performer's trajectory. The tracking sequence generation module 35 is configured to: calculate the performer's crossing time across the region boundary based on the independent trajectory state vector, adjust the filtering parameters of the preset Kalman filter algorithm based on the crossing time, optimize the smoothing strategy of the trajectory covariance matrix, and then smooth the trajectory covariance matrix to obtain a continuous tracking sequence.

[0090] refer to Figure 4 The stage multi-camera collaborative tracking device provided in this embodiment of the invention further includes an initial network construction module 41 and a first network adjustment module 42. The initial network construction module 41 is configured to: acquire three-dimensional model data of the stage space, perform meshing processing, and set boundary buffer distances to obtain a region division result. The first network adjustment module 42 is configured to: based on the region division result, predict the trajectory state vector of the performers using a preset Kalman filter algorithm, and perform priority level division and camera configuration on the overlapping coverage area of ​​the trajectory state vectors to obtain a camera coverage network.

[0091] The first network adjustment module 42 further includes a trajectory state vector generation unit 421, an overlapping area determination unit 422, a level division unit 423, a camera allocation unit 424, and a network integration unit 425. The trajectory state vector generation unit 421 is configured to: based on the area division result, combine the performer's historical motion trajectory data and current position, predict the performer's subsequent motion trajectory using a preset Kalman filter algorithm, and generate a corresponding trajectory state vector. The overlapping area determination unit 422 is configured to: identify areas where multiple trajectory vectors overlap within the grid area covered by the trajectory state vectors, and determine these as overlapping areas. The level division unit 423 is configured to: divide the overlapping areas into high, medium, and low priority levels according to the performer's appearance probability, trajectory concentration, and stage performance area attributes within the overlapping areas. Specifically, for core performance areas, areas with a high probability of performer appearance and high trajectory concentration are designated as high priority; for non-core performance areas, areas with a medium probability of performer appearance and medium trajectory concentration are designated as medium priority; and the remaining overlapping areas are designated as low priority. The level division result is then output. The camera allocation unit 424 is configured to: set a camera allocation scheme for overlapping areas of different priorities based on the classification results and the camera's shooting angle range and performance parameters. The network integration unit 425 is configured to: integrate the trajectory state vector, the classification results, and the camera allocation scheme to form a camera coverage network.

[0092] In one possible implementation, the boundary triggering and transmission module is further configured to: capture the performer's current position in real time using cameras covering the corresponding grid area of ​​the camera coverage network; simultaneously extract the corresponding adjacent area boundary lines and boundary buffer distances from the camera coverage network; calculate the boundary distance between the performer's current position and the adjacent area boundary lines, and compare it with the boundary buffer distance; if the distance between the performer's current position and the adjacent area boundary lines is less than the boundary buffer distance, determine that the performer is about to cross the area boundary and assign a target camera; integrate the performer's trajectory state vector, current position, and historical motion trajectory sequence, generate a standardized fusion data packet according to a preset data format, and transmit it to the target camera; trigger an interaction start signal to instruct the target camera to enter a tracking state.

[0093] In one possible implementation, the first trajectory deduction module is further configured to: enter a tracking-ready state after the target camera responds to the interaction start signal and receives the fusion data packet; dynamically update the particle resampling mechanism weights based on a preset particle filter algorithm, combining the current position of the performer captured in real time by the target camera; perform multi-source fusion processing on the received fusion data packet and the current position captured by the target camera through the updated particle filter algorithm, balance the accuracy differences of different data sources, and generate fusion trajectory data; and complete the performer trajectory fitting calculation based on the fusion trajectory data to obtain the first trajectory deduction result of the performer after multi-camera data fusion.

[0094] In one possible implementation, the second trajectory extrapolation module is further configured to: calculate the deviation value of trajectory points at adjacent time points based on the first trajectory extrapolation result, and determine the continuity error through the deviation value, while constructing a trajectory covariance matrix; compare the continuity error with a preset error threshold, and if the continuity error exceeds the preset error threshold, backtrack and call the fused data packet; based on a preset particle filter algorithm, combined with the backtracked fused data packet and the trajectory data dispersion reflected by the trajectory covariance matrix, readjust the particle resampling mechanism weights to obtain a readjusted particle filter algorithm; and based on the readjusted particle filter algorithm, re-perform multi-source data fusion and trajectory fitting to obtain the second trajectory extrapolation result.

[0095] In one possible implementation, the independent trajectory configuration module is further configured to: based on the second trajectory deduction result, calculate the performer's real-time speed information by calculating the difference in the performer's trajectory position and the time interval between adjacent moments; compare the real-time speed information with a preset speed threshold; if the real-time speed information exceeds the preset speed threshold, activate multi-target separation processing based on particle filtering, and split and identify overlapping trajectories by readjusting the particle filtering algorithm; distinguish the motion trajectories of different performers by combining the performer's historical motion trajectory characteristics, real-time position distribution, and motion direction differences; and construct a corresponding trajectory state vector for each performer's motion trajectory to obtain an independent trajectory state vector corresponding to each performer's trajectory.

[0096] In one possible implementation, the tracking sequence generation module further includes a measurement unit, a time calculation unit, an adjustment unit, a strategy optimization unit, a smoothing unit, and a generation unit. The measurement unit is configured to: extract the performer's real-time position, speed, and direction parameters based on the independent trajectory state vector; and calculate the straight-line distance from the performer to the boundary position by combining this with the boundary position of the corresponding grid region. The time calculation unit is configured to: calculate the crossing time of the performer across the region boundary based on the straight-line distance and the performer's real-time speed; and retain only the crossing time when the performer's direction of movement is towards the boundary position as the effective crossing time. The adjustment unit is configured to: invoke a preset Kalman filter algorithm and adjust the filter parameters accordingly based on the effective crossing time; the filter parameters include the process noise covariance and the observation noise covariance. The strategy optimization unit is configured to: optimize the smoothing strategy of the trajectory covariance matrix based on the adjusted filter parameters; use a low-intensity smoothing strategy for short effective crossing times and a high-intensity smoothing strategy for long effective crossing times. The smoothing unit is configured to smooth the trajectory covariance matrix according to the optimized smoothing strategy, suppressing discrete fluctuations in the trajectory data to obtain smoothed trajectory data. The generation unit is configured to integrate the smoothed trajectory data and the independent trajectory state vectors corresponding to each performer, arrange them in a structured manner according to timestamp order, and generate an optimized continuous tracking sequence.

[0097] It should be noted that the stage multi-camera collaborative tracking device provided in this embodiment of the invention is used to execute all the process steps of the stage multi-camera collaborative tracking method in the above embodiment. The working principle and beneficial effects of the two are one-to-one, so they will not be described again.

[0098] Example 3

[0099] Figure 5 A schematic diagram of the structure of a device provided by the present invention, as shown below. Figure 3 As shown, the device includes a processor 50, a memory 51, an input device 52, and an output device 53; the number of processors 50 in the device can be one or more. Figure 5 Taking a processor 50 as an example; the processor 50, memory 51, input device 52, and output device 53 in the device can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0100] The memory 51, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the stage multi-camera collaborative tracking method in this embodiment of the invention. The processor 50 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 51, thereby realizing the aforementioned stage multi-camera collaborative tracking method.

[0101] The memory 51 may primarily include a program storage area and a data storage area. The program storage area may store applications required for operating the device and at least one function; the data storage area may store data created based on terminal usage. Furthermore, the memory 51 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 51 may further include memory remotely located relative to the processor 50, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0102] Input device 52 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 53 may include display devices such as a display screen.

[0103] Example 4

[0104] The present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a stage multi-camera collaborative tracking method.

[0105] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the method operations described above, but can also perform related operations in the stage multi-camera collaborative tracking method provided in any embodiment of the present invention.

[0106] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0107] It is worth noting that in the above-mentioned embodiments of the stage multi-camera collaborative tracking device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0108] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A stage multi-camera collaborative tracking method, characterized in that, include: Initiate multi-camera collaborative tracking on stage, extract the boundary lines of adjacent areas of the camera coverage network and the corresponding boundary buffer distances. If the distance between the performer's current position and the boundary line of the adjacent area is less than the boundary buffer distance, transmit a fused data packet containing the trajectory state vector, current position and historical motion trajectory sequence to the target camera, and trigger an interactive start signal. The target camera responds to the interactive start signal and receives the fusion data packet, updates the particle resampling mechanism weights of the preset particle filter algorithm, and performs fusion processing to obtain the first trajectory projection result of the performer. Construct the trajectory covariance matrix corresponding to the first trajectory inference result and calculate the continuity error. If the continuity error exceeds the preset error threshold, backtrack and call the fusion data packet to readjust the particle resampling mechanism weights, and perform a second fusion process to obtain the second trajectory inference result. Based on the second trajectory deduction result, the real-time velocity information of the performers is obtained, and multi-target separation processing is performed to determine the independent trajectory state vector corresponding to each performer's trajectory. The traversal time of the performer crossing the boundary of the area is calculated based on the independent trajectory state vector. The filtering parameters of the preset Kalman filter algorithm are adjusted in combination with the traversal time to optimize the smoothing strategy of the trajectory covariance matrix. Then, the trajectory covariance matrix is ​​smoothed to obtain a continuous tracking sequence.

2. The stage multi-camera collaborative tracking method according to claim 1, characterized in that, Before initiating multi-camera collaborative tracking on the stage, the following is also included: The three-dimensional model data of the stage space is acquired, and then meshed and boundary buffer distances are set to obtain the area division results. Based on the region division results, the trajectory state vector of the performer is predicted by a preset Kalman filter algorithm, and the overlapping coverage area of ​​the trajectory state vector is divided into priority levels and camera configurations to obtain a camera coverage network.

3. The stage multi-camera collaborative tracking method according to claim 2, characterized in that, Based on the region segmentation results, the performer's trajectory state vector is predicted using a preset Kalman filter algorithm. The overlapping areas of the trajectory state vectors are then prioritized and camera configurations are applied to obtain a camera coverage network, including: Based on the region division results, combined with the performer's historical motion trajectory data and current position, the performer's subsequent motion trajectory is predicted using a preset Kalman filter algorithm, and a corresponding trajectory state vector is generated. Identify areas in the grid region covered by the trajectory state vector where multiple trajectory vectors overlap and determine them as overlapping areas; Based on the probability of performers appearing in the overlapping area, the concentration of their trajectories, and the attributes of the stage performance area, the overlapping area is divided into three priority levels: high, medium, and low. Specifically, in the core performance area, the area where performers appear with a high probability and the concentration of their trajectories is high priority; in the non-core performance area, the area where performers appear with a medium probability and the concentration of their trajectories is medium priority; and the remaining overlapping area is low priority. The level division results are then output. Based on the classification results and the camera's shooting angle range and performance parameters, a camera allocation scheme is set for overlapping areas of different priorities. The trajectory state vectors, hierarchical classification results, and camera allocation schemes are integrated to form a camera coverage network.

4. The stage multi-camera collaborative tracking method according to claim 1, characterized in that, The process involves extracting the boundary lines of adjacent areas of the camera coverage network and their corresponding boundary buffer distances. If the distance between the performer's current position and the boundary line of the adjacent area is less than the boundary buffer distance, a fused data packet containing the trajectory state vector, current position, and historical motion trajectory sequence is transmitted to the target camera, triggering an interaction start signal, including: The current position of the performer is captured in real time by cameras covering the corresponding grid area of ​​the camera coverage network, and the boundary lines and boundary buffer distances of the adjacent areas are extracted from the camera coverage network. Calculate the boundary distance between the performer's current position and the boundary line of the adjacent area, and compare it with the boundary buffer distance; If the distance between the performer's current position and the boundary line of the adjacent area is less than the boundary buffer distance, it is determined that the performer is about to cross the area boundary and a target camera is assigned. The performer's trajectory state vector, current position, and historical motion trajectory sequence are integrated and a standardized fusion data packet is generated according to a preset data format and transmitted to the target camera. The interaction start signal is triggered, instructing the target camera to enter the tracking state.

5. A stage multi-camera collaborative tracking method according to claim 1, characterized in that, The process of responding to the interactive activation signal and receiving the fusion data packet via the target camera, updating the particle resampling mechanism weights of the preset particle filter algorithm, and fusing the data to obtain the performer's first trajectory projection result includes: After responding to the interaction start signal, the target camera enters the tracking ready state and receives the fused data packet; The target camera captures the performer's current position in real time, and the particle resampling mechanism weights are dynamically updated based on a preset particle filtering algorithm. The received fusion data packet and the current position captured by the target camera are fused using the updated particle filter algorithm to balance the accuracy differences of different data sources and generate fused trajectory data. Based on the fused trajectory data, the performer's trajectory fitting calculation is completed, and the first trajectory deduction result of the performer after the fusion of multi-camera data is obtained.

6. The stage multi-camera collaborative tracking method according to claim 1, characterized in that, The process of constructing the trajectory covariance matrix corresponding to the first trajectory derivation result and calculating the continuity error, and if the continuity error exceeds a preset error threshold, involves backtracking and calling the fused data packet to readjust the particle resampling mechanism weights, and performing a second fusion process to obtain the second trajectory derivation result, including: Based on the first trajectory deduction result, the deviation value of trajectory points at adjacent time points is calculated, and the continuity error is determined through the deviation value. At the same time, the trajectory covariance matrix is ​​constructed. The continuity error is compared with a preset error threshold. If the continuity error exceeds the preset error threshold, the fused data packet is recalled back. Based on the preset particle filter algorithm, combined with the fusion data packet of the backtracking call and the degree of dispersion of trajectory data reflected by the trajectory covariance matrix, the weights of the particle resampling mechanism are readjusted to obtain the readjusted particle filter algorithm. Based on the readjusted particle filter algorithm, multi-source data fusion and trajectory fitting were performed again to obtain the second trajectory inference result; The step of obtaining the performer's real-time velocity information based on the second trajectory derivation result, and performing activation multi-target separation processing to determine the independent trajectory state vector corresponding to each performer's trajectory includes: Based on the second trajectory deduction results, the performer's real-time speed information is obtained by calculating the difference in the performer's trajectory position and the time interval between adjacent moments; The real-time speed information is compared with a preset speed threshold. If the real-time speed information exceeds the preset speed threshold, the multi-target separation processing based on particle filtering is activated, and the overlapping trajectories are split and identified by the readjusted particle filtering algorithm. By combining the historical movement trajectory characteristics, real-time location distribution, and differences in movement direction of the performers, the movement trajectories of different performers can be distinguished. For each performer's motion trajectory, a corresponding trajectory state vector is constructed to obtain the independent trajectory state vector corresponding to each performer's trajectory.

7. A stage multi-camera collaborative tracking method according to claim 1, characterized in that, The process of calculating the performer's crossing time across the region boundary based on the independent trajectory state vector, adjusting the filtering parameters of the preset Kalman filter algorithm based on the crossing time, optimizing the smoothing strategy of the trajectory covariance matrix, and then smoothing the trajectory covariance matrix to obtain a continuous tracking sequence includes: Based on the independent trajectory state vector, the performer's real-time position, speed, and direction of motion parameters are extracted. Combined with the boundary position of the corresponding grid area, the straight-line distance from the performer to the boundary position is calculated. Based on the straight-line distance and the performer's real-time movement speed, the crossing time of the performer across the area boundary is calculated, and only the crossing time when the performer's movement direction is towards the boundary position is retained as the effective crossing time; The preset Kalman filter algorithm is invoked, and the filter parameters are adjusted in a targeted manner based on the effective travel time. The filter parameters include the process noise covariance and the observation noise covariance. The smoothing strategy for the trajectory covariance matrix is ​​optimized based on the adjusted filtering parameters. A low-intensity smoothing strategy is adopted for short effective crossing time, and a high-intensity smoothing strategy is adopted for long effective crossing time. The trajectory covariance matrix is ​​smoothed according to the optimized smoothing strategy to suppress discrete fluctuations in the trajectory data and obtain smoothed trajectory data. The smoothed trajectory data and the independent trajectory state vectors corresponding to each performer are integrated and arranged in a structured manner according to timestamp order to generate an optimized continuous tracking sequence.

8. A stage multi-camera collaborative tracking device, characterized in that, include: The boundary triggering and transmission module is configured to: initiate multi-camera collaborative tracking on stage, extract the boundary lines of adjacent areas of the camera coverage network and the corresponding boundary buffer distances; if the distance between the performer's current position and the boundary line of the adjacent area is less than the boundary buffer distance, transmit a fused data packet containing the trajectory state vector, current position and historical motion trajectory sequence to the target camera, and trigger an interactive start signal. The first trajectory deduction module is configured to: respond to the interactive start signal and receive the fusion data packet through the target camera, update the particle resampling mechanism weight of the preset particle filter algorithm, and perform fusion processing to obtain the first trajectory deduction result of the performer; The second trajectory deduction module is configured to: construct the trajectory covariance matrix corresponding to the first trajectory deduction result and calculate the continuity error; if the continuity error exceeds a preset error threshold, backtrack and call the fusion data packet to readjust the particle resampling mechanism weights, and perform secondary fusion processing to obtain the second trajectory deduction result. The independent trajectory configuration module is configured to: obtain the real-time velocity information of the performers based on the second trajectory deduction result, and perform multi-target separation processing to determine the independent trajectory state vector corresponding to each performer's trajectory; The tracking sequence generation module is configured to: calculate the crossing time of the performer crossing the boundary of the area based on the independent trajectory state vector, adjust the filtering parameters of the preset Kalman filter algorithm in combination with the crossing time, optimize the smoothing strategy of the trajectory covariance matrix, and then smooth the trajectory covariance matrix to obtain a continuous tracking sequence.

9. An electronic device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements a stage multi-camera collaborative tracking method as described in any one of claims 1 to 8.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform a stage multi-camera collaborative tracking method as described in any one of claims 1 to 8.

Images