Stage light automatic tracking system and method

Abstract

The application discloses a stage light automatic tracking system and method, and relates to the technical field of light control and automation.The target representation is split into a stable semantic feature vector and an appearance feature vector, and the appearance branch is updated, slowed down or frozen under the driving of a motion state, so that the tracking feature is not dominated by instantaneous reflection and shadow under the working conditions of stagnation and framing, the cross-frame association rule remains consistent, the triggering probability of identity switching and short-time interruption is reduced, a spatiotemporal context signature is generated and stored before occlusion, a searchable spatial constraint is established according to the relative geometric relationship between the disappearance position and the static marker, and after the occlusion ends, the signature is used to build a candidate spatial gating area to reduce the candidate set, and the signature matching score is integrated into the association score, so that the re-identification process does not depend on short-term appearance memory alone, and the false association between similar candidates is reduced.

Description

Technical Field

[0001] This invention relates to the field of lighting control and automation technology, and in particular to an automatic stage lighting tracking system and method. Background Technology

[0002] In stage performances, spotlights or follow lights need to be pointed and controlled in real time according to the changes in the performer's position. In order to reduce the amount of manual following operations, automatic tracking systems for stage lighting control have emerged in the industry. These systems generally acquire the performer's observation information and perform positioning and continuous tracking. The tracking results are then exchanged with the lighting control system to drive the lights to complete the follow-up operation.

[0003] In existing technologies, common technical approaches can be broadly categorized as: active tagging / labeling tracking, computer vision-based unlabeled tracking, and hybrid tracking that combines multi-source information. Active tagging / labeling typically requires performers to wear specialized devices or markers, which are then collected by a receiver for position estimation. Unlabeled tracking usually relies on camera video streams for human detection, keypoint estimation, or target tracking. Hybrid tracking may simultaneously utilize information from vision, inertial measurement, and ultra-wideband to achieve more stable observations under occlusion or interference conditions.

[0004] However, in real stage environments, factors such as rapid changes in lighting effects, shadows and highly reflective materials, multi-person interactions, and prop obstruction can cause fluctuations in observation information. Some tracking methods are easily affected by short-term visual interference during appearance representation updates and cross-frame association, making it difficult to maintain identity consistency in the long term. After prolonged partial or complete occlusion, when the target reappears, ambiguity may arise among multiple similar candidates, making it difficult to continue the original target trajectory. These issues mean that automatic follow-up control still has uncertainties in certain performance conditions, affecting continuous and stable light following input. Summary of the Invention

[0005] This application provides an automatic stage lighting tracking system and method to solve the problems of difficulty in maintaining identity and easy mis-locking or interruption of follow-up lighting in strong light interference and long obstruction scenes.

[0006] In a first aspect, embodiments of the present invention provide an automatic stage lighting tracking method, comprising: Step S1: Obtain a video sequence containing at least one stage target; The video sequence consists of video frames arranged in chronological order, each video frame having a frame timestamp; the frame timestamp serves as the alignment basis for cross-frame positioning and state determination; the position information is the two-dimensional image coordinates of the stage target in the corresponding video frame or the stage plane coordinates obtained through calibration mapping, and the position information carries the corresponding frame timestamp and validity mark, the validity mark being used to indicate that the position information is obtained by detection and association or by motion prediction; Step S2: Extract stable semantic feature vectors and appearance feature vectors of the stage target in parallel in the current frame; Step S3: Within the time window, determine whether the stage target is in an active motion state, a stationary motion state, or a missing state based on the position information. When in an active motion state, update and fuse the stable semantic feature vector with the appearance feature vector to generate tracking features; When the motion is stationary, reduce or freeze the update of the appearance feature vector and generate the tracking feature based on the stable semantic feature vector; Step S4: Perform cross-frame association and localization based on the tracking features; When a missing measurement state or association failure is detected, the tracking features from the previous cycle are used and the positioning result is output based on motion prediction. The update resumes after a preset number of frames N that continuously meet the normal state. The association failure is when the current frame cannot find a candidate target that meets the association constraints for the existing target trajectory state or the association score is lower than the association threshold rule. The normal state is a non-missing measurement state and a state in which no association failure has occurred. The motion prediction is a displacement extrapolation process based on the historical position information sequence and the frame timestamp interval, outputting the predicted position and marking it as the predicted output in the validity label. Step S5: Generate lighting equipment control commands based on the positioning results.

[0007] In some embodiments, the stable semantic feature vector is obtained by encoding human body key points and their topological connections, and the appearance feature vector is obtained by encoding the appearance information of the image region of the stage target.

[0008] In some embodiments, the active motion state and the stagnant motion state are determined based on time window statistics of velocity, acceleration or key point displacement, and hysteresis rules with different entry and exit thresholds are adopted.

[0009] In some embodiments, the fusion weights are determined by the weight adjustment amount output by a mapping table, a policy library, or a learning model, and upper and lower limits are set for the weight change rate.

[0010] In some embodiments, the method further includes: Before determining that a stage target has entered an occlusion state, a spatiotemporal context signature of the stage target is generated and stored. The spatiotemporal context signature includes at least the disappearance position and the relative distance or relative direction to static landmarks in the scene. During the re-identification process after the occlusion ends, the location information of the candidate target is matched with the spatiotemporal context signature, and the matching result is used as an association constraint. The scene static markers in the spatiotemporal context signature are obtained by at least one of manually placed markers, fixed stage structure corner points, and background feature point set; the spatiotemporal context signature includes the vanishing position of the stage target, as well as the relative distance or relative direction between the vanishing position and at least two static markers, and may include the vanishing time and confidence level; During the re-identification process after the occlusion ends, candidate spatial gating regions are generated based on the spatiotemporal context signature to filter candidate targets, and the matching score between the candidate targets and the spatiotemporal context signature is used as a component of the cross-frame association score for association; when the static marker set changes beyond a preset threshold, the static markers are updated and the static marker set is reconstructed. The change is at least one of the following: a change in the number of visible markers in the static marker set, a change in the degree of marker position drift, or a change in the consistency of background feature point matching. The rules for determining the change are given by the configuration file or strategy library and remain consistent during operation.

[0011] In some embodiments, when there is long-term occlusion or interference targets with similar appearance, multiple associated trajectory hypotheses are maintained in parallel, and the globally optimal trajectory is selected as the final tracking result based on the cumulative score across multiple video frames. The assumptions regarding the parallel maintenance of multiple associated trajectories include: In each video frame, multiple trajectory hypotheses are generated based on the detection candidates, and a maximum number of trajectory hypotheses are set, while a number of hypotheses with higher cumulative scores are retained. The cumulative score is obtained by summing the scores of stable semantic consistency, appearance consistency, motion prediction consistency, and matching constraint scores based on the spatiotemporal context signature. When the number of trajectory hypotheses exceeds the upper limit or the cumulative score difference of multiple trajectory hypotheses is less than the preset threshold, the trajectory hypotheses are pruned or merged; during the period of continuous missing tests, each trajectory hypothesis uses the tracking features of the previous cycle and outputs the position based on motion prediction, and resumes updating after a preset number of frames N of continuous normal state is met. When outputting the final tracking results, anti-shake conditions are set for trajectory switching, including continuous frame count conditions and score difference threshold rules.

[0012] In some embodiments, in the cross-frame association, when there are multiple stage targets with similar appearance features, the association metric is further based on local topological relationships, which include the relative distance and relative angle between the target and at least one other neighboring target.

[0013] In some embodiments, the method further includes global motion compensation: estimating global image transformation parameters between adjacent frames based on background features; performing motion prediction in a virtual stable coordinate system and outputting the predicted position; and transforming the predicted position back to the current frame coordinate system to complete the localization.

[0014] Secondly, embodiments of the present invention provide an automatic stage lighting tracking system, comprising, The video acquisition module is used to acquire video sequences containing at least one stage target; The target tracking processing module is used to detect and correlate the video sequence across frames to output the position information of the stage target; The target tracking processing module includes: a dual-path feature extraction unit, a motion state determination unit, and a gating update fusion unit; A control signal generation module is used to generate control commands for the lighting equipment based on the location information; The gated update fusion unit is used to update and fuse two types of feature vectors when in an active motion state, reduce or freeze the update of appearance feature vectors and fuse them with stable semantic feature vectors when in a stagnant motion state, enter a conservative mode when in a missing test state or when association fails to occur, use the fused features from the previous cycle and output the position based on motion prediction, and exit the conservative mode after continuously satisfying the preset number of frames N of the normal state.

[0015] In some embodiments, the control signal generation module converts the position information into at least one control quantity for the luminaire and outputs it through a network or bus interface. The control quantity includes at least a horizontal angle, a pitch angle, or a beam size.

[0016] Through the above technical solution, the present invention can achieve at least the following beneficial effects: To address the issue of unstable appearance representation caused by abrupt changes in light and shadow and reflective materials: By splitting the target representation into stable semantic feature vectors and appearance feature vectors, and implementing slowdown or freezing of the appearance branch under motion state driving, the tracking features are not dominated by instantaneous reflections and shadows in situations such as stagnation and freeze-frame, and the cross-frame association rules remain consistent, thereby reducing the probability of identity switching and short-term interruption.

[0017] To address the candidate ambiguity problem when a target reappears after long-term occlusion: a spatiotemporal context signature is generated and stored before occlusion, and a searchable spatial constraint is established based on the relative geometric relationship between the disappearance location and the static marker. After occlusion ends, the signature is used to construct a candidate spatial gating region to narrow down the candidate set, and then the signature matching score is incorporated into the association score, so that the re-identification process does not rely solely on short-term appearance memory and reduces false associations between similar candidates.

[0018] To address the issue that single association decisions are susceptible to transient noise when similar targets coexist: By maintaining multiple association trajectory hypotheses in parallel, stable semantic consistency, appearance consistency, motion prediction consistency, and contextual consistency are accumulated as cumulative score components across multiple frames. The best score is then selected and output within a single decision time window, transforming the association decision from a single-frame instantaneous score to cross-frame evidence accumulation, thus reducing erroneous locking caused by short-term detection fluctuations or local occlusion.

[0019] To address the issue of control link breakage during missing tests and association failures: By triggering degradation processing through missing tests or association failures, the tracking features of the previous cycle are reused and the positioning results are output with motion prediction. At the same time, continuous normal frame recovery conditions are set so that the system can still provide continuous position output under abnormal conditions such as missing detection, short-term occlusion, and jitter, thus avoiding the sudden disappearance of control command input that would cause the lamp pointing to become uncontrollable.

[0020] To address the uncertainty in association caused by changes in local relative relationships in multi-person interaction scenarios: When similar-looking targets coexist, local topological relationships are introduced as a component of the association metric, so that the association score can simultaneously refer to the consistency of the relative distance and relative angle between the target and its neighboring targets, thereby reducing candidate confusion caused by relying solely on appearance similarity.

[0021] To address the impact of camera shake or global image changes on motion prediction rules: By estimating the global image transformation parameters of adjacent frames, trajectory prediction is placed in a virtual stable coordinate system and then projected back to the current frame coordinate system. This ensures that motion prediction maintains consistent coordinate rules even when global disturbances exist, and degenerates to prediction in the current frame coordinate system when global motion compensation fails, thus guaranteeing link closure. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the scope of this application.

[0023] Figure 1 This is a flowchart of the automatic stage lighting tracking method in the embodiment.

[0024] Figure 2 This is a framework diagram of the automatic stage lighting tracking system in the embodiment. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] All terms used in this application (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0027] Existing stage automatic tracking systems largely rely on online correlation between appearance and motion, which is prone to inconsistencies in identity and trajectory interruptions under conditions of fluctuating lighting effects and recurring occlusion. While multi-sensor fusion can be complementary, it is still constrained by environmental interference and synchronization calibration consistency. These shortcomings limit stable follow-up control in complex performances. To address these issues, the following implementation method is proposed.

[0028] Example 1: like Figure 1 As shown, this embodiment proposes an automatic stage lighting tracking method, which includes the following steps: Step S1: Obtain a video sequence containing at least one stage target; The video sequence consists of video frames arranged in chronological order, each with a frame timestamp; the frame timestamp serves as the alignment basis for cross-frame localization and state determination. Position information consists of the two-dimensional image coordinates of the stage target in the corresponding video frame or the stage plane coordinates obtained through calibration mapping. The position information carries the corresponding frame timestamp and a validity marker; the validity marker indicates whether the position information was obtained through detection and association or through motion prediction. Step S2: Extract stable semantic feature vectors and appearance feature vectors of the stage target in parallel in the current frame; Step S3: Within the time window, determine whether the stage target is in an active motion state, a stationary motion state, or a missing measurement state based on the position information. When in an active motion state, update and fuse stable semantic feature vectors and appearance feature vectors to generate tracking features; When in a state of motion stagnation, reduce or freeze the update of appearance feature vectors and generate tracking features based on stable semantic feature vectors. Step S4: Perform cross-frame association and localization based on tracking features; When a missing measurement state or association failure is detected, the tracking features from the previous cycle are used, and the localization result is output based on motion prediction. Updates resume after a preset number N frames that continuously meet the normal state condition. Association failure occurs when the current frame cannot find a candidate target that meets the association constraints for the existing target trajectory state, or when the association score is lower than the association threshold rule. The normal state is a non-missing measurement state where no association failure has occurred. Motion prediction is a displacement extrapolation process based on historical location information sequences and frame timestamp intervals, outputting the predicted position and marking it as the predicted output in the validity marker. Step S5: Generate lighting equipment control commands based on the positioning results; Acquiring a video sequence involves reading video frame data and frame timestamps, and using the frame timestamps as the alignment basis for cross-frame positioning and status determination; when a frame is lost or the timestamp is abnormal, the frame is set to a missing test status and a conservative mode is triggered.

[0029] Furthermore, the target candidate information will be referred to as the candidate target set in the following description. The candidate target set consists of multiple candidate entries, and each candidate entry is a record of a candidate target within a single frame.

[0030] The target trajectory state is a cross-frame state record corresponding to a single stage target, containing at least the most recent valid position information, the frame timestamp bound to that position information, a stable semantic feature vector, an appearance feature vector, fusion weights, fusion features, motion state, missing test counts, and association stability counts. Gated region, gated region scale, and candidate space gated region are used under the same semantics, all referring to the regional constraint representation for spatial filtering of candidate items. The cache or database is an abstract storage medium used to store spatiotemporal context signatures and a set of static markers; data writing and reading are both indexed using the target identifier and frame timestamp.

[0031] The configuration file is a structured key-value data that carries the information loaded into the policy library. The policy library is a collection of policies containing a mapping table, default weights, threshold rules, boundary constraints, and version identifiers. The version identifier is used to fix the relevant rules within a single running session and update them when the session switches.

[0032] The input for parallel extraction of stable semantic feature vectors and appearance feature vectors is the target image region and target candidate information within the same frame. The output of the two types of feature vectors is bound to the timestamp of the frame and stored in the target trajectory state.

[0033] The inputs for cross-frame association and localization include the candidate target set for the current frame, the target trajectory state of the previous or historical frames, and the motion prediction results. The candidate target set is the set of target candidate entries output by the current frame detection, and each candidate entry contains at least a candidate bounding box, a candidate position, a candidate confidence score, and a candidate validity flag. Association failure is determined by the inability to find a candidate entry that satisfies the association constraints for the existing target trajectory state in the current frame, or by an association score lower than the association threshold rule. The association threshold rule is given by a configuration file or policy library and remains consistent during runtime. Motion prediction is a displacement extrapolation process performed on the time axis based on the historical position information sequence. The motion prediction output includes a predicted position and a prediction validity flag, which is used to identify the usage status of the prediction output in the control link.

[0034] Specifically, the candidate bounding box represents the enclosed region of the candidate target in the current frame image. The candidate position is a position field calculated from the candidate bounding box, represented by either the coordinates of the candidate bounding box center point or the geometric center coordinates calculated from the set of human keypoints, and maintains the same coordinate system as the candidate bounding box. The candidate confidence is a confidence field from the detection output or a confidence field obtained by combining the detection confidence and the keypoint integrity rate. The candidate validity flag is a flag field indicating whether the candidate item participates in the association scoring and whether it can be used as the localization output. The candidate validity flag includes at least two value rules: usable and unusable, and is transmitted along with the candidate item during cross-frame association and localization. Before entering cross-frame association and localization, the candidate item undergoes an integrity check, which includes the validity of the candidate bounding box, the candidate position, the candidate confidence, and the existence of the frame timestamp. If any field is missing, the candidate validity flag is set to unusable, and a missing test flag is written to the candidate item. The binding method between candidate entries and frame timestamps is to record the frame timestamp at the candidate target set level and reference the frame timestamp at the candidate entry level. When calculating across frames, the difference between frame timestamps is used as the time interval rule. If there is an abnormal time interval, the frame is classified as missing and the conservative mode is triggered.

[0035] In this embodiment, the stable semantic feature vector is obtained by encoding the key points of the human body and their topological connection relationships, and the appearance feature vector is obtained by encoding the appearance information of the image region of the stage target. Human keypoints are the set of joint positions obtained from human pose estimation, and human topological connectivity is the set of connections describing the connections between joints; stable semantic feature vectors are obtained by encoding the set of human keypoints and human topological connectivity, and the encoding input includes keypoint coordinates, keypoint visibility markers and topological connectivity indexes.

[0036] Keypoint visibility markers are used to indicate whether each keypoint has been reliably observed. These markers are generated from pose estimation output or by keypoint confidence threshold rules. When keypoint visibility markers indicate a missing keypoint, the encoding process of the stable semantic feature vector uses a placeholder representation for the missing keypoint and passes the placeholder to the fusion feature generation process. The encoding input for the appearance feature vector is the pixel block of the target image region and its lighting appearance information related to the current frame. The appearance feature vector is bound to the frame timestamp for storage and participates in cross-frame correlation scoring.

[0037] In this embodiment, the active motion state and the stagnant motion state are determined based on the time window statistics of velocity, acceleration or key point displacement, and different hysteresis rules are adopted for entry threshold and exit threshold. The time window is a sliding window counted by frames. The length of the time window is given by the configuration file or policy library and remains consistent during runtime.

[0038] The velocity index is composed of the displacement change of position information within a time window and the timestamp interval; the acceleration index is composed of the change of velocity index within a time window; and the key point displacement index is composed of the displacement change of human body key points within a time window. At least one of the velocity index, acceleration index, and key point displacement index shall be used as the basis for state determination.

[0039] The entry threshold is used to trigger rules for switching from a motion-stagnant state to a motion-active state or vice versa, while the exit threshold is used to trigger rules for exiting a target state. The entry and exit thresholds are given by the configuration file or policy library, respectively, and a counting rule is used to suppress the impact of instantaneous fluctuations on state switching for the number of consecutive frames that meet the conditions. The counting rule is given by the configuration file or policy library and takes effect together with the time window.

[0040] In one implementation, the time window length is measured in frames. The default value and adjustable range of the time window length are given by the configuration file, and the time window length ranges from 5 to 60 frames. The rule for counting the number of consecutively satisfied frames is configured independently of the time window length, and the number of consecutively satisfied frames ranges from 2 to 30 frames. The rules for entry and exit thresholds are defined within the same indicator system. Both entry and exit thresholds use statistical quantities of velocity, acceleration, or keypoint displacement indicators as inputs. The threshold values ​​are determined by historical sample quantiles or validation sets and written into the strategy library. Neither entry nor exit thresholds take extreme values ​​to avoid frequent state switching under noise fluctuations. Entry into a missing test state is triggered by the unavailability of candidate validity markers or invalid location information. Exit from a missing test state is triggered by the availability of candidate validity markers and valid location information for several consecutive frames. The default value and adjustable range of the number of consecutive frames N are given by the configuration file, and N ranges from 2 to 30 frames, and remains consistent within a single running session.

[0041] In this embodiment, the fusion weights are determined by the weight adjustment amount output by the mapping table, policy library, or learning model, and an upper and lower limit is set for the weight change rate. The fusion weight is a set of weight parameters used to combine stable semantic feature vectors and appearance feature vectors. The fusion weight is bound to the frame timestamp and stored along with the target trajectory state.

[0042] The mapping table is a set of entries that maps motion states and their derived indices to fusion weight adjustments. The policy library is a set of policies that include the mapping table, default weights, and boundary constraints. The configuration file is an information structure that carries policy library loading information and policy version identifiers. The mapping table and policy library are loaded by the configuration file and remain consistent during runtime.

[0043] The learning model is an inference model that outputs the fusion weight adjustment. The input of the learning model includes at least motion state, velocity index, key point visibility markers and appearance consistency measure. The output of the learning model is the fusion weight adjustment and its credibility marker. The credibility marker is used to trigger the weight fallback rule when the output is abnormal.

[0044] The training data for the learning model comes from stage performance videos or simulated stage videos. The training annotation categories include target identity annotation, occlusion status annotation, and cross-frame association annotation. The learning objective categories include fusion weight adjustment amount and association score consistency objective. The deployment inference link generates fusion weight adjustment amount in each video frame and is jointly constrained by the upper limit of weight, the lower limit of weight, and the rate of change limit. The rate of change limit is given by the configuration file or policy library and remains consistent during operation.

[0045] Furthermore, the fusion weight is a set of weight parameters, containing two types of fields: stable semantic weight and appearance weight. The fusion weight satisfies a normalization rule to keep the combined result of the stable semantic weight and appearance weight within a fixed scale range. The normalization rule constrains both types of weights to the same scale and performs a normalization process before fusion. The upper and lower limits of the fusion weight are represented by weight value range constraints, which are given by the policy library and range from 0.05 to 0.95, to avoid any branch being completely shielded for a long period, leading to discontinuous updates during the recovery phase. The rate of change constraint is represented by the absolute amount of change in fusion weight between adjacent frames. The rate of change constraint is tuned by the validation set and written into the policy library, and ranges from 0.01 to 0.30 per frame. The credibility flag is a validity flag field output by the learning model. The credibility flag has two value rules: valid and invalid. When the credibility flag is invalid, the weight fallback rule is triggered. The weight fallback rule is to use the fused weights of the previous frame and slowly regress to the default weights according to the change rate limit. The default weights are given by the policy library and remain consistent within a single running session.

[0046] In this embodiment, the method further includes: Before determining that a stage target has entered an occlusion state, a spatiotemporal context signature of the stage target is generated and stored. The spatiotemporal context signature includes at least the disappearance position and the relative distance or relative direction to static landmarks in the scene. During the re-identification process after the occlusion ends, the location information of the candidate target is matched with the spatiotemporal context signature, and the matching result is used as an association constraint. The scene static markers in the spatiotemporal context signature are obtained by at least one of manually placed markers, fixed structural corner points of the stage, and a set of background feature points; the spatiotemporal context signature includes the vanishing position of the stage target, the relative distance or relative direction between the vanishing position and at least two static markers, and may include the vanishing time and confidence level; During the re-identification process after occlusion ends, candidate spatial gating regions are generated based on spatiotemporal context signatures to filter candidate targets, and the matching score between candidate targets and spatiotemporal context signatures is used as a component of cross-frame association scoring to participate in association; when the static marker set changes beyond a preset threshold, the static markers are updated and the static marker set is reconstructed. The change is at least one of the following: a change in the number of visible markers in the static marker set, a change in the degree of marker position drift, or a change in the consistency of background feature point matching. The rules for determining the change are given by the configuration file or strategy library and remain consistent during runtime. Occlusion is a set of states in which the detection confidence of a stage target decreases, the visibility markers of key points are missing, or the position information is invalid in consecutive frames. The entry and exit of occlusion is determined by consecutive frame counting rules and threshold rules. The rules are given by the configuration file or policy library and remain consistent during operation.

[0047] Static markers are sets of marks or feature points whose positions remain stable in the image coordinate system during the same stage scene. Static markers consist of the detection results of fixed stage structures, background texture features, or preset marks, and are stored as marker identifiers and marker position fields.

[0048] The spatiotemporal context signature is a data structure that is bound to the target trajectory state. The spatiotemporal context signature includes at least the target disappearance position, the disappearance frame timestamp, and the positional relationship field of the target disappearance position relative to at least one static marker. The positional relationship field includes a relative distance field or a relative direction field, and contains a set of corresponding static marker identifiers.

[0049] Matching is the process of performing consistency calculations on the positional relationship field between the candidate target location and the spatiotemporal context signature. The matching output is a matching score and a matching validity flag. The association constraint consists of candidate elimination rules driven by the matching validity flag and association weighting rules driven by the matching score. The elimination rules and weighting rules are given by the configuration file or strategy library and remain consistent during runtime.

[0050] In one implementation, before determining that a stage target has entered an occlusion state, the positioning results of the current frame and several neighboring frames are sampled to generate and store a spatiotemporal context signature for the stage target. The spatiotemporal context signature is stored in a cache or database with the target identifier and generation time as indexes. The cache adopts a queue structure partitioned by target and sets an expiration date to prevent expired information from participating in subsequent associations.

[0051] Static markers are obtained through one or a combination of the following methods: Firstly, pre-position artificial markers at the stage edge, curtain support, or ground. These markers are high-contrast patterns, reflective stickers, or coded graphics, and their positions are fixed in the stage coordinate system. Secondly, corner points are extracted from the fixed structure of the stage as markers. The fixed structure includes the stage proscenium edge, truss connection points, prop fixing points, etc. Stable corner points are obtained through edge detection and corner point detection and cross-frame consistency screening is performed. Third, a set of background feature points is extracted from the background region, and a stable set is formed by feature point detection and descriptor matching. Long-term stable points are retained by removing dynamic regions, repeated verification, and lifetime counting.

[0052] Assign a unique identifier number to each static marker and record its pixel coordinates in the current frame, or its coordinates in the stage plane coordinate system obtained through calibration.

[0053] The spatiotemporal context signature must contain at least the following fields: The target's disappearance position, the relative distance or direction between the target and several static landmarks; the disappearance position is the coordinates of the center point of the target in the last frame before occlusion or the geometric center coordinates of the set of key points.

[0054] The relative distance is represented by the distance between the target's disappearance position and the selected static marker position, and the relative direction is represented by the azimuth angle or unit direction vector of the target pointing to the static marker. When the stage plane is used as a reference, the relative distance and relative direction are calculated in the stage plane coordinate system first. If the calibration is not completed, they are calculated in the pixel coordinate system and used together with the global motion compensation results.

[0055] The number of static markers should be two or more, selected in order of distance from the target from near to far or by visibility score from high to low. When manually placed markers are available, they should be selected first, followed by fixed structural corner points, and then background feature points. The signature can also optionally include a disappearance time and a confidence level field. The disappearance time is the video frame number or timestamp, and the confidence level is obtained by combining the detection confidence of the target before disappearance, the key point integrity rate, and the association stability, and is used for subsequent matching weighting.

[0056] When generating a spatiotemporal context signature, the visibility of static markers is first determined, and markers that fall into occlusion areas or drift beyond a preset threshold in several consecutive frames are removed; then the relative distance and relative direction of the retained markers are calculated and a signature entry is formed. When global motion compensation is present, the vanishing position and the position of static markers are first projected onto a virtual stable coordinate system before calculating their relative values, in order to reduce the impact of camera shake and overall translation on the signature. The signature is stored using structured fields, with all fields represented by the same coordinate system and unit system, and the coordinate system type identifier is recorded in the signature.

[0057] In the re-identification process after the occlusion ends, candidate spatial gating regions are first given based on the spatiotemporal context signature, and then the signature matching score is used as part of the association score to participate in cross-frame association.

[0058] The candidate spatial gating region is generated as follows: Using the disappearance location in the signature as the center, and combining it with the motion prediction output location, a gated region is constructed. Simultaneously, the relative distance or direction between the signature and several static markers is used to screen candidate target locations for consistency, ensuring that candidate targets fall within a spatial range that satisfies relative relationship constraints. The gated region can be circular, elliptical, or polygonal. The region scale gradually widens with increasing occlusion duration and is modulated by the disappearance confidence level; a smaller region is used when the confidence level is high, and a larger region is used when the confidence level is low.

[0059] For each candidate target that passes the gate, the matching score between its location information and signature is calculated. The matching score is determined by the relative distance difference and relative direction difference between the candidate target and each static marker. Different markers are weighted and summarized according to visibility score or stability score. The matching score, together with appearance similarity and stable semantic feature similarity, constitutes the association score, so that the matching score constrains the association result but does not determine the association result alone.

[0060] To prevent signature invalidation due to scene changes, set static marker update trigger conditions and reconstruction methods. Trigger conditions include one or a combination of the following: Significant changes in background texture caused by stage lighting scene switching; the number of fixed structure corner points decreases by more than a preset proportion within several consecutive frames; the matching rate of the background feature point set is lower than a preset threshold; manually placed signs are not visible within a preset number of frames.

[0061] Execute reconstruction upon triggering: If manually placed markers exist, their numbers and coordinates should be re-detected and updated first. If manually placed markers are unavailable, then the fixed structural corner points are re-extracted and cross-frame consistency screening is performed; If the set of background feature points is still insufficient, a new stable set is generated and a new set is established.

[0062] After reconstruction, a scene version number is generated for the set of markers. The spatiotemporal context signature is bound to the scene version number during storage. During re-identification, only the signature consistent with the current scene version number is used, or if the versions are inconsistent, the signature is downgraded and the gate area is expanded to reduce the risk of incorrect association caused by scene changes.

[0063] Specifically, the scene version number is the version identifier field of the static marker set. The scene version number and the static marker set are stored together and a consistency check is performed after each frame update detection. The consistency check includes at least one of three indicators: change in the number of static markers, drift in the position of static markers, and consistency of background feature point matching. The indicator calculation results are written into the marker set state record and used as input for update triggering. The weight reduction processing of the spatiotemporal context signature is to proportionally reduce the combined weight of the signature matching score in the association score and relax the upper limit of the scale of the candidate spatial gate region. The proportion and the upper limit of the scale are given by the configuration file and remain consistent within a single running session. The candidate spatial gate region is represented by structured fields for the region center and the region boundary. The region center is jointly determined by the vanishing position and the motion prediction output position, and the region boundary is represented by one of a circle, ellipse, or polygon. When a polygon is used, the polygon is composed of a list of vertex coordinates and is bound to the frame timestamp for storage. The gating region scale relaxes linearly or piecewise as the occlusion duration increases. The relaxation rules are provided by the policy library and include two types of boundary constraints: minimum and maximum scales. The minimum and maximum scales are defined in the same coordinate system and unit system. The vanishing confidence level serves as the modulation input for the relaxation rules; a smaller gating region scale is used when the vanishing confidence level is high, and a larger gating region scale is used when the vanishing confidence level is low. The signature matching validity flag is the validity flag field in the matching output. When the matching validity flag is invalid, only the motion prediction gating is retained, and the context consistency component is set to the penalty value rule. The penalty value rule is provided by the policy library and remains consistent throughout a single running session.

[0064] The above implementation establishes a relative geometric relationship between the last visible position of the target and static markers before occlusion, forming a storable and searchable spatiotemporal context signature. Static markers can be manually placed markers, fixed corner points of stage structures, or a set of stable background feature points. Each marker is assigned a number and coordinate record, facilitating the calculation of the relative distance or direction between the target and markers in the same coordinate system. The signature field includes the disappearance position and relative relationship, and can include the disappearance time and confidence level to control the gating range and matching weighting. After occlusion, the signature is used to construct a candidate spatial gating region to reduce the number of candidates. Then, the matching score is incorporated into the association score, serving as a constraint to improve the reliability of cross-frame association. To address scene transition issues, the marker set is updated based on conditions such as background changes, a decrease in the number of corner points, a decrease in the matching in-point rate, or marker invisibility. The scene version number is bound to the signature to prevent association offsets caused by using old signatures after scene changes, thereby improving the sufficiency and feasibility of the disclosure.

[0065] In this embodiment, when there is long-term occlusion or interference targets with similar appearance, multiple related trajectory hypotheses are maintained in parallel, and the globally optimal trajectory is selected as the final tracking result based on the cumulative score across multiple video frames. The assumptions for maintaining multiple associated trajectories in parallel include: In each video frame, multiple trajectory hypotheses are generated based on the detection candidates, and a maximum number of trajectory hypotheses are set, while a number of hypotheses with higher cumulative scores are retained. The cumulative score is obtained by summing the scores of stable semantic consistency, appearance consistency, motion prediction consistency, and matching constraint scores based on spatiotemporal context signatures; When the number of trajectory hypotheses exceeds the upper limit or the cumulative score difference of multiple trajectory hypotheses is less than the preset threshold, the trajectory hypotheses are pruned or merged; during the period of continuous missing tests, each trajectory hypothesis uses the tracking features of the previous cycle and outputs the position based on motion prediction, and resumes updating after a preset number of frames N of continuous normal state is met. When outputting the final tracking results, set anti-shake conditions for trajectory switching based on consecutive frame count conditions and score difference threshold rules; The associated trajectory hypothesis is a candidate associated sequence data structure of the target on consecutive video frames. Each associated trajectory hypothesis includes at least a hypothesis identifier, a frame sequence index, a candidate entry reference set, and a cumulative score field.

[0066] The cumulative score is a set of score indicators that accumulates association consistency across multiple frames. The cumulative score consists of at least a stable semantic consistency component, an appearance consistency component, a motion prediction consistency component, and a contextual consistency component. The contextual consistency component is generated from the matching score. The combined weights of each component are given by the policy library and remain consistent during runtime. Parallel maintenance includes generating multiple associated trajectory hypotheses for the candidate target set in each frame and updating their cumulative scores, while simultaneously removing low-scoring hypotheses according to the hypothesis number cap and score pruning rules. The hypothesis number cap and score pruning rules are given by the configuration file or policy library and remain consistent during runtime. The globally optimal trajectory is an associated trajectory hypothesis whose cumulative score satisfies the optimal selection rule within a decision time window. The optimal selection rule includes a score maximization rule and a stability constraint rule. The stability constraint rule includes a consecutive frame consistency counting rule and is given by the configuration file or policy library.

[0067] In one implementation, when there are interfering targets that are occluded for extended periods or have similar appearances, multiple associated trajectory hypotheses are maintained in parallel for each single stage target. Each trajectory hypothesis includes a trajectory identifier, the number of the most recently associated candidate target, the position sequence of the most recent frames, the tracking features used, the motion prediction state, and a cumulative score. The trajectory hypotheses are stored in hypothesis pools grouped by target, with a maximum capacity H for each pool to prevent unbounded growth of computational load.

[0068] When each video frame arrives, the set of candidate targets is output first, and the stable semantic feature similarity, appearance feature similarity and position-related information are calculated for each candidate target; then the motion prediction gives the predicted position of the stage target in the current frame.

[0069] Based on the above information, several association allocation schemes are generated: For each existing trajectory hypothesis in the hypothesis pool, candidates satisfying spatial gating are selected from the current frame's candidate target set as expandable branches. Spatial gating is jointly determined by the gating region around the predicted position and the spatiotemporal context signature constraints. For each expandable candidate, a new trajectory branch is formed, inheriting historical information from the previous frame's trajectory hypothesis. To cover missing detection scenarios, an unassociated branch is simultaneously generated for each existing trajectory hypothesis. This branch does not select any candidate targets, uses the tracking features from the previous cycle, predicts the output position based on motion, and records the missing detection count. After generation, all new branches are aggregated to form a candidate trajectory set, and the top K are retained according to cumulative scores, where K is less than or equal to H. When the number of candidate trajectories is less than K, all candidate trajectories are retained.

[0070] In one implementation, it is assumed that the maximum pool capacity H and the number of retained entries K are given by the configuration file and remain consistent within a single runtime session. H ranges from 10 to 300, and K ranges from 5 to 150, with K being less than or equal to H. The trajectory hypothesis identifier is a unique identifier field for the trajectory branch, and it, along with the target identifier, serves as an index field within the hypothesis pool. The candidate target number is the number field for the candidate entry, unique within a single frame and bound to the frame timestamp. The trajectory branch inheritance rule is to inherit the most recent frame position sequence, tracking features, fusion weights, and missing test counts of the corresponding trajectory hypothesis from the previous frame when generating a new branch, and to append candidate entry references or unassociated markers to the current frame. The unassociated marker is a marker field where the trajectory hypothesis is not bound to any candidate entry in the current frame; the missing test count increments after the unassociated marker is written. The scoring and sorting rules are based on the cumulative score as the primary key and the missing test count and stable semantic consistency score as the secondary key. The sorting rules are provided by the policy library and remain consistent within a single running session. When cumulative scores are the same or the difference falls within a preset range, the branch with the lower missing test count is retained first, and the other branch is merged according to the merging rules. The merging rules merge branches whose position sequences and candidate entry reference sets are consistent within the most recent M frames or whose difference is below a preset threshold. M is provided by the configuration file, and its value ranges from 3 to 30 frames. After merging, the identifier of the branch with the higher cumulative score is retained, and the cumulative score of the other branch is merged proportionally. The proportion is provided by the policy library and avoids extreme values ​​to prevent score mutations caused by a single merging. The switching cost is a scoring penalty applied when the branch candidate target number is inconsistent with the current output trajectory candidate target number. The switching cost is provided by the policy library and takes effect in conjunction with the anti-shake condition. The number of consecutive frames T for the anti-shake condition is provided by the configuration file, and its value ranges from 2 to 30 frames.

[0071] The cumulative score is obtained by weighted summation of multiple consistency scores, and its components include: Stable semantic consistency score is used to measure the degree of consistency between the key point topological encoding of the candidate target and the stable semantic features of the trajectory hypothesis; Appearance consistency score is used to measure the degree of consistency between the appearance features of the candidate target and the appearance features or fused features of the trajectory hypothesis; Motion prediction consistency score measures the deviation between the candidate target location and the predicted location; the smaller the deviation, the higher the score. And context matching constraint score, used to measure the consistency between the candidate target location information and the spatiotemporal context signature generated before occlusion.

[0072] For trajectory assumptions in the occlusion recovery phase, increase the proportion of context matching constraint scores in the cumulative score; For the trajectory assumption of being in a state of motion stagnation, increase the proportion of stable semantic consistency score in the cumulative score and decrease the proportion of appearance consistency score.

[0073] The cumulative score is updated using a cross-frame accumulation method. The cumulative score of a new branch is obtained by adding the cumulative score of the trajectory assumption in the previous frame to the incremental score of the current frame. An attenuation coefficient is set for the cumulative score to gradually reduce the impact of errors from earlier frames.

[0074] Pruning is performed when the number of hypotheses begins to explode. Pruning rules include one or a combination of the following: Firstly, capacity pruning: when the candidate trajectory set exceeds H, only the H trajectories with the highest cumulative scores are retained; Secondly, similarity pruning is used to merge trajectory branches whose position sequences and candidate numbers are completely identical or have a difference of less than a preset threshold within the most recent M frames. Only the one with the higher cumulative score is retained, and the cumulative score of the other one is merged in at a small proportion to retain uncertainty information. Third, low-score pruning: when the difference between the cumulative score and the cumulative score of the current best trajectory exceeds a preset threshold, it is directly removed; Fourth, proximity pruning: when the cumulative scores of multiple trajectory branches are close and the difference falls within a preset range, branches with higher stable semantic consistency or lower missing test counts are retained first to reduce the probability of misselection caused by similar appearance.

[0075] To avoid frequent switching between targets with similar appearances, an additional switching cost is set. When the candidate target number of a branch is different from the candidate number of the trajectory output in the previous frame and has no obvious scoring advantage, a penalty is imposed on the cumulative score of that branch.

[0076] When a trajectory hypothesis enters a long period of missing data, the unassociated branch triggers a conservative mode: it continues to use the fusion features of the previous cycle and only uses the motion prediction output position, and the missing data count increases with each frame; when the missing data count exceeds a preset upper limit, the trajectory hypothesis is not terminated, but it is marked as dormant and its association frequency with new detection candidates is reduced. The association frequency can be expressed as attempting candidate expansion only once every preset number of frames to reduce invalid computation. When a detection candidate again meets the spatial gating and continuously meets the preset number of frames N of the normal state, the trajectory assumption is restored from the dormant or conservative mode to the normal mode, and appearance feature updates and fusion weight changes are allowed to change normally again.

[0077] To reduce light jitter, a stabilization strategy is adopted for the output trajectory: The trajectory with the highest cumulative score is selected as the main trajectory from the hypothesis pool, but the output of the main trajectory must meet the anti-shake conditions. The anti-shake conditions include one or a combination of the following: the main trajectory maintains the highest cumulative score for T consecutive frames and the score difference with the second-best trajectory exceeds a preset threshold; or the main trajectory and the current output trajectory are consistent for T consecutive frames and the positional deviation is less than a preset threshold; when a new, better trajectory appears, if its score advantage does not exceed the preset threshold, the original trajectory continues to be output and the ranking is only updated within the hypothesis pool. The output position is further smoothed. The smoothing is achieved by using a moving average of the output position of the most recent few frames or by limiting the rate of change of angular velocity, so that the changes in the horizontal and pitch angles of the lamps do not jump abruptly, thereby maintaining stable lamp pointing during multiple hypothesis competition and switching.

[0078] The above implementation maintains multiple trajectory branches in parallel using a hypothesis pool. In each frame, it generates scalable and unassociated branches for detected candidates, forming various association allocation schemes. The scale is controlled by a maximum capacity H and a mechanism to retain the top K paths. The cumulative score accumulates scores across frames based on stable semantic consistency, appearance consistency, motion prediction consistency, and context matching constraints. The influence ratio of each score is adjusted according to occlusion recovery or stagnation states, making trajectory selection more aligned with the gating update fusion strategy. Pruning employs priority rules based on capacity, similarity merging, low-score removal, and scores being close, and introduces switching costs to reduce frequent jumps between appearance-similar targets. During long-term missing tests, a conservative mode is maintained to output predicted positions. Simultaneously, calculations are controlled through sleep mode and reduced association frequency, and updates resume after a preset number N of frames continuously meeting normal conditions. The final output achieves anti-shake through continuous frame stability conditions and a score difference threshold, and smooths the position or limits the rate of change of angular velocity to reduce lighting jitter.

[0079] In this embodiment, when there are multiple stage targets with similar appearance features in cross-frame association, the association metric is further based on local topological relationships, which include the relative distance and relative angle between the target and at least one other neighboring target. Neighboring targets are the set of other target candidates within the same video frame that satisfy the proximity determination rules with the current target candidate. The proximity determination rules consist of the top K selection rules based on distance sorting or a set selection rule based on a distance threshold. These rules are provided in the configuration file or policy library and remain consistent during runtime. Relative distance is the distance field between the current target candidate position and the neighboring target candidate positions in the same coordinate system. Relative angle is the direction field of the line connecting the current target candidate and the neighboring target candidate relative to the image coordinate axis or stage coordinate axis. The relative distance and relative angle fields are bound to the frame timestamp and participate in the association metric calculation. Association metrics are a set of metrics used to match and score the candidate entries with the target trajectory state. Association metrics include at least appearance consistency metrics, stable semantic consistency metrics, and local topological consistency metrics. Local topological consistency metrics are generated by the consistency of the relative distance and relative angle fields across frames and are constrained by the missing test marker propagation rules.

[0080] Furthermore, the neighboring target set is generated from the candidate target set within the same frame. When there are fewer than one neighboring target, the local topology consistency measure does not participate in the association scoring and is set to a neutral value rule. The neutral value rule is provided by the policy library and remains consistent within a single running session. The unit of the relative distance field is consistent with that of the candidate position field. When the candidate position uses two-dimensional image coordinates, the unit of the relative distance field is pixels; when the candidate position uses stage plane coordinates, the unit of the relative distance field is stage plane coordinate units. The relative angle field is a direction field. The reference axis of the direction field uses either the image coordinate axis or the stage coordinate axis. The reference axis type is consistent with the candidate position coordinate system type and is recorded in the coordinate system type identifier of the association measure. The value range of the relative angle field is 0~360° and is periodically processed before cross-frame consistency calculation to avoid abrupt changes caused by angle crossing boundaries. Local topology consistency measure is linked with the missing test mark propagation rule during cross-frame consistency calculation. When a neighboring target is missing in consecutive frames or the set of neighboring targets changes, the local topology consistency measure is downgraded according to the missing test rule or set to a neutral value rule. The downgrade ratio is given by the policy library and remains consistent within a single running session.

[0081] The method also includes global motion compensation: estimating global image transformation parameters between adjacent frames based on background features; performing motion prediction in a virtual stable coordinate system and outputting the predicted position; and transforming the predicted position back to the current frame coordinate system to complete the localization. The global image transformation parameters are a set of parameters that describe the global geometric relationship between adjacent video frames. The transformation model corresponding to the global image transformation parameters includes either an affine transformation model or a homography transformation model. The transformation model type is given by the configuration file or the policy library and remains consistent during runtime.

[0082] Background features are a set of background region feature points used to estimate global image transformation parameters. The background region is determined by the foreground target candidate region elimination rule. The elimination rule consists of candidate box expansion rule and safety boundary rule and is given by configuration file or policy library.

[0083] The virtual stable coordinate system is a coordinate system established with the reference frame coordinate system as the reference. The reference frame is given by the configuration file or policy library and remains consistent during the same tracking session. Motion prediction is performed in the virtual stable coordinate system using trajectory sequences transformed from historical position information.

[0084] Global motion compensation failure occurs when the global image transformation parameter estimation fails to meet the consistency judgment rules or the estimation result is invalid. The failure state triggers degradation processing, which includes skipping the virtual stable coordinate system transformation and performing motion prediction in the current frame coordinate system, while writing the failure flag into the validity flag of the position information. Exit from the failure state is triggered by the frame counting rule that continuously meets the consistency judgment rules, and the rules are given by the configuration file or policy library.

[0085] Specifically, the consistency determination rule for global image transformation parameters consists of at least one of two conditions: consistency of inlier points in background feature matching and stability of transformation parameters. Inlier point consistency in background feature matching uses the proportion or percentage of inlier points in the background feature matching as input, while stability of transformation parameters uses the magnitude of parameter changes between adjacent frames as input. The threshold rules for these inputs are provided by the policy library and remain consistent throughout a single running session. The candidate box expansion rule in the background region removal rule expands the candidate boxes proportionally to form the removal region. The expansion ratio is provided in the configuration file and ranges from 1.1 to 2.0. The safety boundary rule is used to remove background feature points near the image edge to reduce edge distortion. The safety boundary width is provided in the configuration file and measured in pixels, ranging from 8 to 64 pixels. When the degradation processing for global motion compensation failure is triggered, the virtual stable coordinate system transformation is skipped and motion prediction is performed in the current frame coordinate system. At the same time, the failure flag is written to the validity flag field of the position information. When the failure flag appears consecutively for more than a preset number of frames, the global image transformation parameter estimation frequency is reduced by a preset ratio and restored after a preset number of consecutive frames that meet the consistency judgment rules. The estimation frequency ratio and the preset number of frames are both given by the configuration file and remain consistent within a single running session.

[0086] Example 2: Based on Embodiment 1, this embodiment provides an automatic stage lighting tracking system, including: The video acquisition module is used to acquire video sequences containing at least one stage target; The target tracking processing module is used to detect and correlate video sequences across frames to output the position information of stage targets; The target tracking processing module includes: a dual-path feature extraction unit, a motion state determination unit, and a gating update fusion unit; The control signal generation module is used to generate control commands for the lighting equipment based on the location information; Among them, the gated update fusion unit is used to update and fuse two types of feature vectors when in an active motion state, reduce or freeze the update of appearance feature vectors when in a stagnant motion state and fuse them with stable semantic feature vectors as the main component, and enter conservative mode when in a missing test state or when association fails to occur, so as to use the fused features of the previous cycle and output the position based on motion prediction, and exit conservative mode after continuously satisfying the preset number of frames N of normal state. The video sequence consists of video frames arranged in chronological order, with each video frame having a frame timestamp. The frame timestamp comes from the acquisition clock of the video acquisition module or the unified system clock, and serves as the time reference for cross-frame calculations within the target tracking processing module.

[0087] The location information consists of the two-dimensional image coordinates of the stage target in each video frame or the stage plane coordinates obtained through calibration mapping. The location information carries the corresponding frame timestamp and validity marker; the validity marker indicates whether the location information of that frame was obtained by detection and association or by motion prediction. The stable semantic feature vector is a feature representation used to characterize the stable structural information of the stage target, the appearance feature vector is a feature representation used to characterize the appearance texture and lighting appearance information of the stage target, and the fused feature is a tracking feature representation obtained by the gated update fusion unit combining the stable semantic feature vector and the appearance feature vector according to the fusion weight.

[0088] The motion active state, motion stagnation state, and missing state are determined by the motion state determination unit based on the time series of location information and its validity marker. The missing state includes situations such as target detection failure, target association failure, or invalid location information. The conservative mode is a degraded processing state used during the missing state or association failure. In the conservative mode, the fusion features of the previous period are used and the location information obtained by motion prediction is output. At the same time, the conservative mode marker is written into the validity marker associated with the location information of this frame. The number of consecutive normal state frames N that exit the conservative mode is given by the configuration file or policy library and remains consistent during operation.

[0089] In this embodiment, the control signal generation module converts the position information into at least one control quantity of the lamp and outputs it through a network or bus interface. The control quantity includes at least the horizontal angle, the pitch angle, or the beam size. The control quantity is a set of control fields used by the lighting equipment to complete the follow-up pointing and beam modulation. The control quantity is bound to the frame timestamp and serves as the output of the control signal generation module.

[0090] The conversion from position information to control quantities includes coordinate mapping and limiting processing. Coordinate mapping is generated based on the calibration relationship between the camera coordinate system, stage coordinate system, and lighting installation coordinate system. The calibration relationship includes extrinsic parameter relationship fields and intrinsic parameter relationship fields, which are loaded in the form of configuration files. Limiting processing includes angle change rate limitation and control quantity boundary limitation. The limitation rules are given by the strategy library and remain consistent during operation.

[0091] The network or bus interface output includes encapsulating the control quantity into a control frame data structure and writing it into the transmission timestamp field. The control frame data structure includes at least a target identifier field, a control quantity field, a timestamp field, and a validity flag field. When communication is interrupted or transmission fails, the control signal generation module uses the control quantity from the previous cycle and writes a communication error flag. The exit of the communication error flag is triggered by the consecutively successfully transmitted frame counting rule. The rule is given by the configuration file or policy library and remains consistent during operation.

[0092] Furthermore, the calibration relationships upon which the coordinate mapping is based are stored in structured fields. The extrinsic parameter relationship fields at least include the attitude and position relationships from the camera coordinate system to the stage coordinate system, and the attitude and position relationships from the lighting fixture installation coordinate system to the stage coordinate system. The intrinsic parameter relationship fields at least include camera imaging parameter fields. When the calibration relationships are missing or invalid, the control signal generation module maintains the position information within the two-dimensional image coordinate rules and generates relative control quantities for the horizontal and pitch angles using preset mapping rules. These preset mapping rules are provided by the configuration file and remain consistent throughout a single running session, with a mapping degradation flag written in the validity flag field. Angle change rate limits are represented by the variation amplitude constraints of adjacent output control quantities. These variation amplitude constraints are provided by the policy library and remain consistent throughout a single running session, with a value range of 0.5 to 20.0 per second. Control quantity boundary limits are represented by boundary rules for the reachable angle range and beam size range of the lighting fixtures. These boundary rules are provided by the configuration file and bound to the lighting fixture type identifier. The validity flag field of the control frame data structure must include at least five value categories: normal, predictive output, conservative mode, communication anomaly, and mapping degradation. When communication is interrupted or transmission fails, the control quantity from the previous cycle is used, and the validity flag field is set to communication anomaly. Simultaneously, several recent control frame data structures are cached locally to support continuous output after anomaly recovery. The number of cached frames is specified in the configuration file, ranging from 10 to 600 frames. After the number of consecutive successfully transmitted frames reaches the exit rule, the communication anomaly flag is cleared, and the path for generating control quantities using the current frame position information is restored. The restoration process is constrained by both the angle change rate limit and the control quantity boundary limit to prevent sudden jumps in control quantities.

[0093] Example 3: This embodiment provides an application in a stage scene where modern dance performances and props obscure the view. The automatic stage lighting tracking method takes the video sequence of a fixed-position camera as input and outputs the horizontal angle, pitch angle, and beam size control of the follow spot.

[0094] This embodiment features a scene with three dancers, with the target dancer designated as target identifier D1. The camera resolution is 1920×1080, the frame rate is 30 fps, and the frame timestamp is in milliseconds and increments with each frame. The system has completed the calibration from the camera coordinate system to the stage plane coordinate system, with the stage plane coordinate unit being meters (m). The controlled object is a follow spot light with controls for horizontal angle, pitch angle, and beam size. The attitude and positional relationship between the light's installation coordinate system and the stage coordinate system has been written into the configuration file. The strategy library and mapping table are loaded from the configuration file and have fixed version identifiers at the start of this session.

[0095] This embodiment provides a set of configurations: the time window length is 15 frames; the preset number of frames N that continuously meet the normal state is 6 frames; assuming the maximum pool capacity H is 60 and the number of retained items K is 20; the neighboring target selection rule is to select the top 3 by distance; the initial radius of the gating region is 0.35 m, which linearly widens to a maximum radius of 1.20 m with the duration of occlusion; the initial fusion weights are a stable semantic weight of 0.65 and an appearance weight of 0.35, with the rate of change limited to no more than 0.08 per frame; hysteresis rules are used for active motion and stagnant motion, with the entry threshold rule being an average velocity greater than 0.25 m / s within the time window and the exit threshold rule being an average velocity less than 0.12 m / s within the time window; the entry rule for the missing test state is that the candidate validity marker is unavailable or the location information is invalid for 2 consecutive frames.

[0096] In this embodiment, the set of static markers is derived from a combination of manually placed markers and corner points of fixed stage structures, with scene version number V3. Static markers are recorded using stage plane coordinates and have unique marker numbers: M1 is located at (0.0, 0.0)m, M2 at (10.0, 0.0)m, M3 at (0.0, 6.0)m, and M4 at (10.0, 6.0)m. The candidate target set is obtained from the detection output, and each candidate entry includes at least a candidate bounding box, candidate position, candidate confidence level, and candidate validity marker; the candidate position in this embodiment is represented using stage plane coordinates.

[0097] At frame timestamp of 120000 ms (denoted as frame F0), step S1 acquires the video frame data and frame timestamp. Step S2 extracts stable semantic feature vectors and appearance feature vectors for target D1 in parallel within the current frame: human keypoints are obtained from pose estimation, totaling 17 keypoints, of which 16 keypoint visibility markers show usable and 1 missing; the stable semantic feature vector encoding input includes keypoint coordinates, keypoint visibility markers, and topological connectivity indexes. The appearance feature vector encoding input consists of pixel blocks of the target image region and their lighting appearance information related to the current frame. Both types of feature vectors are bound to the frame timestamp and written into the target trajectory state.

[0098] In frames F0 and several subsequent frames (F1~F14), target D1 performs rapid movement, and the position information sequence in the target trajectory status is displayed as follows: F0: Location (3.10, 2.40)m, candidate confidence level 0.92, validity marked as normal; F7: Location (3.95, 2.95)m, candidate confidence level 0.90, validity marked as normal; F14: Location (4.80, 3.50)m, candidate confidence level 0.91, validity marked as normal.

[0099] Step S3 calculates the average velocity within the time window based on the location information to be approximately 0.58 m / s, classifying it as an active motion state. The gated update fusion unit updates and fuses the two types of feature vectors according to the active motion state branch to generate tracking features: the fusion weights start from the default weights given by the policy library, and the mapping table provides weight adjustment amounts based on the active motion state, so that the stable semantic weights slowly decrease to 0.55 and the appearance weights increase to 0.45 (meeting the rate of change limit). Step S4 uses this tracking feature for cross-frame association and localization; in the candidate target set of each frame, the candidate item corresponding to D1 has the highest association score, the association is successful, and the location information is continuously output.

[0100] During the period from frame timestamp 12 05 00 ms (frame F15) to 12 11 00 ms (F33), the dancer enters a freeze-frame motion, while the costume shows strong reflections and the stage sidelight sweeps across. At this time, the appearance feature similarity fluctuates significantly: for example, at F18, the appearance similarity drops from 0.82 to 0.41, and then rises back to 0.76 at F19; while the stable semantic feature similarity remains in the range of 0.79~0.86. Step S3 calculates the average velocity within the time window to be approximately 0.05 m / s, triggering the hysteresis rule to enter a motion stagnation state. The gated update fusion unit executes the motion stagnation state branch: reducing the appearance feature vector update rate to 0.20 of the original, and freezing the appearance feature vector when the appearance consistency metric is below 0.50 for two consecutive frames; simultaneously, adjusting the fusion weights to a stable semantic weight of 0.85 and an appearance weight of 0.15, and converging frame by frame under a change rate limit of 0.08 per frame. During this period, despite fluctuations in the appearance confidence of candidate items, cross-frame association was mainly supported by stable semantic consistency and motion prediction consistency. The target trajectory state of D1 was not reset, and the localization output remained continuous.

[0101] Starting at frame timestamp 12 12 00 ms (F34), a large prop is pushed onto the stage to obscure target D1. The detection output shows no valid candidate entries for D1 in multiple consecutive frames: F34: No candidate entries satisfying spatial gating are found in the candidate target set; association failed. F35: Candidate validity markers are unavailable; F36: Still unavailable.

[0102] According to the missing-test state entry rule, step S3 determines that the system has entered the missing-test state. Step S4 triggers the conservative mode: using the tracking features from the previous cycle, motion prediction extrapolates the displacement based on the historical position information sequence, outputs the predicted position, and sets the validity flag to the predicted output. The output is: F36 predicts the location to be (5.10, 3.62) m. F40 predicted location: (5.25, 3.68) m; Meanwhile, the number of missing measurements increases with each frame.

[0103] In the last visible frame F33 before F34 enters the occlusion state, a spatiotemporal context signature has been generated and stored, and bound to the scene version number V3. This spatiotemporal context signature includes: the disappearance position (5.00, 3.60)m, the disappearance frame timestamp 12 11 00 ms, the set of static marker identifiers {M2, M4}, the relative distance to M2 is 6.02 m, the relative distance to M4 is 5.62 m, and the disappearance confidence score is 0.88. The signature is written to the buffer using the target identifier D1 and the generation time as indices.

[0104] At frame timestamp 12 19 00 ms (F55), the occlusion ends, and the detection output shows multiple candidate items with similar appearances again. Furthermore, the costume of another dancer on stage, D2, is similar to that of D1. At this point, the current frame's candidate target set contains three main candidate items: Candidate A: Location (5.40, 3.75)m, confidence level 0.80; Candidate B: Location (6.10, 3.90)m, confidence level 0.83; Candidate C: Position (5.35, 2.95)m, confidence level 0.78.

[0105] In step S4, during the re-identification process, candidate spatial gating regions are first generated based on the spatiotemporal context signature: the region center is determined by the disappearance position and the motion prediction position (5.32, 3.70) m, and the gating radius is widened to 0.95 m due to the duration of occlusion. The gating results show that candidates A and C pass, while candidate B is eliminated because its distance from the center exceeds 0.95 m. Signature matching scores are calculated for the candidates that pass the gating: candidate A has a smaller relative distance difference with M2 and M4, resulting in a matching score of 0.84; candidate C has a matching score of 0.52. Subsequently, the matching scores, stable semantic similarity, and appearance similarity are combined to form an association score, with candidate A having the highest overall score.

[0106] Simultaneously, this embodiment enables parallel maintenance of multiple associated trajectory hypotheses. The hypothesis pool generates the following branches in F55: Trajectory hypothesis H1: Associated with candidate A; Trajectory hypothesis H2: Associated with candidate C; Trajectory hypothesis H3: Unrelated branches (conservative mode continuation).

[0107] The cumulative score is a weighted sum of stable semantic consistency, appearance consistency, motion prediction consistency, and contextual consistency components. Due to the occlusion recovery phase, the weight of the contextual consistency component is increased. H1 consistently leads in cumulative score from F55 to F58, but H3 maintains a certain score in the short term to cover detected jitter. At F60, H1 maintains the highest score for six consecutive frames, and the difference between its score and the second-best hypothesis exceeds the score difference threshold rule in the policy library, satisfying the stabilization condition. The system outputs the trajectory corresponding to H1 as the final tracking result and cleans up low-scoring branches to keep the number of hypotheses below K.

[0108] During frames F55-F70, D1 and D2 move in parallel on the stage and have similar appearances. Step S4 introduces local topological relationships in cross-frame association: the relative distance and relative angle between candidate D1 and its neighboring target set (the first 3 neighboring candidates) are calculated, and the local topological consistency metric is calculated across frames. At F62, the relative distance between candidate D1 and neighboring candidate D2 is 0.85 m and the relative angle is 32.0°; at F63, the correspondence remains stable (distance 0.88 m, angle 34.0°), and the local topological consistency metric remains high; however, if mistakenly associated with D2, its relative relationship sequence is inconsistent with the historical trajectory state, the local topological consistency metric decreases and lowers the association score, thereby reducing erroneous switching between similar targets.

[0109] Under the shooting conditions of this embodiment, the camera experiences slight shaking. Step S4, in conjunction with global motion compensation, involves extracting background features in the background region and estimating global image transformation parameters for adjacent frames. The transformation model type is an affine transformation model. Motion prediction is performed in a virtual stable coordinate system, and the predicted position is then transformed back to the current frame coordinate system to complete the localization. When global motion compensation fails to achieve sufficient inlier consistency in individual frames, a failure state degradation process is triggered. The virtual stable coordinate system transformation is skipped, and motion prediction is performed in the current frame coordinate system. Simultaneously, a failure flag is written into the position information validity flag. After continuously satisfying the consistency judgment rules, the failure state is exited, and compensation is resumed.

[0110] Step S5 generates lighting equipment control commands based on the positioning results. The control signal generation module converts the location information into lighting control quantities and outputs a control frame data structure through the network interface. The control frame data structure includes a target identifier field, a control quantity field, a timestamp field, and a validity flag field. The output is: F14 (Active Motion, Normal): Target Identifier D1; Horizontal Angle 32.5°; Pitch Angle 18.2°; Beam Size 0.60°; Timestamp 12 04 66 ms; Validity Marked as Normal; F20 (Motion Freeze, Appearance Frozen): Target Identifier D1; Horizontal Angle 34.1; Pitch Angle 17.9; Beam Size 0.58; Timestamp 12 06 66 ms; Validity Marked as Normal; F40 (Missing Detection, Conservative Mode Prediction): Target Identifier D1; Horizontal Angle 35.6; Pitch Angle 17.4; Beam Size 0.58; Timestamp 12 13 33 ms; Validity is marked as prediction output; F62 (Occlusion Recovery, Multiple Hypothesis Convergence): Target Identifier D1; Horizontal Angle 38.9; Pitch Angle 16.8; Beam Size 0.62; Timestamp 12 20 66 ms; Validity Marked as Normal.

[0111] If communication is interrupted or transmission fails, the control quantity of the previous cycle will be used and the validity flag will be set to communication abnormal. After continuous successful transmissions reach the exit rule, the communication abnormal flag will be cleared and normal output will be restored.

[0112] Based on the above link operation process, it can be observed that: when strong reflections and freeze-frame actions cause instantaneous fluctuations in appearance information, the tracking feature update does not drift synchronously with appearance noise, and cross-frame association tends to maintain the same target trajectory state; when the object reappears after long-term occlusion and there are candidates with similar appearances, candidate space gating and signature matching constraints cause the candidate set to converge, and the cross-frame cumulative scoring of multiple hypotheses changes the decision from single-frame instantaneous judgment to multi-frame evidence accumulation, and the output trajectory switching is constrained by anti-shake conditions; under abnormal conditions such as missing measurements and compensation failure, conservative mode and degradation processing maintain continuous output of control frames and clearly distinguish between predicted output and normal output through validity marking, so that the follow-up control of the lighting equipment remains consistent.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0114] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this application and form different embodiments. For example, all the embodiments above can be used in any combination. The information disclosed in this background section is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

Claims

1. A method for automatic tracking of stage lighting, characterized in that, include: Step S1: Obtain a video sequence containing at least one stage target; The video sequence consists of video frames arranged in chronological order, each video frame having a frame timestamp; the frame timestamp serves as the alignment basis for cross-frame positioning and state determination; the position information is the two-dimensional image coordinates of the stage target in the corresponding video frame or the stage plane coordinates obtained through calibration mapping, and the position information carries the corresponding frame timestamp and validity mark, the validity mark being used to indicate that the position information is obtained by detection and association or by motion prediction; Step S2: Extract stable semantic feature vectors and appearance feature vectors of the stage target in parallel in the current frame; Step S3: Within the time window, determine whether the stage target is in an active motion state, a stationary motion state, or a missing state based on the position information. When in an active motion state, update and fuse the stable semantic feature vector with the appearance feature vector to generate tracking features; When the motion is stationary, reduce or freeze the update of the appearance feature vector and generate the tracking feature based on the stable semantic feature vector; Step S4: Perform cross-frame association and localization based on the tracking features; When a missing measurement state or association failure is detected, the tracking features from the previous cycle are used and the positioning result is output based on motion prediction. The update resumes after a preset number of frames N that continuously meet the normal state. The association failure is when the current frame cannot find a candidate target that meets the association constraints for the existing target trajectory state or the association score is lower than the association threshold rule. The normal state is a non-missing measurement state and a state in which no association failure has occurred. The motion prediction is a displacement extrapolation process based on the historical position information sequence and the frame timestamp interval, outputting the predicted position and marking it as the predicted output in the validity label. Step S5: Generate lighting equipment control commands based on the positioning results.

2. The method according to claim 1, characterized in that, The stable semantic feature vector is obtained by encoding the key points of the human body and their topological connections, and the appearance feature vector is obtained by encoding the appearance information of the image region of the stage target.

3. The method according to claim 1, characterized in that, The active motion state and the stagnant motion state are determined based on time window statistics of velocity, acceleration or key point displacement, and hysteresis rules with different entry and exit thresholds are adopted.

4. The method according to claim 1, characterized in that, The fusion weights are determined by the weight adjustment amount output by the mapping table, policy library, or learning model, and upper and lower limits are set for the weight change rate.

5. The method according to claim 1, characterized in that, The method further includes: Before determining that a stage target has entered an occlusion state, a spatiotemporal context signature of the stage target is generated and stored. The spatiotemporal context signature includes at least the disappearance position and the relative distance or relative direction to static landmarks in the scene. During the re-identification process after the occlusion ends, the location information of the candidate target is matched with the spatiotemporal context signature, and the matching result is used as an association constraint. The scene static markers in the spatiotemporal context signature are obtained by at least one of manually placed markers, fixed stage structure corner points, and background feature point set; the spatiotemporal context signature includes the vanishing position of the stage target, as well as the relative distance or relative direction between the vanishing position and at least two static markers, and may include the vanishing time and confidence level; During the re-identification process after the occlusion ends, candidate spatial gating regions are generated based on the spatiotemporal context signature to filter candidate targets, and the matching score between the candidate targets and the spatiotemporal context signature is used as a component of the cross-frame association score for association; when the static marker set changes beyond a preset threshold, the static markers are updated and the static marker set is reconstructed. The change is at least one of the following: a change in the number of visible markers in the static marker set, a change in the degree of marker position drift, or a change in the consistency of background feature point matching. The rules for determining the change are given by the configuration file or strategy library and remain consistent during operation.

6. The method according to claim 5, characterized in that, When there is long-term occlusion or interference from similar-looking targets, multiple related trajectory hypotheses are maintained in parallel, and the globally optimal trajectory is selected as the final tracking result based on the cumulative score across multiple video frames. The assumptions regarding the parallel maintenance of multiple associated trajectories include: In each video frame, multiple trajectory hypotheses are generated based on the detection candidates, and a maximum number of trajectory hypotheses are set, while a number of hypotheses with higher cumulative scores are retained. The cumulative score is obtained by summing the scores of stable semantic consistency, appearance consistency, motion prediction consistency, and matching constraint scores based on the spatiotemporal context signature. When the number of trajectory hypotheses exceeds the upper limit or the cumulative score difference of multiple trajectory hypotheses is less than the preset threshold, the trajectory hypotheses are pruned or merged; during the period of continuous missing tests, each trajectory hypothesis uses the tracking features of the previous cycle and outputs the position based on motion prediction, and resumes updating after a preset number of frames N of continuous normal state is met. When outputting the final tracking results, anti-shake conditions are set for trajectory switching, including continuous frame count conditions and score difference threshold rules.

7. The method according to claim 1, characterized in that, In the cross-frame association, when there are multiple stage targets with similar appearance features, the association metric is further based on local topological relationships, which include the relative distance and relative angle between the target and at least one other neighboring target.

8. The method according to claim 1, characterized in that, The method further includes global motion compensation: estimating global image transformation parameters between adjacent frames based on background features; performing motion prediction in a virtual stable coordinate system and outputting the predicted position; and transforming the predicted position back to the current frame coordinate system to complete the localization.

9. An automatic stage lighting tracking system, used to perform the method according to any one of claims 1 to 8, characterized in that, include: The video acquisition module is used to acquire video sequences containing at least one stage target; The target tracking processing module is used to detect and correlate the video sequence across frames to output the position information of the stage target; The target tracking processing module includes: a dual-path feature extraction unit, a motion state determination unit, and a gating update fusion unit; A control signal generation module is used to generate control commands for the lighting equipment based on the location information; The gated update fusion unit is used to update and fuse two types of feature vectors when in an active motion state, reduce or freeze the update of appearance feature vectors and fuse them with stable semantic feature vectors when in a stagnant motion state, enter a conservative mode when in a missing test state or when association fails to occur, use the fused features from the previous cycle and output the position based on motion prediction, and exit the conservative mode after continuously satisfying the preset number of frames N of the normal state.

10. The system according to claim 9, characterized in that, The control signal generation module converts the position information into at least one control quantity for the lamp and outputs it through a network or bus interface. The control quantity includes at least the horizontal angle, the pitch angle, or the beam size.

Images