A multi-station dual-spectrum camera networked cascade motion trajectory analysis method

Abstract

The application discloses a kind of multi dual-spectrum camera network cascading motion trajectory analysis methods, it is related to trajectory analysis technical field, for solving the technical problem of target cross lens trajectory tracking under complex scene;Through the networking cooperation of multiple dual-spectrum cameras, cross-view continuous tracking of target and motion trajectory reconstruction are realized, innovatively adopt discrete quality level to carry out the gating splicing of trajectory segment, ensure that trajectory fusion process is accurate and reliable;Conflict detection and evidence backtracking mechanism are introduced, the reliability and auditability of trajectory association are guaranteed, and commitment archiving and on-demand evidence collection process are designed to meet the actual monitoring evidence management requirements;The method can enable a reliable multi-camera cross-domain trajectory analysis system for the skilled person in the art, significantly improve the intelligence and reliability of monitoring target trajectory extraction and evidence management.

Description

Technical Field

[0001] This invention relates to the field of trajectory analysis technology, and more specifically, to a method for analyzing the motion trajectory of a network of multiple dual-spectrum cameras. Background Technology

[0002] In scenarios such as park security, port terminals, airport hubs, rail transit stations, tunnels, and large-scale warehousing and logistics, the passage of people and vehicles often crosses multiple entrances, passages, and functional areas. Management typically needs a multi-point video surveillance system to link the appearance, disappearance, and reappearance of the same target across different monitoring points, forming cross-regional motion trajectories for coordinated response, event tracing, and evidence preservation. To improve observability under conditions such as nighttime, backlighting, rain, fog, and smoke, engineering projects are gradually adopting dual-spectrum cameras combining visible light and infrared thermal imaging. These cameras are then networked via local area networks or dedicated networks to aggregate multiple video streams or trajectory metadata to a platform for analysis. Existing multi-camera trajectory analysis generally includes single-point target detection and tracking, as well as cross-point trajectory association and stitching. Cross-point association typically uses target appearance, spatiotemporal constraints, and point topology information to determine identity, and outputs the global trajectory on the platform. In practical deployments, cross-point association can be affected by differences in the acquisition and transmission links. For example, encoding buffers and network congestion introduce latency differences, and clock synchronization (such as NTP / PTP) may deviate when edge devices restart or the link jitters, leading to unstable timeline alignment of trajectories at different points. Simultaneously, dual-spectral channels are affected by imaging mechanisms and environmental thermal background, causing the appearance and thermal characteristics of targets to change at different points and under different environments, resulting in fluctuations in the consistency of cross-point representations. In areas with narrow passages, mixed pedestrian and vehicle traffic, or targets with similar appearances, short-term occlusion and parallel movement can further amplify candidate confusion. These factors, combined with high-density targets, frequent occlusion, or rapid passage scenarios, increase the number of candidate relationships and narrow the judgment boundary. The platform may experience trajectory jumps, duplicate identifications, or inconsistent associations, revealing insufficient consistency during subsequent review and verification. Meanwhile, engineering systems typically need to control the amount of data transmitted and meet real-time processing requirements, making it difficult to eliminate these uncertainties in the long term by transmitting the entire video or adding redundant computation.

[0003] To address the above problems, this invention proposes a solution. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a motion trajectory analysis method for a network of multiple dual-spectrum cameras, in order to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for motion trajectory analysis of multiple dual-spectrum cameras networked and cascaded, comprising: The trajectory analysis device receives trajectory segments uploaded by each dual-spectrum camera. Each trajectory segment corresponds to one appearance of the target under that camera and includes the target's local identifier, time span, trajectory coordinate sequence, and feature vector. It also selects several clear key frame images that can characterize the target's features from the trajectory segments and stores them in the segment record. The trajectory segment is subjected to discrete quality level determination, and a predefined discrete quality level label is assigned. The discrete quality level label includes high quality, medium quality and low quality. Based on the cross-camera association and stitching of trajectory segments, a phased matching strategy is adopted: First, the candidate set is narrowed down according to the spatiotemporal constraints. Then, the appearance feature similarity is calculated among the candidates that meet the spatiotemporal constraints. The target feature vector distance of the trajectory segments is compared. The target similarity score is obtained by combining appearance and spatiotemporal factors. The target similarity score is compared with a preset threshold. If the target similarity score is higher than the preset threshold, the corresponding trajectory segments are stitched together to generate a cross-camera trajectory. When executing the splicing decision, a segment quality level is introduced as a quality gating mechanism, wherein: if both trajectory segments to be spliced ​​are of high quality, they are merged based on a similarity threshold; if either trajectory segment is of low quality, they are not automatically spliced ​​and are marked as pending, allowing them to exist independently in the trajectory library without being merged with other trajectories; if the two trajectory segments are of medium quality or a combination of high and medium quality, the requirements are increased based on the matching threshold, or a manual verification step is introduced.

[0006] In a preferred embodiment, the monitoring scene is divided into multiple adjacent or overlapping sub-regions, and a dual-spectrum camera is deployed in each sub-region. The field of view coverage of each camera is cascaded according to the regional adjacency relationship. If there are overlapping areas in the field of view of adjacent cameras, the viewing angle calibration needs to be completed in advance, and the target position is uniformly represented in a common world coordinate system.

[0007] In a preferred embodiment, each camera is connected to the central device via a wired or wireless network, and the overlapping field-of-view cameras share some calibration points, mapping their respective coordinate systems onto a unified plane so that spatial position constraints can be used during subsequent trajectory stitching.

[0008] In a preferred embodiment, each trajectory segment corresponds to one appearance of the target under the camera, including the target's local identifier, time span, trajectory coordinate sequence and feature vector, and several clear key frame images that can characterize the target features are selected from the trajectory segment and stored in the segment record.

[0009] In a preferred embodiment, a target quality evaluation mechanism is incorporated during the tracking process to select several frames from the video stream whose target image quality scores are higher than a threshold as target key frames, and the N highest quality frames are saved together with the trajectory segment.

[0010] In a preferred embodiment, the quality score is calculated based on factors such as the sharpness, completeness, and orientation of the target in the image.

[0011] In a preferred embodiment, the discrete quality level determination includes evaluating the trajectory segment based on a combination of factors, including detection reliability, trajectory integrity, field of view, appearance feature stability, and bispectral consistency, and assigning a high quality level, a medium quality level, or a low quality level accordingly.

[0012] In a preferred embodiment, the system also includes trajectory conflict detection and processing. As the trajectory stitching of multiple cameras gradually forms a global trajectory set, the system continuously monitors potential trajectory conflicts. The trajectory conflicts include trajectory duplication or identity conflicts. After a conflict is detected, the system freezes further stitching of the relevant trajectories and enters the evidence verification process.

[0013] In a preferred embodiment, the trajectory conflict detection and processing includes using a grid indexing method to quickly retrieve conflicts in the trajectory, discretizing the global trajectory into a spatial grid sequence and attaching a timestamp, and using a target ID mapping table to track whether the same target ID is assigned to different trajectories.

[0014] In a preferred embodiment, the system further includes an evidence backtracking and conflict resolution mechanism. For discovered trajectory conflicts, an evidence backtracking procedure is initiated. This backtracking includes retrieving the original video, multi-algorithm verification, and manual intervention. Based on the backtracked evidence, a conflict resolution strategy is executed. If it is confirmed that two trajectories belong to the same object, they are merged and the global trajectory is updated. If it is confirmed that a segment from another party is mixed into a certain trajectory, the trajectory is split, and the erroneous segment is either assigned to a new trajectory or placed into the correct object. After adjustment, the system is verified again by conflict detection. The system further includes a trajectory commitment and archiving mechanism, as well as an on-demand evidence retrieval mechanism. Each confirmed global trajectory is assigned a unique identifier and a commitment operation is performed, writing it into the trajectory database for storage. Key evidence data is stored in the evidence repository by trajectory ID. Users can retrieve the corresponding global trajectory and its evidence archive in the trajectory database by querying the trajectory ID or target features, and extract associated evidence materials. An index directory is established in the evidence repository by trajectory ID and time, and retrieval operations are logged.

[0015] The technical effects and advantages of the motion trajectory analysis method for networking and cascading multiple dual-spectrum cameras proposed in this invention are as follows: This invention achieves continuous target tracking and motion trajectory reconstruction across multiple field-of-view cameras through networked collaboration. It innovatively employs discrete quality levels for gated stitching of trajectory segments, ensuring the accuracy and reliability of the trajectory fusion process; introduces conflict detection and evidence backtracking mechanisms to guarantee the reliability and auditability of trajectory association; and designs a committed archiving and on-demand evidence collection process to meet the requirements of actual surveillance evidence management. This method enables those skilled in the art to develop a reliable multi-camera cross-domain trajectory analysis system, significantly improving the intelligence and reliability of surveillance target trajectory extraction and evidence management. The various steps work together, focusing on trajectory processing, quality assessment, and trajectory stitching, comprehensively solving the technical problem of target trajectory tracking across multiple shots in complex scenarios. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the method flow in this embodiment, illustrating the relationship between the steps of the method for trajectory analysis in a multi-camera network cascade. Figure 2 This is a schematic diagram of the system composition structure in this embodiment. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0018] like Figure 1 As shown, this embodiment provides a motion trajectory analysis method for a network of multiple dual-spectrum cameras. This method focuses on multi-camera collaborative tracking and trajectory management. It improves all-weather monitoring capabilities through the fusion of visible light and infrared dual-sensor information; and achieves reliable cross-camera trajectory reconstruction through quality grading and stitching control of trajectory segments, avoiding trajectory errors and loss. The system structure is as follows: Figure 2 As shown, the system includes several dual-spectrum camera nodes distributed across the monitoring area, as well as a central trajectory analysis device or a distributed trajectory analysis module. Each camera node can acquire both high-definition visible light images and infrared thermal imaging sequences, and these multi-source video data are cascaded via network links to be sent to the trajectory analysis device for unified processing. The method flow of this embodiment is as follows: Step S101: Construct a monitoring area model for multi-camera network cascading. Divide the monitoring scene into multiple adjacent or overlapping sub-regions, and deploy one dual-spectrum camera in each sub-region. The field of view coverage of each camera forms a cascading model according to the regional adjacency relationship. For adjacent cameras with overlapping fields of view, the viewing angle calibration needs to be completed in advance, and the target position is uniformly represented in a common world coordinate system. For example, cameras are arranged sequentially along a road to form a chain coverage; for large-scale areas, the areas are divided into grids, and each area is covered by a dual-spectrum camera and cascaded with adjacent areas.

[0019] Furthermore, each camera connects to the central device via wired or wireless network, such as... Figure 2 As shown, this ensures continuous transmission of the video stream. Overlapping field-of-view cameras need to share some calibration points, mapping their respective coordinate systems to a unified plane so that spatial position constraints can be used during subsequent trajectory stitching. Technical effect: This networked regional cascade model guarantees global coverage of the monitoring area, establishes a relay relationship for target movement across regions, and helps solve the problem of trajectory loss after a target goes out of the frame under the limited field of view of a single camera.

[0020] Step S102: Dual-spectrum camera nodes acquire and preprocess video streams. Each camera synchronously acquires visible light and infrared video sequences, performs time-axis alignment and pixel-level registration, and generates strictly synchronized dual-channel video frames. For each frame, a dual-channel information fusion target detection algorithm is used: heat source targets are detected in the infrared channel to obtain preliminary target positions; visual target details are detected in the visible light channel to obtain high-precision bounding boxes. The detection results of the two channels are fused and matched to remove false alarms from a single channel: for example, if a heat source is detected by infrared at the same spatial location but there is no corresponding target in visible light, it can be determined as a false target and removed; conversely, if a target detected by visible light has no thermal response in infrared, its existence can be confirmed by infrared under low light conditions to reduce missed detections. The fused output includes the detection boxes, categories, and initial confidence scores of all moving targets in the monitoring screen. Furthermore, a deep learning model can be used to achieve dual-spectrum feature fusion, such as extracting features from infrared and visible light images through convolution and then fusing them in the network, or a decision-level fusion strategy can be used to dynamically select the best based on environmental conditions. In scenarios such as dense fog and nighttime, the system can automatically increase the weight of the infrared channel to see through obstacles and capture targets; in well-lit conditions, it focuses on visible light detail recognition, thus achieving stable detection in all weather conditions. Technical benefits: Through dual-spectral information fusion, the camera achieves target detection both day and night, enhancing the accuracy and robustness of moving target detection in complex environments. Simultaneously, the complementary use of the two sensors improves the detection rate of small or occluded targets.

[0021] Furthermore, to maintain the usability of subsequent trajectory segment construction and cross-camera stitching in environments such as rain, fog, smoke, low light, and strong backlight, dual-spectral consistency and temperature feature characterization are introduced in the preprocessing stage of step S102. Let the target detection box output by the visible light channel be Bv, and the target detection box output by the infrared channel be Bir. Dual-spectral consistency Rb is used to characterize the spatial consistency of the two channels for the same target.

[0022] In one implementation, Rb is taken as the intersection-union ratio (IoU) of Bv and Bir, where IoU is equal to the intersection area of ​​Bv and Bir divided by the union area of ​​Bv and Bir.

[0023] In another implementation, Rb is taken as an exponential decay form, where Rb is equal to exp(-d) divided by σ, where d is the Euclidean distance between the center point cv of Bv and the center point cir of Bir, cv and cir are taken from the center point coordinates in the same coordinate system, and σ is the calibration parameter.

[0024] Furthermore, infrared temperature features μir are extracted within the Bir coverage area. μir represents the mean or median pixel temperature values ​​within Bir, forming a temperature difference curve Δμirt. Δμirt equals the value of μir at time t minus the value of μir at time t minus Δt, where Δt is the time interval between adjacent sampling times. The variance or root mean square of Δμirt within a preset time window is used as a volatility index, with volatility not exceeding a threshold τΔ as a stability criterion. Since fog and rain cause scattering and absorption, and smoke and dust lead to decreased visible light contrast and obstruction, infrared thermal imaging can provide supplementary observations in such scenarios. However, infrared resolution may decrease when the temperature difference between the target and the background is close. Therefore, Rb not lower than the threshold τrb and Δμirt satisfying the stability criterion are used as common criteria for subsequent keyframe selection, discrete quality level determination, and quality gating mechanisms.

[0025] Step S103: Perform multi-target tracking on the detected targets within a single camera to construct motion trajectory segments. For each target detected in step S102, execute a tracking algorithm based on motion and appearance features in the video sequence of that camera, such as Kalman filter prediction + Hungarian algorithm matching or depth SORT algorithm, to achieve inter-frame correlation tracking of the target. Matching can be performed using the comprehensive similarity of the target's motion consistency and dual-spectral appearance features in consecutive frames. Motion consistency and dual-spectral appearance features include visible light color texture and infrared thermal features to generate stable trajectory segments. Each trajectory segment corresponds to one appearance of the target under that camera, including the target's local identifier (such as camera ID + target ID), time span (start and end frames of appearance), trajectory coordinate sequence (converted to world coordinate system), and feature vector (fusion of visible light and infrared multi-dimensional appearance features), etc. Simultaneously, select several clear key frame images that can characterize the target features from the trajectory segments and store them in the segment record. Furthermore, it is preferable to combine a target quality evaluation mechanism during the tracking process to select several frames with target image quality scores higher than a threshold from the video stream as target key frames. The quality score is calculated based on factors such as the target's sharpness, completeness, and orientation in the image. For example, a clear, frontal target (face / license plate frame) can be assigned a high-quality rating. The highest-quality N frames are saved along with the trajectory fragment for reference during subsequent recognition and stitching. Technical benefits: Through multi-target tracking, each camera integrates the motion trajectory of the same target in continuous video, outputting a trajectory fragment of appropriate length, significantly reducing the computational load of cross-camera correlation. The introduction of a quality screening mechanism ensures the high reliability of the extracted appearance features, improving the accuracy of cross-camera matching. Furthermore, dual-light fusion feature description improves the stability of tracking under challenging conditions such as occlusion and backlighting.

[0026] Furthermore, to ensure the keyframes are verifiable and interpretable under dual-spectral conditions, step S103 introduces dual-spectral mutual verification constraints during quality score calculation and keyframe selection. Let the keyframe quality score be Iq, which is determined by sharpness Iclr, integrity Icmp, orientation Iori, occlusion degree Iocc, and infrared stability Istb. Istb is equal to exp(-Varμir) divided by β, where Varμir is the variance of the infrared temperature characteristic μir within the preset time window of the candidate keyframe, the preset time window length is K consecutive frames or duration Tw, and β is a calibration parameter. Keyframe selection uses a threshold-based rule: first, Rb must be no less than the threshold τrb and Istb must be no less than the threshold τstb. Then, N frames are selected as keyframes from the set of frames that meet the conditions, ranked from highest to lowest Iq. Frames that do not meet the Rb threshold or the Istb threshold are not considered keyframes.

[0027] Step S104: Determine the discrete quality level of the trajectory segments. After receiving the trajectory segments uploaded by each camera, the trajectory analysis device first evaluates the quality based on the monitoring data and tracking performance within the segments, and assigns a predefined discrete quality level label.

[0028] It should be noted that quality assessment considers the following factors: detection reliability, the average confidence level of target detection within the segment, the presence of false positives and false negatives, trajectory integrity, whether there are instances of long-term loss and recapture within the segment span, the smoothness and continuity of the trajectory, field of view, whether the target disappears at the edge of the field of view at the end of the segment, suggesting it may continue to move to other cameras or disappearing midway (lower quality), appearance feature stability, consistency of keyframe image quality within the segment, whether drastic changes in illumination cause unreliable features, dual-spectral consistency, and consistency between visible and infrared channel observations of the target, such as the overlap between heat source and visible target locations. If all the above indicators are good, the trajectory segment is marked as high quality; if there are slight losses or feature blurring, it is classified as medium quality (B); if there are obvious defects such as trajectory interruption or false positives, it is classified as low quality (C). The quality level is a discrete category rather than a continuous score to avoid the algorithm being overly sensitive to subtle score fluctuations, thus simplifying the decision threshold setting. Technically, after quality level assessment, each trajectory segment is assigned an intuitive confidence label, facilitating conditional selection and decision-making during subsequent trajectory stitching. Especially in multi-camera trajectory association, unreliable low-quality segments can be temporarily excluded from automatic stitching to reduce the risk of incorrect association, while high-quality segments can be prioritized for stitching to form a backbone trajectory and improve the reliability of the global trajectory.

[0029] Furthermore, to ensure the discrete quality level determination can be reproducibly applied in bispectral scenarios, step S104 explicitly incorporates the stability of bispectral consistency Rb and temperature difference curve Δμirt into the level mapping rule. The detection reliability index Rd is set to the mean of target detection confidence within a segment, combined with the proportion of low-confidence frames ρlow; Rd is calculated as the mean detection confidence multiplied by 1 minus ρlow. The trajectory integrity index Rc is set to 1 minus the ratio of the segment's loss duration Tmiss to the total segment duration Tlen. The field-of-view index Rv is discretely assigned based on the disappearance type; Rv takes a high value when the target disappears at the end of the segment and the disappearance location is within a preset edge region of the field of view, and a low value when the target disappears in the middle of the segment and in a non-edge region. The appearance feature stability index Rs is a stability measure obtained by exponentially decaying the variance of the keyframe appearance features within the segment; Rs is equal to exp(-VarF) divided by β, where VarF is the variance of the appearance features within the segment, and β is a calibration parameter. The dual-spectral consistency index Rb is taken as the average value of IoUBvBir corresponding to each keyframe in the segment, and combined with the fluctuation of Δμirt within a preset time window. The fluctuation is considered to be stable if it is not higher than the threshold τΔ.

[0030] Discrete quality level labels are determined using a threshold chain rule. First, Rb must be no less than the threshold τrb and Δμirt must satisfy the stability criterion. Then, Rd must be no less than the threshold τd and Rc must be no less than the threshold τc. When the aforementioned conditions are met, Rv is high, and Rs is no less than the threshold τs, it is labeled as high quality level A. When the Rb and Δμirt criteria are met, and only one of Rd, Rc, or Rs is slightly below the corresponding threshold, and there are no trajectory interruptions or false detections, it is labeled as medium quality level B. When the Rb threshold is not met, the Δμirt stability criterion is not met, or there are trajectory interruptions or false detections, it is labeled as low quality level C. This constraint avoids masking the risk of mis-splicing due to inconsistencies in the dual spectra with continuous scores, while maintaining the simplicity of the discrete quality level label decision.

[0031] Step S105: Cross-camera association and stitching based on trajectory segments. For newly introduced target trajectory segments, the trajectory analysis device attempts to stitch them onto existing cross-camera global trajectories or segments from other cameras to achieve cascaded reconstruction of the target trajectory. The association and stitching process involves two cases: If segment A has the same local identifier as an existing historical trajectory (i.e., from the same camera and with the same ID), it is considered a reappearance of the target in the original camera, and segment A is directly appended to the updated global trajectory. Otherwise (no directly matching historical trajectory ID), segment B belonging to the same target as segment A needs to be searched in the trajectory segment set of other cameras.

[0032] Furthermore, to improve the accuracy of the association, a phased matching strategy is adopted: First, the candidate set is narrowed down based on spatiotemporal constraints. For example, the end time and spatial location of segment A should be consistent with or within a reasonable range of the start time and location of segment B (if the fields of view of the two cameras overlap, the same target is required to appear in the overlapping area; if there is no overlap, the temporal sequence and geographical proximity of the two are considered). Among the candidates that meet the spatiotemporal constraints, the similarity of appearance features is calculated, and the distance between the target feature vectors of segment A and segment B is compared. A target similarity score is obtained by combining appearance and spatiotemporal factors. For camera pairs with overlapping fields of view, the consistency of spatial location is given priority, and spatiotemporal similarity can be given a higher weight; while for camera pairs without direct overlapping areas, the weight of appearance features is increased to compensate for insufficient spatiotemporal information. If the target similarity between segment A and a certain segment B is higher than a preset threshold, it is determined that the two belong to the same actual object, and segment B is spliced ​​with segment A to generate the cross-camera trajectory (i.e., global trajectory) of the object. The splicing operation includes: merging the spatiotemporal sequences of the two trajectories to make the target motion path continuous; integrating the keyframes and features of the two segments to improve the information completeness of the global trajectory. Technical Results: The cross-camera association strategy described above enables automatic stitching of the same target's trajectory across different cameras. Utilizing a combination of spatiotemporal constraints and appearance features in the matching process effectively eliminates impossible associations (such as non-overlapping temporal segments), reducing false matching rates and improving association accuracy. The progressively generated global trajectory comprehensively depicts the target's movement path across each camera area, providing foundational data for subsequent behavior analysis and investigation.

[0033] Furthermore, to reflect the unique contribution of infrared thermal imaging to trajectory reconstruction in rain, fog, and dust-covered scenarios, step S105 introduces dual-spectral consistency and temperature feature consistency constraints when calculating the target similarity score. Let the appearance feature similarity Sa be the cosine similarity of the appearance feature vectors of segment A and segment B. Let the spatiotemporal similarity be jointly determined by the consistency of the transfer time window St and the consistency of the transfer distance in the world coordinate system Sp, where St is taken as 1 minus the ratio of the time difference between the two segments switching to the preset maximum transfer time window Tmax, and Sp is taken as 1 minus the ratio of the distance between the two segments switching positions to the preset maximum transfer distance Dmax. Both St and Sp are limited to the range of 0 to 1, and Sst is taken as the smaller value between St and Sp to avoid erroneous association caused by an excessively high value for one item.

[0034] Cross-camera association determination employs a conditional chain rule, first requiring Sa to be no less than the threshold τa, and then requiring Sst to be no less than the threshold τst. Subsequently, keyframe alignment is performed between the end keyframe of segment A and the beginning keyframe of segment B. This alignment selects keyframe pairs as alignment pairs based on the principle of temporal proximity or spatial proximity. Rb is calculated on the alignment pair, and Rb is required to be no less than the threshold τrb. Further, the infrared temperature feature μir corresponding to the alignment pair forms a temperature difference curve Δμirt before and after the cross-lens switch. The consistency criterion for temperature feature is that the mean difference of Δμirt within a preset time window before and after the switch does not exceed the threshold τμ and the fluctuation difference does not exceed the threshold τΔ. For segments where low contrast or occlusion on the visible light side causes keyframe instability, under the premise of satisfying Sa and Sst, the consistency criterion of Rb and temperature features is prioritized as the splicing verification condition, ensuring the reliability of splicing determination even when visible light is affected by fog, rain, or dust.

[0035] Step S106: Quality Gating Mechanism for Trajectory Stitching. To ensure the reliability of the global trajectory, segment quality level is introduced as a gating condition when executing the stitching decision in S105. When the trajectory segments involved in the candidate stitching are of low quality, the system will adopt a stricter or more conservative strategy. Furthermore, the following gating rules are preset: 1) High-quality segments pass directly: If both trajectory segments to be stitched are of high quality (Level A), they can be directly merged based on the similarity threshold without additional approval, because high-quality segments have a low probability of mismatch and the stitching is reliable. 2) Low-quality segments are handled slowly: If any segment is of low quality (Level C), it will not be automatically stitched. The system marks the segment as pending, allowing it to exist independently in the trajectory library without being merged with other trajectories, until new evidence improves its credibility or it is confirmed by manual review. Similarly, if the global trajectory contains low-quality segments, it will not be immediately used for cross-regional association output to users to avoid spreading uncertain information. 3) Medium-Quality Gating Requires Assistance: If two segments are of medium quality (Grade B), or one is a high-quality segment and the other a medium-quality segment, the requirements can be increased based on the matching threshold, or a manual verification step can be introduced. For example, the similarity can be required to be significantly higher than the usual threshold, or the keyframe images of the matching segment can be reviewed by the on-duty personnel before confirming the stitching. Technical Effect: Through the above discrete quality level gating, the system automatically blocks unreliable data during the trajectory stitching process. On the one hand, it ensures that high-quality trajectories can be stitched quickly, giving full play to the system's real-time tracking capabilities; on the other hand, it isolates low-quality segments to prevent them from interfering with the accuracy of the global trajectory. This non-linear gating mechanism is more flexible and secure than simple threshold scoring, ensuring that the final output global trajectory is composed of reliable segments, thus improving the overall quality.

[0036] Furthermore, to make the quality gating mechanism more sensitive to rain, fog, and dust obstruction in dual-spectral scenarios, step S106 introduces stability conditions of Rb and Δμirt in the gating threshold and the positive conversion rule for undetermined segments. Let the gating threshold be θg.

[0037] When the segment to be spliced ​​is of high quality level A and Rb is not lower than the threshold τrb and Δμirt satisfies the stability criterion, θg takes the basic threshold θ0.

[0038] When there is a medium-level B or Rb below the threshold τrb or Δμirt in the segment to be spliced, θg is taken as an increased threshold higher than θ0, and the key frames involved in the splicing judgment are required to satisfy the Rb threshold and Δμirt stability criteria.

[0039] When any segment is of low quality level C, it is directly marked as pending and exists independently in the trajectory library first.

[0040] The conversion rule for pending segments is as follows: when a pending segment and at least two high-quality A-level segments satisfy spatiotemporal continuity and satisfy dual-spectral consistency and temperature characteristic consistency, it becomes a candidate for splicing. The dual-spectral consistency is based on the Rb mean difference not exceeding the threshold τrb difference, and the temperature characteristic consistency is based on the Δμirt mean difference not exceeding the threshold τμ and the fluctuation difference not exceeding the threshold τΔ. Alternatively, after evidence backtracking and confirmation in step S108, it is converted into a splicing decision, thereby avoiding premature incorporation of uncertain segments into the global trajectory when visible light is affected by fog, rain, smoke and dust.

[0041] Step S107: Trajectory Conflict Detection and Handling. As multi-camera trajectory stitching gradually forms a global trajectory set, the system continuously monitors for potential trajectory conflicts. Trajectory conflict refers to the repetition of what should actually be the same object in two global trajectories, or the presence of contradictory segments in a single trajectory. Common scenarios include: (a) Trajectory duplication: Due to the strategy of delaying stitching, the same target may form two parallel global trajectories, for example, low-quality segments may not be stitched in time, resulting in duplication in the trajectory library. (b) Identity conflict: A global trajectory may incorrectly stitch segments from different objects, resulting in unreasonable motion jumps. The system detects conflicts by comparing the consistency inside and outside the global trajectory: For each global trajectory, the system checks the continuity and rationality of the trajectory in time and space. If there is a time reversal or a sudden large shift in spatial position, the trajectory is marked as having an internal conflict; for different global trajectories, the system compares their target appearance features and motion areas. If they are highly similar and overlap in time, it may be that the same object has been divided into two tracks, which is then marked as a duplicate conflict. Furthermore, a grid index method can be used for rapid trajectory conflict retrieval. For example, global trajectories are discretized into a spatial grid sequence and timestamped. If two trajectories are found to have objects appearing on a certain grid at the same time, a conflict is determined. Furthermore, a target ID mapping table is used to track whether the same target ID is assigned to different trajectories. Upon detecting a conflict, the system freezes further stitching of the relevant trajectories and initiates an evidence verification process. Technical benefits: The trajectory conflict detection mechanism can promptly detect errors or duplications in cross-camera associations, enabling the system to eliminate potential problems before outputting the final result. Through conflict marking and interception, the consistency and reliability of trajectory data submitted to users or archived are ensured.

[0042] Step S108: Evidence Retrospection and Conflict Resolution Mechanism. Regarding the trajectory conflict discovered in step S107, the system initiates an evidence retrospection procedure to conduct in-depth analysis and comparison of the trajectories and segments involved in the conflict, in order to confirm the actual situation and correct the trajectory. Furthermore, evidence retrospection includes: 1) Original Video Retrieval: Automatically retrieving original video clips or keyframes from each camera within the relevant time period of the conflict trajectory. For example, for two trajectories suspected to belong to the same person, extracting screenshots from each camera for comparison. 2) Multi-Algorithm Verification: Re-running more refined or different model recognition algorithms on the conflict segments, such as face recognition, clothing color analysis, or vehicle detail comparison, to obtain additional corroborating information. 3) Manual Intervention: Submitting the conflict situation to monitoring center personnel for manual comparison of relevant video evidence to confirm whether it is the same person / vehicle. Based on backtracking evidence, the system executes a conflict resolution strategy: if two trajectories are confirmed to belong to the same object, they are merged (the shorter or lower-quality one is incorporated into the other), and the global trajectory is updated; if it is confirmed that a segment from another object is mixed into a trajectory, the trajectory is split, and the erroneous segment is either assigned to a new trajectory or the correct object. After adjustment, a conflict detection check is performed again to verify that there are no errors. Technical effect: By using evidence backtracking, the system introduces a reconfirmation step, avoiding making final decisions based solely on the initial algorithm results, thereby greatly improving the accuracy of trajectory results. The image evidence retained during the backtracking process also provides a basis for post-event auditing. Once the conflict is resolved, the system can output verified global trajectory data for business use, ensuring the authenticity and reliability of the trajectory analysis results.

[0043] Furthermore, to highlight the evidentiary value of dual-spectrum imaging in the conflict verification stage, step S108 simultaneously retrieves visible light keyframe images and infrared keyframe images during evidence retrospection, and extracts the Rb sequence and temperature difference curve Δμirt from the keyframe sequence corresponding to the conflict fragment for verification. The Rb sequence is a sequence of Rb values ​​calculated pairwise by aligning keyframe pairs, and the Δμirt is taken from the infrared temperature feature μir corresponding to the aligned keyframe pair and calculated according to the sampling interval Δt. For conflict samples where visible light is obscured by fog, rain, smoke, or dust, resulting in unstable appearance comparison, the following criteria are preferentially used as auxiliary evidence: the mean of the Rb sequence is not lower than the threshold τrb and the variance of the Rb sequence is not higher than the fluctuation of the threshold τrb, and the mean difference of Δμirt within the preset time window before and after the switch does not exceed the threshold τμ and the fluctuation difference does not exceed the threshold τΔ. The visible light keyframe index, infrared keyframe index, Rb sequence, and Δμirt used for verification are recorded together as part of the supporting data, making the conflict resolution conclusion traceable.

[0044] Step S109: Trajectory Commitment and Archiving. After the global trajectory is stitched together and conflicts are resolved, the system assigns a unique identifier to each confirmed global trajectory and performs a commitment operation, locking the trajectory so it will not change again and storing it in the trajectory database. Furthermore, the committed trajectory record includes: the global trajectory ID, the involved camera sequence, the time route, the complete trajectory coordinate chain of the target's cross-camera movement, and the target's aggregated features (the total features obtained by fusing multiple appearance feature vectors). Simultaneously, key evidence data accompanying the trajectory is archived: including high-quality keyframe images of each trajectory segment, related video segment indexes, and supporting materials extracted during evidence backtracking. All evidence files are stored in the evidence repository according to the trajectory ID for future reference. For committed trajectories, the system will no longer participate in online matching calculations to reduce resource consumption; however, if the same target re-enters the monitoring range, its historical trajectory identity can be quickly identified by matching its features with the committed trajectory features, achieving trajectory relay updates. Technical effect: Trajectory commitment solidifies valid trajectories, ensuring that no subsequent operations can modify the confirmed trajectory results, meeting the requirements of evidence rigor. Archived trajectory evidence provides full-link data from detection to tracking and association, presenting a complete chain of evidence, which is convenient for public security evidence collection or post-event analysis.

[0045] Furthermore, to solidify and preserve the dual-spectral differential features during the evidence preservation stage, step S109, during archiving and evidence preservation, associates the visible light keyframe image, the infrared keyframe image, and the temperature difference curve Δμirt with the trajectory ID and stores them in the evidence database. The Rb sequence is also stored as verification data for dual-spectral consistency; the Rb sequence is a sequence of Rb values ​​calculated pairwise from aligned keyframe pairs. Optionally, a hash digest is generated for the set of visible light keyframe images, the set of infrared keyframe images, the Rb sequence, and Δμirt corresponding to the same trajectory ID. This hash digest is written into the trajectory record field of the trajectory database for integrity verification when exporting evidence packages or retrieving evidence materials as needed. This ensures that the verifiable contribution of infrared thermal imaging is solidified into a verifiable chain of evidence even in rain, fog, or dust-covered scenarios.

[0046] Step S110: On-Demand Evidence Collection and Access Mechanism. This embodiment provides a user-friendly evidence collection interface, allowing for the on-demand retrieval of trajectory evidence for specific targets when there is an alarm event or law enforcement need. Users can search the trajectory database for the corresponding global trajectory and its evidence files by querying the trajectory ID or target characteristics. The system will quickly locate the panoramic trajectory of the target's movement across cameras and extract associated evidence materials, including keyframe images and video clips from each camera. The evidence can be integrated and played back along a timeline, displaying a series of videos of the target's movement under different cameras. If it is necessary to fix the evidence, the evidence package of the trajectory can be exported with one click, containing all key images and video clips and time stamps, ensuring the integrity and reliability of the evidence chain. Furthermore, a distributed index is used to improve retrieval speed. An index directory is established in the evidence database by trajectory ID and time, supporting multi-condition combined queries. Access operations for sensitive evidence will be logged to meet regulatory audit requirements. Technical Effect: The on-demand evidence collection mechanism ensures that after an event occurs, the data from relevant cameras can be quickly located and extracted, forming a clear chain of evidence, providing strong support for public security or management. In addition, since only key evidence is stored normally rather than the entire video recording, this mechanism reduces the burden on system storage and retrieval, while allowing detailed information to be retrieved when needed, thus achieving a balance between storage efficiency and evidence collection effectiveness.

[0047] In summary, this embodiment achieves continuous target tracking and motion trajectory reconstruction across the field of view through the collaborative networking of multiple dual-spectrum cameras. It innovatively employs discrete quality levels for gated stitching of trajectory segments to ensure the accuracy and reliability of the trajectory fusion process; introduces conflict detection and evidence backtracking mechanisms to guarantee the reliability and auditability of trajectory association; and designs a committed archiving and on-demand evidence collection process to meet the requirements of actual monitoring evidence management. This method enables those skilled in the art to implement a reliable multi-camera cross-domain trajectory analysis system, significantly improving the intelligence and reliability of monitoring target trajectory extraction and evidence management. The various steps work together, focusing on trajectory processing, quality judgment, and trajectory stitching, comprehensively solving the technical problem of target cross-camera trajectory tracking in complex scenes, with significant technical effects. The above embodiments are merely preferred examples of the present invention. Those skilled in the art can adjust and modify the order and details of the steps according to actual needs. As long as the spirit of the technical solution of this application is applied, it should fall within the scope of protection claimed by the present invention.

[0048] It should be noted that, for the sake of brevity, the foregoing method embodiments are described as a series of actions, but this does not mean that the application limits the order of the steps. Based on the ideas of this application, some steps can be executed in different orders or in parallel without affecting the functional implementation. Secondly, those skilled in the art should also understand that the specific embodiments described in the specification are preferred embodiments of the technical solutions of this application, and not limitations on the scope of protection of this application. All equivalent improvements or substitutions made within the spirit and principles of this application should be covered within the scope of protection of this application.

[0049] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for motion trajectory analysis of multiple dual-spectrum cameras networked and cascaded, characterized in that, The system includes a trajectory analysis device that receives trajectory segments uploaded by each dual-spectrum camera. Each trajectory segment corresponds to one appearance of the target under the camera and includes the target's local identifier, time span, trajectory coordinate sequence, and feature vector. The system also selects several clear key frame images that can characterize the target's features from the trajectory segments and stores them in the segment record. The trajectory segment is subjected to discrete quality level determination and assigned a predefined discrete level label, which includes high quality, medium quality and low quality. The discrete quality level determination is based at least on the stability of the bispectral consistency Rb and the temperature difference curve Δμir. The bispectral consistency Rb is obtained by the visible light detection box Bv and the infrared detection box Bir. The temperature difference curve Δμir is obtained by the temperature feature μir extracted within the infrared detection box Bir. Based on the cross-camera association and stitching of trajectory segments, a phased matching strategy is adopted. First, the candidate set is narrowed down according to the spatiotemporal constraints. Then, the appearance feature similarity is calculated among the candidates that meet the spatiotemporal constraints. The target feature vector distance of the trajectory segments is compared. The target similarity score is obtained by combining appearance and spatiotemporal factors. The target similarity score is compared with a preset threshold. If the target similarity score is higher than the preset threshold, the corresponding trajectory segments are stitched together to generate a cross-camera trajectory. When executing the splicing decision, a segment quality level is introduced as a quality gating mechanism. If both trajectory segments to be spliced ​​are of high quality, they are merged based on a similarity threshold. If either trajectory segment is of low quality, they are not automatically spliced ​​and are marked as pending, allowing them to exist independently in the trajectory library without being merged with other trajectories. If the two trajectory segments are of medium quality or a combination of one high and one medium quality, the requirements are increased based on the matching threshold, or a manual verification step is introduced. When the requirements are increased or a manual verification step is introduced, splicing eligibility is restricted based on the bispectral consistency Rb threshold and the temperature difference curve Δμir fluctuation threshold.

2. The motion trajectory analysis method for a network of multiple dual-spectrum cameras according to claim 1, characterized in that: The monitoring scene is divided into multiple adjacent or overlapping sub-regions. A dual-spectrum camera is deployed in each sub-region. The field of view coverage of each camera is cascaded according to the regional adjacency relationship. If there are overlapping areas in the field of view of adjacent cameras, the viewing angle calibration needs to be completed in advance, and the target position is uniformly represented in a common world coordinate system.

3. The motion trajectory analysis method for a network of multiple dual-spectrum cameras according to claim 2, characterized in that: Each camera is connected to the central device via a wired or wireless network. Overlapping field-of-view cameras share some calibration points, mapping their respective coordinate systems onto a unified plane so that spatial position constraints can be used during subsequent trajectory stitching.

4. The motion trajectory analysis method for a network of multiple dual-spectrum cameras according to claim 1, characterized in that: Each trajectory segment corresponds to one appearance of the target under the camera, including the target's local identifier, time span, trajectory coordinate sequence, and feature vector. Several clear keyframe images that can characterize the target's features are selected from the trajectory segment and stored in the segment record. The keyframe images include at least visible light keyframe images and infrared keyframe images. The visible light keyframe images generate a visible light detection box Bv, and the infrared keyframe images generate an infrared detection box Bir.

5. The motion trajectory analysis method for a networked cascade of multiple dual-spectrum cameras according to claim 4, characterized in that: During the tracking process, a target quality evaluation mechanism is combined to select several frames with target image quality scores higher than a threshold from the video stream as target key frames, and save the N highest quality frames along with the trajectory segment. The target image quality score includes at least an infrared temperature feature stability index, which is determined by the variance of the temperature feature μir.

6. The motion trajectory analysis method for a networked cascade of multiple dual-spectrum cameras according to claim 5, characterized in that; The target image quality score is calculated based on factors such as the sharpness, completeness, and orientation of the target in the image, and the stability of the infrared channel is constrained based on the variance of the temperature feature μir.

7. The motion trajectory analysis method for a networked cascade of multiple dual-spectrum cameras according to claim 1, characterized in that; The discrete quality level determination includes evaluating the trajectory segment based on the following factors: detection reliability, trajectory integrity, field of view, appearance feature stability, and dual-spectral consistency. Based on these factors, a high-quality, medium-quality, or low-quality level is assigned. The dual-spectral consistency includes dual-spectral consistency Rb, which is the intersection-union ratio (IU / I) of the visible light detection frame Bv and the infrared detection frame Bir, or the similarity obtained by applying an exponential decay function to the distance between the center point cv of the visible light detection frame Bv and the center point cir of the infrared detection frame Bir. Temperature features μir are extracted within the infrared detection frame Bir to form a temperature difference curve Δμir. The temperature difference curve Δμir at time t is the current temperature feature μir minus the temperature feature μir at the previous sampling interval Δt. The mean of the dual-spectral consistency Rb and the fluctuation of the temperature difference curve Δμir are used as at least one set of criteria for determining the discrete quality level.

8. The motion trajectory analysis method for a networked cascade of multiple dual-spectrum cameras according to claim 1, characterized in that; It also includes trajectory conflict detection and handling. When the trajectory stitching of multiple cameras gradually forms a global trajectory set, the system continuously monitors potential trajectory conflicts. The trajectory conflicts include trajectory duplication or identity conflict. After a conflict is detected, the system freezes further stitching of the relevant trajectory and enters the evidence verification process.

9. The motion trajectory analysis method for a networked cascade of multiple dual-spectrum cameras according to claim 8, characterized in that; The trajectory conflict detection and processing includes using a grid indexing method to quickly retrieve conflicts in the trajectory, discretizing the global trajectory into a spatial grid sequence and attaching a timestamp, and using a target ID mapping table to track whether the same target ID is assigned to different trajectories.

10. The motion trajectory analysis method for a networked cascade of multiple dual-spectrum cameras according to claim 9, characterized in that; It also includes evidence backtracking and conflict resolution mechanisms. For discovered trajectory conflicts, an evidence backtracking procedure is initiated. This backtracking includes retrieving original video, multi-algorithm verification, and manual intervention. Based on the backtracked evidence, a conflict resolution strategy is executed. If it is confirmed that two trajectories belong to the same object, they are merged and the global trajectory is updated. If it is confirmed that a segment from another source is mixed into a trajectory, the trajectory is split, and the erroneous segment is either assigned to a new trajectory or placed into the correct object. After adjustment, a conflict detection verification is performed again to confirm that there are no errors. Furthermore, it includes trajectory commitment, archiving and evidence preservation, and on-demand evidence retrieval mechanisms. Each confirmed global trajectory is assigned a unique identifier and a commitment operation is performed, writing it into the trajectory database for storage. Key evidence data is stored in the evidence repository by trajectory ID. Users can retrieve the corresponding global trajectory and its evidence file in the trajectory database by querying the trajectory ID or target feature, and extract the associated evidence material. The evidence repository is indexed by trajectory ID and time, and the retrieval operation is logged. The evidence retrieval simultaneously retrieves visible light keyframe images and infrared keyframe images, and extracts the dual-spectral consistency Rb sequence and temperature difference curve Δμir for verification. When the trajectory is committed and archived, the visible light keyframe images, infrared keyframe images and temperature difference curve Δμir are stored in the evidence repository by trajectory ID, and a summary hash is generated and written to the trajectory database.

Images