A wharf cross-camera multi-target tracking method and system based on a three-dimensional map

Abstract

The application discloses a wharf cross-camera multi-target tracking method and system based on a three-dimensional map, and belongs to the technical field of intelligent monitoring. The method performs voxelization processing on a three-dimensional model of a wharf, and performs color and texture feature matching on a two-dimensional image in a real-time monitoring video, so that a detection target is accurately projected to a three-dimensional space. In a single camera, an optimized ByteTrack algorithm is combined with Kalman filtering to perform continuous tracking. When a target crosses different cameras, a cost function which fuses a re-identification (ReID) feature, a three-dimensional space distance, a speed constraint and a target appearance frequency weight is used to perform high-robustness identity matching. The application significantly improves target positioning accuracy and cross-camera tracking stability by deeply fusing three-dimensional geographic information and visual analysis, and ensures physical authenticity of a tracking trajectory, and is especially suitable for complex scenes such as wharfs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent monitoring, in particular to a wharf cross-camera multi-target tracking method and system based on a three-dimensional map. BACKGROUND

[0002] With the continuous advancement of smart port construction, real-time, accurate and continuous monitoring of various mobile targets such as ships, vehicles and personnel in the wharf area has become a core requirement to ensure operation safety and efficiency. In existing monitoring technologies, monocular camera-based visual solutions have been widely used due to their flexible deployment and relatively low cost. However, when such solutions are directly applied to the open and complex wharf scene, their inherent limitations become increasingly apparent.

[0003] Firstly, traditional monocular vision is essentially a two-dimensional image analysis, naturally lacking depth information, which leads to serious deficiencies in target positioning accuracy, making it difficult to meet the requirements of wharf fine management for real physical space location. Secondly, in a large-scale scene covered by multiple cameras, when a target disappears from the field of view of one camera and enters another, cross-camera target re-identification (ReID) technology is needed to maintain the continuity of its identity. Existing ReID methods highly depend on clear and stable visual features, but in common situations such as long distance, light changes and target occlusion in the wharf, visual features are easily blurred or ineffective, leading to tracking and matching failures and frequent breaks in target trajectories. More importantly, such purely visual information-based methods fail to effectively utilize the inherent three-dimensional spatial structure of the wharf, and the motion trajectories generated lack physical world constraints, often resulting in incorrect associations such as vehicles penetrating walls and ships landing on shore, which seriously affects the reliability of the entire monitoring system.

[0004] To address the challenges of cross-camera tracking, the industry has made related explorations. For example, Chinese patent CN107689054B discloses a method of constructing a multi-camera topological connected graph, which uses clustering algorithms to determine the entrance and exit areas of the cameras and estimates the transfer time to assist cross-camera tracking. Although this solution provides a solution, it still fundamentally relies on two-dimensional visual information, and its depth estimation is still inferred through the size changes of two-dimensional bounding boxes, with limited accuracy. At the same time, the robustness of its feature matching decreases significantly when the target is far from the camera and the feature pixels are too low, and the problem still exists.

[0005] Therefore, how to break the excessive dependence on single visual features, deeply integrate the existing three-dimensional geographic information of the wharf with two-dimensional video data, and design a cross-camera tracking mechanism that is still robust in low definition and complex environments is a key technical problem that needs to be solved in the current intelligent monitoring field. SUMMARY

[0006] To overcome the existing problems and shortcomings, this invention proposes a multi-target tracking method for docks across cameras based on 3D maps, comprising the following steps:

[0007] S1. 3D map processing: Obtain a 3D model map of the dock, and perform voxelization processing on the 3D model map to generate a voxel cloud containing 3D coordinates and color or texture information;

[0008] S2. Target Detection: Perform real-time target detection on surveillance video acquired by at least one camera, and generate a detection result for each detected target, including its location information and confidence level in a two-dimensional image;

[0009] S3. Three-dimensional spatial mapping: Based on the pre-calibrated camera intrinsic and extrinsic parameters, by matching the color or texture features of the two-dimensional image region with the voxel cloud, a mapping relationship between the surveillance video and the three-dimensional map is established, and the target in the detection result is projected into the three-dimensional map to obtain the three-dimensional spatial coordinates of the target;

[0010] S4. Monocular tracking: Within the field of view of a single camera, the motion trajectory of the target is predicted using Kalman filtering, and the same target in consecutive video frames is continuously tracked and its trajectory status is managed through data association algorithms.

[0011] S5. Cross-camera trajectory matching: When the target switches between different camera views, a matching cost function is constructed that integrates target re-identification (ReID) feature distance, 3D spatial distance and 3D spatial velocity constraints to calculate the trajectory correlation under different cameras and achieve cross-camera target identity matching.

[0012] Furthermore, the process of projecting the target onto the 3D map specifically includes:

[0013] Extract voxel data within the camera's field of view and statistically analyze its color histogram and texture features;

[0014] A sliding window is used on the two-dimensional image to calculate the color histogram and texture features of each window sub-region;

[0015] By calculating the color and texture similarity between voxel regions and image sub-regions, the best matching region is found, thereby establishing the correspondence between voxel clouds and image regions.

[0016] Furthermore, the target detection generates a three-dimensional detection box that includes center coordinates, length, width, and height.

[0017] Further, the continuous tracking within the single camera field of view is optimized based on the ByteTrack algorithm, by first matching and associating high-confidence bounding boxes, and by second matching and associating low-confidence bounding boxes, to enhance the tracking stability of occluded targets.

[0018] Further, the cost function of the cross-camera track matching further comprises a global weight calculated based on historical track data of the target, and the global weight is used to adjust the probability of the target appearing under a specific camera.

[0019] Further, the re-identification (ReID) feature is extracted by a lightweight backbone network, and the average value of the features of the target in the recent number of frames is taken as the template feature.

[0020] A wharf cross-camera multi-target tracking system based on a three-dimensional map, characterized by comprising:

[0021] A three-dimensional coordinate mapping module is configured to voxelize a three-dimensional model map and establish a matching relationship between the voxelized three-dimensional model map and a two-dimensional monitoring image, and project a detected target into a three-dimensional space.

[0022] A target detection module is configured to detect targets in real time from a monitoring video.

[0023] A monocular tracking module is configured to continuously track a target in a single camera.

[0024] A cross-camera matching module is configured to match the same target between different cameras according to a cost function that integrates a re-identification feature, a three-dimensional spatial distance, and a speed constraint.

[0025] A track management and visualization module is configured to maintain and update the track state of all targets, and to visually display the track state on a three-dimensional map.

[0026] Further, the target detection module comprises a feature generation module, a feature fusion module integrating a path aggregation network (PANet) and a bidirectional feature pyramid (BiFPN) structure, and a feature detection module outputting a three-dimensional bounding box.

[0027] Beneficial effects:

[0028] The present application significantly improves the accuracy of target positioning and tracking. By voxelizing a three-dimensional model of a wharf and establishing an accurate matching relationship between a voxel cloud and a two-dimensional image in multiple dimensions such as color and texture, the present application can directly and accurately project a detected target in a video into a real three-dimensional physical space. This transformation fundamentally solves the problem of lack of depth information in traditional monocular vision, making target positioning and track tracking no longer a simulation of a two-dimensional plane, but a real reproduction in a three-dimensional space, greatly improving the accuracy.

[0029] In the process of cross-camera target matching, the application greatly enhances the tracking robustness in low-quality videos and complex environments by constructing a comprehensive cost function that integrates multi-dimensional information. This function innovatively combines three-dimensional spatial distance, three-dimensional spatial speed constraint, and even target high-frequency appearance region weight based on the logic of port operations with traditional ReID visual features through weighted combination. This multi-modal fusion mechanism greatly reduces the dependence on a single high-quality visual feature. Even in extreme cases where the target feature is blurred, the system can complete reliable matching with more robust spatial and logical constraints, thereby effectively avoiding trajectory breaks.

[0030] The application ensures the physical authenticity of the final generated trajectory by placing all tracking activities in a unified framework of a three-dimensional map. The three-dimensional map not only provides accurate coordinates, but also brings rich spatial semantic information such as roads, buildings, and water boundaries. Using this information, the method can automatically filter out a large number of abnormal trajectories that violate physical common sense due to false matching, making the output tracking results fully consistent with the actual scene logic and significantly improving the credibility and reliability of the entire system. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0032] Figure 1 The structure schematic diagram of the target detection framework in the embodiment of the present application.

[0033] Figure 2 The overall flowchart of the port cross-camera multi-target tracking method based on a three-dimensional map in the embodiment of the present application.

[0034] Figure 3 The specific flowchart of the monocular tracking module in the embodiment of the present application. DETAILED DESCRIPTION

[0035] The present application will be described below in conjunction with specific embodiments:

[0036] Embodiment 1:

[0037] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below in conjunction with the drawings and a specific embodiment. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the protection scope of the present application.

[0038] The application will be further described below in conjunction with the accompanying drawings and embodiments. The specific flow chart is shown in Figure 2 .

[0039] The wharf cross-camera multi-target tracking method and system based on a three-dimensional map comprises five core modules, namely:

[0040] The target detection module: multi-target real-time detection based on a single-stage network, which fully considers the advantages of path aggregation network (PANet) and bidirectional feature pyramid (BiFPN) structure for small target features in the monitoring angle, constructs a new cross-scale up-down fusion module, and outputs two-dimensional detection boxes of different detection targets;

[0041] The monocular tracking module: optimizes the ByteTrack algorithm to realize three-dimensional target single-camera multi-target tracking;

[0042] The three-dimensional coordinate mapping module: voxelizes the three-dimensional model map, uses a feature matching algorithm to construct the matching relationship between the two, and generates the three-dimensional coordinate information of the target in the three-dimensional map through a preset target size;

[0043] The cross-lens matching module: fuses the feature distance of the target re-identification algorithm (ReID), and establishes a three-dimensional space constraint cross-camera track association based on the wharf operation logic;

[0044] The trajectory management and visualization module: maintains the target trajectory state, updates, deletes, and creates the trajectory, maintains the trajectory matching effect in monocular and multi-target tracking, and outputs the three-dimensional map visualization result. Next, in conjunction with the flow chart, the process is described step by step.

[0045] First, the corresponding target detection framework is established. Considering the need to match with the three-dimensional map later, the detection framework generates three-dimensional detection results in the regression stage. The target detection framework consists of three modules, namely the feature generation module (BackBone), the feature fusion module (Neck), and the feature detection module (Head), as shown in Figure 1 . The feature generation module mainly generates multiple convolution structures to fully feature the input picture and generate feature layers of different sizes. Three feature layers of different scales are extracted in the feature generation module for detecting targets of different sizes. The feature fusion module fuses the advantages of path aggregation network (PANet) and bidirectional feature pyramid (BiFPN) structure to construct a new cross-scale fusion module. The overall network structure is two groups of up-sampling models and one group of down-sampling models, which includes an intermediate layer for feature matching and fusion between different levels. The overall structure is a top-down and bottom-up network structure, and the specific structure is as followsFigure 1 The feature detection module is a detection head of different scales, and the detection head includes a classifier and a regressor, which are used to generate feature detection structures of different sizes. The classifier is used to generate a corresponding type, and the regressor obtains a corresponding detection result. The final result structure is d = (cent_x, cent_y, length, width, height, conf). The regressor directly generates three-dimensional information of the detection box, which is the center coordinates of the detection box and the corresponding length, width, height and confidence. Non-maximum suppression is used to remove detection results that do not meet the requirements. The three-dimensional coordinate detection result is used in the subsequent monocular matching and three-dimensional map matching processes.

[0046] On the basis of obtaining the three-dimensional coordinates of the target, a monocular tracking module is introduced, which is responsible for continuous tracking of the target in a single camera. The core process includes four steps of tracker initialization, Kalman filter prediction, data association (first matching + second matching), trajectory update and state management. The specific process is shown in Figure 3 . In this case, ByteTrack is used as the basis to optimize the dimension information and establish the target tracking of three-dimensional detection results. The specific steps are as follows:

[0047] First, the tracker is initialized, and the detection result in the previous text is obtained, where each element contains a detection result in the structure of (cent_x, cent_y, length, width, height, conf), and . For the detection result with high confidence in the initial frame, a corresponding tracker is created, and the motion vector , the covariance matrix , and the state transition matrix are initialized. The motion vector is used to describe the trend of each parameter.

[0048] For each tracker in the previous step, Kalman filter prediction is performed. For each active tracker (state is “active”), the detection result in the next frame is predicted according to the previous frame state

[0049]

[0050] which contains the current trajectory prediction result in the next frame.

[0051] For the detection result , select the high-confidence detection box in the detection result, and calculate the ​The optimal matrix is ​​matched using the Hungarian algorithm, where 1- Let be the cost matrix. If a match is found, the tracker is updated; otherwise, the tracker that failed to match is marked. Similarly, select detection boxes with lower confidence and use the Hungarian algorithm again to match them with the remaining detection boxes.

[0052] Based on the matching results from the previous step, the following status update options are selected: 1. For trackers that have a successful match, update the corresponding status; 2. For trackers that have failed to match twice, mark them as not inactive. For trackers that are continuously marked as not inactive, terminate the corresponding tracker. Trackers that have a successful match are marked as active.

[0053] Considering the uncertain lifespan of current surveillance cameras, the 2D coordinates of the cameras are calibrated to improve the accuracy of 3D map matching. Given the fixed camera positions and large number of cameras, Zhang's calibration method is chosen for parameter calculation. A chessboard calibration board larger than 40cm*40cm is generated, and more than 20 photos are taken from different angles using the cameras. OpenCV sub-pixel corner detection is used to obtain pixel coordinates. If we set the plane containing the chessboard squares to Z=0, then we can set the world coordinate system of the corner points of the chessboard to be... After obtaining the pixel coordinate system and world coordinate system, the chessboard is solved using the least squares method:

[0054]

[0055] Obtain the corresponding intrinsic parameter matrix And the corresponding distortion matrix P.

[0056] After calibrating the camera, the next step is to match the 2D target with the 3D map. In practice, 3D mapping is commonly used... Figure 1 Generally, 3D model maps are created using methods such as oblique photogrammetry. While creating 3D maps from point clouds offers higher accuracy, it also incurs higher costs and is therefore less common. Furthermore, 3D models generated through oblique photogrammetry and post-processing cannot be used directly. Therefore, to match them with 2D images, voxelization is performed on the 3D model.

[0057] Generally, 3D model maps are based on triangular meshes. Here, we choose to convert the model to a voxel cloud by mesh voxelization. Let the side length of the selected voxel be... Find the minimum coordinate value of the triangular mesh model in 3D space. and maximum coordinate value A bounding box containing the entire model is determined. A regular voxel grid is constructed in three-dimensional space according to the voxel resolution and the voxelized range. The number of voxels in the x, y, z directions of the voxel grid is determined. The voxels are evenly divided within the corresponding range, and each voxel has a unique three-dimensional index (i, j, k) corresponding to its position in the voxel grid.

[0058] Each triangular patch in the triangular mesh model is processed in turn. Each triangle is defined by three vertices (v1, v2, v3) containing three-dimensional coordinate information. For each triangle, an intersection test is performed for each voxel in the voxel grid. To simplify the calculation process, the Separating Axis Theorem is used to determine the intersection condition, and it is checked whether there is an overlapping part between the projections of the triangle and the voxel on each coordinate axis. If there is an overlap between the projections of the two on all possible separating axes, the triangle and the voxel intersect; if there is no overlap on a certain separating axis, they do not intersect. For voxels intersecting with the triangular mesh, they are marked as "occupied"; voxels that do not intersect are marked as "empty". This state marking is the most basic attribute of voxels, used to distinguish between the interior and exterior space of the model. All point clouds in the occupied state are retained for further processing. At the same time, after determining the two bananas, the color channels (R, G, B) contained in the triangular mesh can be further fused into the voxel information.

[0059] In specific scenarios, the approximate range of the range photographed by the corresponding camera in the map is known. Extracting the voxel data of the camera's field of view can greatly reduce the matching process time of subsequent algorithms and improve accuracy.

[0060] Based on the above data processing results, voxel data of the corresponding three-dimensional model under the monitoring perspective can be obtained. The range contained in this part of the voxel data should be slightly larger than the monitoring range. Subsequent data matching of the two, because the selected voxel has a relatively large range of overlap with the monitoring, the region can be extracted to realize the matching of the two. The matching standard is the color information and the corresponding texture information.

[0061] For the selected voxel region, the color information of the internal voxels is counted. The proportion of the number of voxels in each color interval in the voxel region is counted to form a three-dimensional color histogram.

[0062] The voxel region is converted to a gray scale representation. The center voxel of the voxel region is taken as the reference, different offsets are set, and the gray level co-occurrence matrix is counted. The gray level co-occurrence matrix records the frequency of different gray values appearing at the same time under certain offset conditions. The energy, contrast, correlation, and entropy of the texture feature values are calculated from the gray level co-occurrence matrix.

[0063] For the input image, the RGB color of each pixel is counted in the same way as the voxel region color histogram calculation, and the color histogram of the image is constructed. The image color histogram is based on two-dimensional pixels, and needs to be dynamically updated with the movement of the sliding window.

[0064] The image is converted into a grayscale image, and the gray level co-occurrence matrix and the corresponding texture feature values are calculated for each image sub-region within the sliding window. The size of the sliding window needs to be adjusted according to actual needs, and a window size similar to or slightly larger than the voxel region size is generally selected to ensure that the possible matching region can be fully covered.

[0065] The Bhattacharyya distance is used to calculate the similarity between the voxel region color histogram and the image sub-region color histogram. The smaller the Bhattacharyya distance, the more similar the two color histograms, i.e., the closer the color distribution. The specific expression is:

[0066]

[0067] where P and Q are the color histograms of the voxel region and the image sub-region, respectively, and n is the number of color intervals.

[0068] For texture features, since there are multiple texture feature values, the weighted Euclidean distance is used to calculate the similarity. First, each texture feature value is normalized to be in the same numerical range. The calculation formula of the weighted Euclidean distance is:

[0069]

[0070] where is the weight of the corresponding feature, and are the texture feature values of the voxel region and the image sub-region.

[0071] The color similarity and the texture similarity are weighted and summed according to a certain proportion to obtain the comprehensive similarity S. The larger the value of the comprehensive similarity S, the higher the similarity between the voxel region and the image sub-region.

[0072] After the sliding window traverses the entire image, the comprehensive similarity calculated at all window positions is compared. The window position with the maximum comprehensive similarity is found, and the image sub-region corresponding to this window is the best matching region of the voxel region. Thus, the correspondence between the voxel region and the image sub-region is established, and the region-based matching process is completed.

[0073] Combined with the (cent_x, cent_y, length, width, height, conf) information calculated in the foregoing, the target position information obtained in the image can be directly represented in the voxel cloud, and the voxel cloud is consistent with the coordinates of the three-dimensional map, so the information can also directly project the detected target into the three-dimensional map.

[0074] The matching result in the foregoing can directly mark the three-dimensional detected target in the three-dimensional map, and subsequently fuse the two-dimensional image of the target and the three-dimensional coordinate information to realize cross-camera track matching. In the cross-camera track matching, mainly rely on the two-dimensional re-identification algorithm (ReID) and the three-dimensional spatial window restriction. Considering the performance bottleneck among multiple algorithms, therefore, a lightweight network ResNet-34 is used as the backbone network for feature extraction of the re-identification algorithm. The specific detection process is as follows:

[0075] In the historical tracker, the ROI of the target with the highest confidence in the last five frames is extracted, and the range of interest is greater than 20% than the detection frame;

[0076] The ROI is scaled to 224*224 size by bilinear interpolation, and the normalized output is input into the ReID network;

[0077] The average value of the output of the five frames is taken as the target template feature.

[0078] In the foregoing step, the feature value of the target is obtained, and the matching cost is calculated in combination with the spatial information. The matching relationship between the targets is determined by the cost function. The cost function is determined by three parts:

[0079]

[0080] wherein , , is the weight of the corresponding module loss, wherein is the feature loss, the feature vector distance between different targets is obtained by the ReID algorithm, is the three-dimensional space loss, used for calculating the relative position loss in the world coordinate system, is the speed loss, used for calculating whether the maximum speed of the target in the three-dimensional space meets the requirement of the displacement distance. Meanwhile, considering that in the actual generation of the wharf, generally, there is a fixed path, and different targets have a high-frequency appearing scene, therefore, the global weight t is introduced in combination with the target information of different types, used for determining the probability weight of the appearance of different targets under different cameras. If a type of target appears frequently under some cameras in the previous track, the value will be higher, and vice versa.

[0081] In the foregoing, there is monocular matching and multi-view matching for the trajectory. After each monocular detects the target and multi-view matches the target, the trajectory state transition has a certain update rule. First, the trajectory has three states, active, inactive and termination in monocular. For the trajectory in the matching state in monocular, the state is active, and the state of the trajectory not in the matching state is inactive, and if the inactive frame number is greater than 30 frames, it is in the termination state. Each target has an independent tracker under monocular, and each tracker under monocular has an independent id, which represents the same target under the tracker. Different trackers also have a tracking id under multi-view, which is shared under multi-view conditions. That is, the tracker is independent, and the id is shared. For the multi-view matching tracker, if not all the trackers are in the termination state, it is considered that the target has left the wharf, and all the trackers are discarded. Otherwise, the matching tracker is retained.

[0082] Any person skilled in the art, without departing from the core idea and principle of the technical scheme of the present application, can make various corresponding changes or modifications, and these changes or modifications shall belong to the protection scope of the claims attached to the present application. Therefore, the patent protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for port cross-camera multi-target tracking based on a three-dimensional map, characterized in that, Comprise the following steps: S1. Three-dimensional map processing: obtain the three-dimensional model map of the port, and voxelize the three-dimensional model map to generate a voxel cloud containing three-dimensional coordinates and color or texture information; S2. Target detection: real-time target detection is performed on the monitoring video obtained by at least one camera, and a detection result containing the position information and confidence of each detected target in the two-dimensional image is generated; S3. Three-dimensional space mapping: based on the pre-calibrated camera internal and external parameters, the mapping relationship between the monitoring video and the three-dimensional map is established by matching the color or texture features of the two-dimensional image region and the voxel cloud, and the target in the detection result is projected into the three-dimensional map to obtain the three-dimensional spatial coordinates of the target; S4. Monocular tracking: within the field of view of a single camera, the motion trajectory of the target is predicted using Kalman filtering, and the same target in consecutive video frames is continuously tracked using a data association algorithm to manage its trajectory state; S5. Cross-camera track matching: When the target switches between different camera views, cross-camera target identity matching is achieved by calculating a matching cost function The formula for calculating the cost function is: ； Wherein For feature loss, the distance between feature vectors of different targets is obtained by the ReID algorithm. is the three-dimensional space loss used to compute the relative position loss of the agent in the world coordinate system; For speed loss, to calculate whether the maximum speed of the target in three-dimensional space meets the requirements of displacement distance; t is a global weight calculated based on the historical trajectory data of the target, used to adjust the probability of different targets appearing under a specific camera.

2. The method of claim 1, wherein, The process of projecting the target in the detection result into the three-dimensional map in step S3 specifically comprises: Extract the voxel data within the field of view of the camera and count its color histogram and texture features; On the two-dimensional image, a sliding window is used to calculate the color histogram and texture features of each window sub-region; By calculating the color and texture similarity between the voxel region and the image sub-region, the best matching region is found to establish the correspondence between the voxel cloud and the image region.

3. The method according to claim 1 or 2, characterized in that, The target detection in step S2 generates a three-dimensional detection box containing the center coordinates, length, width and height.

4. The method of claim 1, wherein, The continuous tracking within the field of view of a single camera in step S4 is optimized based on the ByteTrack algorithm, which first matches and associates high-confidence detection boxes, and then secondly matches and associates low-confidence detection boxes to enhance the tracking stability of occluded targets.

5. The method of claim 1, wherein, In step S5, the re-identification feature is extracted by a lightweight backbone network, and the average value of the target's features in the last few frames is taken as its template feature.

6. A three-dimensional map based port cross-camera multi-target tracking system, characterized in that, Comprise: A three-dimensional coordinate mapping module for voxelizing the three-dimensional model map and establishing its matching relationship with the two-dimensional monitoring image, and projecting the detected target into the three-dimensional space; A target detection module for real-time target detection from monitoring video; A monocular tracking module for continuous trajectory tracking of targets within a single camera; A cross-lens matching module for matching the target identity across cameras by calculating a matching cost function when the target switches between different camera fields of view: ； Wherein For feature loss, the distance between feature vectors of different targets is obtained by the ReID algorithm. is the three-dimensional space loss used to compute the relative position loss of the object in the world coordinate system; For speed loss, whether the maximum speed of the target in three-dimensional space meets the requirement of displacement distance is calculated; t is a global weight calculated based on the historical trajectory data of the target, used to adjust the probability of different targets appearing under a specific camera.

7. The system of claim 6, wherein, The target detection module comprises a feature generation module, a feature fusion module combining a path aggregation network (PANet) and a bidirectional feature pyramid (BiFPN) structure, and a feature detection module that outputs a three-dimensional detection box.

Images