Cross-camera multi-target tracking method, device and equipment and readable storage medium

Abstract

The application relates to a cross-camera multi-target tracking method, device and equipment and a readable storage medium, which comprises correcting a picture shot by a camera and generating a field-of-view boundary line and a target transfer line for the camera; a single-camera target detection is performed through a coordinate attention mechanism network to obtain a plurality of detection boxes corresponding to the single camera; a multi-target tracking algorithm is used to track the detection boxes and generate an identity ID for the detection boxes; the to-be-transferred detection boxes entering the overlapping area of the current camera are determined according to the field-of-view boundary line and the target transfer line, and the image coordinates of the to-be-transferred detection boxes and the target detection boxes of the current camera are converted into world coordinates; the centroid position proximity and the detection box overlap between the to-be-transferred detection boxes and the target detection boxes are calculated according to the world coordinates; and the target consistency determination and target transfer between the to-be-transferred detection boxes and the target detection boxes are realized based on the identity ID, the centroid position proximity and the detection box overlap, so that the accurate tracking and identification of the cross-camera multi-target are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target tracking, in particular to a cross-camera multi-target tracking method and device, equipment and a readable storage medium. BACKGROUND

[0002] Multi-target multi-view tracking aims to determine the position and identity of each person from the video stream captured by multiple cameras, which is widely used in the fields of unmanned driving, human-computer interaction and virtual reality. In recent years, due to the analysis results of multi-target multi-view tracking can assist in tactical formulation of sports competition, improve the scientific nature and efficiency of training, be used for enhancing sports video broadcast effect and providing interactive content for audiences, etc., it also plays an important value in the field of sports.

[0003] Among them, as the key technology of high-speed sports motion image analysis system, multi-target multi-view tracking can meet the monitoring needs of large scene and multi-angle, but due to the large relative displacement of high-speed moving targets between frames, and the problems of target sticking, human posture change, light change, etc., it is difficult for the existing tracking algorithm to realize accurate tracking, which may easily cause identity switching or identity loss; in addition, in the process of multi-target multi-view tracking, the following problems make it difficult to track and recognize multi-targets across the view domain: the error of feature matching between adjacent cameras will interfere with the effect of target handover, the target shape mutation caused by the difference of different camera viewpoints, and the low distinguishability of athletes' appearance. Therefore, how to realize accurate tracking and recognition of multi-targets across cameras is a problem to be solved at present. SUMMARY

[0004] The present application provides a cross-camera multi-target tracking method, device, equipment and readable storage medium to realize accurate tracking and recognition of multi-targets across cameras.

[0005] In a first aspect, a cross-camera multi-target tracking method is provided, comprising the following steps:

[0006] Correcting the pictures captured by multiple cameras based on a checkerboard correction method, and generating a field of view boundary line and a target handover line for each corrected camera;

[0007] Performing single-camera target detection through a coordinate attention mechanism network to obtain multiple detection boxes corresponding to the single camera;

[0008] Tracking the targets through a preset multi-target tracking algorithm to generate an identity ID for each target corresponding detection box;

[0009] According to the field of view boundary line and the target handover line, determining the to-be-handover detection box entering the overlapping area of the current camera from the detection box corresponding to the adjacent last camera;

[0010] convert image coordinates of the to-be-handover bounding box and the target bounding box corresponding to the current camera into world coordinates;

[0011] calculate the centroid position proximity and the bounding box overlap between the to-be-handover bounding box and each target bounding box according to the world coordinates respectively;

[0012] perform target consistency determination and target handover on the to-be-handover bounding box and each target bounding box based on the identity ID, the centroid position proximity and the bounding box overlap.

[0013] In some embodiments, the target consistency determination and target handover on the to-be-handover bounding box and each target bounding box based on the identity ID, the centroid position proximity and the bounding box overlap comprises:

[0014] perform target consistency determination on the to-be-handover bounding box and each target bounding box according to the centroid position proximity and the bounding box overlap;

[0015] take the target bounding box having the minimum centroid position proximity and the maximum bounding box overlap with the to-be-handover bounding box as a candidate target box;

[0016] assign the identity ID of the to-be-handover bounding box to the candidate target box to realize target handover across cameras.

[0017] In some embodiments, the determination method of the minimum centroid position proximity is:

[0018] When the world coordinates of the target bounding box and the world coordinates of the to-be-handover bounding box satisfy the following first condition, it is determined that the target bounding box and the to-be-handover bounding box have the minimum centroid position proximity.

[0019] The first condition is:

[0020]

[0021] In the formula, (X' w , Y'w) represents the world coordinates of the target bounding box, represents the world coordinates of the to-be-handover bounding box.

[0022] In some embodiments, the number of to-be-handover bounding boxes is multiple, and after the step of calculating the centroid position proximity and the bounding box overlap between the to-be-handover bounding box and each target bounding box according to the world coordinates respectively, the method further comprises:

[0023] calculate the Euclidean distance between one to-be-handover bounding box and other to-be-handover bounding boxes according to the world coordinates of the to-be-handover bounding boxes;

[0024] if the Euclidean distance is detected to be greater than the distance threshold, performing target consistency determination and target handover of the to-be-handover detection frame and each target detection frame based on the identity ID, the centroid position proximity, and the detection frame overlap based on the one of the to-be-handover detection frames;

[0025] if the Euclidean distance is detected to be less than or equal to the distance threshold, performing target handover of the one of the to-be-handover detection frames and the target detection frame by the Hungarian algorithm and the centroid position proximity.

[0026] In some embodiments, the coordinate attention mechanism network is a YOLO model based on a coordinate attention mechanism.

[0027] In a second aspect, a cross-camera multi-target tracking apparatus is provided, comprising:

[0028] a processing unit configured to correct images captured by multiple cameras based on a checkerboard correction method, and generate a field-of-view boundary line and a target handover line for each corrected camera;

[0029] a detection unit configured to perform single-camera target detection by a coordinate attention mechanism network to obtain multiple detection frames corresponding to the single camera;

[0030] a tracking unit configured to perform target tracking on all the detection frames by a preset multi-target tracking algorithm to generate an identity ID for each target corresponding detection frame;

[0031] a determination unit configured to determine, from detection frames corresponding to an adjacent previous camera, to-be-handover detection frames that enter an overlapping area of a current camera according to the field-of-view boundary line and the target handover line;

[0032] a conversion unit configured to convert image coordinates of the to-be-handover detection frames and target detection frames corresponding to the current camera into world coordinates;

[0033] a calculation unit configured to calculate, according to the world coordinates, a centroid position proximity and a detection frame overlap between the to-be-handover detection frames and each target detection frame, respectively;

[0034] a handover unit configured to perform target consistency determination and target handover of the to-be-handover detection frames and each target detection frame based on the identity ID, the centroid position proximity, and the detection frame overlap.

[0035] In some embodiments, the handover unit is specifically configured to:

[0036] perform target consistency determination of the to-be-handover detection frames and each target detection frame according to the centroid position proximity and the detection frame overlap;

[0037] a target bounding box with minimum centroid position proximity and maximum bounding box overlap degree between the to-be-handled bounding boxes is taken as a candidate target bounding box;

[0038] an identity ID of the to-be-handled bounding box is assigned to the candidate target bounding box to realize target handover across cameras.

[0039] In some embodiments, the number of to-be-handled bounding boxes is multiple, and the computing unit is further configured to:

[0040] a Euclidean distance between one to-be-handled bounding box and other to-be-handled bounding boxes is calculated according to the world coordinates of the to-be-handled bounding boxes;

[0041] if it is detected that the Euclidean distance is greater than a distance threshold, the computing unit is caused to perform the steps of target consistency determination and target handover of the to-be-handled bounding box and each target bounding box based on the identity ID, the centroid position proximity and the bounding box overlap degree based on the one to-be-handled bounding box;

[0042] if it is detected that the Euclidean distance is less than or equal to the distance threshold, the handover unit is further caused to perform target handover of the one to-be-handled bounding box and the target bounding box by using the Hungarian algorithm and the centroid position proximity.

[0043] In a third aspect, a cross-camera multi-target tracking device is provided, which comprises a memory and a processor, at least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the cross-camera multi-target tracking method.

[0044] In a fourth aspect, a computer readable storage medium is provided, and the computer storage medium stores a computer program, when the computer program is executed by a processor, the cross-camera multi-target tracking method is implemented.

[0045] The application provides a cross-camera multi-target tracking method, device and equipment and a readable storage medium. The method comprises correcting pictures captured by multiple cameras based on a checkerboard correction method, generating a field-of-view boundary line and a target handover line for each corrected camera, performing single-camera target detection through a coordinate attention mechanism network to obtain multiple detection boxes corresponding to the single camera, performing target tracking on all detection boxes through a preset multi-target tracking algorithm to generate an identity ID for each target corresponding detection box, determining a to-be-handover detection box entering an overlapping area of a current camera from adjacent last camera corresponding detection boxes according to the field-of-view boundary line and the target handover line, converting image coordinates of the to-be-handover detection box and the current camera corresponding target detection box into world coordinates, respectively calculating a centroid position proximity degree and a detection box overlap degree between the to-be-handover detection box and each target detection box according to the world coordinates, and performing target consistency determination and target handover on the to-be-handover detection box and each target detection box based on the identity ID, the centroid position proximity degree and the detection box overlap degree. The application improves the accuracy of adhesion target detection through a coordinate attention mechanism, and in the cross-camera target switching process, adjacent cameras are associated through the corresponding relationship between image coordinates and world coordinates, so that the dimension information of the unified space world coordinates is added in the overlapping area to constrain target matching. Since the centroid coordinates of the target under the top view angle do not change suddenly with the change of the posture, the cross-camera target consistency determination and target handover through the centroid position proximity degree and the detection box overlap degree of the to-be-handover detection box in the overlapping field of view can effectively improve the accuracy and stability of target tracking identification and handover, and thus the identity of the target can be maintained and accurately positioned continuously. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0047] Figure 1 A flowchart of a cross-camera multi-target tracking method provided by an embodiment of the application;

[0048] Figure 2 A framework diagram of a cross-camera multi-target handover algorithm provided by an embodiment of the application;

[0049] Figure 3 A target adhesion diagram under different angles provided by an embodiment of the application;

[0050] Figure 4 A relationship diagram between camera calibration related coordinate systems provided by an embodiment of the application;

[0051] Figure 5 A schematic diagram of image coordinate conversion to actual site coordinate (i.e. world coordinate) provided for the embodiment of the present application;

[0052] Figure 6 A structural schematic diagram of a cross-camera multi-target tracking device provided for the embodiment of the present application. DETAILED DESCRIPTION

[0053] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0054] The embodiments of the present application provide a cross-camera multi-target tracking method, device, equipment and readable storage medium, to realize accurate tracking and identification of cross-camera multi-target.

[0055] Referring to Figure 1 and Figure 2 , the embodiments of the present application provide a cross-camera multi-target tracking method, comprising the following steps:

[0056] Step S10: correcting the pictures taken by the plurality of cameras based on the checkerboard correction method, and generating a field of view demarcation line and a target handover line for each corrected camera;

[0057] Exemplarily, it can be understood that in high-speed sliding sports such as short track speed skating, since the clothes and postures of athletes are similar, it is difficult to extract features based on the appearance feature of the target re-identification method for non-rigid targets, so that different athletes cannot be effectively distinguished, and the identity ID is likely to jump. In addition, referring to Figure 3 , each circle in the athlete ( Figure 3 ) represents an athlete) is close in high-speed sliding, so there is a sticking problem, and the sliding track is overlapped and crossed due to sports interaction, so the target re-identification method based on track matching often easily causes mismatching or missing matching when performing cross-camera target handover. The embodiments of the present application propose a cross-camera multi-target tracking algorithm based on spatial constraints to improve the accuracy and stability of target handover.

[0058] The following embodiments will take short track speed skating as an example to describe the steps and principles of the cross-camera multi-target tracking method.

[0059] The target handover is a key link in the target continuous tracking process, and the accuracy of the handover determines the effect of the cross-camera multi-target tracking. Therefore, in the embodiment, four high-speed industrial cameras (i.e., high-speed industrial video cameras) are installed in advance on the roof of the short track speed skating venue, the four cameras are distributed directly above the ice track, and there is a certain overlapping field of view between adjacent cameras to ensure smooth switching of the target in the tracking process. It should be noted that the target involved in the embodiment refers to the athlete, and the target will be presented in the form of a detection box.

[0060] In the embodiment, the four monitoring pictures are corrected by using a checkerboard correction method. Specifically, the picture distortion caused by the camera shooting angle is eliminated by plane perspective conversion, and checkerboard calibration images of different angles, postures and positions are collected to detect feature points in the images to obtain the distortion coefficient after obtaining the intrinsic and extrinsic parameters of the camera, and then the pictures taken by the four monitoring cameras are corrected.

[0061] It should be understood that the field of view boundary line refers to the position of the boundary of the field of view of a camera in the adjacent camera, also known as the field of view line, which is key information in the target handover process. In order to distinguish the overlapping area and the non-overlapping area between adjacent cameras, a field of view boundary line needs to be generated for each camera, i.e., when there are multiple cameras, the field of view boundary line needs to be generated before the athlete identity association (which can also be understood as target re-identification) across the cameras, so as to determine the timing of target handover. It should be noted that the projection invariant method will be used in the embodiment to generate a field of view boundary line for each camera to determine when the target appears in the field of view of two cameras; at the same time, a pair of target handover lines are drawn in the overlapping area, i.e., a line suitable for target handover is selected in the overlapping area, so when the target touches the line, it means that the target should be searched in the field of view of another camera at this time.

[0062] Step S20: performing single-camera target detection through a coordinate attention mechanism network to obtain a plurality of detection boxes corresponding to a single camera; wherein the coordinate attention mechanism network is a YOLO model based on a coordinate attention mechanism;

[0063] It should be understood that, demonstratively, the short track speed skating athletes will produce adhesion phenomenon during sliding due to rapid interaction, and the traditional background modeling target detection method has limitations in dealing with the adhesion problem, but the target detection method based on deep learning has higher detection accuracy and speed, which can more easily distinguish the adhesion target. In addition, the motion position of the short track speed skating athletes between frames is very different, which makes the target detector need to have high sensitivity to position information. Therefore, in the embodiment, the CA-YOLO model based on the coordinate attention mechanism is used as the target detector to detect the target, so as to more accurately identify the adhesion target.

[0064] Specifically, the video stream captured by the single camera is input into the CA-YOLO model; the backbone feature extraction network of the CA-YOLO model performs feature extraction on the video stream to output a feature map; then a coordinate attention mechanism layer is added after the backbone feature extraction network of the CA-YOLO model outputs the feature map, the position information is embedded into the channel attention, and the features are aggregated along two spatial directions respectively to capture long-range dependencies and preserve precise position information; at this time, the coordinate attention mechanism splits the global pooling into two feature encoding operations, that is, the obtained feature map is encoded into a pair of direction perception and position sensitive attention maps, and then the correlation between the feature pixels and the positions is established, so that the segmentation ability of the network is significantly improved at the boundary and the details, which helps more accurate target positioning; then the FPN (Feature Pyramid Networks) strengthens the feature fusion of the previous feature layer, and finally the detection head of the CA-YOLO outputs the target category and the detection box. It should be noted that the detection box carries the position information of the target, that is, it can reflect the target position information.

[0065] Step S30: performing target tracking on all detection boxes by a preset multi-target tracking algorithm to generate an identity ID for each detection box corresponding to a target;

[0066] Specifically, in this embodiment, after obtaining the detection box by the target detection algorithm, the multi-target tracking algorithm is used to associate the inter-frame targets to generate an identity ID for each detection box, realize the association of the identities of the athletes, and then continuously track the targets, that is, continuously associate the same target between frames in the subsequent frames to keep the identity consistent. Since the short track speed skating competition rules do not have track restrictions, the positions of the athletes often change rapidly and the rankings often change during the sliding process, and when the positions of adjacent athletes are very close, the confidence of target detection will be reduced, resulting in low tracking performance. Therefore, the Bytetrack algorithm is used for multi-target tracking in this embodiment, that is, low-score confidence boxes are retained.

[0067] Specifically, first, the matching of high-score detection boxes and trajectories is realized based on motion similarity: a video segment V, a target detection Det and a Kalman filter algorithm KF are used as inputs of the association algorithm Byte in Bytetrack; the detection score threshold τ high , τ low and the tracking score threshold ∈ are set; all detection boxes generated by the target detector are divided into two parts: the detection boxes with a detection score higher than the threshold τ high are classified as D high , and the detection boxes with a detection score higher than the threshold τ low and lower than the threshold τ high are classified as D low; then calculate the motion similarity, i.e. calculate the IoU (Intersection over Union) between the detection target frame and the boundary box predicted by the Kalman filter, the larger the IoU, the higher the motion similarity; based on the motion similarity, realize the matching of the high-score detection frame and the track, i.e. for D high and all the tracks T, use the Kalman filter to predict the position information of the target in the next frame, and use the Hungarian algorithm for matching, and each track corresponding to the matching result contains the detection frame and the identity ID of the target.

[0068] Then use the Hungarian algorithm to match the predicted track with the detection frame of the current frame to associate the inter-frame ID, wherein the track can be understood as the position sequence (u, v, r, h) of the target at different times, u, v, r and h respectively represent the center point position of the target rectangular frame, r is the aspect ratio, and h is the height; this embodiment will match through motion information: first, match the detection frame with the nearest tracklet (i.e. small tracking fragment), and then match the detection frame with the lost tracklet. Specifically, use the Kalman filter to predict the track of the target corresponding to the last frame, compare the predicted track state of the next frame target with the detected track state (u, v, r, h) of the next frame target, and only when the comparison value is less than a certain threshold, it can be judged whether the detected target of the next frame corresponds to the target of the last frame (connect the target corresponding to the last frame and the next frame, so as to achieve the tracking effect); use the Mahalanobis distance between the predicted Kalman state and the current detection value to integrate the motion information, and after obtaining the distance value of each detected target corresponding to each track, a threshold is given, and the matched one is selected.

[0069] Wherein, the Mahalanobis distance d (i, j) represents the motion matching degree between the jth detection and the ith track, the track is the covariance matrix of the observation space at the current time predicted by the Kalman filter, y i represents the predicted observation at the current time, d j represents the current track state corresponding to the detected target.

[0070] Finally, realize the second matching of the unmatched track: in the previous matching process, for the detection frame with a motion similarity less than 0.2, the matching track will not be matched, and the unmatched detection frame is stored in D remain , and the unmatched track is stored in T remain ; then the low-score detection frame in D low and the remaining track in T remain are matched for the second time, and the track that has not been matched successfully will be stored in T re-remainand the unmatched low-score bounding boxes are directly deleted.

[0071] where, for the trajectories in T re-remain , it is considered that the target is temporarily lost and the trajectory is put into T lsot ; if the trajectory in T lost exists for more than 30 frames, it is deleted from T lost , otherwise it is kept in T lost ; if it can be matched later, it is also deleted from T lost ; for the detections in D remain , if the detection score is higher than the tracking score threshold ∈ and exists for more than two frames, it is initialized as a new trajectory. For the output of each frame, only the detection box and the corresponding ID of the trajectory with respect to the current frame are output.

[0072] Step S40: determining the to-be-transferred detection box from the detection box corresponding to the adjacent previous camera according to the field-of-view boundary and the target transfer line, the to-be-transferred detection box entering the overlapping region of the current camera;

[0073] For example, when the detection box 1 corresponding to the target A in the adjacent previous camera Cam1 touches the field-of-view boundary corresponding to the current camera Cam2, it indicates that the target A has reached the overlapping field of view of the current camera; when the detection box 1 corresponding to the target A touches the target transfer line corresponding to the current camera Cam2, it indicates that the target A needs to be transferred across the cameras, i.e., when the target passes through the overlapping region of the adjacent cameras, it will enter the target transfer work; therefore, the detection box 1 is taken as the to-be-transferred detection box.

[0074] Step S50: converting the image coordinates of the to-be-transferred detection box and the target detection box corresponding to the current camera into world coordinates;

[0075] For example, it should be understood that in a multi-camera tracking system, global calibration of multiple cameras is needed to unify all the cameras and map the image coordinates of each video frame to the world coordinate system to establish the corresponding relationship between adjacent cameras. The relationship between the commonly used coordinate systems such as the world coordinate system, the camera coordinate system, the image physical coordinate system and the image pixel coordinate system related to camera calibration is as follows: Figure 4As shown, the camera's imaging process is essentially a coordinate system transformation. Points in space are transformed from the world coordinate system to the camera coordinate system, then projected onto the image plane. Finally, the data from the physical coordinate system of the image on the virtual image plane is transformed to the image pixel coordinate system. It can be understood that in this embodiment, the world coordinate system is used to describe the actual coordinates of the target (i.e., the athlete) on the ice rink, while the image coordinate system is used to describe the projection relationship of the target from the camera coordinate system to the image coordinate system during the imaging process.

[0076] Specifically, for any point P in space, its camera coordinates are (X... c Y c Z c ), world coordinates are (X w Y w Z w The transformation from the world coordinate system to the camera coordinate system can be achieved through rotation and translation. The transformation formula is shown in equation (1), where the rotation matrix R is a 3×3 orthogonal identity matrix and t represents the three-dimensional translation vector.

[0077]

[0078] Let the image coordinates of any point P in space be (x, y). The transformation from the camera coordinate system to the image coordinate system is the projection perspective process from three-dimensional to two-dimensional. The shape is projected onto the projection plane using the central projection method to form a single-sided projection image that closely resembles the visual effect. The transformation formula is shown in equation (2):

[0079]

[0080] It should be noted that the speed skating scene involved in this embodiment is shot from a top-down perspective, with the camera approximately 20 meters above the ice surface. Therefore, the transformation between the field coordinates (i.e., world coordinates) and the image coordinates can be considered as a two-dimensional affine translation. Therefore, see... Figure 5 As shown, in order to describe the position of the athlete (i.e., the target) on the ice rink, a field coordinate system (X) is established with the center of the ice rink as the center and according to a two-dimensional Cartesian coordinate system. w Y w The site coordinates of the center points of the four cameras are obtained based on the mapping data from the geodetic instrument; then the image coordinates corresponding to each camera are obtained from the site coordinates. Then, the image coordinates are mapped to a unified world coordinate system to establish a coordinate relationship between the four cameras and obtain the world coordinates corresponding to each detection box. Figure 5 This demonstrates the field of view of the four surveillance cameras in the venue, as well as the sub-monitoring areas (i.e., Figure 5 This is a diagram showing the mapping of the target centroid's image coordinates to the site coordinates in the partial monitoring images of Cam1 to Cam4 presented in the video.

[0081] In the embodiment, when the target (i.e. the detection frame to be handed over) enters the handover stage, it needs to be connected across the cameras, at this time, the image coordinates corresponding to the tracking target in the image frames of the adjacent two handover cameras need to be converted into unified field coordinates, and through the analysis of the field coordinates, kinematic parameters such as the tracking trajectory, speed and sliding distance of the speed skating player can also be obtained. Wherein, the conversion formula of the image coordinates of the sub monitoring area corresponding to the unified field coordinates (X w , Y w ) is as follows:

[0082]

[0083]

[0084] In the formula, X and Y represent the center point coordinate values of each camera in the actual field coordinates obtained by surveying, and Y correspond to the center point coordinate values of the single image frame, μ is the unit pixel distance, λ is the mapping coefficient, and R is the projection deviation caused by the displacement of the image point due to vertical shooting.

[0085] In the embodiment, when the target matching is performed in the handover area, the correlation results and error values based on the field coordinate matching are fed back to the image coordinates of the target, at this time, the matching results can be used to determine the targets successfully handed over and the targets unsuccessfully handed over. As can be seen, the above feedback can be obtained by converting the field coordinates into image coordinates. Wherein, the image coordinates corresponding to the sub monitoring area obtained from the field coordinates (X w , Y w ) are as follows:

[0086]

[0087]

[0088] In the formula, X and Y represent the center point coordinate values of the actual field coordinates of each camera, and Y correspond to the center point coordinate values of the single image frame, μ is the unit pixel distance, λ is the mapping coefficient, and c is the correction parameter.

[0089] Step S60: According to the world coordinates, the centroid position proximity and the detection frame overlap between the detection frame to be handed over and each target detection frame are calculated respectively;

[0090] Exemplarily, it can be understood that in a regular (non-relay) short track skating competition, the number of participating athletes is fixed, and no new target is added. Therefore, the evaluation criteria of the cross-camera target handover in the embodiment will be composed of the centroid position proximity and the detection box overlap. Among them, the centroid position proximity score refers to the difference between the distance between the world coordinates of the candidate target center point in the current view and the world coordinates of the target in the previous camera view, that is, by calculating the difference between the distance between the world coordinates of the center point of the detection box to be handed over in the current camera and the world coordinates of the center point of the target detection box in the adjacent previous camera, the centroid position proximity between the detection box to be handed over and each target detection box can be obtained.

[0091] The detection box overlap refers to the area coincidence degree between the target box to be matched in the current camera and the target box with the identity ID determined in the previous camera, that is, by calculating the area coincidence degree between the detection box to be handed over in the adjacent previous camera and the target detection box in the current camera, the detection box overlap between the detection box to be handed over and each target detection box can be obtained. Specifically, the IoU (i.e. the detection box overlap) size from the coordinate boxes in the two views and

[0092]

[0093] Among them, the larger the IoU value, the greater the possibility that the two detection boxes are the same target.

[0094] Step S70: performing target consistency determination and target handover on the detection box to be handed over and each target detection box based on the identity ID, the centroid position proximity, and the detection box overlap.

[0095] Exemplarily, in the embodiment, the centroid position proximity and the detection box overlap are combined to determine the target consistency; among them, the smaller the centroid position proximity and the larger the detection box overlap, the greater the possibility that the target detection box is the target (i.e. the target to be handed over) being tracked in the previous view, that is, the higher the probability of target consistency; by assigning the identity ID of the target to be handed over to the target detection box with high consistency with the detection box to be handed over, the target handover can be completed.

[0096] ​It can be seen that the embodiment improves the accuracy of adhesion target detection through the coordinate attention mechanism, and in the cross-camera target switching process, the adjacent cameras are associated through the corresponding relationship between the image coordinates and the world coordinates, so that the dimension information of the unified space world coordinates is added in the overlapping area to constrain the target matching. Since the centroid coordinates of the target under the top view angle will not change abruptly with the change of the posture, the cross-camera target consistency determination and target handover are performed through the proximity of the centroid positions of the detection boxes to be handed over in the overlapping field of view and the detection box overlap degree, which can effectively improve the accuracy and stability of target tracking identification and handover, and further maintain the identity and accurately position the target.

[0097] Further, the target consistency determination and target handover of the detection box to be handed over and each target detection box based on the identity ID, the proximity of the centroid positions and the detection box overlap degree, comprising:

[0098] determining the target consistency of the detection box to be handed over and each target detection box according to the proximity of the centroid positions and the detection box overlap degree;

[0099] the target detection box with the minimum proximity of the centroid positions and the maximum detection box overlap degree with the detection box to be handed over is taken as the candidate target box; wherein the determination method of the minimum proximity of the centroid positions is:

[0100] when the world coordinates of the target detection box and the world coordinates of the detection box to be handed over satisfy the following first condition, it is determined that the target detection box and the detection box to be handed over have the minimum proximity of the centroid positions;

[0101] the first condition is:

[0102]

[0103] wherein, (X' w , Y' w ) represents the world coordinates of the target detection box, represents the world coordinates of the detection box to be handed over;

[0104] the identity ID of the detection box to be handed over is assigned to the candidate target box to realize the cross-camera target handover.

[0105] For example, when the target consistency determination is made between the handover detection frame and each target detection frame according to the centroid position proximity and the detection frame overlap, the target detection frame with the minimum centroid position proximity and the maximum detection frame overlap is determined to be the same target as the handover detection frame, and the target detection frame is taken as the candidate target frame. The minimum centroid position proximity is determined by the position of the center point of the n target detection frames in the image coordinate system in the output result of the target detection and the position in the world coordinate system is calculated The position of the handover detection frame in the world coordinate system is denoted as The world coordinate of the target detection frame in the current view is denoted as When the following condition is met:

[0106]

[0107] It is considered that the handover detection frame and the target detection frame are closest in the physical position, and it is determined that the handover detection frame and the target detection frame have the minimum centroid position proximity.

[0108] Finally, the identity ID of the handover detection frame is assigned to the candidate target frame, so as to realize the target handover between the cameras, and the same target is continuously associated between frames, so as to keep the identity consistent and continuously track the target.

[0109] Further, the number of handover detection frames is multiple, and after the step of calculating the centroid position proximity and the detection frame overlap between the handover detection frame and each target detection frame according to the world coordinates, the method further comprises:

[0110] The Euclidean distance between one handover detection frame and other handover detection frames is calculated according to the world coordinates of the handover detection frame.

[0111] If the Euclidean distance is greater than the distance threshold, the step of performing the target consistency determination and the target handover between the handover detection frame and each target detection frame based on the identity ID, the centroid position proximity and the detection frame overlap is performed based on the one handover detection frame.

[0112] If the Euclidean distance is less than or equal to the distance threshold, the target handover between the one handover detection frame and the target detection frame is performed by the Hungarian algorithm and the centroid position proximity.

[0113] Exemplarily, in the embodiment, when multiple athletes arrive at the target handover line at the same time (i.e., multiple handover detection boxes exist at the same time), a multi-target data fusion problem needs to be processed. A distance threshold τ of a distance between target center coordinates is preset, when the Euclidean distance between a handover detection box in a certain view and other surrounding handover detection boxes is greater than the distance threshold τ, the target matching is performed by using the centroid position proximity and the detection box overlap degree; and when the Euclidean distance between the center coordinates of the handover detection box and other surrounding handover detection boxes is less than or equal to the distance threshold τ, the nearest optimal matching is performed by combining the Hungarian algorithm with the centroid position proximity, and then the target handover is realized, so as to reduce the false matching and the missed matching, and improve the accuracy of the target handover.

[0114] wherein the Hungarian algorithm can be used to solve the maximum matching problem of a bipartite graph, and the maximum matching refers to a subset that makes the number of edges matched between nodes of the bipartite graph maximum. Assuming that the set of all target detection boxes in the current view is U, and the set of all target detection boxes in the last view is V, the detection boxes in the two sets are matched as much as possible by the Hungarian algorithm.

[0115] In the weighted bipartite graph, the edge weights connected by the nodes are different, and therefore the problem of considering the maximum or minimum total weight while requiring the nodes to be matched, i.e., the optimal matching problem. Specifically, assuming that the set of all target detection boxes in the current view is U={u1, u2, u3, u4}, the set of all target detection boxes in the last view is V={v1, v2, v3, v4}, and a bipartite graph G=(U, V, S) is formed, wherein there are edges between U and V, there are no edges inside U and inside V, S represents a set of matched edges, the edges in the S set have no common nodes, and a function f(s) is used to represent the sum of the weights of the edges in the matching, and therefore the target of completing the maximum weight matching is to find a matched edge set S that makes f(s) maximum.

[0116] It can be understood that the matching by the Hungarian algorithm actually defines a bijection of U-V, and the most matched vertices are found for u1…u4 in turn. When some vertices in U have the possibility of matching with the vertices in V, the matching degree can be represented by the weight of the edge, i.e., the weight of U i -V j is denoted as W ij . Since the essence of the Hungarian algorithm is to complete the minimum weight matching, i.e., the purpose is to find a matched edge set S that makes f(s) minimum, if the Hungarian algorithm is used to solve the maximum weight matching problem, the maximum weight matching needs to be converted into the minimum matching, i.e., the weight of the edge is added with a negative sign, and therefore the final result of the minimum weight matching by the Hungarian algorithm is actually the maximum weight matching.

[0117] The minimum matching of the Hungarian algorithm is realized by operating the adjacency matrix, and the specific process is as follows: (1) subtract the minimum value from the value of each row in the adjacency matrix (the value is the weight value including a negative sign), so that each row has a 0; (2) subtract the minimum value from each column to make each column have a 0, and obtain a new adjacency matrix; (3) perform a loop operation on the adjacency matrix: (3.1) cover all 0 elements in the adjacency matrix with as few lines as possible; (3.2) determine whether the loop needs to be terminated: if the number of lines L meets the node number n, the loop is terminated; if the number of lines L is less than the node number n, the loop is continued, assuming that the current L is 3 and n is 4, so the loop step needs to be continued; (3.3) transform the 0 elements in the adjacency matrix to obtain more 0 elements: let the minimum value of the elements not covered by the line be k, subtract the minimum value k from the elements not covered by the line, and add k to the elements corresponding to the line intersection position, and return to step (3.1); assuming that after step (3.1) is performed, L is 4, and since L=n, the loop needs to be terminated, and the result of the current adjacency matrix is the result of the minimum matching, that is, the near-optimal matching is realized.

[0118] It can be understood that the weight W of the edge between the nodes of the bipartite graph is obtained by calculating the centroid position proximity ij , and the smaller the weight is, the greater the probability of correct correspondence between the two nodes is. Assuming that there are 4 to-be-handed-over detection boxes in the last camera, and the coordinates of the center points of the 4 to-be-handed-over detection boxes in the world coordinate system are a1=(90, 82), a2=(120, 110), a3=(98, 122), and a4=(136, 150); there are 4 target detection boxes to be matched in the current camera, and the coordinates of the center points of the 4 target detection boxes in the world coordinate system are b1=(138, 153), b2=(95, 119), b3=(91, 84), and b4=(122, 110), and let W ij represent the Euclidean distance between the i-th to-be-handed-over detection box and the j-th target detection box coordinate in the current camera, x ai and y ai represent the horizontal and vertical coordinates of the to-be-handed-over detection box, x bj and y bj represent the horizontal and vertical coordinates of the current target detection box, then

[0119]

[0120] Calculate the distance matrix A:

[0121]

[0122] Define the decision variable:

[0123]

[0124]

[0125] Through the step of operating the adjacency matrix, an optimal matching with the minimum W value is obtained, that is, the optimal matching based on the combination of the Hungarian algorithm and the centroid position proximity is realized, and the matching result is: a1→b3, a2→b4, a3→b2, a4→b1.

[0126] In summary, the cross-camera multi-target tracking algorithm provided in the embodiment mainly includes three parts of target detection, multi-target tracking under a single camera, and cross-camera target handover: a view field demarcation line and a target handover line are generated after the cross-camera picture is corrected; a bounding box position information is output by using a target detector based on deep learning, and then multi-target tracking under a single camera is expanded, and an identity ID is assigned and maintained for each detected target; when the target passes through the overlapping area of adjacent cameras, the target handover work is entered, that is, by establishing the corresponding relationship between the camera image coordinates and the actual site coordinates, the stability of the target handover is improved by using the spatial dimension constraint to determine the consistency of the target according to the centroid position proximity and the detection box overlap degree of the corresponding frame; after the target handover is completed, the tracking is continued in the next camera. Among them, whether the target reaches the overlapping view field of the next adjacent camera is judged according to the view field demarcation line, if not, multi-target tracking under a single camera is performed; if the overlapping view field is reached, cross-camera target handover is performed; the above steps are repeated until all targets pass through the finish line, and the tracking process is ended. As can be seen, the embodiment has excellent tracking performance for high-speed moving targets, and can continuously maintain the identity and accurately position the target.

[0127] The following embodiment also tests the cross-camera multi-target tracking algorithm.

[0128] In the embodiment, 34 complete short track sliding athlete sliding videos are collected, and each contains four time-synchronized high-speed industrial camera pictures, a total of 136 monitoring videos, each video has a length of about 3 minutes, a size of 860x720 pixels, and a frame rate of 60 frames / second. More than 4000 images containing multiple athletes are collected from the videos to form a Skater training data set, the athletes in the data set are manually labeled bounding boxes, xml format labeled files containing target coordinate values and width and height are obtained, and the training set and the test set are divided according to a ratio of 9:1. Among them, the number of athletes in each video is 4-10, and the cross-camera multi-target tracking test can be performed.

[0129] In the process of training the Skater dataset using the CA-YOLO model, the size of the input image is 788x530, the maximum epoch is 200, the batch size and the initial learning rate are set to 4 and 0.0001 respectively, data augmentation is performed using Mosaic and MixUp in the first 170 epochs, and the dataset is further expanded through rotation, translation and other methods.

[0130] The trained model is used for prediction, where the NMS (Non Maximum Suppression) threshold is set to 0.3 and the confidence threshold is set to 0.65; the target class and confidence score will be displayed on the top of the detection box in the CA-YOLO detection result. This embodiment will use AP (Average Precision) to evaluate the detection effect of different target detection models, and the evaluation results are shown in Table 1. As shown in Table 1, compared with mainstream models such as YOLO series, the CA-YOLO with coordinate attention mechanism proposed in this embodiment achieves an AP precision of 91.1% when the IoU threshold is 0.5, and an average AP of 69.2% when the IoU threshold is 0.5 to 0.95, showing stable detection performance and higher sensitivity to target position differences, which is more conducive to detecting adherent targets.

[0131] Table 1 Comparison of target detection precision

[0132]

[0133] It can be understood that in multi-target multi-camera tracking, the evaluation index related to the target identity ID is the most important. In this embodiment, the cross-camera target handover algorithm proposed in this application will be evaluated using the IDF1, IDP, IDR and IDsw indexes. Among them, IDF1 represents the average ratio, specifically the ratio of the number of correct target detections to the average of the true number and the sum of the calculated detection number, which is the first index to evaluate the performance of the tracker; IDP (Identification Precision) refers to the accuracy of ID recognition in each detection box; IDR (Identification Recall) refers to the recall rate of ID recognition in each detection box; IDsw (ID Switches) represents the number of times the target identity ID has an error jump, that is, the identity ID of the target has an error switch in the tracking process, and the optimal state of IDsw is 0.

[0134] The calculation formulas of IDF1, IDP and IDR are shown in equations (9) to (11) respectively:

[0135]

[0136]

[0137]

[0138] In the formula, IDTP represents the number of IDs of the detected target in the video that are correctly assigned as positive, IDFP represents the number of IDs that are incorrectly assigned as positive, IDFN represents the number of IDs that are incorrectly assigned as negative (i.e. the number of missed assignments), and IDTN represents the number of IDs that are correctly assigned as negative. The optimal state of the IDF1, IDP, and IDR indicators is 100%.

[0139] In the embodiment, the CFT method based on feature matching, the MTMC_ReID and TRACTA methods based on trajectory matching, and the HMT method based on perspective transformation and motion features are included in the experiment. The evaluation results of the specific target handover method are shown in Table 2.

[0140] Table 2 Comparison results of target handover algorithm experiment

[0141]

[0142] As can be seen from Table 2, the cross-camera target handover method based on spatial constraints proposed in the application has certain advantages in the indicators IDF1, IDP, IDR, and IDsw compared with other existing handover methods. The method can accurately map image coordinates to unified site coordinates, thereby accurately performing target re-identification, and has better robustness in the problem of continuous tracking of multiple targets in a multi-view camera system with overlapping views.

[0143] As can be seen from Table 2, the cross-camera target handover method based on spatial constraints proposed in the application has certain advantages in the indicators IDF1, IDP, IDR, and IDsw compared with other existing handover methods. The method can accurately map image coordinates to unified site coordinates, thereby accurately performing target re-identification, and has better robustness in the problem of continuous tracking of multiple targets in a multi-view camera system with overlapping views.

[0144] The embodiment of the application further provides a cross-camera multi-target tracking device, which comprises:

[0145] a processing unit configured to correct pictures captured by a plurality of cameras based on a checkerboard correction method, and generate a field-of-view boundary and a target handover line for each corrected camera;

[0146] a detection unit configured to perform single-camera target detection by a coordinate attention mechanism network to obtain a plurality of detection boxes corresponding to the single camera;

[0147] a tracking unit configured to perform target tracking on all detection boxes by a preset multi-target tracking algorithm to generate an identity ID for each target corresponding detection box;

[0148] a judgment unit configured to determine a to-be-handover detection box from detection boxes corresponding to an adjacent last camera according to the field-of-view boundary and the target handover line, the to-be-handover detection box entering an overlapping area of a current camera;

[0149] a conversion unit configured to convert image coordinates of the to-be-handover detection box and target detection boxes corresponding to the current camera into world coordinates;

[0150] a calculation unit configured to calculate a centroid position proximity and a detection box overlap between the to-be-handover detection box and each target detection box according to the world coordinates, respectively;

[0151] a handover unit configured to perform target consistency determination and target handover on the to-be-handover detection box and each target detection box based on the identity ID, the centroid position proximity, and the detection box overlap.

[0152] Further, the handover unit is specifically configured to:

[0153] perform target consistency determination on the to-be-handover detection box and each target detection box according to the centroid position proximity and the detection box overlap;

[0154] assign a target detection box having a minimum centroid position proximity and a maximum detection box overlap with the to-be-handover detection box as a candidate target box;

[0155] assign an identity ID of the to-be-handover detection box to the candidate target box to realize cross-camera target handover.

[0156] Further, the determination method of the minimum centroid position proximity is:

[0157] when world coordinates of a target detection box and world coordinates of a to-be-handover detection box satisfy the following first condition, it is determined that the target detection box and the to-be-handover detection box have a minimum centroid position proximity;

[0158] the first condition is:

[0159]

[0160] (X w , Y w ) represents the world coordinates of the target detection frame, (X

[0161] Further, the number of the to-be-handover detection frames is multiple, and the computing unit is further configured to:

[0162] According to the world coordinates of the to-be-handover detection frame, calculate the Euclidean distance between one of the to-be-handover detection frames and other to-be-handover detection frames;

[0163] If it is detected that the Euclidean distance is greater than the distance threshold, the computing unit is caused to perform the steps of target consistency determination and target handover of the to-be-handover detection frame and each target detection frame based on the identity ID, the centroid position proximity, and the detection frame overlap degree based on the one of the to-be-handover detection frames;

[0164] If it is detected that the Euclidean distance is less than or equal to the distance threshold, the handover unit is further caused to perform target handover of the one of the to-be-handover detection frames and the target detection frame through the Hungarian algorithm and the centroid position proximity.

[0165] Further, the coordinate attention mechanism network is a YOLO model based on a coordinate attention mechanism.

[0166] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described device and each unit can refer to the corresponding processes in the foregoing cross-camera multi-target tracking method embodiments, which will not be described here.

[0167] The device provided in the foregoing embodiments can be implemented in the form of a computer program, which can run on a cross-camera multi-target tracking device as shown in Figure 6 .

[0168] The embodiments of the present application also provide a cross-camera multi-target tracking device, which comprises a memory, a processor and a network interface connected through a system bus, and the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to realize all steps or part of steps of the foregoing cross-camera multi-target tracking method.

[0169] The network interface is configured to perform network communication, such as sending an assigned task. Those skilled in the art can understand that, Figure 6The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0170] The processor can be a CPU, and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor, etc. The processor is the control center of the computer device, and connects all parts of the computer device through various interfaces and lines.

[0171] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a video playing function, an image playing function, etc.), etc. The data storage area can store data created according to the use of the mobile phone (such as video data, image data, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device or other volatile solid-state memory device.

[0172] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, all steps or part of steps of the cross-camera multi-target tracking method are realized.

[0173] The embodiments of the present application realize all or part of the foregoing processes, and can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer readable storage medium, and the computer program can realize the steps of the above methods when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0174] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, a server or a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer usable program codes.

[0175] The present application is described with reference to flowcharts and / or block diagrams according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device for realizing the functions specified in the flowcharts and / or block diagrams. Figure 1 The device for realizing the functions specified in one flow or multiple flows and / or blocks. Figure 1 The device for realizing the functions specified in one flow or multiple flows and / or blocks.

[0176] It has to be noted that, as used herein, the terms "includes", "including", "has", "having" or "has" are intended to be open-ended terms that specifically permit the inclusion of one or more elements, steps, features, components, or the like. As used herein, the term "exemplary" is used to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations. The terms "comprises", "comprising", "includes", "including", "has", "having", "contains", "containing" or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, contains or includes, but does not consist of, one or more elements, steps, features, components, or the like, is and is not limited to those elements, steps, features, components, or the like. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an overly literal or restricted way unless expressly so defined herein.

[0177] The foregoing detailed description has set forth various embodiments of the devices and / or methods via the use of specific terminology. However, embodiments thereof should not be construed as limited to the foregoing aspects, and the terminology used herein should not be read to limit the descriptive scope of the application. The description is intended to cover any and all modifications and equivalents, and is limited only by the claims.

Claims

1. A method for multi-target tracking across cameras, characterized in that, Includes the following steps: The image captured by multiple cameras is corrected based on the checkerboard correction method, and a field of view boundary line and target intersection line are generated for each corrected camera. Single-camera target detection is performed using a coordinate attention mechanism network to obtain multiple detection boxes corresponding to a single camera. A preset multi-target tracking algorithm is used to track targets in all detection boxes, so as to generate an identity ID for each target-corresponding detection box. Based on the field of view boundary line and the target intersection line, determine the intersection detection box that enters the overlapping area of ​​the current camera from the detection box corresponding to the adjacent previous camera; Convert the image coordinates of the detection box to be handed over and the target detection box corresponding to the current camera into world coordinates; Calculate the centroid proximity and overlap between the detection box to be handed over and each target detection box based on the world coordinates. Based on the identity ID, the centroid proximity, and the detection box overlap, target consistency is determined and target handover is performed between the detection box to be handed over and each target detection box.

2. The cross-camera multi-target tracking method as described in claim 1, characterized in that, The process of determining target consistency and transferring targets based on the identity ID, the centroid proximity, and the detection box overlap includes: Based on the centroid proximity and the detection box overlap, target consistency is determined between the detection box to be handed over and each target detection box. The target detection box with the minimum centroid proximity and the maximum detection box overlap with the detection box to be handed over is selected as the candidate target box. Assign the identity ID of the detection box to be handed over to the candidate target box to achieve target handover across cameras.

3. The cross-camera multi-target tracking method as described in claim 2, characterized in that, The method for determining the proximity of the minimum centroid position is as follows: If the world coordinates of the target detection box and the world coordinates of the detection box to be intersected satisfy the following first condition, it is determined that the target detection box and the detection box to be intersected have a minimum centroid position proximity. The first condition is: In the formula, (X' w Y' w () represents the world coordinates of the object detection box. This represents the world coordinates of the detection frame to be handed over.

4. The cross-camera multi-target tracking method as described in claim 1, characterized in that, The number of detection boxes to be handed over is multiple. After the step of calculating the centroid proximity and detection box overlap between the detection box to be handed over and each target detection box according to the world coordinates, the method further includes: Calculate the Euclidean distance between one of the detection frames to be handed over and the other detection frames to be handed over, based on the world coordinates of the detection frames to be handed over. If the detected Euclidean distance is greater than the distance threshold, then the step of determining the target consistency and handing over the target detection box to be handed over and each target detection box based on the identity ID, the centroid position proximity and the detection box overlap is performed based on one of the detection boxes to be handed over; If the detected Euclidean distance is less than or equal to the distance threshold, then the target detection box is transferred to one of the detection boxes to be transferred using the Hungarian algorithm and the centroid proximity.

5. The cross-camera multi-target tracking method as described in claim 1, characterized in that: The coordinate attention mechanism network is a YOLO model based on the coordinate attention mechanism.

6. A multi-target tracking device across cameras, characterized in that, include: The processing unit is used to correct the images captured by multiple cameras based on the checkerboard correction method, and to generate a field of view boundary line and a target intersection line for each corrected camera. The detection unit is used to perform single-camera target detection through a coordinate attention mechanism network to obtain multiple detection boxes corresponding to a single camera. The tracking unit is used to track all detection boxes using a preset multi-target tracking algorithm, so as to generate an identity ID for each detection box corresponding to each target; The judgment unit is used to determine the detection box to be intersected from the detection box corresponding to the adjacent previous camera based on the field of view boundary line and the target intersection line; The transformation unit is used to convert the image coordinates of the detection box to be handed over and the target detection box corresponding to the current camera into world coordinates; The calculation unit is used to calculate the centroid proximity and overlap between the detection box to be handed over and each target detection box according to the world coordinates. The handover unit is used to determine the consistency of the target and hand over the target based on the identity ID, the centroid proximity, and the detection box overlap.

7. The cross-camera multi-target tracking device as described in claim 6, characterized in that, The handover unit is specifically used for: Based on the centroid proximity and the detection box overlap, target consistency is determined between the detection box to be handed over and each target detection box. The target detection box with the minimum centroid proximity and the maximum detection box overlap with the detection box to be handed over is selected as the candidate target box. Assign the identity ID of the detection box to be handed over to the candidate target box to achieve target handover across cameras.

8. The cross-camera multi-target tracking device as described in claim 6, characterized in that: The number of the detection frames to be handed over is multiple, and the calculation unit is also used for: Calculate the Euclidean distance between one of the detection frames to be handed over and the other detection frames to be handed over, based on the world coordinates of the detection frames to be handed over. If the detected Euclidean distance is greater than the distance threshold, the calculation unit is used to perform the steps of determining the consistency of the target detection box and each target detection box based on the identity ID, the centroid position proximity and the detection box overlap based on one of the detection boxes to be handed over; If the detected Euclidean distance is less than or equal to the distance threshold, the handover unit is further configured to perform target handover between one of the detection boxes to be handed over and the target detection box using the Hungarian algorithm and the centroid position proximity.

9. A multi-target tracking device across cameras, characterized in that, include: A memory and a processor, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the cross-camera multi-target tracking method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer storage medium stores a computer program that, when executed by a processor, implements the cross-camera multi-target tracking method as described in any one of claims 1 to 5.

Images