Non-rigid spatial mapping based calibration-free multi-camera multi-target tracking method

Abstract

The application discloses a kind of non-rigid space mapping-based multi-camera multi-target tracking methods without calibration, it is related to computer vision and intelligent video analysis technical field.The method includes: obtaining the position of target bounding box in each camera and the corresponding appearance feature representation;Extract the effective area that can be used for geometric modeling, generate binary effective area mask;Get dense displacement field and spatial correspondence confidence;Fixed step is constructed on the source camera image Sampling grid, the grid point is screened, and the high-confidence control point set from source view to target view is formed;According to the control point set, establish the cross-camera space mapping relationship under non-calibration condition;The spatial geometric similarity of target bounding box in different cameras is calculated, and the cross-camera target association is carried out by combining the appearance feature similarity, to realize the multi-camera multi-target tracking under non-calibration condition.The method can establish the cross-camera space mapping relationship, realize stable multi-camera multi-target tracking.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent video analysis technology, and in particular to a calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping. Background Technology

[0002] Multi-camera multi-target tracking is a crucial foundational technology in fields such as intelligent video surveillance, smart transportation, and public space perception. Its goal is to correlate local observations or trajectories of the same pedestrian from different camera perspectives across different viewpoints, thereby obtaining a globally continuous trajectory for the entire multi-camera system. Compared to single-camera tracking, multi-camera multi-target tracking not only needs to address issues such as occlusion, appearance changes, and scale differences, but also needs to solve the cross-camera correlation challenges caused by camera distortion, viewpoint differences, and lighting variations. Especially in real-world open environments, multi-camera systems typically have wide spatial distribution, complex installation locations, significant differences in imaging viewpoints, and limited field-of-view overlap, severely limiting the performance of cross-camera target correlation.

[0003] Existing multi-camera multi-target tracking methods can be broadly categorized into two types: appearance feature matching-based methods and spatial geometric constraint-based methods. Appearance feature matching-based methods typically utilize the target's color, texture, and shape features across different cameras to associate targets. Although the performance of these appearance features has been greatly improved with the development of deep learning models, these methods are still prone to identity confusion in situations involving dense crowds, similar clothing, severe occlusion, or changing lighting. To improve the stability of cross-camera association, some methods further introduce spatial geometric constraints. This typically involves establishing a reference plane mapping relationship using pre-acquired camera intrinsic and extrinsic parameters, or extracting sparse corresponding points to calculate a homography matrix, projecting targets from different viewpoints onto a unified reference coordinate system, and utilizing spatial position consistency to assist in identity matching.

[0004] However, the applicability of existing technologies in open and complex scenarios remains limited. On the one hand, geometrically constrained methods often rely on accurate camera calibration parameters, but in real-world large-scale multi-camera monitoring systems, the number of cameras is large and their installation and deployment are not uniform, often making it difficult or impossible to obtain high-precision calibration information. On the other hand, the homography matrix commonly used in existing uncalibrated multi-camera tracking methods belongs to a rigid plane mapping model, which is based on the assumption that the scene satisfies approximately absolute flatness. When the monitoring area contains slopes, steps, undulating ground, or strong perspective distortion, the spatial mapping based on homography transformation often produces large deviations, thus affecting the cross-camera target association performance.

[0005] Therefore, existing multi-camera multi-target tracking technologies still have significant shortcomings in terms of geometric adaptability and trajectory correlation capabilities in uncalibrated complex scenes. Summary of the Invention

[0006] The technical problem to be solved by the present invention is how to provide a calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping that does not depend on calibration parameters and can adapt to non-planar scenes.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping, comprising the following steps:

[0008] Acquire multi-channel video data and obtain the target bounding box positions and corresponding appearance feature representations in each camera;

[0009] The image used for cross-camera spatial mapping is segmented to extract the effective region that can be used for geometric modeling and generate a binary effective region mask.

[0010] For camera pairs, a dense displacement estimation model is used to calculate pixel-level correspondences, resulting in a dense displacement field and spatial correspondence confidence.

[0011] A sampling grid is constructed on the source camera image at a fixed step size. The grid points are then filtered to form a set of high-confidence control points from the source view to the target view.

[0012] Based on the set of control points, a non-rigid spatial mapping is fitted to establish a cross-camera spatial mapping relationship under uncalibrated conditions;

[0013] By utilizing the spatial mapping relationship, the spatial geometric similarity of the target bounding boxes in different cameras is calculated, and cross-camera target association is performed by combining the appearance feature similarity, thereby obtaining the global continuous trajectory in the multi-camera system and realizing multi-camera multi-target tracking under calibration conditions.

[0014] Preferably, the effective area includes one or more of roads, ground, grass, carpet, mats, or step surfaces, and people, vehicles, sky, trees, buildings, and water areas are excluded from the effective area.

[0015] A further technical solution is that the point pairs in the high-confidence control point set must simultaneously meet the following conditions: the spatial correspondence confidence satisfies the preset threshold constraint; the source point is located within the effective area of ​​the source camera; and the target point obtained by dense displacement field mapping is located within the effective area of ​​the target image.

[0016] A further technical solution is that the non-rigid mapping is established by combining affine terms and non-rigid radial basis terms, and a smoothing regularization term is introduced during the parameter solution process to reduce the influence of local abnormal control points on the overall mapping result.

[0017] A further technical solution involves the following steps for constructing a master control mesh structure for each pair of reference cameras and target cameras: setting several discrete topological anchor points within the effective area of ​​the reference camera at fixed step sizes; and using the pixel coordinates of each topological anchor point in the reference camera as fixed reference coordinates to describe the cross-camera ground topology skeleton.

[0018] A further technical solution involves using a dense displacement estimation model to continuously estimate the observation position of the topological anchor point in the target camera in subsequent video frames, and storing the anchor point observation results obtained at each moment in a hierarchical manner to form a multi-layer observation set oriented towards a time sliding window.

[0019] A further technical solution involves robustly fusing multiple frames of observations for each topological anchor point within the window after accumulating a sufficient number of observations in the sliding window; if multiple observations of similar spatial locations exist for the same anchor point, these consistent observations are fused to obtain the final position; if no stable and consistent observation cluster exists, the observation with the highest confidence is selected as the target position estimate for that anchor point; and based on the correspondence between the fused anchor points, the non-rigid spatial mapping between camera pairs is refitted and updated online.

[0020] A further technical solution involves, after establishing the cross-camera spatial mapping relationship, a step of performing a two-way loopback consistency evaluation on the cross-camera spatial mapping relationship. The evaluation step includes: sampling test points in the effective area of ​​the camera pair respectively; performing two-way loopback projection using forward mapping and reverse mapping to obtain the loopback error; and statistically analyzing the proportion of loopback points, the magnitude of the error, and the consistency of the two-way error based on the loopback error to evaluate the reliability of the current cross-camera spatial mapping relationship of the camera pair.

[0021] A further technical solution is that the cross-camera target association includes: calculating appearance similarity based on the appearance features of targets in different cameras; calculating spatial geometric similarity based on the spatial mapping to obtain the spatial positional relationship of target foot points; and matching and associating local trajectories in different cameras based on the appearance similarity and spatial geometric similarity, thereby obtaining a global continuous trajectory in the multi-camera system and realizing multi-camera multi-target tracking under calibration-free conditions.

[0022] The beneficial effects of the above technical solution are as follows: The method establishes a non-rigid spatial mapping relationship without camera calibration parameters by introducing a dense displacement field across viewpoints under effective region constraints, overcoming the problem of insufficient geometric adaptability of traditional homography matrices in complex non-planar scenes; by introducing a master control mesh structure and a hierarchical observation storage mechanism, it achieves continuous accumulation, robust fusion, and online refitting of cross-camera topological relationships, effectively reducing the damage to the overall geometric relationship caused by single-frame noise and local mismatches; by evaluating the consistency of bidirectional loop closures, the reliability of cross-camera spatial mapping relationships is clarified, thereby improving identity preservation capability and association stability. This invention, without requiring camera intrinsic and extrinsic parameters or additional calibration parameters, can realize the construction, online maintenance, and robust target association of cross-camera spatial relationships, and is suitable for multi-camera, multi-target tracking tasks in open scenes. Attached Figure Description

[0023] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0024] Figure 1 This is an overall flowchart of the method described in the embodiments of the present invention;

[0025] Figure 2 This is a diagram showing the results of multi-camera multi-target tracking in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0027] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0028] like Figure 1 As shown in the figure, this invention discloses a calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping, the method comprising the following steps:

[0029] Step 1: This invention first acquires multi-channel video information collected by a multi-camera system, and then uses object detection to obtain the target bounding box positions and corresponding appearance feature descriptors in each camera. Assume the multi-camera system contains... The camera, the first Each camera is recorded as For each video sequence, the present invention uses object detection to obtain the target bounding box position and the corresponding appearance feature descriptor in the camera.

[0030] In one specific embodiment, the target footstep location can be determined by the midpoint of the bottom edge of the target bounding box, used for subsequent cross-camera spatial mapping and geometric consistency calculation; the appearance feature representation can be extracted by the re-identification network, used for subsequent cross-camera identity association. The purpose of this step is to provide basic target observation information for subsequent cross-camera spatial mapping and target association.

[0031] Step Two: Since cross-camera spatial relationships are mainly determined by stable regions such as the ground, while dynamic or ineffective regions such as pedestrians, vehicles, sky, trees, and building facades are prone to introducing mismatch noise, this invention first performs segmentation on the image used to initialize the cross-camera spatial mapping before constructing the cross-camera spatial mapping. This extracts effective regions that can be used for geometric modeling and generates corresponding binary effective region masks to constrain the estimation range of subsequent cross-camera spatial mappings. Specifically, let the first... The initial image corresponding to each camera is Then, by segmentation, the corresponding binary effective region mask can be obtained:

[0032]

[0033] in, Represents pixels Located within the effective area, This indicates that the pixel will not participate in subsequent cross-camera spatial mapping estimation.

[0034] In a preferred embodiment, the semantic segmentation model may employ the SegFormer model. The effective region may include areas capable of supporting target movement, such as roads, ground, grass, carpets, mats, and step surfaces; areas such as people, vehicles, trees, sky, buildings, and water surfaces are excluded. Through this method, the present invention can constrain the geometric modeling region, allowing only pixels from physical surfaces to participate in subsequent cross-view spatial mapping estimation, thereby reducing the interference of dynamic targets and unstructured elements on geometric modeling.

[0035] Step 3: After obtaining the effective region mask, this invention applies to any camera pair A dense displacement field across viewpoints is calculated to obtain the pixel-level correspondence between the two viewpoints. Specifically, a dense displacement estimation model is used to match the two images to obtain the correspondence from the source camera. To the target camera Dense displacement fields and corresponding spatial confidence levels:

[0036]

[0037] Similarly, we can also obtain data from the camera. To the camera Displacement field and its confidence level:

[0038]

[0039] In one specific embodiment, the dense displacement estimation model can employ the UniFlowMatch model. For any point in the source image... Its predicted location in the target image can be represented as

[0040]

[0041] The purpose of this step is to obtain cross-view pixel-level correspondences using a data-driven approach without camera calibration parameters, laying the foundation for subsequent control point selection and non-rigid space mapping fitting.

[0042] Step 4: Since the dense displacement field results inevitably contain local noise and unreliable correspondences, this invention does not directly use all pixels for spatial mapping fitting, but instead uses regular grid sampling and multiple constraints to screen high-confidence control points.

[0043] Specifically, a regular sampling grid is constructed on the source camera image with a fixed step size. Only point pairs that satisfy the spatial correspondence confidence constraint, the source domain effective region constraint, and the target domain effective region constraint are retained, forming a set of high-confidence control points from the source view to the target view. Let a certain grid center point in the source image be denoted as... Therefore, for a point to be accepted as a valid control point, it must simultaneously meet the following conditions:

[0044]

[0045] A point pair is valid only if all of the above constraints are satisfied. Only those pairs are retained as valid control points. Thus, a set of high-confidence control points from the source perspective to the target perspective can be obtained.

[0046] In one specific embodiment, if the number of control points is less than a preset minimum threshold If the current camera pair lacks sufficiently reliable geometric support, then no further spatial mapping fitting will be performed. Through the above steps, this invention can extract a set of control points with uniform spatial distribution and reliable geometric meaning from dense displacement field results, reducing the impact of local noise and mismatches on the mapping results.

[0047] Step 5: After obtaining the set of high-confidence control points, this invention employs thin-plate spline transformation to construct a non-rigid spatial mapping relationship between camera pairs. Specifically, for the camera... To the camera The mapping, assuming any input point Then its mapping result can be expressed as

[0048]

[0049] in, This is the affine parameter matrix, used to describe global translation, rotation, and scale changes; The non-rigid deformation weights corresponding to each control point; is the radial basis function for the thin plate spline.

[0050] To improve the stability of the fit, this invention introduces a smoothing regularization term during the parameter solution process. This reduces the impact of local anomaly control points on the overall mapping, thereby improving the smoothness and robustness of the spatial mapping. Similarly, a system based on the camera can be further constructed. To the camera Inverse space mapping

[0051] Using the above method, this invention establishes a non-rigid spatial mapping relationship between camera pairs without relying on camera intrinsic and extrinsic parameters or manual calibration information. Compared with traditional homography matrices, this bidirectional non-rigid spatial mapping can more flexibly adapt to non-planar terrain, local perspective distortion, and complex viewpoint differences.

[0052] Step Six: Since the initially constructed cross-camera spatial mapping relationship may be affected by occlusion, local noise, or changes in the visible area, this invention further introduces a master control mesh structure to model and maintain the cross-camera topology relationship online. The master control mesh structure of each pair of reference cameras and target cameras is constructed, and several discrete topology anchor points are set in the effective area of ​​the reference camera at a fixed step size. The pixel coordinates of each anchor point in the reference camera are used as fixed reference coordinates.

[0053] Specifically, for each pair of reference cameras and target cameras, several discrete topological anchor points are set within the effective area of ​​the reference camera at fixed step sizes to construct the master control mesh structure. Let the first... The coordinates of each anchor point in the reference camera are as follows: These coordinates serve as fixed reference coordinates, used to describe the ground topology skeleton beneath the camera. In one specific embodiment, only one topological anchor point is retained for each grid cell to reduce redundancy and maintain the compactness of the topological representation. The purpose of this step is to discretize the continuous ground region into several fixed topological anchor points, thereby providing a foundation for subsequent multi-frame observation accumulation, robust fusion, and online refitting.

[0054] Step 7: After the master control grid structure is established, the present invention continuously estimates the observation position of each topological anchor point in the target camera in subsequent video frames, and stores the observation results in a hierarchical manner to form a multi-layer observation set oriented towards a time sliding window.

[0055] Specifically, for the first In this observation, the dense displacement estimation model is used to estimate the observation position of each anchor point in the target camera, and the observation layer at the current moment is formed.

[0056]

[0057] in, Indicates the first The observation position of each anchor point in the target camera This represents the corresponding observation confidence level. Multiple observation layers at different times constitute a multi-layered observation set within a time-sliding window.

[0058]

[0059] in, This is the length of the sliding window.

[0060] By explicitly storing observations at different times in a hierarchical manner, this invention avoids the direct contamination of the overall cross-camera topology by abnormal observations in a single frame, and improves the tolerance to local noise and instantaneous mismatches.

[0061] Step 8: After accumulating a sufficient number of observations within the sliding window, this invention robustly fuses multiple frames of observations for the same anchor point within the window. If multiple spatially close observations exist for the same anchor point, these consistent observations are fused to obtain the final position. If no stable and consistent observation cluster exists, the observation with the highest confidence level is selected as the target position estimate for that anchor point. Specifically, for any topological anchor point... Collect all available observations within the window to form an observation set:

[0062] ,

[0063] If a given anchor point has only a single observation, that observation is directly used as its target location estimate. If multiple observations exist, the spatial distances between these observations are further compared, and the observation cluster with the highest consistency is found within a given distance threshold. If the consistent cluster contains at least two observations, the mean of the coordinates within that cluster is taken as the fused position of the anchor point in the target view. Otherwise, it is considered that the current observations lack stable consensus, and the observation with the highest confidence is retained as the final result. This fusion strategy effectively suppresses outlier interference within the time window.

[0064] Step 9: After obtaining the fused anchor point correspondence, the present invention performs online refitting of the non-rigid spatial mapping between camera pairs based on the fusion result, and updates the cross-camera spatial mapping model of the current camera pair to achieve gradual optimization of cross-camera geometric relationships.

[0065] Specifically, the control point set is reconstructed using the fused anchor point correspondence, and the non-rigid spatial mapping from the source camera to the target camera and from the target camera to the source camera is refitted accordingly. This allows the cross-camera geometric relationship to be gradually optimized to a more stable state over time. Through the above online refitting and mapping update steps, this invention can continuously correct and update the cross-camera spatial relationship in subsequent videos, improving the stability and reliability of geometric constraints under long-term operating conditions.

[0066] Step 10: To evaluate the reliability of the constructed cross-camera spatial mapping relationship, this invention further performs a bidirectional loopback consistency assessment. Specifically, this assessment is conducted at each camera... and camera Sampling test points within the effective area. Using a camera. Medium sampling point For example, let's first use forward mapping. Map it to the camera Then use reverse mapping Map it back to the camera The loop result is obtained. Therefore, the closure error at this point is defined as...

[0067]

[0068] Similarly, it can be done in the camera Perform symmetric Loop closure mapping is performed, and the corresponding loop closure error set is obtained. Based on bidirectional loop closure errors, this invention statistically analyzes the proportion of loop in-loop points, the mean error, and the median error in both directions. Let the loop closure error threshold be... Then the proportions of the inner points of the loops in the two directions are respectively

[0069]

[0070] in, For indicator functions, and These represent the number of valid sampling points in the two directions, respectively. To ensure good consistency in the bidirectional spatial mapping, this invention uses the smaller proportion of loopback points in the two directions as the loopback consistency ratio.

[0071] Furthermore, let the mean values ​​of the two-way closure errors be respectively... and The medians are respectively and Then the error magnitude score is defined as

[0072]

[0073] This is a scaling constant for error attenuation. Furthermore, to avoid a situation where the mapping effect is good in only one direction while the other direction is significantly distorted, this invention also defines a bidirectional balance score.

[0074]

[0075] Finally, the loop closure consistency ratio, error magnitude score, and bidirectional balance score are weighted and fused to obtain the camera's consistency score. Spatial mapping reliability evaluation results:

[0076]

[0077] in, , and These are preset weighting coefficients. The reliability evaluation results... This is used to characterize the self-consistency of the current non-rigid spatial mapping on the effective support region. When the loop closure error is small, the proportion of bidirectional interior points is high, and the bidirectional errors are relatively balanced, it indicates that the spatial mapping relationship of the current camera pair has high reliability.

[0078] The reliability evaluation results The geometric quality score is used to characterize the spatial mapping relationship of the current camera. Based on this score, appearance similarity and geometric similarity are dynamically weighted and fused. When the geometric quality score is high, the weight of geometric similarity in the overall affinity is increased; when the geometric quality score is low, the weight of geometric similarity is decreased, causing the association process to degenerate into a mode dominated by appearance similarity. Based on the obtained overall affinity, local trajectories in different cameras are matched and associated to obtain the global continuous trajectory of the same target in the multi-camera system, achieving the output of multi-camera multi-target tracking results.

[0079] This invention has been validated on multiple publicly available multi-camera datasets. Experimental results show that, without requiring camera intrinsics and extrinsic parameters or additional calibration processes, this invention can effectively establish cross-camera spatial relationships and achieve global target tracking in multi-camera systems. Even in situations with significant viewpoint differences, severe local occlusion, and uneven scene terrain, this invention maintains good identity preservation capabilities and association stability.

[0080] Although the above embodiments have been described, those skilled in the art, once they understand the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the above descriptions are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made using the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping, characterized in that... Includes the following steps: Acquire multi-channel video data and obtain the target bounding box positions and corresponding appearance feature representations in each camera; The image used for cross-camera spatial mapping is segmented to extract the effective region that can be used for geometric modeling and generate a binary effective region mask. For camera pairs, a dense displacement estimation model is used to calculate pixel-level correspondences, resulting in a dense displacement field and spatial correspondence confidence. A sampling grid is constructed on the source camera image at a fixed step size. The grid points are then filtered to form a set of high-confidence control points from the source view to the target view. Based on the set of control points, a non-rigid spatial mapping is fitted to establish a cross-camera spatial mapping relationship under uncalibrated conditions; By utilizing the spatial mapping relationship, the spatial geometric similarity of the target bounding boxes in different cameras is calculated, and cross-camera target association is performed by combining the appearance feature similarity, thereby obtaining the global continuous trajectory in the multi-camera system and realizing multi-camera multi-target tracking under calibration conditions.

2. The calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping as described in claim 1, characterized in that... The method includes the following steps: Acquire multi-channel video data collected by a multi-camera system, and use object detection to obtain the target bounding box positions and corresponding appearance feature descriptors in each camera; The image used to initialize the cross-camera spatial mapping is segmented, the effective region that can be used for geometric modeling is extracted, and the corresponding binary effective region mask is generated to constrain the estimation range of subsequent cross-camera spatial mapping. For any pair of cameras, the pixel-level correspondence is calculated using a dense displacement estimation model to obtain the dense displacement field and the spatial correspondence confidence. A sampling grid is constructed on the source camera image with a fixed step size for filtering. Only point pairs that satisfy the spatial correspondence confidence constraint, the source domain effective area constraint, and the target domain effective area constraint are retained to form a set of high-confidence control points from the source view to the target view. Based on the obtained set of control points, non-rigid mappings from the source camera to the target camera and from the target camera to the source camera are fitted respectively to establish cross-camera spatial mapping relationships under calibration-free conditions. Construct the master control mesh structure for each pair of reference cameras and target cameras, set several discrete topological anchor points in the effective area of ​​the reference camera at a fixed step size, and use the pixel coordinates of each anchor point in the reference camera as fixed reference coordinates. In subsequent video frames, the dense displacement estimation model is used to continuously estimate the observation position of the topological anchor point in the target camera, and the anchor point observation results obtained at each time moment are stored in a hierarchical manner to form a multi-layer observation set oriented towards a time sliding window. After accumulating enough observations in the sliding window, robust fusion of multi-frame observations of each topological anchor point within the window is performed; if there are multiple observations with spatially close locations for the same anchor point, then these consistent observations are fused to obtain the final location; if there is no stable and consistent cluster of observations, then the observation with the highest confidence is selected as the target location estimate for that anchor point. Based on the obtained anchor point correspondence, the non-rigid spatial mapping between cameras is refitted online, and the corresponding cross-camera spatial mapping model is updated to achieve gradual optimization of cross-camera geometric relationships. Test points are sampled in the effective areas of each pair of cameras. The non-rigid mapping is used to perform bidirectional loop closure consistency evaluation. The loop closure error, the proportion of points in the loop closure, and the bidirectional error consistency are statistically analyzed. The reliability of the spatial mapping between cameras is evaluated to obtain a geometric quality score. The appearance similarity and geometric similarity are dynamically weighted and fused. Based on the obtained comprehensive affinity, the trajectory is associated across cameras to achieve the output of multi-camera multi-target tracking results.

3. The calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping as described in claim 2, characterized in that, A method for segmenting an image used to initialize a cross-camera spatial mapping and extracting a valid region suitable for geometric modeling includes the following steps: Let the first The initial image corresponding to each camera is Then, by segmentation, the corresponding binary effective region mask can be obtained: in, Represents pixels Located within the effective area, This indicates that the pixel will not participate in subsequent cross-camera spatial mapping estimation; The effective area includes one or more of the following: roads, ground, grass, carpet, mats, or steps, and people, vehicles, sky, trees, buildings, and water areas are excluded from the effective area.

4. The calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping as described in claim 2, characterized in that, The methods for obtaining the dense displacement field and its spatially corresponding confidence level include: Two images are matched using a dense displacement estimation model to obtain the result from the source camera. To the target camera Dense displacement fields and corresponding spatial confidence levels: ； And get from the camera To the camera Displacement field and its confidence level: 。 5. The calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping as described in claim 2, characterized in that, The method for obtaining the set of high-confidence control points includes the following steps: A regular sampling grid is constructed on the source camera image with a fixed step size. Let the center point of a certain grid in the source image be denoted as . Therefore, for a point to be accepted as a valid control point, it must simultaneously meet the following conditions: A point pair is valid only if all of the above constraints are satisfied. Only then are they retained as valid control point pairs, resulting in a set of high-confidence control points from the source perspective to the target perspective.

6. The calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping as described in claim 2, characterized in that, The non-rigid mapping includes the following steps: For from the camera To the camera The mapping, assuming any input point Then its mapping result can be expressed as: in, This is the affine parameter matrix, used to describe global translation, rotation, and scale changes; The non-rigid deformation weights corresponding to each control point; is the radial basis function for the thin plate spline.

7. The calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping as described in claim 2, characterized in that, The method for continuously estimating the observed position of the topological anchor point in the target camera using a dense displacement estimation model includes the following steps: For the In this observation, the observation position of each anchor point in the target camera is estimated using a dense displacement estimation model, and the observation layer at the current moment is formed: in, Indicates the first The observation position of each anchor point in the target camera This indicates the corresponding observation confidence level; Observations at multiple times constitute a multi-layered observation set within a time-sliding window: in, This is the length of the sliding window.

8. The calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping as described in claim 2, characterized in that, The method for robustly fusing multi-frame observations of each topological anchor point within a window includes the following steps: For any topological anchor point Collect all available observations within the window to form an observation set. If an anchor point has only a single observation, that observation is directly used as its target position estimate. If there are multiple observations, the spatial distances between these observations are further compared, and the observation cluster with the greatest consistency is found under a given distance threshold. If the consistency cluster contains no less than two observations, the mean of the coordinates within the cluster is taken as the fusion position of the anchor point in the target view. Otherwise, it is considered that the current observations lack stable consensus, and the observation with the highest confidence is retained as the final result.

9. The calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping as described in claim 2, characterized in that, Methods for updating the cross-camera space mapping model of the current camera pair include: The control point set is reconstructed using the fused anchor point correspondence, and the non-rigid spatial mapping from the source camera to the target camera and from the target camera to the source camera is refitted accordingly, so that the cross-camera geometric relationship is gradually optimized to a more stable state over time.

10. The calibration-free multi-camera multi-target tracking method based on non-rigid spatial mapping as described in claim 2, characterized in that, The method for performing a two-way loopback consensus assessment includes the following steps: In the camera and camera Sampling test points within the effective area, camera Medium sampling point First, use forward mapping Map it to the camera Then use reverse mapping Map it back to the camera The loop result is obtained. Therefore, the closure error at this point is defined as: Similarly, in the camera Perform symmetric Loop closure mapping is performed, and the corresponding set of loop closure errors is obtained. Based on the bidirectional loop closure errors, the proportion of loop in-loop points, the mean error, and the median error are calculated in both directions. set up The loop closure error threshold is Then the proportions of the inner points of the loops in the two directions are respectively: in, For indicator functions, and These represent the number of valid sampling points in the two directions, respectively. Let the mean values ​​of the two-way closure errors be respectively and The medians are respectively and Then the error magnitude score is defined as: Define the two-way balance score as the scaling constant for error decay: The camera's consistency score, error magnitude score, and bidirectional balance score are weighted and fused to obtain the camera's... Spatial mapping reliability evaluation results: in, , and The reliability evaluation results are based on preset weighting coefficients. It is used to characterize the degree of self-consistency of the current non-rigid spatial mapping on the effective support region.

Images