A multi-body association matching method, system, terminal and medium introducing uncertainty

Abstract

The application discloses a multi-motion body correlation matching method, system, terminal and medium introducing uncertainty, and belongs to the technical field of target tracking. The method comprises the following steps: acquiring detection information and filtering prediction information; judging whether the track length is greater than a preset threshold; if not, executing a first strategy, fusing a current matching metric and a speed prediction matching metric to obtain a first cost matrix; if yes, executing a second strategy, acquiring a historical matching metric, determining a weight according to a target proportion, fusing three types of matching metrics to obtain a second cost matrix, and then determining a scaling factor according to the target proportion to correct the second cost matrix; and outputting the cost matrix for correlation matching. The application improves the track continuity and matching recognition degree of small weak multi-target in an edge deployment scene, and effectively reduces the ID switching frequency and deployment cost.

Description

Technical Field

[0001] This application relates to the field of target tracking technology, specifically to a method, system, terminal, and medium for matching multiple moving bodies with introduced uncertainty. Background Technology

[0002] Currently, there are many technical solutions for multi-target photoelectric tracking navigation. Generally, software uses mainstream algorithms such as sort, deepsort, bytetracker, and botsort, inserting a ReID (Re-Identification) module. Hardware-wise, it employs dual-modal cameras, leveraging the advantages of dual-modal operation and using a strategy of complementary image characteristics. These solutions largely rely on high-performance detectors and computing cards. However, these methods often fail when encountering weak, multi-target objects.

[0003] The main reasons for this phenomenon are as follows: First, most current methods are designed for tracking and navigation of conventional large targets such as pedestrians or vehicles. Therefore, they are not very applicable to small targets with similar models, shapes, colors, materials, or structures, and whose proportion in the field of view is less than 1%. Second, small targets are inherently difficult to detect, resulting in high false alarm and false negative rates. Over-reliance on detector performance can lead to discontinuous tracking and frequent batching. Third, current methods use the optimal value of appearance similarity factor or IOU (Intersection over Union) factor for matching. For small targets with similar models, shapes, colors, materials, or structures, the calculation results of the above factors will be very close, causing matching chaos regardless of the gate settings, resulting in ID switching errors.

[0004] In summary, the current algorithm's insertion of the ReID module into the business process is unsuitable for weak, multi-target scenarios. Furthermore, when deploying ReID at the edge, its weight model requires separate training, quantization, and deployment, increasing training, time, and deployment costs. The secondary detection by the ReID module further consumes limited edge computing power, leading to decreased system frame rate, reduced real-time performance, and even failure to deploy and run properly on edge devices. To address these issues, there is an urgent need for an association matching method that is independent of the ReID module, adaptable to the characteristics of weak targets, and considers the limitations of edge computing power. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a method, system, terminal, and medium for multi-moving object association matching that introduces uncertainty. By introducing a velocity prediction matching metric with uncertainty correction, dynamic weighted fusion of multiple matching metrics driven by target proportion, and scaling factor correction, it improves the track continuity and matching recognition of weak multi-target objects in edge deployment scenarios without requiring a ReID module or relying on high-performance computing power, effectively reducing the number of ID switching times and deployment costs.

[0006] In a first aspect, the technical solution of the present invention provides a multi-moving-body association matching method that introduces uncertainty, applied to association matching scenarios where the area proportion of the moving body in the image field of view is less than or equal to a preset proportion, comprising the following steps: The detection information of multiple moving objects in the current frame and the filtered prediction information of existing motion trajectories are obtained. The detection information includes the position information and size information of each moving object in the image. Determine whether the length of the existing motion trajectory is greater than a preset length threshold; When the trajectory length is less than or equal to a preset length threshold, the first matching calculation strategy is executed. This strategy is configured to: calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information; introduce an uncertainty parameter to correct the change in detection information between adjacent frames to calculate the speed prediction matching metric; and fuse the current matching metric and the speed prediction matching metric to obtain the first cost matrix. When the trajectory length exceeds a preset length threshold, a second matching calculation strategy is executed. This strategy is configured as follows: calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information; introduce an uncertainty parameter to correct the change in detection information between adjacent frames to calculate the velocity prediction matching metric; obtain the historical matching metrics of the trajectory information of the previous frame and the detection information of the current frame; determine the fusion weight of each matching metric according to the target proportion of the moving object; fuse the current matching metric, the velocity prediction matching metric, and the historical matching metric based on the weight to obtain the second cost matrix; and determine the scaling factor according to the target proportion to correct the second cost matrix. Output the first cost matrix or the corrected second cost matrix as the cost matrix for multi-moving body association matching. Based on this cost matrix, perform association matching between multiple moving bodies in the current frame and existing motion trajectories to determine the matching relationship between each moving body and each existing motion trajectory.

[0007] Secondly, the technical solution of the present invention provides a multi-moving-body association matching system that introduces uncertainty, applied to association matching scenarios where the area ratio of the moving body in the image field of view is less than or equal to a preset ratio, including: The information acquisition module is used to acquire detection information of multiple moving objects in the current frame and filtered prediction information of existing motion trajectories. The detection information includes the position information and size information of each moving object in the image. The trajectory length determination module is used to determine whether the trajectory length of an existing motion trajectory is greater than a preset length threshold. The first matching calculation module is used to execute the first matching calculation strategy when the trajectory length is less than or equal to a preset length threshold. The strategy is configured to: calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information; introduce uncertainty parameters to correct the change in detection information between adjacent frames to calculate the speed prediction matching metric; and fuse the current matching metric and the speed prediction matching metric to obtain the first cost matrix. The second matching calculation module is used to execute a second matching calculation strategy when the trajectory length is greater than a preset length threshold. This strategy is configured as follows: calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information; introduce an uncertainty parameter to correct the change in detection information between adjacent frames to calculate the velocity prediction matching metric; obtain the historical matching metrics of the trajectory information of the previous frame and the detection information of the current frame; determine the fusion weight of each matching metric according to the target proportion of the moving object; fuse the current matching metric, the velocity prediction matching metric, and the historical matching metric based on the weight to obtain the second cost matrix; and determine the scaling factor according to the target proportion to correct the second cost matrix. The association matching module is used to output a first cost matrix or a corrected second cost matrix as the cost matrix for multi-moving body association matching. Based on the cost matrix, the module performs association matching between multiple moving bodies in the current frame and existing motion trajectories to determine the matching relationship between each moving body and each existing motion trajectory.

[0008] Thirdly, the technical solution of the present invention provides a terminal, including: The memory is used to store the multi-moving-body association matching program that introduces uncertainty; A processor is configured to implement the steps of the multi-moving body association matching method with introduced uncertainty as described above when executing the multi-moving body association matching procedure with introduced uncertainty.

[0009] Fourthly, the present invention provides a computer-readable storage medium storing a multi-moving body association matching program that introduces uncertainty. When the multi-moving body association matching program that introduces uncertainty is executed by a processor, it implements the steps of the multi-moving body association matching method that introduces uncertainty as described in any of the above claims.

[0010] As can be seen from the above technical solutions, this application has the following advantages: This method is compatible with the filtering algorithms in conventional methods and supports maintaining track continuity using filtered prediction values ​​when targets are missed or falsely detected. Furthermore, by introducing an uncertainty parameter to correct for changes in detection information between adjacent frames and calculating a velocity prediction matching metric, the velocity prediction box can cover the possible nonlinear motion range of the target, solving the problem of inaccurate prediction of nonlinear random motion by conventional filtering algorithms. Even when a target is briefly missed, the corrected prediction box still maintains a high degree of spatial overlap with the re-emerging detection box, allowing the track to continue and reducing frequent batching caused by track interruptions. Instead of relying on appearance features, it constructs matching criteria from three dimensions: spatial location (current matching metric), motion trend (velocity prediction matching metric), and temporal continuity (historical matching metric). This significantly increases the numerical differences in the cost matrix for targets with different motion patterns. For two targets with similar appearances but different motion trajectories, the numerical differences in their corresponding cost matrices are amplified, improving matching recognition and effectively reducing ID switching errors. By eliminating the need for the ReID module, the aforementioned training, deployment, and inference processes are eliminated, reducing the computational load and storage consumption of edge chips. System frame rate and real-time performance are guaranteed, making it fully adaptable to edge deployment scenarios with low computing power chips. Attached Figure Description

[0011] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic flowchart of a multi-moving-body association matching method that introduces uncertainty, provided in an embodiment of the present invention.

[0013] Figure 2 This is a schematic diagram of the execution process of the first matching calculation strategy.

[0014] Figure 3 This is a schematic diagram of the execution process of the second matching calculation strategy.

[0015] Figure 4 This is a schematic block diagram of a multi-moving-body association matching system with introduced uncertainty, provided as an embodiment of the present invention.

[0016] Figure 5 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The various terms used in this specification are merely descriptive of specific embodiments and do not constitute a limitation thereof.

[0019] Figure 1 This is a schematic flowchart illustrating a multi-moving-body association matching method introducing uncertainty, provided as an embodiment of the present invention. Figure 1 The executing entity can be a multi-moving body association matching system that introduces uncertainty. The multi-moving body association matching method with introduced uncertainty provided in this embodiment of the invention is executed by a computer device; correspondingly, the multi-moving body association matching system with introduced uncertainty runs on the computer device. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0020] The method in this embodiment is applied to association matching scenarios where the area of ​​a moving object in the image field of view is less than or equal to a preset ratio, such as... Figure 1 As shown, the method includes the following steps.

[0021] S1, acquire detection information of multiple moving objects in the current frame and filtered prediction information of existing motion trajectories. The detection information includes the position information and size information of each moving object in the image.

[0022] S2, determine whether the length of the existing motion trajectory is greater than the preset length threshold.

[0023] S3, when the trajectory length is less than or equal to a preset length threshold, execute the first matching calculation strategy. This strategy is configured as follows: calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information, introduce uncertainty parameters to correct the change in detection information between adjacent frames to calculate the velocity prediction matching metric, and fuse the current matching metric and the velocity prediction matching metric to obtain the first cost matrix.

[0024] S4. When the trajectory length is greater than the preset length threshold, the second matching calculation strategy is executed. This strategy is configured as follows: calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information; introduce an uncertainty parameter to correct the change in detection information between adjacent frames to calculate the velocity prediction matching metric; obtain the historical matching metrics of the trajectory information of the previous frame and the detection information of the current frame; determine the fusion weight of each matching metric according to the target proportion of the moving object; fuse the current matching metric, the velocity prediction matching metric, and the historical matching metric based on the weight to obtain the second cost matrix; and determine the scaling factor according to the target proportion to correct the second cost matrix.

[0025] S5, output the first cost matrix or the corrected second cost matrix as the cost matrix for multi-moving body association matching. Based on the cost matrix, perform association matching between multiple moving bodies in the current frame and existing motion trajectories to determine the matching relationship between each moving body and each existing motion trajectory.

[0026] The method in this embodiment is based on trajectories with different maturity levels, each with varying confidence levels in available information. Different matching strategies should be adapted accordingly. For new trajectories (short trajectory length), historical information is insufficient, so matching is performed solely based on the spatial overlap of the current frame and velocity prediction to avoid interference from unstable information. For mature trajectories (long trajectory length), historical information is sufficient, so the method integrates current matching, velocity prediction matching, and historical matching information, and dynamically allocates weights based on the target size. At the same time, scaling factors are used to amplify the numerical differences in the cost matrix, thereby improving the identification of small targets with high similarity.

[0027] Specifically, uncertainty-corrected velocity prediction is introduced to address the difficulty in predicting the nonlinear random motion of small targets. Four-dimensional uncertainty parameters (α, β, γ, δ) are defined to correct for changes in detection information between adjacent frames, generating velocity prediction boxes and calculating velocity prediction matching metrics, ensuring the prediction results cover the target's nonlinear motion range. New and mature targets are distinguished based on trajectory length: new targets only integrate current matching metrics and velocity prediction matching metrics; mature targets further incorporate historical matching metrics, achieving weighted fusion of three types of information. Furthermore, dynamic weight allocation is performed based on target station proportion. Targets are divided into medium-large, small, and micro categories according to their area proportion in the field of view, with different weight rankings configured for each category, adapting the matching strategy to the detection characteristics of targets of different sizes. Simultaneously, a scaling factor amplifies differentiation; that is, a scaling factor is determined based on the target proportion to correct the cost matrix, amplifying the numerical differences in cost between different targets and improving the matching and identification accuracy of highly similar small targets.

[0028] As a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process of this embodiment, the following will provide possible embodiments to describe the specific implementation of the above steps in a non-limiting manner.

[0029] The method in this embodiment is applied to association matching scenarios where the area of ​​the moving object in the image field of view is less than or equal to a preset ratio, preferably 1%.

[0030] In this embodiment, step S1 involves acquiring relevant information. First, an infrared imaging detector is used to acquire the current frame image. The infrared imaging detector is preferably a 640×512 resolution infrared focal plane array detector, used to acquire the infrared radiation image of the scene. Then, a target detector is used to perform target detection on the current frame infrared image to obtain detection information for multiple moving objects in the current frame. The detection information includes the position and size information of each moving object in the image. The position information includes the x-coordinate of the center point of the moving object detection box. and the center point ordinate Size information includes the width of the moving object detection frame. and height The bounding box is the smallest bounding rectangle output by the object detection algorithm, used to define the spatial extent of the target in an image.

[0031] For example, the object detector can be implemented using object detection algorithms based on convolutional neural networks, including but not limited to the YOLO series, Cascade R-CNN series, CenterNet, etc. Since this embodiment targets small targets (field of view ≤1%), a lightweight detection network is preferably used, which simplifies the number of network layers (convolutional layers, pooling layers) and improves the multi-scale feature fusion strategy to suit the characteristics of small targets, thus adapting to the limited computing power of edge chips. The detector outputs the coordinate information of each object detection box. And its corresponding confidence score.

[0032] Simultaneously, for the established motion trajectories, a filtering algorithm is used to recursively predict the state of each trajectory at the current frame time, obtaining filtered prediction information for the existing motion trajectories. The filtered prediction information includes the position and size information of the prediction box, specifically the x-coordinate of the center point of the filtered prediction box. , center point ordinate ,width and height .

[0033] In a preferred embodiment, the filtering algorithm employs a Kalman filter, suitable for multi-target tracking in edge deployment scenarios. In other embodiments, the filtering algorithm may also employ other predictive filtering algorithms such as α-β filtering or particle filtering.

[0034] In this embodiment, the preset length threshold is 2. The trajectory length is less than 2 as the judgment mark. When the trajectory length is less than or equal to 2, it is judged as a new target and the first matching calculation strategy is executed. Otherwise, it is judged as a mature target and the second matching calculation strategy is executed.

[0035] See Figure 2 In this embodiment, step S3 executes a first matching calculation strategy for the new target. This strategy relies only on the current frame intersection-over-union ratio and velocity prediction to avoid introducing unstable factors due to the short track and improve the success rate of weak multi-target tracking and matching. Specifically, it includes the following steps.

[0036] S311, calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information.

[0037] First, extract the position and size information of the moving object detection box in the current frame from the detection information obtained in step S1, denoted as detection box i, with the x-coordinate of its center point as... The center point's ordinate is Width is Height is Extract the position and size information of the existing motion trajectory filtering prediction box from the filtering prediction information obtained in step S1, denoted as prediction box j, with the x-coordinate of its center point as... The center point's ordinate is Width is Height is .

[0038] Then, the Intersection over Union (IoU) ratio between the detection box i and the predicted box j is calculated. The formula for calculating the intersection-union ratio is:

[0039] in, Let i be the area of ​​the detection box. To predict the area of ​​box j, Let be the intersection area of ​​the detection box i and the predicted box j.

[0040] Since the cost matrix in this embodiment is used to interface with the cost input of the Hungarian algorithm, and the Hungarian algorithm aims to minimize the cost (i.e., the smaller the cost, the better the match), while the larger the crossover ratio (CRR) itself, the better the match, the opposite direction is achieved. Therefore, the CRR is converted into cost form, resulting in the current matching metric = 1 - .

[0041] S312 introduces an uncertainty parameter to correct the change in detection information between adjacent frames in order to calculate the speed prediction matching metric.

[0042] In this embodiment, the uncertainty parameter includes the first uncertainty of the moving object in the horizontal coordinate direction of the image. The second uncertainty in the ordinate direction The third uncertainty of the detection frame width The fourth uncertainty of the detection frame height Each parameter ranges from 0 to 1, representing the root mean square error of the nonlinear random motion of the tracked object compared to the average value.

[0043] The uncertainty parameter is obtained by any of the following methods.

[0044] Method 1 (Small Sample Learning): Collect video sample frames where the area of ​​a moving object in the field of view is less than or equal to a preset proportion; extract the change in the detection box of the same moving object in adjacent sample frames to form a dataset of changes in four dimensions: x-axis, y-axis, width, and height; calculate the mean square error of the change dataset for each dimension, and use the mean square error as the initial uncertainty value for each dimension. During the runtime phase, determine the scene correction coefficient based on the actual environmental interference state, and use the scene correction coefficient to linearly correct the initial uncertainty value to obtain the corrected uncertainty parameter.

[0045] Specifically, video sample frames of moving objects consistent with a preset application scenario are collected. The area of ​​the moving object in the field of view is less than or equal to a preset proportion, preferably 1%, and the sample size is 500 to 1000 frames. The moving objects in each sample frame are labeled and normalized preprocessed to remove false alarm frames and missed detection frames. The change in the detection box of the same moving object in adjacent sample frames is extracted to form a dataset of changes in four dimensions: horizontal coordinate, vertical coordinate, width, and height. The mean and standard deviation of the change datasets in each dimension are calculated, and the standard deviation is used as the initial uncertainty value for each dimension. The formulas for calculating the mean and standard deviation are:

[0046]

[0047] Where n is the number of valid sample frames, μ is the mean of the change dataset, and σ is the standard deviation of the change dataset.

[0048] During the operation phase, based on environmental interference factors in the actual application scenario (such as atmospheric turbulence, infrared noise, and target motion characteristics), a scenario correction coefficient is used to linearly correct the initial uncertainty value, resulting in the corrected uncertainty parameter, expressed as:

[0049] Where k is the scene correction coefficient, with a value ranging from 0.8 to 1.2: k=1 when there is no obvious environmental interference, k is 1.1 to 1.2 when there is strong interference (such as severe atmospheric turbulence), and k is 0.8 to 0.9 when there is weak interference. Finally, the corrected uncertainty parameters are solidified into the algorithm program to adapt to edge chip deployment; it supports incremental updates of small samples every 1000 frames.

[0050] Method 2 (Engineering Experience Setting Method): Select from the preset value range based on the speed level of the moving object.

[0051] Specifically, in conventional applications of infrared imaging detectors, uncertainty parameters are directly selected from a preset range based on the speed level of the moving object. For low-speed, weak target scenarios: α∈[0.02,0.05], β∈[0.02,0.05], γ∈[0.01,0.03], δ∈[0.01,0.03]; for high-speed, weak target scenarios: α∈[0.05,0.1], β∈[0.05,0.1], γ∈[0.03,0.06], δ∈[0.03,0.06]; for scenarios with strong noise interference: α∈[0.08,0.12], β∈[0.08,0.12], γ∈[0.05,0.08], δ∈[0.05,0.08].

[0052] First, calculate the change in detection information between adjacent frames. This change represents the trend of a moving object's position and size changes from the previous frame to the current frame, and the calculation formula is:

[0053] in, These represent the changes in the x-coordinate, y-coordinate, detection box width, and detection box height of the moving object between adjacent frames, respectively. Velocity here is represented by the rate of change of the target's position and size per unit time (between adjacent frames). In this embodiment, one frame is used as the unit time, meaning the change in target position and size is obtained using the adjacent frame difference method. In other embodiments, the unit time can be set to several milliseconds depending on engineering requirements. The core principle is to use adjacent frames... The target's position and size changes are quantified to predict the target's trend. Two-dimensional relationships are used to describe the target's motion and attitude changes in the image, simplifying complex kinematic physical modeling in three-dimensional space.

[0054] After obtaining the uncertainty parameters, the velocity prediction box is calculated using the following formula. Location and size information:

[0055] in, These represent the x-coordinate, y-coordinate, width, and height of the center point of the velocity prediction box, respectively.

[0056] Using the current frame detection box as a reference, the change in detection information between adjacent frames is... The predicted bounding box is multiplied by an uncertainty parameter and then superimposed onto the current frame's detection box to obtain a predicted bounding box that is moderately offset along the target's motion trend. The uncertainty parameter controls the offset magnitude; the larger the parameter value, the greater the offset of the predicted bounding box relative to the current frame's detection box, and the wider the range that the target may reach; the smaller the parameter value, the closer the predicted bounding box is to the current frame's detection box, and the more conservative the prediction.

[0057] Calculate the intersection-over-union ratio (IoU) between the velocity prediction box and the detection box in the current frame based on the velocity prediction box information. , is represented as:

[0058] The Cross-Union Ratio (CIRR) ranges from [0, 1]. A higher value indicates a greater spatial overlap between the detected bounding box and the velocity-predicted bounding box, suggesting a better match between the current detection result and the target's historical movement trend. Converting the CIRR to a cost form yields the velocity prediction matching metric. The smaller the velocity prediction matching metric, the more closely the current frame detection matches the target's historical motion trend, and the higher the probability of a match.

[0059] S313, the first cost matrix is ​​obtained by fusing the current matching metric and the speed prediction matching metric.

[0060] Obtain the current matching metric and the velocity prediction matching metric, and determine their respective weights. Since the new target's trajectory is short and historical information is insufficient, and the current frame detection information represents actual observations with high reliability, the current matching metric should dominate; the velocity prediction matching metric, as a motion trend constraint, plays a supporting role. In a preferred embodiment, the first weight of the current matching metric is... The second weight of the speed prediction matching metric That is, the first weight is greater than the second weight.

[0061] In other embodiments, the weights can also be dynamically adjusted based on the detector's confidence level: when the detection confidence level is higher than a preset threshold, Take a value between 0.85 and 0.95. Use a value between 0.05 and 0.15; when the detection confidence is low, increase it appropriately. The proportion of motion prediction was reduced to 0.2% to 0.3%, which enhanced the guiding role of motion prediction in matching.

[0062] First cost matrix The calculation method involves summing the weighted matching metrics of the two items, and the formula is as follows:

[0063] The data format is connected to the cost matrix input interface of the Hungarian algorithm. The calculated value reflects the matching between the target and the existing trajectory. The smaller the value, the better the match.

[0064] The first matching calculation strategy constructs a cost matrix by fusing the current matching metric and the velocity prediction matching metric. Compared with the matching method that only relies on a single IoU, it adds motion trend constraints, so that the new target can still maintain the continuity of matching when it is briefly occluded or detects fluctuations. At the same time, since the historical information of the new trajectory is insufficient, this strategy does not introduce historical matching metrics, thus avoiding the interference of unstable factors introduced by insufficient historical information on the matching results.

[0065] See Figure 3 In this embodiment, step S4 executes a second matching calculation strategy for mature targets, which specifically includes the following steps.

[0066] S411, calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information.

[0067] S412 introduces an uncertainty parameter to correct for changes in detection information between adjacent frames in order to calculate the speed prediction matching metric.

[0068] Steps S411 and S412 are the same as steps S311 and S312 above, and will not be repeated here.

[0069] S413, obtain the historical matching metric between the trajectory information of the previous frame and the detection information of the current frame.

[0070] Read the previous frame from the storage unit, i.e. At any given time, the trajectory bounding box information corresponding to that trajectory is denoted as... Its location and size information includes the x-coordinate of the center point. , center point ordinate ,width and height This information can be either the filtered prediction result of the previous frame or the detection result of the previous frame, depending on the trajectory management strategy.

[0071] Calculate the intersection-over-union ratio (IoU) between the trajectory bounding box of the previous frame and the detection bounding box of the current frame. The calculation formula is:

[0072] Transforming the intersection-union ratio into a cost form yields the historical matching metric. The smaller this value, the better the spatial continuity and smoothness between the current frame detection and the trajectory of the previous frame, and the higher the probability that the two belong to the same target.

[0073] S414. Determine the fusion weight of each matching metric based on the target proportion of the moving body, and fuse the current matching metric, the velocity prediction matching metric, and the historical matching metric based on the weight to obtain the second cost matrix.

[0074] First, calculate the area proportion of the moving object in the current frame image. The formula for calculating the target proportion R is:

[0075] in, W and H are the width and height of the detection box in the current frame, respectively, and the total width and total height of the image frame, respectively.

[0076] Then, the target type is determined based on the target proportion R, and the weight corresponding to the current matching metric is determined accordingly. Weights corresponding to historical matching metrics Weights corresponding to the speed prediction matching metric The weighting is assigned according to the following rules: (1) When the target proportion is greater than the first preset threshold and less than or equal to the second preset threshold, it is determined to be a medium-to-large target, and the weight ranking is as follows: That is, the weight of the current matching metric is greater than the weight of the velocity prediction matching metric, and the weight of the velocity prediction matching metric is greater than the weight of the historical matching metric. This is because medium and large targets occupy more pixels in the image, the detection box is more stable, and the velocity prediction reliability is higher, while the historical changes between adjacent frames are relatively smooth, and the contribution of historical matching metrics is smaller. In a preferred embodiment, the first preset threshold is 0.5%, and the second preset threshold is 1%, that is, when the target proportion is greater than 0.5% and less than or equal to 1%, it is determined to be a medium and large target.

[0077] (2) When the target proportion is greater than the third preset threshold and less than or equal to the first preset threshold, it is determined to be a small target, and the weight ranking is as follows: That is, the weight of the current matching metric is greater than the weight of the historical matching metric, and the weight of the historical matching metric is greater than the weight of the velocity prediction matching metric. This is because the detection box of small targets fluctuates more, reducing the reliability of velocity prediction, while the historical matching metric provides additional temporal constraints using trajectory information from the previous frame, helping to maintain matching stability. In a preferred embodiment, the third preset threshold is 0.1%, meaning that a target is determined to be a small target when its proportion is greater than 0.1% and less than or equal to 0.5%.

[0078] (3) When the target proportion is less than or equal to the third preset threshold, it is determined to be a micro-target, and the weight ranking is as follows: ,and In other words, the weight of the current matching metric is greater than the weight of the historical matching metric, while the weight of the velocity prediction matching metric is zero. This is because micro-targets occupy only a very small number of pixels in an image, and the detection box is highly susceptible to interference factors such as image noise, quantization errors, and atmospheric jitter. Predictions based on velocity information have extremely low reliability for such targets and may even introduce misleading matches. Therefore, matching should not rely on velocity prediction information, but only on the current and historical matching metrics. In a preferred embodiment, a target is identified as a micro-target when its proportion is less than or equal to 0.1%.

[0079] In a preferred embodiment, the weight values ​​for each type of target are as follows: Medium to large-sized targets: Typical value ; Small goals: Typical value ; Miniature targets: Typical value .

[0080] In scenarios with strong environmental interference (such as severe atmospheric turbulence or strong infrared noise), the weighting value can be adjusted appropriately: increase it during strong interference. The value of is adjusted to enhance the reliability of real-time detection results; the value is appropriately reduced when there is weak interference. The value of is chosen to balance the contribution of each matching metric.

[0081] The second cost matrix is ​​obtained by weighting and summing the three matching metrics based on the determined weights. The calculation formula is:

[0082] The data format is consistent with the first cost matrix, and both are connected to the cost matrix input interface of the Hungarian algorithm.

[0083] S415, the scaling factor is determined based on the target proportion to correct the second cost matrix.

[0084] The target type is determined based on the area ratio of the moving object in the current frame image. The target types include medium-to-large targets, small targets, and micro targets, and the target type changes sequentially as the area ratio decreases.

[0085] Determine the scaling factor based on the target type. Scaling factor The scaling factor is negatively correlated with the target proportion R; the smaller the target proportion, the larger the scaling factor value. The smaller the target, the fewer pixels the detection box occupies in the image, and the smaller the absolute difference between the various intersection-union ratios. Therefore, it is more necessary to amplify the small differences through the scaling factor to improve the matching and recognition of small targets with high similarity and avoid matching confusion.

[0086] In a preferred embodiment, the scaling factor is determined according to the following rules: Medium to large-sized targets (0.5%) <R≤1%): A typical value is 0.2; Small target (0.1%) <R≤0.5%): The typical value is 0.4; Miniature target (R≤0.1%): The typical value is 0.8.

[0087] In special scenarios, when the false alarm rate of the detection box is greater than 30%, a preset offset is added to the scaling factor to further amplify the numerical difference of the cost matrix and cope with the matching in scenarios with high false alarm rates.

[0088] The second cost matrix is ​​corrected using a scaling factor, which is achieved by multiplying the second cost matrix by the scaling factor, as follows:

[0089] Revised As the final output of the second cost matrix, the differences between its values ​​are significantly amplified, which helps the Hungarian algorithm make more accurate matching decisions between weak targets with high similarity.

[0090] The second matching calculation strategy constructs a cost matrix by integrating three types of information: current matching metric, speed prediction matching metric, and historical matching metric. It dynamically allocates the weight of each matching metric according to the target proportion and uses a scaling factor to correct the cost matrix, thereby achieving multi-dimensional and adaptive matching of mature targets and significantly improving the matching and identification of weak targets with high similarity.

[0091] The first cost matrix obtained in step S313 or the second cost matrix corrected in step S415 is output as the cost matrix for multi-moving object association matching and input into the Hungarian algorithm. The Hungarian algorithm uses this cost matrix as input and minimizes it to find the optimal one-to-one matching relationship between multiple moving object detection boxes and existing motion trajectories in the current frame, determining the matching relationship between each moving object and each existing motion trajectory. The matched trajectories are then entered into the trajectory management module to perform operations such as trajectory updating, deletion, or creation.

[0092] The foregoing has described in detail an embodiment of a multi-moving body association matching method that introduces uncertainty. Based on the multi-moving body association matching method with introduced uncertainty described in the above embodiment, this invention also provides a multi-moving body association matching system with introduced uncertainty corresponding to the method.

[0093] Figure 4 This is a schematic block diagram of a multi-moving body association matching system introducing uncertainty, provided as an embodiment of the present invention. In this embodiment, the multi-moving body association matching system introducing uncertainty is applied to association matching scenarios where the area ratio of the moving body in the image field of view is less than or equal to a preset ratio, depending on the function it performs. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and is stored in memory.

[0094] The information acquisition module is used to acquire detection information of multiple moving objects in the current frame and filtered prediction information of existing motion trajectories. The detection information includes the position information and size information of each moving object in the image.

[0095] The trajectory length determination module is used to determine whether the length of an existing motion trajectory is greater than a preset length threshold.

[0096] The first matching calculation module is used to execute a first matching calculation strategy when the trajectory length is less than or equal to a preset length threshold. The strategy is configured to: calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information; introduce uncertainty parameters to correct the change in detection information between adjacent frames to calculate the velocity prediction matching metric; and fuse the current matching metric and the velocity prediction matching metric to obtain the first cost matrix.

[0097] The second matching calculation module is used to execute a second matching calculation strategy when the trajectory length is greater than a preset length threshold. This strategy is configured as follows: calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information; introduce an uncertainty parameter to correct the change in detection information between adjacent frames to calculate the velocity prediction matching metric; obtain the historical matching metrics of the trajectory information of the previous frame and the detection information of the current frame; determine the fusion weight of each matching metric according to the target proportion of the moving object; fuse the current matching metric, the velocity prediction matching metric, and the historical matching metric based on the weight to obtain the second cost matrix; and determine the scaling factor according to the target proportion to correct the second cost matrix.

[0098] The association matching module is used to output a first cost matrix or a corrected second cost matrix as the cost matrix for multi-moving body association matching. Based on the cost matrix, the module performs association matching between multiple moving bodies in the current frame and existing motion trajectories to determine the matching relationship between each moving body and each existing motion trajectory.

[0099] The multi-moving body association matching system with introduced uncertainty in this embodiment is used to implement the aforementioned multi-moving body association matching method with introduced uncertainty. Therefore, the specific implementation of this system can be found in the embodiment section of the multi-moving body association matching method with introduced uncertainty mentioned above. Thus, the specific implementation can be referred to the description of the corresponding embodiments, and will not be elaborated here.

[0100] Furthermore, since the multi-moving body association matching system with introduced uncertainty in this embodiment is used to implement the aforementioned multi-moving body association matching method with introduced uncertainty, its function corresponds to the function of the above method, and will not be repeated here.

[0101] Figure 5 This is a schematic diagram of a terminal 500 provided in an embodiment of the present invention, including: a processor 510, a memory 520, and a communication unit 530. The processor 510 is used to implement the process steps of the above-described embodiment of the multi-moving body association matching method with introduced uncertainty when implementing the multi-moving body association matching program with introduced uncertainty stored in the memory 520.

[0102] This invention also provides a computer storage medium, which may be a magnetic disk, an optical disk, read-only memory (ROM), or random access memory (RAM), etc. The computer storage medium stores a multi-moving body correlation matching program that introduces uncertainty. When the multi-moving body correlation matching program that introduces uncertainty is executed by a processor, it implements the process steps of the above-described embodiment of the multi-moving body correlation matching method that introduces uncertainty.

[0103] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for matching associations of multiple moving bodies that introduces uncertainty, characterized in that, For association matching scenarios where the area of ​​a moving object in the image's field of view is less than or equal to a preset ratio, the following steps are included: The detection information of multiple moving objects in the current frame and the filtered prediction information of existing motion trajectories are obtained. The detection information includes the position information and size information of each moving object in the image. Determine whether the length of the existing motion trajectory is greater than a preset length threshold; When the trajectory length is less than or equal to a preset length threshold, the first matching calculation strategy is executed. This strategy is configured to: calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information; introduce an uncertainty parameter to correct the change in detection information between adjacent frames to calculate the speed prediction matching metric; and fuse the current matching metric and the speed prediction matching metric to obtain the first cost matrix. When the trajectory length exceeds a preset length threshold, a second matching calculation strategy is executed. This strategy is configured as follows: calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information; introduce an uncertainty parameter to correct the change in detection information between adjacent frames to calculate the velocity prediction matching metric; obtain the historical matching metrics of the trajectory information of the previous frame and the detection information of the current frame; determine the fusion weight of each matching metric according to the target proportion of the moving object; fuse the current matching metric, the velocity prediction matching metric, and the historical matching metric based on the weight to obtain the second cost matrix; and determine the scaling factor according to the target proportion to correct the second cost matrix. Output the first cost matrix or the corrected second cost matrix as the cost matrix for multi-moving body association matching. Based on this cost matrix, perform association matching between multiple moving bodies in the current frame and existing motion trajectories to determine the matching relationship between each moving body and each existing motion trajectory.

2. The multi-moving-body association matching method introducing uncertainty according to claim 1, characterized in that, The uncertainty parameter includes the first uncertainty of the moving object in the horizontal axis direction of the image. The second uncertainty in the ordinate direction The third uncertainty of the detection frame width The fourth uncertainty of the detection frame height .

3. The multi-moving-body association matching method introducing uncertainty according to claim 2, characterized in that, The uncertainty parameter is obtained by any of the following methods; Method 1: Collect video sample frames in which the area of ​​the moving object in the field of view is less than or equal to a preset proportion; Extract the change in the detection box of the same moving object in adjacent sample frames to form a dataset of changes in four dimensions: horizontal axis, vertical axis, width, and height. Calculate the mean squared error of the dataset for each dimension of change, and use the mean squared error as the initial uncertainty value for each dimension. The scenario correction coefficient is determined based on the actual environmental disturbance state. The initial uncertainty value is linearly corrected using the scenario correction coefficient to obtain the corrected uncertainty parameter. Method 2: The value is selected from a preset range based on the speed level of the moving object.

4. The multi-moving-body association matching method introducing uncertainty according to claim 2, characterized in that, An uncertainty parameter is introduced to correct for changes in detection information between adjacent frames in order to calculate the speed prediction matching metric, specifically including: Calculate the change in detection information between adjacent frames , represented as: in, These represent the changes in the x-coordinate, y-coordinate, detection box width, and detection box height of the moving object between adjacent frames, respectively. These represent the x-coordinate, y-coordinate, width, and height of the center point of the moving object detection box in the current frame, respectively. These represent the x-coordinate, y-coordinate, width, and height of the center point of the existing motion trajectory filtering prediction box, respectively. The velocity prediction box information is calculated using the following formula. This includes location and size information: in, These are the x-coordinate of the center point, y-coordinate of the center point, width, and height of the velocity prediction box, respectively. Calculate the intersection-over-union ratio (IoU) between the velocity prediction box and the detection box in the current frame based on the velocity prediction box information. Therefore, the speed prediction matching metric is obtained as follows: .

5. The multi-moving-body association matching method introducing uncertainty according to claim 4, characterized in that, The first cost matrix is ​​obtained by fusing the current matching metric and the speed prediction matching metric, specifically including: Determine the weights of the current matching metric and the speed prediction matching metric; The first cost matrix is ​​obtained by weighting and summing the current matching metric and the velocity prediction matching metric based on the determined weights; where the current matching metric is 1 minus the intersection-union ratio of the existing trajectory filtering prediction value and the detection box of the current frame.

6. The multi-moving-body association matching method introducing uncertainty according to claim 4, characterized in that, The fusion weights of each matching metric are determined based on the target proportion of the moving object. The current matching metric, the velocity prediction matching metric, and the historical matching metric are then fused based on these weights to obtain the second cost matrix, which specifically includes: When the target proportion is greater than the first preset threshold and less than or equal to the second preset threshold, it is determined to be a medium-to-large target. The weight of the current matching metric is greater than the weight of the speed prediction matching metric, and the weight of the speed prediction matching metric is greater than the weight of the historical matching metric. When the target proportion is greater than the third preset threshold and less than or equal to the first preset threshold, it is determined to be a small target. The weight of the current matching metric is greater than the weight of the historical matching metric, and the weight of the historical matching metric is greater than the weight of the speed prediction matching metric. When the target proportion is less than or equal to the third preset threshold, it is determined to be a micro target, the weight of the current matching metric is greater than the weight of the historical matching metric, and the weight of the speed prediction matching metric is zero. The second cost matrix is ​​obtained by weighting and summing the current matching metric, the speed prediction matching metric, and the historical matching metric based on the determined weights.

7. The multi-moving-body association matching method introducing uncertainty according to claim 6, characterized in that, The scaling factor is determined based on the target proportion to modify the second cost matrix, specifically including: The target type is determined based on the area ratio of the moving object in the current frame image. The target types include medium-to-large targets, small targets, and micro targets, and the target type changes sequentially as the area ratio decreases. The scaling factor is determined based on the target type, where the scaling factor is negatively correlated with the area ratio; Multiply the second cost matrix by a scaling factor to increase the difference between the values ​​in the second cost matrix.

8. A multi-moving-body association matching system introducing uncertainty, characterized in that, This applies to association matching scenarios where the area of ​​a moving object in the image's field of view is less than or equal to a preset ratio, including: The information acquisition module is used to acquire detection information of multiple moving objects in the current frame and filtered prediction information of existing motion trajectories. The detection information includes the position information and size information of each moving object in the image. The trajectory length determination module is used to determine whether the trajectory length of an existing motion trajectory is greater than a preset length threshold. The first matching calculation module is used to execute the first matching calculation strategy when the trajectory length is less than or equal to a preset length threshold. The strategy is configured to: calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information; introduce uncertainty parameters to correct the change in detection information between adjacent frames to calculate the speed prediction matching metric; and fuse the current matching metric and the speed prediction matching metric to obtain the first cost matrix. The second matching calculation module is used to execute a second matching calculation strategy when the trajectory length is greater than a preset length threshold. This strategy is configured as follows: calculate the current matching metric based on the current frame detection information and the existing trajectory filtering prediction information; introduce an uncertainty parameter to correct the change in detection information between adjacent frames to calculate the velocity prediction matching metric; obtain the historical matching metrics of the trajectory information of the previous frame and the detection information of the current frame; determine the fusion weight of each matching metric according to the target proportion of the moving object; fuse the current matching metric, the velocity prediction matching metric, and the historical matching metric based on the weight to obtain the second cost matrix; and determine the scaling factor according to the target proportion to correct the second cost matrix. The association matching module is used to output a first cost matrix or a corrected second cost matrix as the cost matrix for multi-moving body association matching. Based on the cost matrix, the module performs association matching between multiple moving bodies in the current frame and existing motion trajectories to determine the matching relationship between each moving body and each existing motion trajectory.

9. A terminal, characterized in that, include: The memory is used to store the multi-moving-body association matching program that introduces uncertainty; A processor, configured to implement the steps of the multi-moving-body association matching method with introduced uncertainty as described in any one of claims 1 to 7 when executing the multi-moving-body association matching procedure with introduced uncertainty.

10. A computer-readable storage medium, characterized in that, The readable storage medium stores a multi-moving body association matching program that introduces uncertainty, which, when executed by a processor, implements the steps of the multi-moving body association matching method that introduces uncertainty as described in any one of claims 1 to 7.

Images