Multi-camera cooperative perception dynamic target real-time ranging tracking method and system

Abstract

The application discloses a dynamic target real-time ranging tracking method and system based on multi-camera cooperative sensing, relates to the technical field of target tracking, and is based on a target function to calculate a target value of each initial decision variable configuration, perform non-dominated sorting on all initial decision variable configurations, divide the initial decision variable configurations into different Pareto levels, perform content randomization operation on all initial decision variable configurations, iteratively optimize newly generated decision variable configurations, output all Pareto frontier solution sets when a convergence condition is met, and apply the Pareto frontier solution sets to corresponding scenes according to scene changes. The tracking system comprehensively considers multiple targets such as ranging accuracy, tracking robustness and real-time performance, generates a group of non-dominated solutions, can dynamically adapt to different scene requirements, realizes multi-dimensional balanced optimization of performance, and significantly improves the flexibility and adaptability of the system in combination with specific requirements of a real-time scene.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, specifically to a method and system for real-time ranging and tracking of dynamic targets using multi-camera collaborative sensing. Background Technology

[0002] In fields such as intelligent transportation, autonomous driving, security monitoring, and industrial automation, real-time dynamic target ranging and tracking are crucial for achieving accurate perception. Traditional single-camera ranging and target tracking methods are limited by issues such as field of view, occlusion, and insufficient depth information, making it difficult to provide reliable perception results in complex dynamic scenes. To address these problems, multi-camera collaborative perception systems have emerged.

[0003] The existing technology has the following shortcomings:

[0004] 1. Existing technologies typically focus on a single optimization objective, such as maximizing ranging accuracy or achieving optimal real-time performance, while neglecting the interrelationships between multiple objectives. In systems where ranging accuracy is paramount, excessive pursuit of computational details may lead to a decrease in real-time performance, making it unable to meet the demands of scenarios with rapidly changing dynamic targets. In systems where real-time performance is paramount, reducing latency often sacrifices the robustness of target tracking, resulting in unstable performance in occluded or complex scenarios. Single-objective optimization cannot adapt to the changing demands of real-world scenarios, especially in applications where the performance of multiple objectives needs to be dynamically balanced (such as intelligent transportation and drone swarms), where the system struggles to achieve optimal performance.

[0005] 2. Existing technologies are usually based on static configuration and preset parameters, lacking the ability to adjust to dynamic scene changes. For example, when the number of targets increases or the occlusion situation worsens, the system cannot dynamically adjust the camera field of view and task allocation strategy, resulting in an increased tracking loss rate. In complex scenes (such as densely trafficked intersections or high-speed crossings of moving targets), traditional methods cannot optimize weights and parameters in real time, resulting in a significant decrease in system performance, rigid system operation, and difficulty in adapting to complex real-time changes, leading to unstable ranging or tracking results.

[0006] Based on this, this application proposes a dynamic target real-time ranging and tracking method and system with multi-camera collaborative perception. It comprehensively considers multiple objectives such as ranging accuracy, tracking robustness and real-time performance, and generates a set of non-dominated solutions. It can dynamically adapt to different scenario requirements, achieve multi-dimensional balanced optimization of performance, and significantly improve the flexibility and adaptability of the system by combining the specific requirements of real-time scenarios. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for real-time ranging and tracking of dynamic targets using multi-camera collaborative sensing, in order to address the shortcomings in the prior art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a real-time ranging and tracking method for dynamic targets based on multi-camera collaborative sensing, the tracking method comprising the following steps:

[0009] The tracking system generates multiple initial decision variable configurations through initialization, calculates the target value of each initial decision variable configuration based on the objective function, performs non-dominated sorting on all initial decision variable configurations, and divides them into different Pareto levels;

[0010] After randomizing the content of all initial decision variable configurations, the newly generated decision variable configurations are iteratively optimized. When the convergence condition is met, all Pareto front solutions are output, and the Pareto front solutions are dynamically selected and applied to the corresponding scenarios according to the changes in the scenarios.

[0011] In a preferred embodiment, the objective value for each initial decision variable configuration is calculated based on an objective function, the function expression of which is: In the formula, For the target value, The robustness index In response to the latency index, , These are the proportional coefficients for the robustness index and the response delay index, respectively. , All are greater than 0.

[0012] In a preferred embodiment, all initial decision variable configurations are non-dominated and ranked into different Pareto levels, including the following steps:

[0013] After selecting the current initial decision variable configuration, compare the current initial decision variable configuration one by one;

[0014] If the current initial decision variable configuration is no worse than the comparison initial decision variable configuration in all parameters, and the current initial decision variable configuration has at least one parameter value that is better than the comparison initial decision variable configuration, then the current initial decision variable configuration dominates the comparison initial decision variable configuration.

[0015] If the initial decision variable configuration is no worse than the current initial decision variable configuration in all parameters, and the initial decision variable configuration has at least one parameter value that is better than the current initial decision variable configuration, then the initial decision variable configuration dominates the current initial decision variable configuration.

[0016] Otherwise, the current initial decision variable configuration does not dominate the comparison initial decision variable configuration, or the comparison initial decision variable configuration does not dominate the current initial decision variable configuration;

[0017] If there is no dominance relationship between the initial decision variable configurations, it is recorded as a non-dominated solution, that is, there is no solution in which any other initial decision variable configuration dominates the current initial decision variable configuration;

[0018] Sort according to the dominance relationship among the initial decision variable configurations;

[0019] First Pareto Front: Includes initial decision variable configurations that are not dominated by any other initial decision variable configurations;

[0020] The second Pareto frontier includes initial decision variable configurations that are governed by the initial decision variable configurations of the first Pareto frontier, but are not governed by other initial decision variable configurations.

[0021] In a preferred embodiment, at each Pareto front, the crowding degree of the initial decision variable configuration is calculated, expressed as: In the formula, The crowding level configured for the current initial decision variables. , Assign target values ​​to adjacent initial decision variables for the current initial decision variable. The maximum objective value in the Pareto front. The minimum objective value in the Pareto front;

[0022] In each Pareto front, all initial decision variables are sorted in descending order of crowding.

[0023] In a preferred embodiment, after performing content randomization on all initial decision variable configurations, the newly generated decision variable configurations are iteratively optimized. When the convergence condition is met, all Pareto front solutions are output, including the following steps:

[0024] Based on the initial decision variable configuration, randomization operations are performed to generate new initial decision variable configurations. Randomization operations include random perturbation, probability-based changes, and mutation operations.

[0025] The initial decision variable configuration after the content randomization operation is re-performed with the target value calculation, non-dominated sorting and Pareto rank division steps to generate a new Pareto frontier;

[0026] When the number of iterations equals the threshold, the change in the target value is less than the threshold, or the change in congestion is less than the threshold, the convergence condition is determined to be met. Once the convergence condition is met, all Pareto front solutions are output.

[0027] In a preferred embodiment, the robustness index is calculated as follows: The ranging error and tracking failure rate of the initial decision variable configuration are obtained; the ranging error and tracking failure rate are normalized to map their values ​​to the range [0,1]; the normalized values ​​of the ranging error and tracking failure rate are obtained; and the robustness index is calculated, expressed as: In the formula, The robustness index This is the normalized value of the ranging error. To track the normalized value of the failure rate;

[0028] The calculation logic for the response latency index is as follows: obtain the algorithm latency and network transmission latency of the initial decision variable configuration, and sum the algorithm latency and network transmission latency to obtain the response latency index.

[0029] In a preferred embodiment, the initial decision variable configuration includes the resolution, frame rate, field of view coverage, weight allocation for multi-camera depth estimation, and camera task priority allocation for each camera.

[0030] A dynamic target real-time ranging and tracking system with multi-camera collaborative perception includes a configuration generation module, a non-dominated sorting module, and a scene application module.

[0031] Configuration generation module: Generates multiple initial decision variable configurations through initialization, and calculates the target value of each initial decision variable configuration based on the objective function;

[0032] Non-dominated sorting module: performs non-dominated sorting on all initial decision variable configurations and divides them into different Pareto level blocks;

[0033] Scenario Application Module: After randomizing the content of all initial decision variable configurations, the newly generated decision variable configurations are iteratively optimized. When the convergence condition is met, all Pareto front solutions are output, and the Pareto front solutions are dynamically selected and applied to the corresponding scenarios according to the changes in the scenarios.

[0034] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0035] This invention generates multiple initial decision variable configurations through initialization. After calculating the target value of each initial decision variable configuration based on the objective function, all initial decision variable configurations are non-dominated and sorted into different Pareto levels. After content randomization of all initial decision variable configurations, the newly generated decision variable configurations are iteratively optimized. When the convergence condition is met, all Pareto front solutions are output, and the Pareto front solutions are dynamically selected and applied to the corresponding scenarios according to changes in the scenario. The tracking system comprehensively considers multiple objectives such as ranging accuracy, tracking robustness, and real-time performance, generating a set of non-dominated solutions that can dynamically adapt to different scenario requirements, achieving multi-dimensional balanced optimization of performance. Combined with the specific needs of real-time scenarios, it significantly improves the system's flexibility and adaptability. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0037] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] Example 1: Please refer to Figure 1 As shown in this embodiment, the dynamic target real-time ranging and tracking method based on multi-camera collaborative perception includes the following steps:

[0040] The tracking system generates multiple initial decision variable configurations during initialization. Each initial decision variable configuration includes the resolution, frame rate, field of view coverage, weight allocation for multi-camera depth estimation, and camera task priority allocation for each camera. After calculating the target value of each initial decision variable configuration based on the objective function, all initial decision variable configurations are non-dominated and sorted into different Pareto levels. After randomizing the content of all initial decision variable configurations, the newly generated decision variable configurations are iteratively optimized. When the convergence condition is met, all Pareto front solutions are output, and the Pareto front solutions are dynamically selected and applied to the corresponding scene according to scene changes.

[0041] This application generates multiple initial decision variable configurations through initialization. After calculating the target value of each initial decision variable configuration based on the objective function, all initial decision variable configurations are non-dominated and sorted into different Pareto levels. After content randomization of all initial decision variable configurations, the newly generated decision variable configurations are iteratively optimized. When the convergence condition is met, all Pareto front solutions are output, and the Pareto front solution set is dynamically selected and applied to the corresponding scenario according to the scenario changes. The tracking system comprehensively considers multiple objectives such as ranging accuracy, tracking robustness, and real-time performance, generating a set of non-dominated solutions that can dynamically adapt to different scenario requirements, achieving multi-dimensional balanced optimization of performance. Combined with the specific needs of real-time scenarios, it significantly improves the system's flexibility and adaptability.

[0042] Example 2: The tracking system generates multiple initial decision variable configurations during initialization. Each initial decision variable configuration includes the resolution, frame rate, field of view coverage, weight allocation for multi-camera depth estimation, and camera task priority allocation for each camera. The process includes the following steps:

[0043] Generate diverse configuration combinations for each camera to cover different scenario requirements.

[0044] Decision variables include:

[0045] Resolution: The pixel resolution of the camera (e.g., 720p, 1080p, 4K).

[0046] Frame rate: The number of image frames captured per second (e.g., 15 FPS, 30 FPS, 60 FPS).

[0047] Viewing range: The horizontal and vertical viewing angles of the camera (e.g., 90°, 120°).

[0048] Depth estimation weights: The weights of different cameras' contributions to depth information in multi-view fusion (e.g., higher weights can be assigned to highly occluded areas).

[0049] Task Priority: Priority for different camera tasks (such as tracking of occluded areas or global monitoring).

[0050] Random sampling is performed using a parameter range (e.g., resolution [720p, 1080p, 4K]) to generate an initial population. Population diversity is improved using orthogonal experimental design or Latin square design.

[0051] Assume the parameter range of the multi-camera system is as follows:

[0052] Resolution: [720p, 1080p, 4K];

[0053] Frame rate: [15FPS, 30FPS, 60FPS];

[0054] Viewing angle coverage: [90°, 120°, 150°];

[0055] Depth estimation weights: [0.1, 0.3, 0.5, 0.7, 0.9];

[0056] Task priority: [1,2,3,4] (1 represents the highest priority, and 4 represents the lowest priority);

[0057] Method 1: Random sampling to generate the initial population

[0058] Randomly select values ​​from a parameter range and combine them to generate diverse configurations. For example:

[0059] Camera A configuration: Resolution: 1080p, Frame rate: 30FPS, Field of view: 120°, Depth estimation weight: 0.7, Task priority: 2;

[0060] Camera B configuration: Resolution: 720p, Frame rate: 15FPS, Field of view: 150°, Depth estimation weight: 0.5, Task priority: 1;

[0061] Camera C configuration: Resolution: 4K, Frame rate: 60FPS, Field of view coverage: 90°, Depth estimation weight: 0.9, Task priority: 3.

[0062] The generated populations are: [A:1080p,30FPS,120°,0.7,2],[B:720p,15FPS,150°,0.5,1],[C:4K,60FPS,90°,0.9,3].

[0063] Orthogonal arrays are used to evenly distribute parameter combinations, thereby improving the representativeness of configuration combinations. Assume each parameter has 3 levels:

[0064] Resolutions: 720p, 1080p, 4K;

[0065] Frame rates: 15 FPS, 30 FPS, 60 FPS;

[0066] Viewing angle coverage: 90°, 120°, 150°;

[0067] The orthogonal array is shown in Table 1:

[0068] distribute:

[0069] Group number resolution Frame rate View coverage 1 720p 15FPS 90° 2 720p 30FPS 120° 3 720p 60FPS 150° 4 1080p 15FPS 120° 5 1080p 30FPS 150° 6 1080p 60FPS 90° 7 4K 15FPS 150° 8 4K 30FPS 90° 9 4K 60FPS 120°

[0070] Table 1

[0071] Combining depth estimation weights and task priorities, the above configuration is further extended by adding depth estimation weights and task priorities: Three values ​​are selected from the depth estimation weights [0.1, 0.3, 0.5, 0.7, 0.9] and distributed (e.g., 0.3, 0.7, 0.9). Task priorities are assigned from [1, 2, 3]. For example, the configuration of the first group of the orthogonal array can be extended as follows:

[0072] Configuration 1: Resolution: 720p, Frame Rate: 15FPS, View Coverage: 90°, Depth Estimation Weight: 0.3, Task Priority: 1;

[0073] Configuration 2: Resolution: 720p, Frame Rate: 15FPS, Field of View Coverage: 90°, Depth Estimation Weight: 0.7, Task Priority: 2;

[0074] Configuration 3: Resolution: 720p, Frame Rate: 15FPS, View Coverage: 90°, Depth Estimation Weight: 0.9, Task Priority: 3.

[0075] The following populations can be generated using the methods described above:

[0076] [1:720p,15FPS,90°,0.3,1],[2:1080p,30FPS,120°,0.7,2],[3:4K,60FPS,150°,0.9,3],[4:720p,30FPS,120°,0.5,2],[5:1080p,15FPS,150°,0.7,1], This population not only includes diversity in parameter ranges but also takes into account the balanced distribution of the experimental design, providing a good foundation for subsequent optimization and adaptation to different scenarios.

[0077] The non-dominated allocation of all initial decision variables is performed and divided into different Pareto levels, including the following steps:

[0078] The objective value for each initial decision variable is calculated based on the objective function, the function expression of which is:

[0079] In the formula, For the target value, The robustness index In response to the latency index, , These are the proportional coefficients for the robustness index and the response delay index, respectively. , All are greater than 0.

[0080] The robustness index is calculated as follows: The ranging error and tracking failure rate of the initial decision variables are obtained. The ranging error and tracking failure rate are normalized to map their values ​​to the range [0,1]. The normalized values ​​of the ranging error and tracking failure rate are then obtained. Finally, the robustness index is calculated, expressed as: In the formula, The robustness index This is the normalized value of the ranging error. To track the failure rate normalization value, the larger the robustness index, the smaller the ranging error and the tracking failure rate of the initial decision variable configuration, and the better the robustness.

[0081] The expression for calculating the ranging error is: In the formula, For ranging error, The number of times the object is measured. No. The distance to the object measured this time. For the first The larger the distance measurement error value, the lower the distance measurement accuracy.

[0082] The calculation logic of the response delay index is as follows: obtain the algorithm delay and network transmission delay of the initial decision variable configuration, sum the algorithm delay and network transmission delay to obtain the response delay index. The larger the response delay index, the worse the overall response of the initial decision variable configuration.

[0083] The non-dominated allocation of all initial decision variables is performed and divided into different Pareto levels, including the following steps:

[0084] After selecting the current initial decision variable configuration, compare the current initial decision variable configuration one by one;

[0085] If the current initial decision variable configuration is no worse than the comparison initial decision variable configuration in all parameters, and the current initial decision variable configuration has at least one parameter value that is better than the comparison initial decision variable configuration, then the current initial decision variable configuration dominates the comparison initial decision variable configuration.

[0086] For example, configuration A dominates configuration B if, for all parameters in all objective functions, A's values ​​for all parameters in the objective function are no worse than B's (e.g., A's robustness exponent is greater than or equal to B's robustness exponent, and A's response latency exponent is less than or equal to B's response latency exponent), and A is superior to B on at least one objective function on all parameters (e.g., A's robustness exponent is greater than B's robustness exponent, or A's response latency exponent is less than B's response latency exponent). In other words, configuration A performs better than configuration B on all parameters in the objective function, or at least is equal to B on all parameters in some objective functions. Therefore, A dominates B.

[0087] If the initial decision variable configuration is no worse than the current initial decision variable configuration in all parameters, and the initial decision variable configuration has at least one parameter value that is better than the current initial decision variable configuration, then the initial decision variable configuration dominates the current initial decision variable configuration.

[0088] Otherwise, the current initial decision variable configuration does not dominate the comparison initial decision variable configuration, or the comparison initial decision variable configuration does not dominate the current initial decision variable configuration;

[0089] If there is no dominance relationship between the initial decision variable configurations, it is recorded as a non-dominated solution, that is, there is no solution in which any other initial decision variable configuration dominates the current initial decision variable configuration;

[0090] Sort according to the dominance relationship among the initial decision variable configurations;

[0091] First Pareto Front: Includes initial decision variable configurations that are not dominated by any other initial decision variable configurations;

[0092] The second Pareto frontier includes initial decision variable configurations that are dominated by the initial decision variable configurations of the first Pareto frontier, but not by other initial decision variable configurations, and so on, to obtain all Pareto frontiers.

[0093] In each Pareto front, the crowding degree of the initial decision variable placement is calculated, expressed as:

[0094] In the formula, The crowding level configured for the current initial decision variables. , Configure the target values ​​of adjacent (i.e., before and after configuring the current initial decision variable) initial decision variables for the current initial decision variable. The maximum objective value in the Pareto front. The minimum objective value in the Pareto front;

[0095] In each Pareto front, all initial decision variables are sorted in descending order of their crowding. The greater the crowding of the initial decision variables, the sparser the initial decision variables are in the target space, which may be better.

[0096] After randomizing the content of all initial decision variable configurations, the newly generated decision variable configurations are iteratively optimized. When the convergence condition is met, all Pareto front solutions are output, and the Pareto front solutions are dynamically selected and applied to the corresponding scenarios according to the changes in the scenarios. This includes the following steps:

[0097] Based on the initial configuration of decision variables, a randomization operation is performed to generate new initial configurations of decision variables. The randomization operation aims to increase population diversity and avoid getting trapped in locally optimal initial configurations. This process can be achieved through the following methods:

[0098] Random perturbation: Apply small random perturbations to each decision variable. For example, make minor adjustments to parameters such as resolution, frame rate, and depth estimation weights.

[0099] Probability-based changes: Randomly select and update decision variables according to certain probabilities to avoid making the same changes every time.

[0100] Mutation operation: Using the mutation method in genetic algorithms, select some decision variables to undergo random changes to generate new candidate initial decision variable configurations.

[0101] Randomly select a certain proportion of the initial decision variable configurations to perturb them. Repair the perturbed initial decision variable configurations (if constraints apply) to ensure the generated initial decision variable configurations are valid.

[0102] The initial decision variable configuration after the content randomization operation is re-performed with the target value calculation, non-dominated sorting and Pareto rank division steps to generate a new Pareto frontier;

[0103] When the number of iterations equals the threshold, the change in the target value is less than the threshold, or the change in the crowding is less than the threshold, the convergence condition is determined to be met. Once the convergence condition is met, all Pareto front solutions are output.

[0104] Because a dynamic target real-time ranging and tracking system that relies on multi-camera collaborative perception needs to adapt to changes in the actual scene, it first needs to detect and classify these changes. Different scenes may have different requirements for the system; for example, they may require higher resolution, faster frame rates, or more accurate depth estimation.

[0105] Scene monitoring: Real-time monitoring of scene changes through the system's sensors, data acquisition modules, or external environment perception. For example, factors such as lighting conditions, target movement speed, and occlusion within the scene.

[0106] Scene classification: Based on the detected scene information, scenes are categorized into different types. For example:

[0107] Static scenes: The target changes slowly, and the background is relatively fixed.

[0108] Dynamic scenarios: The target changes rapidly, and the background may also change significantly.

[0109] High occlusion scene: The target is partially or completely occluded.

[0110] High-precision requirements: scenarios requiring higher precision ranging and target tracking.

[0111] Depending on the changing scenarios, it is necessary to dynamically select an appropriate Pareto front solution set to address different requirements. Different scenarios have different performance requirements, therefore, it is necessary to select the solution most suitable for the current scenario from the output Pareto front solution set.

[0112] Scenario requirement mapping: Define a set of performance requirements for each scenario type, for example:

[0113] Static scenarios: may favor low-power, low-latency configurations.

[0114] Dynamic scenes require high frame rates and high resolution configurations.

[0115] In high-precision scenarios, priority should be given to the accuracy of depth estimation and low ranging error.

[0116] Based on the requirements of the scenario, select the most suitable solution from the current Pareto front solution set. Usually, the solution with the best performance to meet the scenario requirements is selected, but other solutions can also be considered as alternatives to allow for rapid adjustment if the scenario changes.

[0117] If the current scene is static, the system can choose a solution with lower resolution but lower response latency to save computational resources. In dynamic scenes, the system will choose a solution with higher resolution and higher frame rate to ensure tracking accuracy and real-time performance. For heavily occluded scenes, the system will prioritize solutions with higher weights for depth estimation to improve target recognition and ranging accuracy under occlusion conditions.

[0118] Based on the selected Pareto front solution set, the system adjusts its various parameter configurations (such as camera resolution, frame rate, depth estimation weights, etc.) to adapt to the needs of the current scenario. Adjusting parameters such as camera resolution, frame rate, field of view, and task priority ensures the system can effectively adapt to the current scenario. For example, in high-precision scenarios, the system can increase camera resolution and depth estimation weights; in low-latency scenarios, the system can decrease resolution and increase frame rate. Based on the selected solution set, the system dynamically optimizes the allocation of system resources. For example, in low-resource scenarios, the system may reduce the number of tasks processed in parallel, while in high-load scenarios, it may increase parallelism and computing power. Based on real-time feedback from the scenario, parameters are continuously adjusted to ensure system stability and performance.

[0119] When the scene changes, the system should be able to quickly adapt and update the selected solution set. For example, when the scene changes from a static environment to a dynamic environment, the system should reselect a solution set suitable for dynamic target tracking and adjust the parameters.

[0120] The system monitors dynamic changes in the scene, such as target velocity and occlusion. Based on new scene information, it reselects the Pareto front solution set best suited for the current scene. It performs online optimization or iteration based on scene changes to ensure optimal performance in new scenarios. Depending on scene requirements, the final selected solution set is applied to specific system settings for real-time dynamic target tracking, ranging, and feedback. The system continuously monitors the scene to ensure optimal performance throughout the entire operation.

[0121] Example 3: The dynamic target real-time ranging and tracking system with multi-camera collaborative perception described in this example includes a configuration generation module, a non-dominated sorting module, and a scene application module.

[0122] Configuration generation module: Generates multiple initial decision variable configurations through initialization. The content of each initial decision variable configuration includes the resolution, frame rate, field of view coverage, weight allocation of multi-camera depth estimation, and camera task priority allocation for each camera. After calculating the target value of each initial decision variable configuration based on the objective function, the initial decision variable configuration and target value are sent to the non-dominated sorting module.

[0123] Non-dominated ranking module: Performs non-dominated ranking on all initial decision variable configurations and divides them into different Pareto levels. The non-dominated ranking and Pareto level division results are sent to the scenario application module.

[0124] Scenario Application Module: After randomizing the content of all initial decision variable configurations, the newly generated decision variable configurations are iteratively optimized. When the convergence condition is met, all Pareto front solutions are output, and the Pareto front solutions are dynamically selected and applied to the corresponding scenarios according to the changes in the scenarios.

[0125] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0126] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0127] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0128] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0129] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for dynamic target real-time ranging tracking of multi-camera cooperative perception, characterized in that: The tracking method comprises the following steps: The tracking system generates a plurality of initial decision variable configurations by initialization, calculates a target value of each initial decision variable configuration based on a target function, and the function expression is: , wherein, is the target value, is a robustness index, is a response delay index, , are proportional coefficients of the robustness index and the response delay index respectively, and , are greater than 0, non-dominated sorting is performed on all initial decision variable configurations, and different Pareto levels are divided. The non-dominated sorting is performed on all initial decision variable configurations, and the initial decision variable configurations are divided into different Pareto levels, which comprises the following steps: After the current initial decision variable configuration is selected, the current initial decision variable configuration is compared one by one; If the current initial decision variable configuration is not worse than the compared initial decision variable configuration in all parameters, and the current initial decision variable configuration is better than the compared initial decision variable configuration in at least one parameter, the current initial decision variable configuration dominates the compared initial decision variable configuration; If the compared initial decision variable configuration is not worse than the current initial decision variable configuration in all parameters, and the compared initial decision variable configuration is better than the current initial decision variable configuration in at least one parameter, the compared initial decision variable configuration dominates the current initial decision variable configuration; Otherwise, the current initial decision variable configuration does not dominate the compared initial decision variable configuration, or the compared initial decision variable configuration does not dominate the current initial decision variable configuration; If there is no dominance relationship between the initial decision variable configurations, the initial decision variable configuration is recorded as a non-dominated solution, that is, there is no solution in which any other initial decision variable configuration dominates the current initial decision variable configuration; The initial decision variable configurations are sorted according to the dominance relationship between the initial decision variable configurations, and the Pareto levels comprise a first Pareto level and a second Pareto level; the first Pareto level comprises initial decision variable configurations which are not dominated by any other initial decision variable configuration; and the second Pareto level comprises initial decision variable configurations which are dominated by the initial decision variable configurations of the first Pareto front, but are not dominated by other initial decision variable configurations. After the content randomization operation is performed on all initial decision variable configurations, the newly generated decision variable configurations are iteratively optimized, and when the convergence condition is met, all Pareto front solution sets are output, and the Pareto front solution sets are dynamically selected according to scene changes and applied to corresponding scenes.

2. The method of claim 1, wherein: In each Pareto front, the crowding distance of the initial decision variable configuration is calculated, expressed as: , wherein, is the crowding distance of the current initial decision variable configuration, , is the target value of the adjacent initial decision variable configuration of the current initial decision variable configuration, is the maximum target value in the Pareto front, is the minimum target value in the Pareto front; In each Pareto front, all initial decision variable configurations are sorted according to the crowding degree from large to small.

3. The multi-camera co-perception dynamic target real-time ranging and tracking method of claim 2, wherein: After the content randomization operation is performed on all initial decision variable configurations, the newly generated decision variable configurations are iteratively optimized, and when the convergence condition is met, all Pareto front solution sets are output, which comprises the following steps: On the basis of the initial decision variable configurations, a randomization operation is performed to generate new initial decision variable configurations, and the randomization operation comprises random disturbance, probability-based change and mutation operation; The target value calculation, non-dominated sorting and Pareto level division steps are performed again on the initial decision variable configurations which have completed the content randomization operation, to generate new Pareto fronts; When the number of iterations is equal to the number threshold, the target value change is less than the target value change threshold, or the crowding degree change is less than the crowding degree change threshold, it is judged that the convergence condition is met, and when the convergence condition is met, all Pareto front solution sets are output.

4. The multi-camera co-perception dynamic target real-time ranging and tracking method of claim 3, wherein: The calculation logic of the robustness index is: obtaining ranging error and tracking failure rate of the initial decision variable configuration, performing normalization processing on the ranging error and the tracking failure rate, mapping the value range of the ranging error and the tracking failure rate to [0, 1], obtaining ranging error normalized value and tracking failure rate normalized value, and calculating the robustness index, the expression is: , wherein, is the robustness index, is the ranging error normalized value, is the tracking failure rate normalized value; The calculation logic of the response delay index is that the algorithm delay and the network transmission delay of the initial decision variable configuration are obtained, and the algorithm delay and the network transmission delay are summed to obtain the response delay index.

5. The multi-camera co-perception dynamic target real-time ranging and tracking method of claim 4, wherein: The content of the initial decision variable configuration includes the resolution, frame rate, angle coverage range of each camera, weight distribution of multi-camera depth estimation and camera task priority distribution.

6. A multi-camera cooperative perception dynamic target real-time ranging and tracking system for implementing the tracking method of any one of claims 1-5, characterized in that: The method comprises a configuration generation module, a non-dominated sorting module and a scene application module. The configuration generation module generates a plurality of initial decision variable configurations through initialization, and calculates the target value of each initial decision variable configuration based on a target function. The non-dominated sorting module performs non-dominated sorting on all initial decision variable configurations and divides them into different Pareto level blocks. The scene application module performs content randomization operation on all initial decision variable configurations, iteratively optimizes the newly generated decision variable configurations, outputs all Pareto frontier solution sets when the convergence condition is met, and dynamically selects the Pareto frontier solution sets to be applied to the corresponding scene according to the scene changes.

Images