Vehicle-mounted target tracker based on computer vision

Abstract

The invention discloses a vehicle-mounted target tracker based on computer vision, and relates to the technical field of computer vision, and the vehicle-mounted target tracker comprises a sensor data collection module which is used for collecting real-time data from various sensors; the data fusion module is used for integrating the data from the plurality of sensors of the sensor data collection module and generating comprehensive information about the environment and the target; the data processing module is used for processing the complex data, extracting features and carrying out target identification and classification; according to the invention, by fusing the environment information and the target information, a more comprehensive and accurate context environment is provided for the system, the current scene can be understood more accurately, the authenticity of the target object can be judged more accurately, so that a more reasonable decision can be made, and the accuracy of the system is improved. And through combined use of an advanced target tracking algorithm, the precision, robustness and real-time performance of target tracking can be significantly improved.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision, and in particular to a vehicle-mounted target tracker based on computer vision. Background Art

[0002] The vehicle-mounted target tracker based on computer vision is a system that uses computer vision technology to identify, track and locate specific targets in real time in the vehicle environment. It integrates cameras, sensors, processors and advanced algorithms, and can accurately capture and track targets such as vehicles, pedestrians, obstacles, etc. in complex road and traffic environments. By real-time tracking of pedestrians, other vehicles and obstacles around the vehicle, the vehicle-mounted target tracker can warn of potential dangerous situations in advance, such as pedestrians crossing and vehicles changing lanes, so as to help the driver or the autonomous driving system respond in time and avoid collision accidents.

[0003] However, in actual use, the traffic environment changes in various ways. When tracking a target, it is in a complex environment such as lighting changes, occlusion, noise, etc., which can easily lead to tracking failures. The system may mistakenly identify certain objects as targets, or fail to identify the real target, resulting in false alarms or missed alarms. Therefore, a vehicle-mounted target tracker based on computer vision is proposed. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art that tracking failure is prone to occur and false alarms or missed alarms are generated, and a vehicle-mounted target tracker based on computer vision is proposed.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A vehicle-mounted target tracker based on computer vision, comprising:

[0007] A sensor data collection module, used to collect real-time data from a variety of sensors;

[0008] A data fusion module for integrating data from multiple sensors in the sensor data collection module to generate comprehensive information about the environment and the target;

[0009] Data processing module, used to process complex data, extract features, and perform target recognition and classification;

[0010] State estimation module, used to update the state information of the target in real time;

[0011] The target tracking module is used to continuously track the target according to the target state of the state estimation module, and predict and update its motion trajectory;

[0012] Machine learning model module, used to train and optimize models and predict the future behavior of the target;

[0013] The decision and feedback module is used to make decisions and give feedback based on the tracking and prediction results.

[0014] The above technical solution further includes:

[0015] The sensor data collection module includes a sensor interface unit, a data synchronization unit and a data preprocessing unit. The sensor interface unit is responsible for receiving raw data from different sensors, such as radar point cloud data, LiDAR distance and angle data, camera video stream, GPS latitude and longitude information, and IMU attitude and acceleration data. The data synchronization unit performs timestamp alignment and spatial coordinate conversion on different data to ensure the temporal and spatial consistency of the data. The data preprocessing unit performs preliminary processing on the raw data, such as denoising, filtering, and compression, to improve data quality.

[0016] The data fusion module includes a data calibration unit, a fusion algorithm unit and a context construction unit. The data calibration unit accurately calibrates different data to eliminate inherent errors between data. The fusion algorithm unit implements and optimizes the Bayesian fusion algorithm to efficiently fuse multi-source data. The context construction unit combines environmental information and fused target information to construct a complete context environment.

[0017] The construction process of the context construction unit is as follows:

[0018] Data collection: Collect environmental information, including map data, terrain information, and meteorological data;

[0019] Context modeling: Build an environment model based on the collected environmental information to reflect the terrain, roads, buildings, and obstacles. In the environment model, analyze the position and status of each target based on the fused target information.

[0020] Contextual relationship analysis: Analyze the interaction between the target and environmental elements, such as approaching obstacles, entering specific areas, analyzing the relative position, speed, and direction relationship between multiple targets, and judging potential conflicts or cooperation opportunities;

[0021] Context generation and update: Based on the above analysis, a complete context environment including environmental information, target information and the relationship between them is generated, and the context environment is updated in real time to reflect the latest changes in environmental information and target information;

[0022] Decision and feedback support: The generated context environment is provided to the decision and feedback module to support risk assessment and decision making. Based on the decision execution results and new observation data, the context environment is optimized to improve its accuracy and reliability.

[0023] The data processing module includes an image processing unit, a target recognition and classification unit, and a trajectory analysis unit. The image processing unit performs enhancement, edge detection, and feature extraction on the captured video image. The target recognition and classification unit uses a CNN deep learning model to perform target recognition and classification on the processed image. The trajectory analysis unit uses historical trajectory data to analyze the target's motion pattern and behavioral characteristics.

[0024] The state estimation module includes a state prediction unit and a state update unit. The state prediction unit predicts the future state based on the current state and the model, and the state update unit updates the target state in real time using new observations and the prediction model.

[0025] The target tracking module includes a deep learning fusion unit, a feature fusion unit, a multimodal tracking unit and an anti-interference unit. The deep learning fusion unit extracts complex features of the target by constructing a deep neural network, and uses these features to track the target. The feature fusion unit effectively integrates feature information from different sources to improve the performance of target tracking. The multimodal tracking unit uses data from different forms to track the target. The anti-interference unit handles illumination changes, occlusion, and noise.

[0026] The implementation steps of the deep learning fusion unit are:

[0027] Data preparation: Collect and annotate a large amount of video sequence data, including various lighting conditions, occlusion situations, and motion patterns to ensure the generalization ability of the model;

[0028] Model design: Design a deep learning model architecture and use Siamese network for similarity matching;

[0029] Feature extraction: Use deep neural networks to extract deep features of the target from the input image and distinguish between the target and the background;

[0030] Object tracking: Based on the extracted features, search and track the object frame by frame in the video sequence;

[0031] The implementation steps of the feature fusion unit are:

[0032] Feature extraction: extract different types of features, such as color features, texture features, and shape features;

[0033] Feature fusion strategy: Select the addition method for fusion, and perform weighted summation on the two feature maps F1 and F2 according to a certain weight coefficient to obtain the fused feature map Ffused. The formula is as follows:

[0034] Ffused=α·F1+β·F2;

[0035] Among them, Ffused is the fused feature, F1 and F2 are the features to be fused, and α and β are weight coefficients used to balance the importance of different features;

[0036] Application of fused features: Use the fused features for target tracking;

[0037] The implementation steps of the multimodal tracking unit are:

[0038] Data synchronization: ensuring that data from different sensors are synchronized in time and space for fusion processing;

[0039] Feature extraction and representation: extract and represent features of data of different modalities respectively;

[0040] Multimodal fusion: effectively integrate the features of different modalities to form a more comprehensive description of the target, achieved through splicing, addition, and multiplication;

[0041] Target tracking: Target tracking based on fused multimodal features;

[0042] The implementation steps of the anti-interference unit are:

[0043] Robust feature design: construct features that are robust to lighting and occlusion;

[0044] Data preprocessing: enhancing data quality, such as denoising and contrast adjustment;

[0045] Online learning and updating: adjust model parameters in real time to cope with changes, use contrast loss function to improve model robustness, and encourage similar samples to have similar features and different samples to have different features. The formula is as follows:

[0046]

[0047] Among them, d is the feature distance between sample pairs, y is the label that the sample pairs belong to the same class, and m is the margin, which is used to control the minimum distance between samples of different classes;

[0048] Anomaly Detection and Recovery: Detects anomalies and reinitializes tracking.

[0049] The machine learning model module includes a model selection unit, a feature engineering unit and a model training and optimization unit. The model selection unit selects a suitable RNN machine learning model. The feature engineering unit designs and extracts effective features for model training, such as historical trajectory features and environmental context features. The model training and optimization unit uses a large amount of data for model training and adopts cross-validation and hyperparameter tuning methods to optimize model performance.

[0050] The decision and feedback module includes a decision logic unit, a feedback mechanism unit and an adaptive adjustment unit. The decision logic unit formulates clear decision rules or strategies. The feedback mechanism unit designs and implements feedback mechanisms, such as user interfaces, alarm systems, etc., to provide real-time feedback. The adaptive adjustment unit continuously optimizes decision logic and feedback mechanisms based on system performance and user feedback.

[0051] The present invention has the following beneficial effects:

[0052] 1. In the present invention, by integrating environmental information and target information, a more comprehensive and accurate context environment is provided for the system. This rich context information helps the system to understand the current scene more accurately and judge the authenticity of the target object, so as to make more reasonable decisions. For example, in a complex traffic environment, the system can judge the authenticity and behavioral intention of the target based on the context information and make corresponding decisions in advance.

[0053] 2. In the present invention, the advanced target tracking algorithms such as deep learning fusion, feature fusion, multimodal tracking, and anti-interference technology are introduced and used in combination, which can significantly improve the accuracy, robustness and real-time performance of target tracking, extract high-level features, and describe the target more accurately. Even in complex environments, the target and background can be effectively distinguished. The features from different sources or different levels are integrated to combine the advantages of multiple features and provide a more comprehensive and accurate target description, thereby improving the tracking accuracy. By introducing anti-interference technology, the algorithm can still stably track the target in the presence of interference factors such as lighting changes, occlusion, and noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A system block diagram of a vehicle-mounted target tracker based on computer vision proposed by the present invention;

[0055] Figure 2 Schematic diagram of the specific steps of the target tracking module in the present invention.

[0056] In the figure: 1. Sensor data collection module; 2. Data fusion module; 3. Data processing module;

[0057] 4. State estimation module; 5. Target tracking module; 6. Machine learning model module; 7. Decision and feedback module. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] Embodiment 1

[0060] like Figure 1-Figure 2 As shown, the present invention proposes a vehicle-mounted target tracker based on computer vision, comprising:

[0061] A sensor data collection module 1, used to collect real-time data from a variety of sensors;

[0062] A data fusion module 2 is used to integrate the data from multiple sensors in the sensor data collection module 1 to generate comprehensive information about the environment and the target;

[0063] Data processing module 3, used to process complex data, extract features, and perform target recognition and classification;

[0064] A state estimation module 4 is used to update the state information of the target in real time;

[0065] A target tracking module 5, used to continuously track the target according to the target state of the state estimation module 4, and predict and update its motion trajectory;

[0066] Machine learning model module 6, used to train and optimize the model to predict the target's future behavior;

[0067] The decision and feedback module 7 is used to make decisions and give feedback based on the tracking and prediction results.

[0068] The working principle of a vehicle-mounted target tracker based on computer vision proposed in the present invention is that when the car is driving, the sensor data collection module 1 is responsible for collecting data from multiple sensors, such as radar, LiDAR, camera, GPS, IMU, in real time. These data are the basis for subsequent processing and analysis. After collecting the data from the sensor data collection module 1, the data fusion module 2 is responsible for integrating these data to generate comprehensive information about the environment and the target. This process includes multi-source data calibration, that is, calibrating the errors between different sensors, such as timestamp alignment and spatial coordinate conversion, to ensure the accuracy and consistency of the data. The data is then processed to improve the accuracy and robustness of the state estimation. At the same time, combined with environmental information, a complete context of the target and its surrounding environment is constructed to provide support for subsequent processing and decision-making. The context information helps the system understand the current scene more accurately, thereby making more reasonable decisions. For example, in a complex traffic environment, the system can judge the authenticity and behavioral intention of the target based on the context information, and make decisions such as avoidance or acceleration in advance;

[0069] For complex data such as images, the data processing module 3 is responsible for further processing and refining. Through image processing techniques such as image enhancement, edge detection, and feature extraction, the image quality is improved and useful information is extracted. Then, the targets in the image are identified and classified. In addition, the historical trajectory data is used to conduct in-depth analysis of the target's motion pattern and behavior characteristics, providing strong support for the continuous tracking and prediction of the target. Then, the state estimation module 4 is responsible for real-time updating of the target's position, speed and other state information, predicting the future state based on the current state and model, and making real-time corrections through new observations to ensure the accuracy and real-time performance of the state estimation.

[0070] The target tracking module 5 is responsible for continuously tracking the target and predicting its motion trajectory, extracting the complex features of the target from the video data, and tracking the target based on these features. It effectively integrates features from different sources or different levels by adding them together to improve the accuracy and robustness of target tracking. It combines data from different sensors or in different forms, such as images, sounds, and radars, extracts and fuses these data to form a more comprehensive description of the target, and tracks the target. It designs robust features, preprocesses the data, and implements online learning and updating of the tracking algorithm to improve the stability and reliability of the target tracking algorithm in complex and interfering environments.

[0071] The machine learning model module 6 trains and optimizes the model to predict the future behavior of the target, selects the RNN machine learning model and performs feature engineering to design and extract effective features for model training, uses a large amount of data for model training and adopts cross-validation, hyperparameter tuning and other methods to improve model performance. The trained model can predict the future behavior pattern of the target based on historical data and current observations to provide strong support for the decision-making and feedback module. Finally, the decision-making and feedback module 7 makes decisions based on the tracking and prediction results and provides real-time feedback to the user or other parts of the system, and has the ability to adaptively adjust and continuously optimize the decision logic and feedback mechanism according to system performance and user feedback to improve the overall performance of the system and user experience.

[0072] In one embodiment, the sensor data collection module 1 includes a sensor interface unit, a data synchronization unit and a data preprocessing unit. The sensor interface unit is responsible for receiving raw data from different sensors, such as point cloud data of radar, distance and angle data of Li DAR, video stream of camera, longitude and latitude information of GPS, and attitude and acceleration data of I MU. The data synchronization unit performs timestamp alignment and spatial coordinate conversion on different data, and the data preprocessing unit performs preliminary processing on the raw data.

[0073] In one embodiment, the data fusion module 2 includes a data calibration unit, a fusion algorithm unit and a context construction unit. The data calibration unit accurately calibrates different data to eliminate inherent errors between data. The fusion algorithm unit implements and optimizes the Bayesian fusion algorithm to efficiently fuse multi-source data. The context construction unit combines environmental information and fused target information to construct a complete context environment.

[0074] In one embodiment, the construction process of the context construction unit is:

[0075] Data collection: Collect environmental information, including map data, terrain information, and meteorological data;

[0076] Context modeling: Build an environment model based on the collected environmental information to reflect the terrain, roads, buildings, and obstacles. In the environment model, analyze the position and status of each target based on the fused target information.

[0077] Contextual relationship analysis: Analyze the interaction between the target and environmental elements, such as approaching obstacles, entering specific areas, analyzing the relative position, speed, and direction relationship between multiple targets, and judging potential conflicts or cooperation opportunities;

[0078] Context generation and update: Based on the above analysis, a complete context environment including environmental information, target information and the relationship between them is generated, and the context environment is updated in real time to reflect the latest changes in environmental information and target information;

[0079] Decision and feedback support: The generated context environment is provided to the decision and feedback module 7 to support it in risk assessment and decision making, and feedback is provided to optimize the context environment based on the decision execution results and new observation data.

[0080] In one embodiment, the data processing module 3 includes an image processing unit, a target recognition and classification unit, and a trajectory analysis unit. The image processing unit performs enhancement, edge detection, and feature extraction on the captured video image. The target recognition and classification unit uses a CNN deep learning model to perform target recognition and classification on the processed image. The trajectory analysis unit uses historical trajectory data to analyze the target's motion pattern and behavioral characteristics.

[0081] In one embodiment, the state estimation module 4 includes a state prediction unit and a state update unit. The state prediction unit predicts the future state based on the current state and the model, and the state update unit updates the target state in real time using new observations and the prediction model.

[0082] In one embodiment, the target tracking module 5 includes a deep learning fusion unit, a feature fusion unit, a multimodal tracking unit and an anti-interference unit. The deep learning fusion unit extracts complex features of the target by constructing a deep neural network, and uses these features to track the target. The feature fusion unit effectively integrates feature information from different sources, the multimodal tracking unit uses data from different forms to track the target, and the anti-interference unit processes illumination changes, occlusion, and noise.

[0083] In one embodiment, the implementation steps of the deep learning fusion unit are:

[0084] Data preparation: Collect and annotate a large amount of video sequence data, including various lighting conditions, occlusion situations, and motion patterns to ensure the generalization ability of the model;

[0085] Model design: Design a deep learning model architecture and use the Simese network for similarity matching;

[0086] Feature extraction: Use deep neural networks to extract deep features of the target from the input image and distinguish between the target and the background;

[0087] Object tracking: Based on the extracted features, search and track the object frame by frame in the video sequence;

[0088] The implementation steps of the feature fusion unit are:

[0089] Feature extraction: extract different types of features, such as color features, texture features, and shape features;

[0090] Feature fusion strategy: Select the addition method for fusion, and perform weighted summation on the two feature maps F1 and F2 according to a certain weight coefficient to obtain the fused feature map Ffused. The formula is as follows:

[0091] Ffused=α·F1+β·F2;

[0092] Among them, Ffused is the fused feature, F1 and F2 are the features to be fused, and α and β are weight coefficients used to balance the importance of different features;

[0093] Application of fused features: Use the fused features for target tracking;

[0094] The implementation steps of the multimodal tracking unit are:

[0095] Data synchronization: ensuring that data from different sensors are synchronized in time and space for fusion processing;

[0096] Feature extraction and representation: extract and represent features of data of different modalities respectively;

[0097] Multimodal fusion: effectively integrate the features of different modalities to form a more comprehensive description of the target, achieved through splicing, addition, and multiplication;

[0098] Target tracking: Target tracking based on fused multimodal features;

[0099] The implementation steps of the anti-interference unit are:

[0100] Robust feature design: construct features that are robust to lighting and occlusion;

[0101] Data preprocessing: enhancing data quality, such as denoising and contrast adjustment;

[0102] Online learning and updating: adjust model parameters in real time to cope with changes, use contrast loss function to improve model robustness, and encourage similar samples to have similar features and different samples to have different features. The formula is as follows:

[0103]

[0104] Among them, d is the feature distance between sample pairs, y is the label that the sample pairs belong to the same class, and m is the margin, which is used to control the minimum distance between samples of different classes;

[0105] Anomaly Detection and Recovery: Detects anomalies and reinitializes tracking.

[0106] In one embodiment, the machine learning model module 6 includes a model selection unit, a feature engineering unit, and a model training and optimization unit. The model selection unit selects a suitable RNN machine learning model. The feature engineering unit designs and extracts effective features for model training, such as historical trajectory features and environmental context features. The model training and optimization unit uses a large amount of data for model training, and adopts cross-validation and hyperparameter tuning methods to optimize model performance.

[0107] In one embodiment, the decision and feedback module 7 includes a decision logic unit, a feedback mechanism unit and an adaptive adjustment unit. The decision logic unit formulates clear decision rules or strategies. The feedback mechanism unit designs and implements feedback mechanisms, such as user interfaces, alarm systems, etc., to provide real-time feedback. The adaptive adjustment unit continuously optimizes the decision logic and feedback mechanism based on system performance and user feedback.

[0108] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A vehicle-mounted target tracker based on computer vision, characterized in that: include: A sensor data collection module (1), used for collecting real-time data from a variety of sensors; A data fusion module (2) for integrating data from multiple sensors of the sensor data collection module (1) to generate comprehensive information about the environment and the target; Data processing module (3), used to process complex data, extract features, and perform target recognition and classification; A state estimation module (4), used for updating the state information of the target in real time; A target tracking module (5) is used to continuously track the target according to the target state of the state estimation module (4), and predict and update its motion trajectory; Machine learning model module (6), used to train and optimize the model and predict the future behavior of the target; The decision and feedback module (7) is used to make decisions and give feedback based on the tracking and prediction results.

2. A vehicle-mounted target tracker based on computer vision according to claim 1, characterized in that: The sensor data collection module (1) comprises a sensor interface unit, a data synchronization unit and a data preprocessing unit. The sensor interface unit is responsible for receiving raw data from different sensors, such as radar point cloud data, LiDAR distance and angle data, camera video stream, GPS latitude and longitude information, and IMU attitude and acceleration data. The data synchronization unit performs time stamp alignment and spatial coordinate conversion on different data, and the data preprocessing unit performs preliminary processing on the raw data.

3. The vehicle-mounted target tracker based on computer vision according to claim 1, characterized in that: The data fusion module (2) comprises a data calibration unit, a fusion algorithm unit and a context construction unit. The data calibration unit accurately calibrates different data to eliminate inherent errors between data. The fusion algorithm unit implements and optimizes the Bayesian fusion algorithm to efficiently fuse multi-source data. The context construction unit combines environmental information and fused target information to construct a complete context environment.

4. A vehicle-mounted target tracker based on computer vision according to claim 3, characterized in that: The construction process of the context construction unit is as follows: Data collection: Collect environmental information, including map data, terrain information, and meteorological data; Context modeling: Build an environment model based on the collected environmental information to reflect the terrain, roads, buildings, and obstacles. In the environment model, analyze the position and status of each target based on the fused target information. Contextual relationship analysis: Analyze the interaction between the target and environmental elements, such as approaching obstacles, entering specific areas, analyzing the relative position, speed, and direction relationship between multiple targets, and judging potential conflicts or cooperation opportunities; Context generation and update: Based on the above analysis, a complete context environment including environmental information, target information and the relationship between them is generated, and the context environment is updated in real time to reflect the latest changes in environmental information and target information; Decision and feedback support: The generated context environment is provided to the decision and feedback module (7) to support risk assessment and decision making, and to provide feedback to optimize the context environment based on the decision execution results and new observation data.

5. The vehicle-mounted target tracker based on computer vision according to claim 1, characterized in that: The data processing module (3) comprises an image processing unit, a target recognition and classification unit and a trajectory analysis unit. The image processing unit performs enhancement, edge detection and feature extraction processing on the captured video image. The target recognition and classification unit uses a CNN deep learning model to perform target recognition and classification on the processed image. The trajectory analysis unit uses historical trajectory data to analyze the movement pattern and behavior characteristics of the target.

6. The vehicle-mounted target tracker based on computer vision according to claim 1, characterized in that: The state estimation module (4) comprises a state prediction unit and a state update unit. The state prediction unit predicts the future state based on the current state and the model, and the state update unit updates the target state in real time using new observation values ​​and the prediction model.

7. The vehicle-mounted target tracker based on computer vision according to claim 1, characterized in that: The target tracking module (5) comprises a deep learning fusion unit, a feature fusion unit, a multimodal tracking unit and an anti-interference unit. The deep learning fusion unit extracts complex features of the target by constructing a deep neural network and uses these features to track the target. The feature fusion unit effectively integrates feature information from different sources. The multimodal tracking unit uses data from different forms to track the target. The anti-interference unit processes illumination changes, occlusion and noise.

8. The vehicle-mounted target tracker based on computer vision according to claim 7, characterized in that: The implementation steps of the deep learning fusion unit are: Data preparation: Collect and annotate a large amount of video sequence data, including various lighting conditions, occlusion situations, and motion patterns to ensure the generalization ability of the model; Model design: Design a deep learning model architecture and use Siamese network for similarity matching; Feature extraction: Use deep neural networks to extract deep features of the target from the input image and distinguish between the target and the background; Object tracking: Based on the extracted features, search and track the object frame by frame in the video sequence; The implementation steps of the feature fusion unit are: Feature extraction: extract different types of features, such as color features, texture features, and shape features; Feature fusion strategy: Select the addition method for fusion, and perform weighted summation on the two feature maps F1 and F2 according to a certain weight coefficient to obtain the fused feature map Ffused. The formula is as follows: Ffused=α·F1+β·F2; Among them, Ffused is the fused feature, F1 and F2 are the features to be fused, and α and β are weight coefficients used to balance the importance of different features; Application of fused features: Use the fused features for target tracking; The implementation steps of the multimodal tracking unit are: Data synchronization: ensuring that data from different sensors are synchronized in time and space for fusion processing; Feature extraction and representation: extract and represent features of data of different modalities respectively; Multimodal fusion: effectively integrate the features of different modalities to form a more comprehensive description of the target, achieved through splicing, addition, and multiplication; Target tracking: Target tracking based on fused multimodal features; The implementation steps of the anti-interference unit are: Robust feature design: construct features that are robust to lighting and occlusion; Data preprocessing: enhancing data quality, such as denoising and contrast adjustment; Online learning and updating: adjust model parameters in real time to cope with changes, use contrast loss function to improve model robustness, and encourage similar samples to have similar features and different samples to have different features. The formula is as follows: Among them, d is the feature distance between sample pairs, y is the label that the sample pairs belong to the same class, and m is the margin, which is used to control the minimum distance between samples of different classes; Anomaly Detection and Recovery: Detects anomalies and reinitializes tracking.

9. The vehicle-mounted target tracker based on computer vision according to claim 8, characterized in that: The machine learning model module (6) includes a model selection unit, a feature engineering unit and a model training and optimization unit. The model selection unit selects a suitable RNN machine learning model. The feature engineering unit designs and extracts effective features for model training, such as historical trajectory features and environmental context features. The model training and optimization unit uses a large amount of data to train the model and adopts cross-validation and hyperparameter tuning methods to optimize model performance.

10. The vehicle-mounted target tracker based on computer vision according to claim 9, characterized in that: The decision and feedback module (7) includes a decision logic unit, a feedback mechanism unit and an adaptive adjustment unit. The decision logic unit formulates clear decision rules or strategies. The feedback mechanism unit designs and implements feedback mechanisms, such as user interfaces, alarm systems, etc., to provide real-time feedback. The adaptive adjustment unit continuously optimizes the decision logic and feedback mechanism based on system performance and user feedback.

Images