Implementation method and system of high-precision moving object tracking AI camera

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to an implementation method and system of a high-precision moving object tracking AI camera, and the method comprises the steps: obtaining a three-dimensional coordinate, a visual image sequence and motion state data of a target object; constructing a neural network dynamic weight model, and adjusting data priorities of the millimeter wave radar, the camera and the inertial sensor; performing weighted calculation on the calibrated three-axis coordinates, and generating a continuous tracking path in combination with a velocity vector; constructing a motion mode prediction model for predicting a future motion track of the target object; screening an effective candidate path set; outputting a credibility score of the prediction path; if the credibility score does not reach the threshold value, replacing and correcting; and selecting an effective prediction path with the minimum deviation as a main scheme, and reserving a suboptimal prediction path as a standby scheme. According to the method, the problem that a traditional method lacks an effective flow correction strategy when the tracking error is relatively large is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and relates to an implementation method and system of a high-precision moving object tracking AI camera. Background Art

[0002] With the rapid development of computer vision and artificial intelligence technology, mobile object tracking technology is widely used in security monitoring, intelligent transportation, sports training, human-computer interaction and other fields. Traditional mobile object tracking technology faces many challenges. Due to the complex and changeable environment, factors such as lighting conditions, occlusion, motion blur, etc. will seriously affect the tracking accuracy and robustness. The data fusion problem of different sensors is also a key factor restricting the tracking performance.

[0003] Traditional methods often use fixed-weight sensor data fusion strategies to deal with these problems, which cannot dynamically adjust the priority of each sensor according to real-time environmental changes, resulting in large tracking errors. Traditional mobile object tracking algorithms often lack sufficient contextual information and environmental perception capabilities when predicting the future trajectory of the target object, making it difficult to accurately predict the trajectory of objects in complex scenarios.

[0004] Based on the above problems, traditional methods lack effective correction process strategies when the tracking error is large. Summary of the invention

[0005] In order to solve the above problems, the present invention provides an implementation method and system of a high-precision moving object tracking AI camera.

[0006] In a first aspect, the present invention provides an implementation method of a high-precision moving object tracking AI camera, which adopts the following technical solutions: A method for implementing a high-precision moving object tracking AI camera includes the following steps: S1, obtain the three-dimensional coordinates, visual image sequence, and motion state data of the target object, perform cross-modal spatiotemporal alignment and calibration, and eliminate the differences in sensor coordinate systems; S2, build a neural network dynamic weight model to adjust the data priority of millimeter wave radar, camera and inertial sensor in real time; S3, performing weighted calculation on the calibrated three-axis coordinates and generating a continuous tracking path in combination with the velocity vector; S4, constructing a motion pattern prediction model to obtain a motion pattern prediction model for predicting the future motion trajectory of the target object; S5, predict and generate three candidate paths, and screen a valid set of candidate paths; S6. Calculate the average deviation coefficient of the candidate paths, combine the deviation threshold and the effective path ratio, and output the credibility score of the predicted path; S7. If the credibility score does not reach the threshold, correction is performed through iterative model training, weight redistribution, or sensor replacement; S8. Select the effective predicted path with the smallest three-axis coordinate deviation as the main solution, and retain the suboptimal predicted path as a backup solution.

[0007] A further solution of the present invention, step S1, comprises the following steps: S11, obtaining the three-dimensional position coordinates and speed information of the target object, capturing the visual image sequence of the target object, and recording the self-motion state data of the target object; Specifically, the millimeter-wave radar emits continuously changing frequency waves and receives the signal reflected by the object to obtain the three-dimensional coordinates and radial velocity of the target object; a high-speed camera is used to capture the visual image sequence of the target object, and an anti-shake inertial sensor is used to record the target object's own motion state data; Specifically, when the anti-shake inertial sensor is installed on the target object, the actual lateral acceleration and displacement increment of the target object are recorded; S12, performing spatial alignment calibration on radar point cloud data, pixel data of visual images, and displacement increments calculated by anti-shake inertial sensors; Specifically, through the unified time signal, the data time difference of the millimeter wave radar, high-speed camera, and anti-shake inertial sensor does not exceed 1 millisecond, ensuring that the millimeter wave radar, high-speed camera, and anti-shake inertial sensor collect data at the same time; Based on the three-dimensional coordinates of the radar point cloud, the two-dimensional pixel coordinates of the visual image, and the displacement increment coordinates calculated by the anti-shake inertial sensor, cross-modal associations are established by extracting common feature points to try to solve the problem of spatial posture differences among the radar, camera coordinate system, and displacement increment coordinate system; combined with the key points of the target object, the mapping relationship between the radar point cloud coordinates (X, Y, Z), the camera image coordinate system (U, V), and the displacement increment coordinate system (A, B) is converted into a homogeneous transformation matrix, and the matching error is eliminated through least squares optimization.

[0008] A further solution of the present invention, step S2, comprises the following steps: The integrated light sensor and weather perception unit obtain the current light intensity and visibility. The moving speed of the target object is calculated based on the speed information output by the millimeter-wave radar and the self-motion state data of the anti-shake inertial sensor. Then, a dynamic weight allocation model is established to calculate the dynamic weight coefficients of the measurement data of the millimeter-wave radar, the measurement data of the high-speed camera, and the measurement data of the anti-shake inertial sensor. The dynamic weight allocation adjusts the priority of different sensors in real time.

[0009] A further solution of the present invention, step S3, comprises the following steps: S31, performing three-axis coordinate weighted calculation according to the radar point cloud coordinates, the pixel coordinates of the image, the displacement increment coordinates extracted from the spatiotemporal alignment data, and the dynamic weight parameters measured in step S2; Specifically, the displacement increment coordinates calculated by the anti-shake inertial sensor ( )、radar point cloud coordinates( ), pixel coordinates of the image ( ); Combined with the dynamic weight parameter: the weight coefficient of the anti-shake inertial sensor is , the weight coefficient of millimeter wave radar is , the weight coefficient of the high-speed camera is ,satisfy ; Perform weighted calculations on the X, Y, and Z axes respectively to satisfy the following formula:

[0010]

[0011]

[0012] S32, converting the millimeter wave radar into a velocity vector using a spatial coordinate system, the weighted three-axis coordinate data, the velocity vector measured by the millimeter wave radar ( ) to generate a tracking path containing three-dimensional coordinates and velocity vectors.

[0013] A further solution of the present invention, step S4, comprises the following steps: Based on the generated tracking path, 100 consecutive frames of historical motion data are intercepted as the basis for model training. Each frame of data contains the three-dimensional coordinates of the target object ( )、Velocity vector( )、Acceleration( ), ensure that 100 frames of data cover a motion time window of at least 10 seconds to capture the complete motion pattern of the target; The three-dimensional coordinates in the historical valid motion data of a single frame ( )、Velocity vector( )、Acceleration( ) are combined into a nine-dimensional feature vector to form a historical training set of the time series; an LSTM network is used to train and generate a motion pattern prediction model, with the nine-dimensional feature vectors of the first 70 frames as input and the three-dimensional coordinates of the next 30 frames as output, and the two-layer LSTM network is trained using the historical training set to obtain a motion pattern prediction model for predicting the future motion trajectory of the target object; Collect the three-dimensional coordinates of the current motion data of the target object ( )、Velocity vector( )、Acceleration( ) are combined into a nine-dimensional feature vector; the nine-dimensional feature vector is input into the motion pattern prediction model to obtain the future motion trajectory of the target object.

[0014] A further solution of the present invention, step S5, comprises the following steps: Predict the future motion trajectory of the target object according to the motion pattern prediction model, input the target object's recent 3-second motion data into the motion pattern prediction model, and obtain the target object's future 0.5-second motion trajectory; combine the target object's future 0.5-second motion trajectory and real-time environmental data to generate three candidate path solutions; Among them, the three candidate path solutions correspond to the three strategies of inertial continuation prediction, neighboring feature matching prediction, and scene map pre-judgment prediction; The actual detection positions of the millimeter-wave radar and visual sensor will be obtained, and the predicted coordinates of the three candidate path solutions will be compared with the actual detection coordinates point by point; The X / Y plane deviation is calculated in pixels of the image. The total X / Y plane deviation between the candidate path and the actual detection result satisfies the following formula:

[0015] in, Indicates the total deviation in the X / Y plane between the candidate path and the actual detection result. Indicates the difference between the predicted coordinates of the candidate path and the actual detected coordinates in the X-axis direction. Indicates the difference between the predicted coordinates of the candidate path and the actual detected coordinates in the Y-axis direction; The X / Y plane deviation is in units of image pixels, and the maximum allowable deviation threshold of the total X / Y plane deviation is 5 pixels; the Z axis deviation is in units of meters, and the maximum allowable deviation threshold of the Z axis deviation is 0.3 meters; determine whether the candidate path meets the threshold conditions, mark the valid candidate paths that do not exceed the threshold conditions, and build a valid candidate path solution set.

[0016] A further solution of the present invention, step S6, comprises the following steps: Combined with the effective candidate path solution set, the average deviation coefficient of the candidate prediction path solution is calculated to satisfy the following formula:

[0017] in, represents the average coefficient of deviation, represents the number of valid candidate paths, Indicates The difference between the predicted coordinates of the candidate path and the actual detected coordinates in the X-axis direction, Indicates The difference between the predicted coordinates of the path and the actual detected coordinates in the Y-axis direction, Indicates The difference between the predicted coordinates of the path and the actual detected coordinates on the Z axis, Indicates The total deviation between the predicted coordinates of the path and the actual detected coordinates, Indicates the variance of the actual detection value; Combined with the average deviation coefficient of the candidate prediction path scheme, the credibility score of the candidate prediction path scheme is calculated to satisfy the following formula:

[0018] in, represents the credibility score, represents the number of valid candidate paths, represents the total number of candidate paths, represents the average coefficient of deviation, It indicates the preset allowable deviation threshold of the system, which is usually the result obtained by multiple calculations in the laboratory.

[0019] A further solution of the present invention, step S7, comprises the following steps: Set the credibility score threshold of the candidate prediction path solution Compare the credibility scores of candidate prediction path solutions and credibility score threshold The size between; If the credibility score , the predicted trajectory is determined to be reliable; If the credibility score , determine that the predicted trajectory is unreliable, execute the correction request, and select the appropriate compensation plan, including replacing or upgrading new sensors, reallocating dynamic weights, and iterating predictions; Among them, iterative prediction, re-collecting new historical training sets, training and generating a revised motion pattern prediction model; Redistribute dynamic weights, redistribute the dynamic weight coefficients of millimeter-wave radar, high-speed camera, and anti-shake inertial sensor to generate a revised predicted trajectory; Replace or upgrade new sensors, detect sensor data anomalies, replace or upgrade new sensors, and regenerate a revised predicted trajectory based on the new combination; Re-calibrate the prediction path and re-evaluate the credibility , if the credibility score , the predicted path is determined to be reliable; if the credibility score , if the predicted trajectory is determined to be unreliable, a second correction request is executed; if three consecutive corrections fail, a single correction will trigger a maximum of three iterations, and the reinitialization process will be executed.

[0020] A further solution of the present invention, step S8, comprises the following steps: Calculate the error between each valid candidate path solution and the actual position, select the solution with the smallest error as the main path, and retain the suboptimal solution as the backup path; calculate the absolute value of the three-axis coordinate deviation between each valid candidate path solution and the actual detection position,

[0021] in, is the absolute value of the three-axis coordinate deviation, is the difference between the predicted coordinates of the candidate path and the actual detected coordinates in the X-axis direction. is the difference between the path prediction coordinates and the actual detection coordinates in the Y-axis direction. It is the difference between the path prediction coordinate and the actual detection coordinate on the Z axis; Preset error value Maximum deviation threshold , compare the error values Maximum deviation threshold The size between; if When the target object is in a complex motion state, two backup paths are retained; if When the time comes, execute the operation of reinitialization process.

[0022] In a second aspect, the present invention provides an implementation system of a high-precision moving object tracking AI camera, which adopts the following technical solutions: The data acquisition module is used to obtain the three-dimensional coordinates, visual image sequences, and motion state data of the target object, and perform cross-modal spatiotemporal alignment and calibration; Dynamic weight model building module, used to build a neural network dynamic weight model to adjust the data priority of millimeter wave radar, camera and inertial sensor in real time; A weighted calculation module, used to perform weighted calculation on the calibrated three-axis coordinates and generate a continuous tracking path in combination with a velocity vector; A motion pattern prediction model building module is used to obtain a motion pattern prediction model for predicting the future motion trajectory of a target object; The candidate path generation and screening module is used to predict and generate three candidate paths and screen the valid candidate path set; The credibility score calculation module is used to calculate the average deviation coefficient of the candidate paths, combine the deviation threshold and the effective path ratio, and output the credibility score of the predicted path; The process correction module, if the credibility score does not reach the threshold, makes corrections through iterative model training, weight redistribution or sensor replacement; The effective prediction path selection module is used to select the effective prediction path with the smallest three-axis coordinate deviation as the main solution, and retain the suboptimal prediction path as a backup solution.

[0023] In summary, the present invention includes the following beneficial technical effects: 1. Through the spatiotemporal alignment of cross-modal data of millimeter-wave radar, high-speed camera, and anti-shake inertial sensor, combined with the adaptive weight distribution model of light, weather, and motion speed, the sensor data with the highest reliability is prioritized in rain, fog, low light, or high-speed scenes; 2. The motion pattern prediction model built based on the LSTM network learns the target motion law through the historical nine-dimensional feature vector, and combines inertial continuation, neighboring feature matching and scene map semantic prediction to generate multi-strategy candidate paths. Through dynamic deviation threshold screening and credibility scoring mechanism, the system can automatically select the predicted path that best fits the actual trajectory; 3. When the credibility score does not meet the standard, the system triggers multiple real-time correction strategies, including iterative adjustment of sensor dynamic weights, incremental training optimization of LSTM model, and rapid replacement of sensors in case of failure. It uses three-iteration fault tolerance and initialization mechanism to effectively avoid cumulative errors and ensure continuous tracking under extreme conditions such as high-speed mutations or sensor abnormalities. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. The drawings are used to provide a further understanding of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A flowchart of an implementation method of a high-precision moving object tracking AI camera is disclosed.

[0026] Figure 2 A structural schematic diagram of an implementation system of a high-precision moving object tracking AI camera is disclosed. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0028] The following is combined with Figure 1-Figure 2 The preferred embodiments of the present invention are described in detail.

[0029] See attached Figure 1 The present invention proposes a method for implementing a high-precision moving object tracking AI camera, comprising the following steps: S1, obtain the three-dimensional coordinates, visual image sequence, and motion state data of the target object, perform cross-modal spatiotemporal alignment and calibration, and eliminate the differences in sensor coordinate systems; S2, build a neural network dynamic weight model to adjust the data priority of millimeter wave radar, camera and inertial sensor in real time; S3, performing weighted calculation on the calibrated three-axis coordinates and generating a continuous tracking path in combination with the velocity vector; S4, constructing a motion pattern prediction model to obtain a motion pattern prediction model for predicting the future motion trajectory of the target object; S5, predict and generate three candidate paths, and screen a valid set of candidate paths; S6. Calculate the average deviation coefficient of the candidate paths, combine the deviation threshold and the effective path ratio, and output the credibility score of the predicted path; S7. If the credibility score does not reach the threshold, correction is performed through iterative model training, weight redistribution, or sensor replacement; S8. Select the effective predicted path with the smallest three-axis coordinate deviation as the main solution, and retain the suboptimal predicted path as a backup solution.

[0030] In one embodiment of the present invention, step S1 includes the following steps: S11, obtaining the three-dimensional position coordinates and speed information of the target object, capturing the visual image sequence of the target object, and recording the self-motion state data of the target object; Specifically, the millimeter-wave radar emits continuously changing frequency waves and receives the signal reflected by the object to obtain the three-dimensional coordinates and radial velocity of the target object; a high-speed camera is used to capture the visual image sequence of the target object, and the high-speed camera turns on the HDR mode to avoid overexposure or darkening of the picture caused by strong light or shadow; Use anti-shake inertial sensors to record the target object's own motion state data; Specifically, when the anti-shake inertial sensor is installed on the target object, the actual lateral acceleration and displacement increment of the target object are recorded.

[0031] S12, performing spatial alignment calibration on radar point cloud data, pixel data of visual images, and displacement increments calculated by anti-shake inertial sensors; Specifically, through the unified time signal, the data time difference of the millimeter wave radar, high-speed camera, and anti-shake inertial sensor does not exceed 1 millisecond, ensuring that the millimeter wave radar, high-speed camera, and anti-shake inertial sensor collect data at the same time; Based on the three-dimensional coordinates of the radar point cloud, the two-dimensional pixel coordinates of the visual image, and the displacement increment coordinates calculated by the anti-shake inertial sensor, cross-modal associations are established by extracting common feature points to try to solve the problem of spatial posture differences among the radar, camera coordinate system, and displacement increment coordinate system; combined with the key points of the target object, the mapping relationship between the radar point cloud coordinates (X, Y, Z), the camera image coordinate system (U, V), and the displacement increment coordinate system (A, B) is converted into a homogeneous transformation matrix, and the matching error is eliminated through least squares optimization; the radar point cloud coordinates, camera image coordinate system, and displacement increment coordinate system after spatial alignment and calibration are packaged into a unified format.

[0032] In one embodiment of the present invention, step S2 comprises the following steps: The integrated light sensor and weather sensing unit (for example, visibility sensor) obtain the current light intensity and visibility; the moving speed of the target object is calculated based on the speed information output by the millimeter-wave radar and the self-motion state data of the anti-shake inertial sensor; and then a dynamic weight allocation model is established to calculate the dynamic weight coefficients of the measurement data of the millimeter-wave radar, the measurement data of the high-speed camera, and the measurement data of the anti-shake inertial sensor. The dynamic weight allocation significantly improves the performance of the multi-sensor system by adjusting the priority of different sensors in real time.

[0033] In one embodiment of the present invention, establishing a dynamic weight allocation model includes the following steps: Collect historical light intensity, historical visibility, and historical target object movement speed around the target object. Experts assign and annotate dynamic weight coefficients for millimeter-wave radar measurement data, high-speed camera measurement data, and anti-shake inertial sensor measurement data based on historical light intensity, historical visibility, and historical movement speed. Collect the historical light intensity, historical visibility, and historical target object movement speed around the target object to build a historical data set; build a dynamic weight allocation model based on a three-layer neural network model (input layer, hidden layer, and output layer), using the light intensity, visibility, and movement speed of the target object as input, and the dynamic weight coefficients of millimeter-wave radar, high-speed camera, and anti-shake inertial sensor as output. Use the historical data set to train the neural network model to obtain a dynamic weight allocation model; Collect the current light intensity, current visibility, and current movement speed of the target object around the target object to obtain an input data set; Input the input data set into the dynamic weight allocation model to obtain the dynamic weight coefficient allocation values ​​of the millimeter wave radar, high-speed camera, and anti-shake inertial sensor; The dynamic weight coefficient allocation values ​​of each sensor (millimeter wave radar, high-speed camera, anti-shake inertial sensor) are integrated, and the normalized weight value output by the dynamic weight allocation model is converted into the weight percentage of each sensor, which is a significant improvement compared to the fixed weight allocation scheme.

[0034] For example, the environmental parameters are associated with optical sensors. When the light intensity is >1000 lux, the measurement data of the high-confidence millimeter-wave radar is used. The environmental parameters are associated with meteorological sensing units. When the visibility in rainy and foggy weather is <50 meters, the measurement data of the millimeter-wave radar is used first. When the target object's moving speed is >30m / s, the image blur or radar lag error caused by high-speed movement is compensated by the measurement data of the anti-shake inertial sensor; Among them, when rain, fog and high-speed motion conditions coexist, the priority of the environmental state is higher than the priority of the motion state (for example, when it rains and the target speed is >30m / s, the dynamic weight coefficient of the millimeter-wave radar measurement data is still allocated as 80%). The exponential decay method is used to avoid sudden changes in weights (for example, when the light suddenly drops, the dynamic weight coefficient of the measurement data of the high-speed camera gradually decreases from 70% to 30% within 10 frames), which can reduce the tracking error of the high-precision moving object tracking AI camera.

[0035] 1. Input of the dynamic weight allocation model: light intensity = 1500 lux, visibility in rainy and foggy weather = 2000 meters, target object movement speed = 35 m / s; Output of the dynamic weight allocation model: weight of millimeter wave radar measurement data = 70% (high light ensures clear image), weight of high-speed camera measurement data = 25%, weight of anti-shake inertial sensor measurement data = 5%; High-speed cameras are used to guide positioning, while millimeter-wave radar assists in correcting motion deviations.

[0036] 2. Input of the dynamic weight allocation model: light intensity = 300 lux, visibility in rainy and foggy weather = 30 meters, target object movement speed = 15 m / s; Output of the dynamic weight allocation model: weight of millimeter wave radar measurement data = 80% (heavy fog weakens visual reliability), weight of high-speed camera measurement data = 15%, weight of anti-shake inertial sensor measurement data = 5%; Millimeter-wave radar is used for positioning, and anti-shake inertial sensors eliminate airflow disturbance errors.

[0037] 3. Input of the dynamic weight allocation model: the target object's moving speed = 0 m / s (suddenly accelerated to 40 m / s after being stationary); Output of the dynamic weight allocation model: The weight of the millimeter wave radar measurement data = 15% (the fog weakens the reliability of vision), the weight of the high-speed camera measurement data = 35%, and the weight of the anti-shake inertial sensor measurement data = 50%; The anti-shake inertial sensor takes over urgently to avoid visual tracking failure caused by high-speed sudden changes.

[0038] In one embodiment of the present invention, step S3 includes the following steps: S31, extracting radar point cloud coordinates, image pixel coordinates, displacement increment coordinates from the spatiotemporal alignment data of step S1, and the dynamic weight parameters measured in step S2, and performing three-axis coordinate weighted calculation; Specifically, the displacement increment coordinates calculated by the anti-shake inertial sensor ( )、radar point cloud coordinates( ), pixel coordinates of the image ( ); Combined with the dynamic weight parameter: the weight coefficient of the anti-shake inertial sensor is , the weight coefficient of millimeter wave radar is , the weight coefficient of the high-speed camera is ,satisfy ; Perform weighted calculations on the X, Y, and Z axes respectively to satisfy the following formula:

[0039]

[0040]

[0041] For example, rainy and foggy weather scene ( =80%, =15%, =5%); The predetermined radar detection coordinates (10.235m, 5.678m, 2.314m), visual detection coordinates (10.240m, 5.665m, 2.320m), and displacement increment coordinates (10.238m, 5.670m, 2.315m) are used for weighted calculation. The final coordinates are (10.236m, 5.675m, 2.316m).

[0042] S32, converting the millimeter wave radar into a velocity vector using a spatial coordinate system, the weighted three-axis coordinate data, the velocity vector measured by the millimeter wave radar ( ) to generate a tracking path containing 3D coordinates and velocity vectors; update the weighted results at a frequency of 200 Hz to form a frame-by-frame continuous path data stream; encapsulate the 3D coordinates and velocity vectors into a JSON structure, including the sensor's weight parameters and timestamp.

[0043] In one embodiment of the present invention, step S4 comprises the following steps: Based on the tracking path generated in step S3, 100 consecutive frames of historical motion data are intercepted as the basis for model training. Each frame of data contains the three-dimensional coordinates of the target object ( )、Velocity vector( )、Acceleration( ), ensure that 100 frames of data cover a motion time window of at least 10 seconds to capture the complete motion pattern of the target; The three-dimensional coordinates in the historical valid motion data of a single frame ( )、Velocity vector( )、Acceleration( ) are combined into a nine-dimensional feature vector to form a historical training set of the time series; an LSTM network is used to train and generate a motion pattern prediction model, with the nine-dimensional feature vectors of the first 70 frames as input and the three-dimensional coordinates of the next 30 frames as output. The two-layer LSTM network is trained using the historical training set to obtain a motion pattern prediction model that predicts the future motion trajectory of the target object.

[0044] Collect the three-dimensional coordinates of the current motion data of the target object ( )、Velocity vector( )、Acceleration( ) are combined into a nine-dimensional feature vector; the nine-dimensional feature vector is input into the motion pattern prediction model to obtain the future motion trajectory of the target object, which supports the motion pattern learning of different types of target objects.

[0045] In one embodiment of the present invention, step S5 comprises the following steps: Predict the future motion trajectory of the target object according to the motion pattern prediction model, input the target object's recent 3-second motion data (60 frames of high-speed camera data) into the motion pattern prediction model, and obtain the target object's future 0.5-second motion trajectory; combine the target object's future 0.5-second motion trajectory and real-time environmental data to generate three candidate path solutions; Among them, the three candidate path solutions correspond to the three strategies of inertial continuation prediction, neighboring feature matching prediction, and scene map pre-judgment prediction; Inertial continuation prediction path, based on the target object's motion inertia (including: speed and acceleration) in the past 3 seconds, the target object is calculated to predict its future motion trajectory. The target object maintains its current motion state unchanged, and the effective path extends along the straight path direction. It relies on physical motion inertia and is suitable for scenarios without environmental interference. Neighboring feature matching prediction path: When the target object is partially blocked or disturbed by the environment, search for other target objects with similar features (shape, speed, size) in the neighboring area, and select the motion trajectory of the similar target object with the highest matching degree as the candidate prediction path; relying on feature matching, it is suitable for target occlusion or group motion scenes; The scene map predicts the path, combines high-precision scene maps (such as lane lines, intersections, and obstacle distribution) to predict the reasonable path of the target object, and generates candidate routes that meet the scene constraints through path planning. The trajectory curvature is dynamically adjusted to ensure that the maximum steering angle meets the target object. It relies on scene semantic understanding and is suitable for structured roads or fixed route scenarios.

[0046] The actual detection positions of the millimeter-wave radar and visual sensor will be obtained, and the predicted coordinates of the three candidate path solutions will be compared with the actual detection coordinates point by point; The X / Y plane deviation is calculated in pixels of the image. The total X / Y plane deviation between the candidate path and the actual detection result satisfies the following formula:

[0047] in, Indicates the total deviation in the X / Y plane between the candidate path and the actual detection result. Indicates the difference between the predicted coordinates of the candidate path and the actual detected coordinates in the X-axis direction. Indicates the difference between the predicted coordinates of the candidate path and the actual detected coordinates in the Y-axis direction.

[0048] The X / Y plane deviation is in units of image pixels, and the maximum allowable deviation threshold of the total X / Y plane deviation is 5 pixels. The Z axis (depth direction) deviation is in units of meters, and the maximum allowable deviation threshold of the Z axis deviation is 0.3 meters. Determine whether the candidate path meets the threshold conditions, mark the valid candidate paths that do not exceed the threshold conditions, and build a valid candidate path solution set.

[0049] In one embodiment of the present invention, step S6 comprises the following steps: Combined with the effective candidate path solution set, the average deviation coefficient of the candidate prediction path solution is calculated to satisfy the following formula:

[0050] in, represents the average coefficient of deviation, represents the number of valid candidate paths, Indicates The difference between the predicted coordinates of the candidate path and the actual detected coordinates in the X-axis direction, Indicates The difference between the predicted coordinates of the path and the actual detected coordinates in the Y-axis direction, Indicates The difference between the predicted coordinates of the path and the actual detected coordinates on the Z axis, Indicates The total deviation between the predicted coordinates of the path and the actual detected coordinates, Indicates the variance of the actual detection value; Combined with the average deviation coefficient of the candidate prediction path scheme, the credibility score of the candidate prediction path scheme is calculated to satisfy the following formula:

[0051] in, represents the credibility score, represents the number of valid candidate paths, represents the total number of candidate paths, represents the average coefficient of deviation, It indicates the preset allowable deviation threshold of the system, which is usually the result obtained by multiple calculations in the laboratory.

[0052] For example, three paths are set: Indicates the effective path, with a total deviation of 0.07m; Indicates an invalid path with a total deviation of 0.4m; Indicates a valid path, with a total deviation of 0.15m; the number of valid paths = 2, the total number of paths = 3, and the average deviation coefficient of the candidate predicted path solutions is calculated =0.11; Set the system preset allowable deviation threshold to = 0.3m, calculate the credibility score of the candidate prediction path solution =76.7%.

[0053] In one embodiment of the present invention, step S7 includes the following steps: Set the credibility score threshold of the candidate prediction path solution Compare the credibility scores of candidate prediction path solutions and credibility score threshold The size between; If the credibility score , the predicted trajectory is determined to be reliable; If the credibility score , determine that the predicted trajectory is unreliable, execute the correction request, and select the appropriate compensation plan, including replacing or upgrading new sensors, reallocating dynamic weights, and iterating predictions; Among them, iterative prediction, re-collecting new historical training sets, training and generating a revised motion pattern prediction model; Redistribute dynamic weights, redistribute the dynamic weight coefficients of millimeter-wave radar, high-speed camera, and anti-shake inertial sensor to generate a revised predicted trajectory; Replace or upgrade new sensors, detect sensor data anomalies, replace or upgrade new sensors, and regenerate a revised predicted trajectory based on the new combination; Re-calibrate the prediction path and re-evaluate the credibility , if the credibility score , the predicted path is determined to be reliable; if the credibility score , if the predicted trajectory is determined to be unreliable, a second correction request is executed; if three consecutive corrections fail, a single correction will trigger a maximum of three iterations, and the reinitialization process will be executed.

[0054] In one embodiment of the present invention, step S8 comprises the following steps: Calculate the error between each valid candidate path solution and the actual position, select the solution with the smallest error as the main path, and retain the suboptimal solution as the backup path; calculate the absolute value of the three-axis coordinate deviation between each valid candidate path solution and the actual detection position,

[0055] in, is the absolute value of the three-axis coordinate deviation, is the difference between the predicted coordinates of the candidate path and the actual detected coordinates in the X-axis direction. is the difference between the path prediction coordinates and the actual detection coordinates in the Y-axis direction. It is the difference between the path prediction coordinate and the actual detection coordinate on the Z axis; Preset error value Maximum deviation threshold , compare the error values Maximum deviation threshold The size between; if When the target object is in a complex motion state, two backup paths are retained; if When the time comes, execute the operation of reinitialization process.

[0056] See attached Figure 2 The present invention also proposes an implementation system of a high-precision moving object tracking AI camera, comprising the following modules: The data acquisition module is used to obtain the three-dimensional coordinates, visual image sequences, and motion state data of the target object, and perform cross-modal spatiotemporal alignment and calibration; Dynamic weight model building module, used to build a neural network dynamic weight model to adjust the data priority of millimeter wave radar, camera and inertial sensor in real time; A weighted calculation module, used to perform weighted calculation on the calibrated three-axis coordinates and generate a continuous tracking path in combination with a velocity vector; A motion pattern prediction model building module is used to obtain a motion pattern prediction model for predicting the future motion trajectory of a target object; The candidate path generation and screening module is used to predict and generate three candidate paths and screen the valid candidate path set; The credibility score calculation module is used to calculate the average deviation coefficient of the candidate paths, combine the deviation threshold and the effective path ratio, and output the credibility score of the predicted path; The process correction module, if the credibility score does not reach the threshold, makes corrections through iterative model training, weight redistribution or sensor replacement; The effective prediction path selection module is used to select the effective prediction path with the smallest three-axis coordinate deviation as the main solution, and retain the suboptimal prediction path as a backup solution.

[0057] The modules described above may be implemented in whole or in part through software, hardware, or a combination thereof, supporting a processor embedded in or independent of a computer device in hardware form, and also supporting a memory stored in a computer device in software form, so that the processor can call and execute operations corresponding to the modules described above.

[0058] It should be noted that the user information (including but not limited to user device information and personal information, etc.) and data (including but not limited to data used for analysis, stored data and displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of relevant data require relevant legal standards.

[0059] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention is described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for implementing a high-precision moving object tracking AI camera, characterized in that: The following steps are involved: S1, obtain the three-dimensional coordinates, visual image sequence, and motion state data of the target object, perform cross-modal spatiotemporal alignment and calibration, and eliminate the differences in sensor coordinate systems; S2, build a neural network dynamic weight model to adjust the data priority of millimeter wave radar, camera and inertial sensor in real time; S3, performing weighted calculation on the calibrated three-axis coordinates and generating a continuous tracking path in combination with the velocity vector; S4, constructing a motion pattern prediction model to obtain a motion pattern prediction model for predicting the future motion trajectory of the target object; S5, predict and generate three candidate paths, and screen a valid set of candidate paths; S6. Calculate the average deviation coefficient of the candidate paths, combine the deviation threshold and the effective path ratio, and output the credibility score of the predicted path; S7. If the credibility score does not reach the threshold, correction is performed through iterative model training, weight redistribution, or sensor replacement; S8. Select the effective predicted path with the smallest three-axis coordinate deviation as the main solution, and retain the suboptimal predicted path as a backup solution.

2. The implementation method of a high-precision moving object tracking AI camera according to claim 1, characterized in that: Step S1 includes the following steps: S11, obtaining the three-dimensional position coordinates and speed information of the target object, capturing the visual image sequence of the target object, and recording the self-motion state data of the target object; Specifically, the millimeter-wave radar emits continuously changing frequency waves and receives the signal reflected by the object to obtain the three-dimensional coordinates and radial velocity of the target object; a high-speed camera is used to capture the visual image sequence of the target object, and an anti-shake inertial sensor is used to record the target object's own motion state data; Specifically, when the anti-shake inertial sensor is installed on the target object, the actual lateral acceleration and displacement increment of the target object are recorded; S12, performing spatial alignment calibration on radar point cloud data, pixel data of visual images, and displacement increments calculated by anti-shake inertial sensors; Specifically, through the unified time signal, the data time difference of the millimeter wave radar, high-speed camera, and anti-shake inertial sensor does not exceed 1 millisecond, ensuring that the millimeter wave radar, high-speed camera, and anti-shake inertial sensor collect data at the same time; Based on the three-dimensional coordinates of the radar point cloud, the two-dimensional pixel coordinates of the visual image, and the displacement increment coordinates calculated by the anti-shake inertial sensor, cross-modal associations are established by extracting common feature points to try to solve the problem of spatial posture differences among the radar, camera coordinate system, and displacement increment coordinate system; combined with the key points of the target object, the mapping relationship between the radar point cloud coordinates (X, Y, Z), the camera image coordinate system (U, V), and the displacement increment coordinate system (A, B) is converted into a homogeneous transformation matrix, and the matching error is eliminated through least squares optimization.

3. The implementation method of a high-precision moving object tracking AI camera according to claim 2, characterized in that: Step S2 includes the following steps: The integrated light sensor and weather perception unit obtain the current light intensity and visibility. The moving speed of the target object is calculated based on the speed information output by the millimeter-wave radar and the self-motion state data of the anti-shake inertial sensor. Then, a dynamic weight allocation model is established to calculate the dynamic weight coefficients of the measurement data of the millimeter-wave radar, the measurement data of the high-speed camera, and the measurement data of the anti-shake inertial sensor. The dynamic weight allocation adjusts the priority of different sensors in real time.

4. The implementation method of a high-precision moving object tracking AI camera according to claim 3 is characterized in that: Step S3 includes the following steps: S31, performing three-axis coordinate weighted calculation according to the radar point cloud coordinates, the pixel coordinates of the image, the displacement increment coordinates extracted from the spatiotemporal alignment data, and the dynamic weight parameters measured in step S2; Specifically, the displacement increment coordinates calculated by the anti-shake inertial sensor ( )、radar point cloud coordinates( ), pixel coordinates of the image ( ); Combined with the dynamic weight parameter: the weight coefficient of the anti-shake inertial sensor is , the weight coefficient of millimeter wave radar is , the weight coefficient of the high-speed camera is ,satisfy ; Perform weighted calculations on the X, Y, and Z axes respectively to satisfy the following formula: ； ； ； S32, converting the millimeter wave radar into a velocity vector using a spatial coordinate system, the weighted three-axis coordinate data, the velocity vector measured by the millimeter wave radar ( ) to generate a tracking path containing three-dimensional coordinates and velocity vectors.

5. The implementation method of a high-precision moving object tracking AI camera according to claim 4, characterized in that: Step S4 includes the following steps: Based on the generated tracking path, 100 consecutive frames of historical motion data are intercepted as the basis for model training. Each frame of data contains the three-dimensional coordinates of the target object ( )、Velocity vector( )、Acceleration( ), ensure that 100 frames of data cover a motion time window of at least 10 seconds to capture the complete motion pattern of the target; The three-dimensional coordinates in the historical valid motion data of a single frame ( )、Velocity vector( )、Acceleration( ) are combined into a nine-dimensional feature vector to form a historical training set of the time series; an LSTM network is used to train and generate a motion pattern prediction model, with the nine-dimensional feature vectors of the first 70 frames as input and the three-dimensional coordinates of the next 30 frames as output, and the two-layer LSTM network is trained using the historical training set to obtain a motion pattern prediction model for predicting the future motion trajectory of the target object; Collect the three-dimensional coordinates of the current motion data of the target object ( )、Velocity vector( )、Acceleration( ) are combined into a nine-dimensional feature vector; the nine-dimensional feature vector is input into the motion pattern prediction model to obtain the future motion trajectory of the target object.

6. The implementation method of a high-precision moving object tracking AI camera according to claim 5, characterized in that: Step S5 includes the following steps: Predict the future motion trajectory of the target object according to the motion pattern prediction model, input the target object's recent 3-second motion data into the motion pattern prediction model, and obtain the target object's future 0.5-second motion trajectory; combine the target object's future 0.5-second motion trajectory and real-time environmental data to generate three candidate path solutions; Among them, the three candidate path solutions correspond to the three strategies of inertial continuation prediction, neighboring feature matching prediction, and scene map pre-judgment prediction; The actual detection positions of the millimeter-wave radar and visual sensor will be obtained, and the predicted coordinates of the three candidate path solutions will be compared with the actual detection coordinates point by point; The X / Y plane deviation is calculated in pixels of the image. The total X / Y plane deviation between the candidate path and the actual detection result satisfies the following formula: ； in, Indicates the total deviation in the X / Y plane between the candidate path and the actual detection result. Indicates the difference between the predicted coordinates of the candidate path and the actual detected coordinates in the X-axis direction. Indicates the difference between the predicted coordinates of the candidate path and the actual detected coordinates in the Y-axis direction; The X / Y plane deviation is in units of image pixels, and the maximum allowable deviation threshold of the total X / Y plane deviation is 5 pixels; the Z axis deviation is in units of meters, and the maximum allowable deviation threshold of the Z axis deviation is 0.3 meters; determine whether the candidate path meets the threshold conditions, mark the valid candidate paths that do not exceed the threshold conditions, and build a valid candidate path solution set.

7. The implementation method of a high-precision moving object tracking AI camera according to claim 6, characterized in that: Step S6 includes the following steps: Combined with the effective candidate path solution set, the average deviation coefficient of the candidate prediction path solution is calculated to satisfy the following formula: ； in, represents the average coefficient of deviation, represents the number of valid candidate paths, Indicates The difference between the predicted coordinates of the candidate path and the actual detected coordinates in the X-axis direction, Indicates The difference between the predicted coordinates of the path and the actual detected coordinates in the Y-axis direction, Indicates The difference between the predicted coordinates of the path and the actual detected coordinates on the Z axis, Indicates The total deviation between the predicted coordinates of the path and the actual detected coordinates, Indicates the variance of the actual detection value; Combined with the average deviation coefficient of the candidate prediction path scheme, the credibility score of the candidate prediction path scheme is calculated to satisfy the following formula: ； in, represents the credibility score, represents the number of valid candidate paths, represents the total number of candidate paths, represents the average coefficient of deviation, It indicates the preset allowable deviation threshold of the system, which is usually the result obtained by multiple calculations in the laboratory.

8. The implementation method of a high-precision moving object tracking AI camera according to claim 7, characterized in that: Step S7 includes the following steps: Set the credibility score threshold of the candidate prediction path solution Compare the credibility scores of candidate prediction path solutions and credibility score threshold The size between; If the credibility score , the predicted trajectory is determined to be reliable; If the credibility score , determine that the predicted trajectory is unreliable, execute the correction request, and select the appropriate compensation plan, including replacing or upgrading new sensors, reallocating dynamic weights, and iterating predictions; Among them, iterative prediction, re-collecting new historical training sets, training and generating a revised motion pattern prediction model; Redistribute dynamic weights, redistribute the dynamic weight coefficients of millimeter-wave radar, high-speed camera, and anti-shake inertial sensor to generate a revised predicted trajectory; Replace or upgrade new sensors, detect sensor data anomalies, replace or upgrade new sensors, and regenerate a revised predicted trajectory based on the new combination; Re-calibrate the prediction path and re-evaluate the credibility , if the credibility score , the predicted path is determined to be reliable; if the credibility score , if the predicted trajectory is determined to be unreliable, a second correction request is executed; if three consecutive corrections fail, a single correction will trigger a maximum of three iterations, and the reinitialization process will be executed.

9. The implementation method of a high-precision moving object tracking AI camera according to claim 5, characterized in that: Step S8 includes the following steps: Calculate the error between each valid candidate path solution and the actual position, select the solution with the smallest error as the main path, and retain the suboptimal solution as the backup path; calculate the absolute value of the three-axis coordinate deviation between each valid candidate path solution and the actual detection position, ； in, is the absolute value of the three-axis coordinate deviation, is the difference between the predicted coordinates of the candidate path and the actual detected coordinates in the X-axis direction. is the difference between the path prediction coordinates and the actual detection coordinates in the Y-axis direction. It is the difference between the path prediction coordinate and the actual detection coordinate on the Z axis; Preset error value Maximum deviation threshold , compare the error values Maximum deviation threshold The size between; if When the target object is in a complex motion state, two backup paths are retained; if When the time comes, execute the operation of reinitialization process.

10. A high-precision moving object tracking AI camera implementation system, characterized in that: Includes the following modules: The data acquisition module is used to obtain the three-dimensional coordinates, visual image sequences, and motion state data of the target object, and perform cross-modal spatiotemporal alignment and calibration; Dynamic weight model building module, used to build a neural network dynamic weight model to adjust the data priority of millimeter wave radar, camera and inertial sensor in real time; A weighted calculation module, used to perform weighted calculation on the calibrated three-axis coordinates and generate a continuous tracking path in combination with a velocity vector; A motion pattern prediction model building module is used to obtain a motion pattern prediction model for predicting the future motion trajectory of a target object; The candidate path generation and screening module is used to predict and generate three candidate paths and screen the valid candidate path set; The credibility score calculation module is used to calculate the average deviation coefficient of the candidate paths, combine the deviation threshold and the effective path ratio, and output the credibility score of the predicted path; The process correction module, if the credibility score does not reach the threshold, makes corrections through iterative model training, weight redistribution or sensor replacement; The effective prediction path selection module is used to select the effective prediction path with the smallest three-axis coordinate deviation as the main solution, and retain the suboptimal prediction path as a backup solution.

Images