Domain knowledge assisted ground multi-target air monitoring method

Abstract

The application discloses a kind of field knowledge auxiliary ground multi-target air monitoring methods, comprising the following steps: S1 establishes target sensor model, establishes field knowledge set;S2 according to the target information obtained by sensor, matches target corresponding field knowledge;S3 according to the target information obtained by sensor, utilizes target sensor model, using improved JPDA method carries out multi-target tracking, in the tracking process, the state estimation value of model is corrected by field knowledge;S4 updates target corresponding field knowledge;repeat steps S3-S4, continuously track target.The field knowledge auxiliary ground multi-target air monitoring method disclosed in the application improves the multi-target tracking precision while enhancing the robustness of target tracking and trajectory updating process.

Description

TECHNICAL FIELD

[0001] The application relates to a ground multi-target air monitoring method assisted by domain knowledge and belongs to the technical field of target monitoring. BACKGROUND

[0002] Small unmanned aerial vehicles (UAVs) have been widely applied in many aspects such as traffic monitoring, post-disaster search and rescue, wild animal habitat protection, anti-terrorism operations and battlefield situation monitoring.

[0003] However, the traditional airborne air monitoring method can only track the trajectory of a single target, and the method used is mostly a simple filtering technology, such as Kalman filtering (for linear systems), extended Kalman filtering, unscented Kalman filtering, particle filtering technology (for nonlinear systems) and interactive multi-model technology (for highly maneuverable targets). The above methods cannot effectively track the trajectories of multiple targets.

[0004] Although there are methods for tracking multiple targets, these methods generally lack accuracy and have weak robustness.

[0005] In addition, the traditional airborne air monitoring method is greatly affected by external interference and sensor noise, and is prone to tracking precision decline.

[0006] Therefore, it is necessary to further study the airborne air monitoring method to realize the tracking of multiple targets and reduce the influence of external interference and sensor noise on monitoring precision. SUMMARY

[0007] In order to overcome the above problems, the present application has been researched in depth, and a ground multi-target air monitoring method assisted by domain knowledge is proposed, which comprises the following steps:

[0008] S1, establishing a target sensor model and establishing a domain knowledge set,

[0009] The target sensor model is used to describe the relationship between the target state and the sensor observation value, and the domain knowledge set is a collection of domain knowledge used to describe the geographical information constraint of the ground target;

[0010] S2, matching the target corresponding domain knowledge according to the target information obtained by the sensor;

[0011] S3, using the target sensor model to track multiple targets by using the improved JPDA method according to the target information obtained by the sensor, and correcting the state estimation value of the model by the domain knowledge in the tracking process;

[0012] S4, updating the target corresponding domain knowledge;

[0013] Steps S3-S4 are repeated to continuously track the target.

[0014] Further, in S1, the target sensor model is represented as:

[0015]

[0016] wherein, xi(k) represents the state value of target i at kth scan time,

[0017] xi(k) represents the position of target i in the three-dimensional environment at kth scan time, xi(k) represents the velocity of target i in the three-dimensional environment at kth scan time,

[0018] xi(k) represents the system noise, which is a Gaussian white noise sequence with a covariance matrix Q k ;

[0019] xi(k) represents the observation value of the sensor at kth scan time, the observation value of the onboard sensor is the distance pitch angle and azimuth angle

[0020] xi(k) represents the observation noise, which is a Gaussian white noise sequence with a corresponding covariance matrix R k ;

[0021] F k represents the transition matrix of the system, is a nonlinear mapping relationship between the system space and the observation space.

[0022] Further, the set of domain knowledge includes a road constraint subset,

[0023] The road constraint subset contains all road information in the monitoring area, and the road information is a set of multiple road segments, each road segment is represented by a straight line segment, so that each road segment has position and direction information.

[0024] Preferably, the set of domain knowledge further includes a speed constraint, and the speed constraint sets a maximum speed of the target.

[0025] Further, in S2, according to the target position obtained by the sensor, a road segment containing the target position in the road constraint subset is found, and the road segment is taken as one constraint in the domain knowledge road constraint corresponding to the target.

[0026] Preferably,

[0027] ​In S2, the road constraints in the target corresponding domain knowledge further include road segments adjacent to the road segment where the target position is located.

[0028] Preferably, in S3, the target state estimation obtained in the improved JPDA method is corrected in the target tracking process using the improved JPDA method, and the corrected target state estimation is used to replace the original target state estimation for target tracking.

[0029] The correction is performed on the target state estimation by one or more constraint information in the domain knowledge constraints.

[0030] Preferably, the correction is achieved by incorporating the constraint information into the state estimation result through a projection method.

[0031] Preferably, the domain knowledge constraints include road segment direction constraints, road segment position constraints, and speed constraints.

[0032] The road segment direction constraint in the domain knowledge constraints can be expressed as:

[0033]

[0034] The road segment position constraint in the domain knowledge constraints can be expressed as:

[0035]

[0036] The speed constraint in the domain knowledge constraints can be expressed as:

[0037]

[0038] wherein, represents the state value of the target i at the kth scan time;

[0039] g1 represents the road segment direction constraint equation, represents the road segment direction constraint matrix, represents the upper bound of the road segment direction constraint;

[0040] g2 represents the road segment position constraint equation, represents the road segment position constraint matrix, represents the upper bound of the road segment position constraint;

[0041] g3 represents the speed constraint equation, represents the speed constraint matrix, v inf represents the lower bound of the speed vector, v sup represents the upper bound of the speed vector;

[0042] wherein, the road segment direction constraint matrix is the road segment position constraint matrix is Velocity constraint matrix To

[0043] Further, the updating refers to updating the road segment and the next adjacent road segment in the corresponding domain knowledge of the target.

[0044] The present application has the beneficial effects including:

[0045] (1) By setting the domain knowledge constraint, the multi-target tracking accuracy is greatly improved, and the robustness of the target tracking and trajectory updating process is enhanced;

[0046] (2) The target trajectory updating logic is improved, and the full-automatic tracking of multiple targets can be realized, including trajectory confirmation, removal and re-confirmation. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 Fig. 1 shows a flowchart of a domain knowledge assisted ground multi-target air monitoring method according to a preferred embodiment of the present application;

[0048] Figure 2 Fig. 3 shows a test area map in Example 1;

[0049] Figure 3 Fig. 4 shows the road segments obtained in S1 in Example 1;

[0050] Figure 4 Fig. 5 shows the average error between target tracking and actual trajectory of the target in Example 1;

[0051] Figure 5 Fig. 6 shows a graph of the change in the number of targets and the number of monitored targets in Example 1. DETAILED DESCRIPTION

[0052] The present application will be further described in detail by the accompanying drawings and examples. Through these descriptions, the features and advantages of the present application will become more apparent.

[0053] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations. Although various aspects of an implementation can be described herein as being a preferred implementation or mode, for example, this and similar terminology is used herein to more readily describe the implementation as it is one implementation among numerous implementations possible.

[0054] The present application provides a domain knowledge assisted ground multi-target air monitoring method, characterized in that it comprises the following steps:

[0055] S1, establishing a target sensor model, establishing a domain knowledge constraint set,

[0056] The target sensor model is used to describe the relationship between the target state and the sensor observation value, and the domain knowledge constraint set is used to describe the set of geographic information constraints on the ground target;

[0057] S2, matching the target corresponding domain knowledge based on the target information obtained by the sensor;

[0058] S3. Based on the target information obtained by the sensor, the target sensor model is used to perform multi-target tracking using an improved JPDA method, such as the method mentioned in the article He, S., Shin, H.-S., and Tsourdos, A., "Joint probabilistic data association filter with unknown detection probability and clutter rate," Sensors, Vol. 18, No. 1, 2018, p. 269. During the tracking process, the state estimate of the model is corrected by domain knowledge constraints;

[0059] S4. Update target domain knowledge

[0060] Repeat steps S3 to S4 to continuously track the target.

[0061] In the present invention, on the basis of traditional multi-target tracking, domain knowledge constraints are added to improve the multi-target tracking accuracy and enhance the robustness of target tracking.

[0062] Specifically, in S1, the target sensor model is the basic model for target tracking. In the present invention, the target sensor model is set to:

[0063]

[0064] in, represents the state value of target i at the k-scan time,

[0065] represents the position of target i in the three-dimensional environment at the k-scan time, represents the speed of target i in the three-dimensional environment at the k-scan time,

[0066] Represents the system noise, which is a Gaussian white noise sequence, and its covariance matrix is ​​Q k ;

[0067] represents the observation value of the sensor at the k-scan moment, and the observation value of the airborne sensor is the position of target i relative to the drone distance Pitch angle and azimuth angle

[0068] represents observation noise, which is a Gaussian white noise sequence, and the corresponding covariance matrix is R k ;

[0069] F k represents the transition matrix of the system, is a nonlinear mapping relationship between the system space and the observation space.

[0070] It should be noted that: and are independent of the system state and are independent of each other.

[0071] Further, F k represents the transition matrix of the system, and is represented as:

[0072]

[0073] where t k|k-1 represents the time interval between adjacent scanning moments, I3 represents a 3x3 unit matrix corresponding to the dimension, and 03 represents a 3x3 zero matrix corresponding to the dimension.

[0074] Further, the nonlinear mapping relationship between the system space and the observation space is represented as:

[0075]

[0076] Further, unlike traditional multi-target tracking methods, in the present application, the target is also subject to domain knowledge constraints, and in the present application, the constraints on the target are represented as:

[0077]

[0078] where g represents the constraint equation, and d represents the upper bound of the constraint.

[0079] Further, the domain knowledge set includes a road constraint subset for representing the influence of roads on target states, for example, in areas without highways, targets generally cannot reach, and in areas with rivers, targets need to walk on bridges.

[0080] Further preferably, the road constraint subset contains all road information in the monitoring area, and the road information is a collection of multiple road segments, each road segment is represented by a straight line segment, so that each road segment has position and direction information.

[0081] Further preferably, the road constraint subset is obtained by the following steps:

[0082] S11, extracting a certain number of coordinate points from each road center line based on known map information;

[0083] S12, generating a curve of each road according to the extracted coordinate points, and simplifying the actual road into a curve;

[0084] S13, dividing the curve equation into multiple segments according to the change of the orientation angle of the curve.

[0085] In a preferred embodiment, the number of coordinate points in S11 is determined according to the actual precision requirement, and is preferably set to one coordinate point every 10 meters in horizontal distance;

[0086] Preferably, in S11, each coordinate point is projected from the earth coordinate system to the Cartesian coordinate system, and in the present application, the projection method is not limited, as long as the projection can be realized, for example, the Gauss-Kruger projection method is adopted.

[0087] Preferably, the projection method mentioned in the article Karney, C. F. F., “Transverse Mercator with an accuracy of a few nanometers,” Journal of Geodesy, Vol. 85, No. 8, 2011, pp. 475-485 is adopted.

[0088] In S12, preferably, the curve of each road in the Cartesian coordinate system is generated, and any method can be used to generate the curve, for example, the curve can be generated by a polynomial fitting method, and the polynomial fitting method is a basic method for fitting a curve with multiple points, and the specific parameters can be selected by experience by those skilled in the art.

[0089] In S13 of the present application, the angle between the direction of the curve and the X coordinate axis is referred to as the orientation angle, and it is emphasized that in the fitting process, the orientation angle of the curve satisfies a fixed range, and if the range of the orientation angle of the curve changes, the curve needs to be divided into two segments, the first segment is set to (-90°, 90°), and the second segment is set to (90°, 270°), thus, according to the direction of the curve, it is divided into multiple combinations of curves, for example, a circular road curve, the direction of the curve does not change, which can be divided into two parts of upper and lower circular arcs, so as to ensure that the orientation angle of the curve satisfies a single range, and then it is fitted by a polynomial fitting method.

[0090] In a preferred embodiment, a threshold value is set, when the change of the orientation angle between the position of a point on the curve and the starting point is equal to the threshold value, the curve is divided at the point, and the divided point is taken as the starting point of the new curve; the process is repeated until the curve is completely divided, and multiple curves are obtained.

[0091] Preferably, the threshold is set to 3°.

[0092] Further, the obtained curve is simplified into a plurality of straight line segments by connecting the start point and the division point of the curve in sequence with straight lines, so as to reduce the operation amount, and each straight line segment represents a road segment.

[0093] According to the present application, each obtained straight line segment has position and direction information, and can be represented as:

[0094]

[0095] wherein r represents different road segments, represents the direction information of the road segment r, κ r represents the position information of the road segment r, [x r , f(x r )] represents the start point of the road segment r in the curve, [x r+1 , f(x r+1 )] represents the end point of the road segment r in the curve.

[0096] In a preferred embodiment, the set of domain knowledge further comprises a speed constraint subset, and the target maximum speed is set in the speed constraint subset.

[0097] In S2, according to the target position obtained by the sensor, the road segment containing the target position in the road constraint subset is found, and the road segment is taken as the target corresponding domain knowledge.

[0098] In a preferred embodiment, the target corresponding domain knowledge further comprises a road segment adjacent to the road segment where the target position is located, and is represented as

[0099] In a preferred embodiment, the target corresponding domain knowledge further comprises the maximum speed of the target.

[0100] According to the present application, in S3, the improved JPDA method refers to any method for realizing multi-target tracking based on the JPDA method, and is preferably the method described in the article He, S., Shin, H.-S., and Tsourdos, A., “Joint probabilistic data association filter with unknown detection probability and clutter rate,” Sensors, Vol. 18, No. 1, 2018, p. 269.

[0101] In the improved JPDA method, the general process comprises: obtaining a target state estimation, then obtaining existence probability density of different trajectories according to the target state estimation, then performing trajectory updating, and initializing a new target state.

[0102] For example, the method mentioned in Joint probabilistic data association filter with unknown detection probability and clutter rate comprises the following sub-steps:

[0103] S31, based on the target state value at the last moment and the covariance matrix thereof predict each target state by extended Kalman filtering and the corresponding covariance matrix

[0104] S32, obtain an edge probability density based on target existence by the improved JPDA method;

[0105] S33, update the target state based on the association result by extended Kalman filtering;

[0106] S34, obtain a target state estimation;

[0107] S36, obtain existence probability density of different trajectories;

[0108] S37, perform trajectory updating, and initialize a new target state.

[0109] In the present application, the specific steps involved in the traditional improved JPDA method are described in detail, and only the differences between the present application and the traditional method are described below.

[0110] According to the present application, the difference from the traditional improved JPDA method is that, in the process of target tracking by using the improved JPDA method, the target state estimation obtained by the improved JPDA method is corrected, and the corrected target state estimation is used to replace the original target state estimation for target tracking,

[0111] Specifically, after obtaining the target state estimation, before obtaining the existence probability density of different trajectories according to the target state estimation, there is also:

[0112] S35, correct the target state estimation by domain knowledge constraint.

[0113] Specifically, the target state estimation is corrected by one or more constraints in the domain knowledge;

[0114] According to S2, the knowledge field comprises road constraints and speed constraints, and in the road constraints, the position and direction of a road segment are included, that is, the knowledge field constraints comprise road segment direction constraints, road segment position constraints and speed constraints, and in the application, one or more of the constraint information is used to modify the target state estimation.

[0115] The modification is achieved by incorporating the constraint information into the state estimation result by a projection method, and the projection method is one of the common solving methods for constraint linear equations, and the solving process can refer to the article [Chen Yonglin, Yuan Yongxin. Projection method for solving constraint linear equations [J]. Journal of Nanjing University: Natural Science Edition, 1996, 19(1): 4.], and the process is not described herein.

[0116] Further, the road segment direction constraint can be expressed as:

[0117]

[0118] The road segment position constraint can be expressed as:

[0119]

[0120] The speed constraint can be expressed as:

[0121]

[0122] wherein, Xi represents the state value of the target i at the k scanning moment;

[0123] g1 represents a road segment direction constraint equation, Gi represents a road segment direction constraint matrix, Gi represents an upper limit of the road segment direction constraint;

[0124] g2 represents a road segment position constraint equation, Gi represents a road segment position constraint matrix, Gi represents an upper limit of the road segment position constraint;

[0125] g3 represents a speed constraint equation, Gi represents a speed constraint matrix, v inf vi represents a lower limit of the speed vector, v sup vi represents an upper limit of the speed vector.

[0126] wherein, the road segment direction constraint matrix is the road segment position constraint matrix is the speed constraint matrix is

[0127] In one preferred embodiment, unlike the conventional way, in the improved JPDA method, in the track updating, the disappeared confirmed track is not immediately deleted, but a time threshold is set, when the time of track disappearance is less than the time threshold, the disappeared track will be kept as a predicted estimate of the to-be-confirmed track,

[0128] If the to-be-confirmed track corresponding to the target is detected again within the time threshold, the to-be-confirmed track is restored as a confirmed track and is updated; if the time threshold is exceeded, the to-be-confirmed track is deleted, and it is considered that the target has completely disappeared.

[0129] This updating method greatly reduces the influence of external interference and sensor noise on tracking accuracy, and enhances the robustness in the target tracking and track updating process.

[0130] In S4, the updating refers to updating the road segment and the next adjacent road segment in the domain knowledge corresponding to the target,

[0131] Preferably, the following steps can be performed:

[0132] S41, the variable structure multiple model method (VS-MM) is used to construct the road segment ζ r and the next adjacent road segment ζ r+1 in the domain knowledge corresponding to the target into a model, which is represented as and obtain the posterior probability of the corresponding road segment as the posterior probability of the road segment at the last time;

[0133] The variable structure multiple model method (VS-MM) is a classical state estimation model method, and specific reference can be made to the paper LI X R, Bar-Shalom Y. Mode-Set Adaptation in Multiple-Model Estimators for Hybrid Systems [C] / / In Proceed-ings of the 1992 American Control Conference, Chicago, IL, June 1992: 1794-1799.

[0134] S42, based on the matching posterior probability of the road segment at the last time the prior probability of each road segment in the model matching the corresponding target is estimated through a Markov chain transmission model

[0135] The Markov chain transmission is a common statistical method, and is a classical method for probability modeling and data analysis, and specific methods thereof will not be described herein.

[0136] S43, based on the improved JPDA method obtained in the data association results And target prior probability Obtained by Bayesian method target matching each section of the posterior probability density

[0137] Bayesian method is a method of calculating the probability of assumptions, which can be found in the specific method of 'Yang Xianze. 21st century university characteristic teaching materials artificial intelligence and machine translation: Southwest Jiaotong University Press, February 2006: 1st edition, 233 pages'.

[0138] S44, set the section update threshold p u , the section update threshold p u Update the model in the target corresponding domain knowledge If Then update the set {ζ r+1 ,ζ r+2}, otherwise keep {ζ r ,ζ r+1};

[0139] S45, output The direction and position of the section, as the constraint of the target domain knowledge.

[0140] Embodiments

[0141] Embodiments 1

[0142] The small unmanned aerial vehicle is used to monitor the ground multi-target in the air, and the experimental area map is shown in Figure 2 The monitoring area is between A and B, and the number of targets is increased or decreased by covering and other methods in the monitoring experiment. The detection process includes the following steps:

[0143] S1, establish a target sensor model, establish a domain knowledge set,

[0144] The target sensor model is used to describe the relationship between the target state and the sensor observation value, and the domain knowledge set is a set of domain knowledge, which is used to describe the geographical information constraint of the ground target;

[0145] S2, according to the target information obtained by the sensor, match the target corresponding domain knowledge;

[0146] S3, based on the target information obtained by the sensor, using the target sensor model, using the improved JPDA method for multi-target tracking mentioned in the article He, S., Shin, H.-S., and Tsourdos, A., "Joint probabilistic data association filter with unknown detection probability and clutter rate," Sensors, Vol. 18, No. 1, 2018, p. 269 to modify the state estimate value of the model through domain knowledge in the tracking process;

[0147] S4, updating the target corresponding domain knowledge;

[0148] Repeat steps S3-S4 to continuously track the target.

[0149] In S1, the target sensor model is represented as:

[0150]

[0151] The transition matrix F of the system k is represented as:

[0152]

[0153] The nonlinear mapping relationship between the system space and the observation space is represented as:

[0154]

[0155] The road constraint subset contains all road information in the monitoring area, and the road information is a set of multiple road segments, each road segment is represented by a straight line segment, so that each road segment has position and direction information,

[0156] The road constraint subset is obtained by the following steps:

[0157] S11, based on the known map information, such as Figure 2 , extract a certain number of coordinate points from each road center line;

[0158] S12, generate a curve for each road according to the extracted coordinate points, and simplify the actual road into a curve;

[0159] S13, divide the curve equation into multiple segments according to the change of the curve orientation angle;

[0160] A threshold is set in S13, when the change of orientation angle between a certain point on the curve and the starting point is equal to the threshold, the curve is divided at the point, and the divided point is taken as the starting point of the new curve; the process is repeated until the curve is completely divided, a plurality of curves are obtained, and the threshold is set to 3°,

[0161] The starting point and the divided point of the curve are connected in sequence by straight lines, the obtained curve is simplified into a plurality of straight line segments, and the result is as shown in Figure 3 The number of segmented points is proportional to the road curvature, which proves the effectiveness of the map information extraction module.

[0162] The position and direction information of each straight line segment obtained can be expressed as:

[0163]

[0164] The domain knowledge set also includes a speed constraint subset, and the target maximum speed is set in the speed constraint subset.

[0165] In S2, according to the target position obtained by the sensor, a road segment containing the target position in the road constraint subset is found, and the road segment is taken as the target corresponding domain knowledge, and the target corresponding domain knowledge also includes a road segment adjacent to the road segment where the target position is located, and is expressed as

[0166] In S3, the improved JPDA method includes the following sub-steps:

[0167] S31, based on the target state value and the covariance matrix thereof at the last moment, the state of each target is predicted by extended Kalman filtering and the corresponding covariance matrix

[0168] S32, the edge probability density based on the existence of the target is obtained by the improved JPDA method;

[0169] S33, the target state based on the association result is updated by extended Kalman filtering;

[0170] S34, the target state estimation is obtained;

[0171] S35, the target state estimation is corrected by the constraint of the domain knowledge;

[0172] S36, the existence probability density of different trajectories is obtained;

[0173] S37, trajectory updating is performed, and the state of the new-born target is initialized.

[0174] In S35, the target state estimation is corrected by one or more constraints in the domain knowledge;

[0175] wherein the road segment direction constraint can be represented as:

[0176]

[0177] The road segment position constraint can be represented as:

[0178]

[0179] The speed constraint can be represented as:

[0180]

[0181] The road segment direction constraint matrix is The road segment position constraint matrix is The speed constraint matrix is

[0182] The modification, by oblique projection method, the constraint information into the state estimation results.

[0183] In S37, not immediately delete the confirmed trajectory disappears, but set a time threshold, when the trajectory disappears time is less than the time threshold, the disappearing trajectory will be as a to-be-confirmed trajectory to maintain the prediction estimate,

[0184] If the to-be-confirmed trajectory corresponding target is detected again within the time threshold, the to-be-confirmed trajectory is restored to the confirmed trajectory and updated; if the time threshold is exceeded, the to-be-confirmed trajectory is deleted, and it is considered that the target has completely disappeared.

[0185] In S4, the following steps are taken:

[0186] S41, the variable structure multi-model method (VS-MM) is used to construct the model of the road segment ζ r and the next road segment ζ r+1 adjacent to the road segment, represented as and obtain the posterior probability of the corresponding road segment as the posterior probability of the road segment at the previous time;

[0187] S42, based on the posterior probability of the road segment matching at the previous time the prior probability of each road segment matching the corresponding target is estimated by Markov chain transmission model

[0188] S43, based on the data association results obtained by the improved JPDA method and the target prior probability Obtain the posterior probability density of target matching for each road segment using the Bayesian method

[0189] S44. Setting the road segment update threshold p u , update the threshold p based on the road segment u Update the model in the target corresponding domain knowledge like Update the collection For {ζ r+1 ,ζ r+2}, otherwise keep {ζ r ,ζ r+1};

[0190] S45, output The direction and position of the road segment are used as domain knowledge constraints of the target.

[0191] The final result is as follows Figure 4 、 5 As shown, condition 1 is that domain knowledge correction is not used in S35, condition 2 is that only section direction constraint correction is used in S35, condition 3 is that section direction and section position constraint correction is used in S35, and condition 4 is that section direction, section position and speed constraint correction is used in S35.

[0192] Figure 4 The average error between the target tracking and the actual target trajectory is shown. It can be seen from the figure that domain constraints can greatly reduce tracking errors and improve tracking quality, and the tracking quality improves with the increase of constraints in domain knowledge.

[0193] Figure 5 The graph shows the change in the number of targets and the change in the number of monitored targets in the experiment. It can be clearly seen that new targets can be detected and initialized, disappeared targets will be removed, and targets that disappear briefly in the middle (corresponding to about 30s-40s and 55s-65s) can be re-detected and updated, which proves the effectiveness of the trajectory update in this method and shows that the existence of domain knowledge constraints can enhance the robustness of the algorithm for trajectory update.

[0194] In the description of the present invention, it should be noted that the terms "upper," "lower," "inner," "outer," "front," and "rear" and the like, indicating positions or locations, are based on the operating state of the present invention and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0195] In the description of the application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connecting" should be understood in a broad sense, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be direct connection, or indirect connection through intermediate medium; it can be internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the application can be understood according to the specific circumstances.

[0196] The above describes the application in combination with the preferred embodiments, but these embodiments are only exemplary and serve only to illustrate. On this basis, various substitutions and improvements can be made to the application, which all fall within the protection scope of the application.

Claims

1. A domain knowledge-assisted ground multi-target aerial monitoring method, characterized in that: The following steps are involved: S1. Establish target sensor model and domain knowledge set. The target sensor model is used to describe the relationship between the target state and the sensor observation value. The domain knowledge set is a collection of domain knowledge, which is used to describe the ground target subject to geographic information constraints. S2, matching the target corresponding domain knowledge based on the target information obtained by the sensor; S3. Based on the target information acquired by the sensor, the target sensor model is used to perform multi-target tracking using an improved JPDA method. During the tracking process, the state estimate of the model is corrected using domain knowledge. The correction is performed using one or more constraint information in the knowledge domain constraints to correct the target state estimate. The knowledge domain constraints include a road section direction constraint, a road section position constraint, and a speed constraint. S4, update the target corresponding domain knowledge; Repeat steps S3 to S4 to continuously track the target.

2. The domain knowledge-assisted ground multi-target aerial monitoring method according to claim 1 is characterized in that: In S1, the target sensor model is expressed as: in, represents the state value of target i at the k-scan time, represents the position of target i in the three-dimensional environment at the k-scan time, represents the speed of target i in the three-dimensional environment at the k-scan time, Represents the system noise, which is a Gaussian white noise sequence, and its covariance matrix is ​​Q k ; represents the observation value of the sensor at the k-scan moment, and the observation value of the airborne sensor is the position of target i relative to the drone distance Pitch angle and azimuth Represents the observation noise, which is a Gaussian white noise sequence, and the corresponding covariance matrix is ​​R k ; F k represents the transfer matrix of the system, is the nonlinear mapping relationship between the system space and the observation space.

3. The domain knowledge-assisted ground multi-target aerial monitoring method according to claim 1 is characterized in that: The domain knowledge set includes a road constraint subset, The road constraint subset includes all road information in the monitoring area. The road information is a collection of multiple road segments, and each road segment is represented by a straight line segment, so that each road segment has position and direction information.

4. The domain knowledge-assisted ground multi-target aerial monitoring method according to claim 3 is characterized in that: The domain knowledge set further includes a speed constraint, in which a target maximum speed is set.

5. The domain knowledge-assisted ground multi-target aerial monitoring method according to claim 1 is characterized in that: In S2, based on the target position obtained by the sensor, the road section containing the target position is found in the road constraint subset, and the road section is used as a constraint in the road constraint of the target corresponding domain knowledge.

6. The domain knowledge-assisted ground multi-target aerial monitoring method according to claim 5, characterized in that: In S2, the road constraints in the target corresponding domain knowledge also include road sections adjacent to the road section where the target location is located.

7. The domain knowledge-assisted ground multi-target aerial monitoring method according to claim 1, characterized in that: In S3, during the target tracking process using the improved JPDA method, the target state estimate obtained by the improved JPDA method is corrected, and the corrected target state estimate replaces the original target state estimate for target tracking.

8. The domain knowledge-assisted ground multi-target aerial monitoring method according to claim 7, characterized in that: The correction is achieved by integrating constraint information into the state estimation result through an oblique projection method.

9. The domain knowledge-assisted ground multi-target aerial monitoring method according to claim 7, characterized in that: The road direction constraint in the knowledge domain constraint can be expressed as: The road segment location constraint in the knowledge domain constraint can be expressed as: The speed constraint in the knowledge domain constraint can be expressed as: in, represents the state value of target i at the k-scan time; g1 represents the road section direction constraint equation, represents the road section direction constraint matrix, Indicates the upper bound of the road section direction constraint; g2 represents the road segment position constraint equation, represents the road segment position constraint matrix, Indicates the upper bound of the road segment position constraint; g3 represents the velocity constraint equation, represents the velocity constraint matrix, v inf Represents the lower bound of the velocity vector, v sup represents the upper bound of the velocity vector; Among them, the road section direction constraint matrix for Road segment position constraint matrix for Velocity constraint matrix for 10. The domain knowledge-assisted ground multi-target aerial monitoring method according to claim 1, characterized in that: The updating refers to updating the road segment and the next adjacent road segment in the domain knowledge corresponding to the target.

Images