An Incremental Learning Balance Inference Method for Adversarial Applications

Abstract

The present invention discloses an incremental learning balance deduction method for confrontation, which belongs to the field of flight behavior recognition and prediction technology. Based on the flight trajectory information of the target party, the present invention uses trajectory difference calculation and a double threshold difference discrimination method based on a buffer zone to enable the model to perform incremental learning more efficiently. On the other hand, the sample mixing processing of the present invention ensures that the model retains the memory of old knowledge while learning new knowledge. The present invention can be used to fully deduce the game and complex changes in the process of aerial confrontation drills of unmanned aerial vehicles.

Description

Technical Field

[0001] The present invention belongs to the technical field of flight behavior recognition and prediction, and specifically relates to an incremental learning balance deduction method for confrontation based on flight behavior recognition. Background Art

[0002] Data-driven intelligent algorithms usually face a severe challenge, namely the number of data samples. Single situation prediction cannot effectively provide the prediction algorithm with sufficient and diverse sample data, and thus cannot complete the training of the intelligent prediction algorithm. Therefore, in view of the strong uncertainty and multiple possibilities in the aerial confrontation exercise of drone swarms, it is necessary to design a corresponding parallel deduction framework, run it in practice, and realize the parallel deduction of multiple simulation instances based on key event nodes, so as to provide the original data source for subsequent data processing modules or behavior prediction.

[0003] The parallel simulation and deduction system reconstructs the target state information space according to the target's current situation information, further combines the current behavior understanding results, and combines the expert knowledge in the data support module to generate multiple possible branch routes in the future and the proportion of related branch possibilities, forming a corresponding simulation and deduction plan, and performing multi-threaded or multi-process simulation and parallel deduction. Through the parallel deduction of multiple possibilities in the future, the corresponding future state data can be obtained.

[0004] Considering the game-playing and uncertainty of the aerial confrontation exercise process of drone swarms, it is a difficult task to obtain useful information from massive data to analyze and process the data quickly and accurately. Traditional batch learning can no longer meet the timeliness requirements of scenarios with high uncertainty. If parallel deduction is to be able to make real-time adjustments according to the actual situation of the confrontation scenario, it is necessary to use new samples obtained from the environment to correct the existing prediction model and realize online incremental learning. However, online incremental learning has the following two disadvantages: on the one hand, the algorithm is prone to overfitting the training samples; on the other hand, due to the lack of a selective forgetting and elimination mechanism for training samples, the classification accuracy is greatly affected. Summary of the invention

[0005] The present invention provides an incremental learning balance deduction method for confrontation, which can be used to improve the behavior recognition and behavior prediction performance of aerial targets.

[0006] The technical solution adopted by the present invention is:

[0007] An incremental learning balance deduction method for confrontation, the method comprising the following steps:

[0008] Step 1: Periodically collect the flight status data of the target party and perform data preprocessing on the collected flight status data, including data cleaning and data normalization;

[0009] Step 2: Input the flight status data collected at time t and preprocessed into the behavior recognition model, and obtain the behavior A at time t based on the output behavior recognition result of the behavior recognition model. t , the behavior recognition model is an end-to-end model based on deep learning;

[0010] Step 3: Based on the response behavior configured for each behavior type, based on behavior A at time t t Get the coping behavior D at time t t ;

[0011] Step 4: Set a time-based sliding window w, and store the target party’s real behavior sequence (A t-w+1 ,...,A t ) and coping behavior sequence (D t-w+1 ,...,D t ) as the input of the prediction model, and multiple deduction behavior sequences (A′) are obtained based on parallel deduction. t ,...,A′ t+w-1 )1,(A′ t ,...,A′ t+w-1 )2, ...; where the deduction behavior sequence is from time t to the next w moments; A′ t ~A′ t+w-1 Indicates that the prediction model outputs the prediction behavior, and the prediction model is an end-to-end model based on deep learning;

[0012] Step 5: Compare multiple deduced behavior sequences with the target party’s real behavior sequence (A t-w+1 ,...,A t ) performs trajectory difference calculation, takes the average of the obtained multiple differences, and obtains the final difference;

[0013] Step 6, generating a mixed sample set based on the final difference, and using the mixed sample set as a training set to train the prediction model;

[0014] The specific method of generating a mixed sample set based on the final difference is:

[0015] If the final difference is lower than the predefined minimum difference threshold, the sample mixing process is not performed;

[0016] If the final difference is higher than the predefined maximum difference threshold, the target party's real behavior sequence and the responder's real response sequence are used as new samples to perform sample mixing processing, and after obtaining the mixed sample, the new sample is stored in the database;

[0017] If the final difference is between the minimum difference threshold and the maximum difference threshold, the target party's real behavior sequence and the responder's real response sequence are stored in the cache. If the cache exceeds the preset cache threshold, all data in the cache are used as new samples and sample mixing is performed. After the mixed sample is obtained, the new sample is stored in the database and the cache is cleared.

[0018] The sample mixing process is as follows: based on the configured sample selection strategy, some old samples in the database are read and mixed with the obtained new samples to obtain a mixed sample set.

[0019] Furthermore, in step 1, the flight status data includes: longitude, latitude, speed, heading angle, elevation angle, tangential acceleration and horizontal normal acceleration.

[0020] Furthermore, in step 1, data normalization refers to: performing maximum and minimum normalization processing on each dimension of the flight status data. For example, for the dimension of longitude, normalization is performed based on the maximum and minimum values ​​of the longitude.

[0021] Furthermore, in step 4, multiple deduction behavior sequences are obtained based on parallel deduction, specifically:

[0022] Step 401: If the current time is t, set the first time in the sliding window to T = t-w+1;

[0023] Step 402: The target behavior sequence (A T ,...,A T+w-1 ), and the responder's coping behavior sequence (D T ,...,D T+w-1 ) Input the prediction model and obtain the predicted behavior (behavior prediction result) A′ of the target party at time T+w T+w ;

[0024] Step 403: Based on behavior A′ T+w , and get the corresponding response behavior D of the responding party T+w , for different D T+w Multiple branches are formed to obtain different defensive behavior sequences;

[0025] Step 404: set T to T+1;

[0026] Step 405: If T>t+w-1, end the parallel deduction, otherwise continue to execute step 402.

[0027] Furthermore, in step 5, the trajectory difference calculation is specifically as follows:

[0028] Step 501: One-hot encode each behavior to obtain the encoding of each behavior of the target party;

[0029] Step 502: Encode each behavior in the two behavior sequences to obtain two 0-1 matrices of shape m×w, where m is the length of the behavior code;

[0030] Step 503: Connect the positions with a value of 1 in the two matrices in column order to form two behavior trajectories L1 and L2. The coordinates of the points on the trajectories are (x 1 ,y 1 ) and (x 2 ,y 2 );

[0031] The difference between the deduced behavior sequence and the target party’s actual behavior sequence is calculated according to the formula: in, Representative trajectory L t The horizontal coordinate of the i-th point on the Represents the horizontal coordinate of the i-th point on trajectory L2.

[0032] Furthermore, in step 6, the sample mixing process is specifically as follows:

[0033] Divide the old samples in the database into multiple categories according to the labels. For each category of samples: if there are fewer than the specified number D of samples in this category, select all of them; otherwise, select D samples randomly.

[0034] The selected old samples and new samples are mixed and randomly shuffled to form a mixed sample set.

[0035] The technical solution provided by the present invention brings at least the following beneficial effects:

[0036] Based on the flight trajectory information of the target party, the present invention can make the model more efficiently perform incremental learning by using trajectory difference calculation and a double-threshold difference discrimination method based on a buffer. On the other hand, the sample mixing processing of the present invention ensures that the model retains the memory of old knowledge while learning new knowledge. The present invention can be used to fully deduce the game and complex changes in the process of aerial confrontation drills of drones. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1It is a schematic diagram of a processing process of an incremental learning balance deduction method for confrontation provided by an embodiment of the present invention;

[0039] Figure 2 is a processing flow chart of an embodiment of the present invention for an air confrontation exercise;

[0040] Figure 3 It is a parallel demonstration behavior sequence diagram of a drone swarm aerial confrontation exercise in an embodiment of the present invention;

[0041] Figure 4 Schematic diagram of the trajectory difference calculation process in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0043] In order to solve the shortcomings of traditional deep learning prediction models in coping with the game-playing and uncertainty in the confrontation process during aerial target confrontation exercises, as well as the inaccuracy of incremental model updates and catastrophic forgetting, an embodiment of the present invention provides an incremental learning balance deduction method for confrontation, which is suitable for target groups with flight trajectories.

[0044] The incremental learning balance deduction method for confrontation provided by the embodiment of the present invention includes five aspects: behavior recognition, behavior prediction, parallel deduction of air confrontation exercises, sample mixing and online incremental learning, wherein behavior recognition is implemented based on a set behavior recognition model, and behavior prediction is implemented based on a set behavior prediction model, and the two models are composed of an end-to-end model based on deep learning; the parallel deduction of air confrontation exercises mainly utilizes the behaviors of the two parties at multiple current moments (one of which adopts a corresponding response strategy based on the behavior of the other party), and the responding party conducts multi-branch ultra-real-time deduction based on the behavior prediction model of the target party, and deduces the behavior sequence of the target party at multiple future moments according to the different response plans of the responding party; online incremental learning mainly refers to using the trajectory difference calculation method to calculate a series of behavior sequences obtained by parallel deduction. The difference between the sequence and the real behavior sequence is calculated; if the difference is lower than the predefined minimum difference threshold, no processing is performed; if the difference is higher than the predefined maximum difference threshold, the real behavior sequence is used as a new sample to continue sample mixing, and the new sample is stored in the database after the mixed sample is obtained; if the difference is between the minimum difference threshold and the maximum difference threshold, the real behavior sequence is stored in the cache area, and if the cache area exceeds the preset cache threshold, all data in the cache area are used as new samples and sample mixing is performed, and the new sample is stored in the database after the mixed sample is obtained, and the cache area is cleared; when the samples are mixed, some old samples in the database are read through the sample selection algorithm, and they are mixed with the obtained new samples to obtain a mixed sample set; the mixed sample set is used as a training set to train the target behavior prediction module to achieve online learning.

[0045] like Figure 1 As shown, the incremental learning balance deduction method for confrontation provided by the embodiment of the present invention includes the following steps:

[0046] 1. Periodically collect the flight status data of the target party, which is usually a cluster of multiple targets, such as a swarm of drones;

[0047] 2. Perform data preprocessing on the collected flight status data, including data cleaning and data normalization;

[0048] 3. Input the flight status data after data preprocessing (the sampling time is represented as t) into the behavior recognition model, and obtain the behavior A at time t based on the output behavior recognition result (predicted behavior category) of the behavior recognition model. t ;

[0049] 4. Based on the response behavior configured for each behavior type, the corresponding A t Coping behavior D t ;

[0050] 5. Set the input data of the prediction model based on the sliding window method, and obtain multiple deduction behavior sequences based on the output of the prediction model based on parallel deduction;

[0051] 6. Calculate the trajectory difference between the multiple deduced behavior sequences and the target party's real behavior sequence, take the average of the multiple differences obtained, and obtain the final difference;

[0052] Seventh, generate a mixed sample set based on the final difference, and use the mixed sample set as a training set to train the prediction model.

[0053] As a possible implementation, in the embodiment of the present invention, the data storage format in the database is as follows:

[0054] Property Name Data Types meaning ID Interger Data No. TimeId Interger Data acquisition time Action Char Target Behavior Defense Char Counterparty defensive behavior (counterparty coping behavior) NextAction Char The target behavior at the next moment (label)

[0055] As a possible implementation method, the incremental learning balance deduction method for confrontation provided by the embodiment of the present invention is used in the incremental learning parallel deduction of the confrontation exercise of the unmanned cluster, see Figure 2 , the specific implementation steps are as follows:

[0056] Step S1: periodically collect flight status data of the target party (for the convenience of description, the target party is defined as the blue party, and the responding party is defined as the red party) drone group in the sensor, wherein the flight status data can select the flight information of the aerial target based on actual needs. In the embodiment of the present invention, the flight status includes data of seven dimensions: longitude long, latitude lat, speed v, heading angle head, elevation angle elev, tangential acceleration tang, and horizontal normal acceleration hori;

[0057] Step S2: Clean and normalize the flight status data of the drone swarm: perform consistency check, invalid value and missing value processing on the data, and perform maximum and minimum normalization processing on each dimension of the flight status data of the drone swarm;

[0058] Step S3: For the flight status data of the drone group at time t, input the behavior recognition model M dis , we get the behavior A at time t t The behavior recognition model can adopt the end-to-end model based on deep learning commonly used in the field. The embodiment of the present invention does not limit the specific network structure. The behavior type is also set according to the actual application scenario. In this embodiment, there are seven types of agreed behaviors, namely A t ∈{a1,a2,a3,a4,a5,a6,a7}, the behavior recognition model can be expressed as: M dis =f(long,lat,v,head,elev,tang,hori), the output is the behavior A at the current moment t ;

[0059] Step S4: Behavior recognition result A at time t in S3 t , the Red UAV group makes the corresponding defensive action D t There are seven types of agreed defensive behaviors, namely D t ∈{d1, d2, d3, d4, d5, d6, d7-. That is, for each flight behavior, at least one corresponding response behavior (defense behavior) is pre-configured.

[0060] Step S5: For the current time t, set a time-based sliding window w, and convert the blue side's real behavior sequence (A t-w+1 ,...,A t ) and the defensive behavior sequence of the Red team against the Blue team's behavior (D t-w+1 ,...,D t ) as the prediction model M fore The blue team's behavior sequence (A t ,...,A t+w-1 ), parallel push behavior is as follows Figure 3 As shown;

[0061] The prediction model can be expressed as: dis =f((A t-w+1 ,...,A t ), (D t-w+1 ,...,D t )), the output is the blue team behavior prediction result A′ at time t+1 t+1 ;

[0062] The parallel deduction steps are:

[0063] Step S501: If the current time is t, set the first time in the sliding window to T = t-w+1;

[0064] Step S502: The blue team behavior sequence (A T ,...,A T+w-1 ) and the Red side's defensive behavior sequence (D T ,...,D T+w-1 ) Input M fore , and get the blue team's behavior prediction result A' at time T+w T+w ;

[0065] Step S503: For prediction result A T+w , Red's defensive behavior D T+w There may be multiple decisions for different D T+w By forming multiple branches, different defensive behavior sequences can be obtained;

[0066] Step S504: set T to T+1;

[0067] Step S505: If T>t+w-1, end the parallel deduction, otherwise go to step S502;

[0068] Step S6: Multiple deduction behavior sequences (A′) obtained in step S5 t ,...,A′ t+w-1 )1,(A′ t ,...,A′ t+w-1 )2…, respectively, with the real behavior sequence (A t-w+1 ,...,A t ) to calculate the trajectory difference:

[0069] Step S601: One-hot encode each behavior using the sequential relationship between discrete intents:

[0070] There are many combinations of drone behaviors, and one behavior sequence corresponds to one coding method. Based on expert knowledge, it is assumed that the order of the seven blue behaviors is: a1, a2, a3, a4, a5, a6, a7, and their respective coding conventions are as follows:

[0071] intention coding <![CDATA[a1]]> 1000000 <![CDATA[a2]]> 0100000 <![CDATA[a3]]> 0010000 <![CDATA[a4]]> 0001000 <![CDATA[a5]]> 0000100 <![CDATA[a6]]> 0000010 <![CDATA[a7]]> 0000001

[0072] Step S602: using the encoding method in step S601, encode each behavior in the two behavior sequences to obtain two 0-1 matrices of shape m×w, where m is the length of the behavior code and w is the length of the behavior sequence, which is consistent with the sliding window length;

[0073] Step S603: Connect the positions where the values ​​are 1 in the two matrices in column order to form two behavior trajectories L1 and L2. The coordinates of the points on the trajectories are (x 1 ,y 1 ) and (x 2 ,y 2 ) is the row and column number of the position corresponding to the value 1 in the matrix. The two tracks are aligned by column and the difference is calculated in the row direction, such as Figure 4 As shown:

[0074] The difference calculation formula is:

[0075]

[0076] in, represents the horizontal coordinate of the i-th point on L1, Represents the horizontal coordinate of the i-th point on L2;

[0077] Step S604: taking the average of the obtained multiple differences to obtain a final difference (difference, diff);

[0078] Step S7: Determine whether diff is greater than a preset minimum difference threshold (minimum difference threshold, min_th). If so, continue (execute step S8), otherwise, go to step S12;

[0079] Step S8: Determine whether diff is greater than a maximum difference threshold (maximum difference threshold, man_th). If so, continue (execute step S9). Otherwise, go to step S13.

[0080] Step S9: Take the blue team's real behavior sequence as a new sample (the amount of data is defined as D), and perform sample mixing processing;

[0081] Step S10: Sample mixing processing: select some old samples in the database and mix them with new samples with a data volume of D:

[0082] Step S10-1: Divide the old samples in the database into multiple categories according to the labels (N is defined to represent the number of categories). For each category of samples: if there are less than D samples in the category, all of them are selected; otherwise, D samples are randomly selected.

[0083] Step S10-2: Mix the selected old samples and new samples and randomly shuffle them to form a mixed sample set;

[0084] Step S11: Use the mixed sample set as a training set to train the model M fore For training, model M fore After being updated, go to step S17.

[0085] Step S12: storing the blue team's real behavior sequence and the red team's defensive behavior sequence as new samples in the database, and then proceeding to step S17;

[0086] Step S13: storing the blue side's real behavior sequence and the red side's defensive behavior sequence in a buffer area;

[0087] Step S14: Determine whether the data length (data_len) of the cache area is greater than a preset cache threshold (cache threshold, cache_th): if yes, continue; otherwise, go to step S17;

[0088] Step S15: Read the data in the buffer area as a new sample;

[0089] Step S16: Clear the cache area and go to step S10;

[0090] Step S17: Check whether the drone swarm aerial confrontation exercise is finished. If so, continue; otherwise, go to step S1.

[0091] Step S18: End.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0093] The above are only some embodiments of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the creative concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. An incremental learning balance deduction method for confrontation, characterized in that: The following steps are involved: Step 1: Periodically collect the flight status data of the target party and perform data preprocessing on the collected flight status data, including data cleaning and data normalization; Step 2: Input the flight status data collected at time t and preprocessed into the behavior recognition model, and obtain the behavior recognition result at time t based on the output behavior recognition result of the behavior recognition model. , the behavior recognition model is an end-to-end model based on deep learning; Step 3: Based on the coping behaviors configured for each behavior type, based on the behavior at time t Get the coping behavior at time t ; Step 4: Set a time-based sliding window w, and store the target party's real behavior sequence with a length of w from t-w+1 to t ( ,... , ) and coping behavior sequences ( ,... , ) as the input of the prediction model, and multiple deduction behavior sequences are obtained based on parallel deduction ( ,... , )1,( ,... , )2, ...; where the deduction behavior sequence is from time t to the next w moments; ~ Represents the prediction behavior output by the prediction model, and the prediction model is an end-to-end model based on deep learning; Step 5: Compare multiple deduced behavior sequences with the target party’s real behavior sequence ( ,... , ) performs trajectory difference calculation, takes the average of the obtained multiple differences, and obtains the final difference; Step 6, generating a mixed sample set based on the final difference, and using the mixed sample set as a training set to train the prediction model; The specific method of generating a mixed sample set based on the final difference is: If the final difference is lower than the predefined minimum difference threshold, the sample mixing process is not performed; If the final difference is higher than the predefined maximum difference threshold, the target party's real behavior sequence and the responder's real response sequence are used as new samples to perform sample mixing processing, and after obtaining the mixed sample, the new sample is stored in the database; If the final difference is between the minimum difference threshold and the maximum difference threshold, the target party's real behavior sequence and the responder's real response sequence are stored in the cache. If the cache exceeds the preset cache threshold, all data in the cache are used as new samples and sample mixing is performed. After the mixed sample is obtained, the new sample is stored in the database and the cache is cleared. The sample mixing process is as follows: based on the configured sample selection strategy, some old samples in the database are read and mixed with the obtained new samples to obtain a mixed sample set.

2. The method according to claim 1, characterized in that In step 1, the flight status data includes longitude, latitude, speed, heading angle, elevation angle, tangential acceleration and horizontal normal acceleration.

3. The method according to claim 1 or 2, characterized in that In step 1, data normalization refers to: performing maximum and minimum normalization processing on each dimension of the flight status data.

4. The method according to claim 1, characterized in that In step 4, multiple deduction behavior sequences are obtained based on parallel deduction, specifically: Step 401: If the current time is t, set the first time in the sliding window T = t-w+1; Step 402: The target behavior sequence ( ,... , ), and the responder's sequence of coping behaviors ( ,... , ) Input the prediction model to obtain the predicted behavior of the target party at time T+w ; Step 403: Behavior-based , and get the corresponding response behavior of the responder , for different Multiple branches are formed to obtain different defensive behavior sequences; Step 404: Set T to T+1; Step 405: If T>t+w-1, end the parallel deduction, otherwise continue to execute step 402.

5. The method according to claim 1, characterized in that In step 5, the trajectory difference calculation is specifically as follows: Step 501: One-hot encode each behavior to obtain the encoding of each behavior of the target party; Step 502: Encode each behavior in the two behavior sequences to obtain two 0-1 matrices of shape m×w, where m is the length of the behavior code; Step 503: Connect the positions with a value of 1 in the two matrices in column order to form two behavior trajectories. and The coordinates of the points on the trajectory are ( , )and( , ); The difference between the deduced behavior sequence and the target party’s actual behavior sequence is calculated according to the formula: ,in, Representative trajectory The horizontal coordinate of the i-th point on the Representative trajectory The horizontal coordinate of the i-th point on the graph.

6. The method according to claim 1, characterized in that In step 6, the sample mixing process is specifically as follows: Divide the old samples in the database into multiple categories according to the labels. For each category of samples: if there are fewer than the specified number D of samples in this category, select all of them; otherwise, select D samples randomly. The selected old samples and new samples are mixed and randomly shuffled to form a mixed sample set.

Images