Unmanned aerial vehicle target recognition method based on pulse category perception boundary evolution

Abstract

This invention discloses a UAV target recognition method based on pulse category perception boundary evolution, relating to the field of UAV technology. The method includes: acquiring an image to be detected; inputting the image to be detected into a pre-trained spiking neural network model for analysis and processing, to output the UAV target category of the target; the technical solution of this invention solves the problem of poor stability in UAV target recognition in dynamic scenes, achieving the technical effect of improving the accuracy of UAV target category recognition.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a UAV target recognition method based on pulse category perception boundary evolution. Background Technology

[0002] With the rapid development of UAV technology, the method of using UAVs to identify and judge ground and / or air targets in real time is becoming increasingly popular. Therefore, how to improve the accuracy and reliability of UAV target identification has become an important research topic in the current UAV application field.

[0003] Currently, most target recognition methods for UAVs are based on traditional deep neural network models. However, such methods are computationally complex and energy-intensive, and they are difficult to fully characterize the event-driven characteristics and temporal evolution of the target during its motion. In high-speed motion or complex dynamic scenarios, their recognition stability is poor. Summary of the Invention

[0004] This invention provides a UAV target recognition method based on pulse category perception boundary evolution to achieve accurate UAV target recognition and improve the accuracy of UAV target recognition.

[0005] In a first aspect, embodiments of the present invention provide a UAV target recognition method based on pulse category-aware boundary evolution, comprising: Acquire the image to be detected; wherein the image to be detected is an image containing the target to be detected, collected by a UAV. The image to be detected is input into a pre-trained spiking neural network model for analysis and processing to output the UAV target category of the target. The spiking neural network model is trained based on perturbation boundary values ​​created and updated based on training iterations. These perturbation boundary values ​​include category-level shared boundary values ​​and sample-level residual values. The category-level shared boundary values ​​provide a baseline perturbation constraint for each sample category, and the sample-level residual values ​​are used to perform sample-level corrections to the baseline perturbation constraint. The perturbation boundary values ​​constrain the upper limit of the perturbation amplitude when multiple adversarial examples are generated. The category-level shared boundary values ​​and the sample-level residual values ​​are adaptively adjusted based on the classification results of the multiple adversarial examples. The multiple adversarial examples are multiple second samples, which are generated after perturbating a first sample. The first sample is the original sample.

[0006] Secondly, embodiments of the present invention provide a UAV target recognition device based on pulse category perception boundary evolution, comprising: The image to be detected module is used to acquire the image to be detected; wherein, the image to be detected is an image containing the target to be detected, collected by a UAV. The target category output module is used to input the image to be detected into a pre-trained spiking neural network model for analysis and processing, so as to output the UAV target category of the target to be detected; wherein, the spiking neural network model is trained based on perturbation boundary values ​​created and updated based on training iterations, the perturbation boundary values ​​include category-level shared boundary values ​​and sample-level residual values, the category-level shared boundary values ​​are used to provide a baseline perturbation constraint for each sample category, and the sample-level residual values ​​are used to perform sample-level correction on the baseline perturbation constraint; the perturbation boundary values ​​are used to constrain the upper limit of the perturbation amplitude when multiple adversarial examples are generated, and the category-level shared boundary values ​​and the sample-level residual values ​​are adaptively adjusted based on the classification results of the multiple adversarial examples, the multiple adversarial examples are multiple second samples, the second samples are generated after perturbation processing of the first samples, and the first samples are the original samples.

[0007] Thirdly, embodiments of the present invention also provide an electronic device, comprising: At least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a UAV target recognition method based on pulse category perception boundary evolution as provided in any embodiment of the present invention.

[0008] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions for causing a processor to execute a UAV target recognition method based on pulse category perception boundary evolution as provided in any embodiment of the present invention.

[0009] Fifthly, embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements a UAV target recognition method based on pulse category perception boundary evolution as described in any one of the embodiments of the present invention.

[0010] This invention addresses the problem of poor stability in drone target recognition in dynamic scenarios by acquiring a target image from a drone and inputting it into a pre-trained spiking neural network model for analysis and processing to output the drone target category of the target. This improves the accuracy of drone target category recognition.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic flowchart of a UAV target recognition method based on pulse category perception boundary evolution provided by an embodiment of the present invention; Figure 2 A flowchart of perturbation boundary value update for a UAV target recognition method based on pulse category perception boundary evolution provided in an embodiment of the present invention; Figure 3 A flowchart illustrating a UAV target recognition method based on pulse category perception boundary evolution provided in an embodiment of the present invention; Figure 4 A schematic diagram of the training process for a UAV target recognition method based on pulse category perception boundary evolution provided in an embodiment of the present invention; Figure 5 A schematic diagram of the structure of a UAV target recognition device based on pulse category perception boundary evolution provided in an embodiment of the present invention; Figure 6 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] Before introducing the UAV target recognition method based on pulse category perception boundary evolution, we will first elaborate on the training process of the spiking neural network.

[0017] First, the second sample used to train the spiking neural network is introduced; further, based on the second sample, the method for determining the category-level shared boundary value and the sample-level residual value is explained; further, after generating the third sample, how to dynamically update the category-level shared boundary and the sample-level residual according to the classification results is introduced; finally, the process of determining the parameters of the spiking neural network model is explained.

[0018] Figure 1 This is a schematic flowchart of a UAV target recognition method based on pulse category perception boundary evolution, provided as an embodiment of the present invention. This embodiment elaborates on the generation process of the second sample used to train the spiking neural network, and further describes the training process of the spiking neural network based on the second sample. For detailed implementation methods, please refer to the description of this embodiment. Figure 1 As shown, the method includes: S110. Input multiple first samples into the current spiking neural network model so that the current spiking neural network model outputs the classification result of the first samples.

[0019] In this embodiment, the first sample is raw input data collected from the original dataset without any adversarial perturbation or data augmentation. Optionally, the first sample in the original dataset can be an image containing at least one target collected by a drone. This can be understood as the original dataset being composed of multiple first samples, obtained by collecting raw image data during flight using an imaging sensor mounted on the drone. The first sample may contain one or more targets to be identified, or it may only contain the background environment. The targets to be identified can be drones, ground vehicles, buildings, and people, etc.

[0020] The current spiking neural network model is a neural network constructed using spiking neurons. In this embodiment, based on the spiking neural network model, features are extracted from the first sample to output the classification result corresponding to the first sample. It should be noted that the current spiking neural network model is the spiking neural network model of the current training iteration.

[0021] Next, the processing procedure for the first sample based on the current spiking neural network will be described.

[0022] First, the first sample is preprocessed, which involves converting each pixel value in the first sample into a pulse sequence over a time step. For each time step, a 0 / 1 pulse is randomly generated using the pixel value as the firing probability, ultimately generating the input pulse matrix.

[0023] Furthermore, for each time step, the input pulse matrix is ​​input into the input layer of the current spiking neural network, and layer-by-layer computation is performed. Taking one layer as an example, the input layer neurons receive the input pulse matrix. For the l-th layer, all neurons in the (l-1)-th layer are traversed. Based on the weights connecting the neurons in the (l-1)-th layer to the neurons in the l-th layer and the pulse output of the neurons in the (l-1)-th layer at the current time step, the input pulse signal of the l-th layer is determined.

[0024] After acquiring the input pulse signal of the l-th layer, the membrane potential state of each spiking neuron is updated in a discrete-time step based on the input pulse signal. That is, the membrane potential of each neuron is updated according to the defined integral-fire model: ; in, Let i be the membrane potential at the current time step (t), and i be the neuron index. Let be the membrane potential at time step t-1. The input pulse signal received by neuron i at the current time step t. The membrane potential is the firing threshold; when the membrane potential exceeds the firing threshold, the neuron fires a pulse. This refers to the pulse output at time step t-1. It can be understood that if the updated membrane potential reaches the discharge threshold, then the pulse output at the current time step... Set to 1 and trigger a delay reset; otherwise, output the pulse for the current time step. It is 0, that is: ; in, Let be the pulse output of neuron i at time step t, where i is the neuron index and t is the time step index. Let be the membrane potential of neuron i at time step t. This is the discharge threshold.

[0025] After completing the forward propagation for all time steps, the output result for each time step is obtained. To comprehensively reflect the response characteristics of the spiking neural network throughout the entire time window, the pulse outputs at each time step are accumulated or averaged to obtain the average firing rate of each output neuron, and this vector is used as the final classification confidence of the spiking neural network model. The category corresponding to the maximum classification confidence is obtained as the first sample classification result.

[0026] ; Where f(x) is the classification result of the spiking neural network for the first sample x, x is the input sample, T is the total number of time steps, and t is the index of the current time step. This is the pulse output at the t-th time step.

[0027] The first sample classification result is the predicted category of the current spiking neural network model. It should be noted that the category of the first sample classification result is derived from the combination of categories defined by the label of the first sample in the original dataset. For example, the first sample classification result could be one of the following: drone, vehicle, building, or person.

[0028] Specifically, in the current training iteration, multiple first samples from the original dataset are sequentially input into the current spiking neural network model. The current spiking neural network extracts features from the first samples and outputs the corresponding classification result for each first sample. This allows for real-time evaluation of the classification accuracy of the current spiking neural network model for the first samples under the current model parameters, providing a reliable basis for subsequent selection of correctly classified samples.

[0029] S120. Select multiple first samples whose classification results are consistent with the first sample labels as third samples, and use the first sample classification results as the corresponding second sample labels.

[0030] In this embodiment, the first sample label is the true category annotation of the first sample, typically provided manually or by a reliable annotation system. The first sample label can come from the set of target categories involved in the images acquired by the UAV. The third sample is a set of first samples after filtering. This sample can be a sample correctly classified by the current spiking neural network model. The strategy for determining the third sample can be to use the first sample whose classification result matches the first sample label as the third sample.

[0031] The second sample label is the reassigned label to the selected third sample. Since the third sample is the one whose classification result matches the first sample label, the second sample label corresponding to the third sample is the same as the first sample label, and is used for subsequent training of the spiking neural network model.

[0032] For example, if 10 first samples are input into the current spiking neural network model, 10 corresponding first sample classification results are output. These 10 first sample classification results are compared with their corresponding first sample labels. The 7 first samples whose labels match their classification results are considered third samples; that is, there are 7 third samples. For example, if the first sample label is "vehicle" and the first sample classification result is also "vehicle," then the first sample label matches its classification result.

[0033] Specifically, multiple first samples are input into the current spiking neural network model to obtain the first sample classification result for each first sample. The first sample whose classification result matches the first sample label is selected as the third sample, and its first sample label is used as the second sample label for the corresponding third sample. Samples that the current spiking neural network model can correctly classify are then selected for subsequent processing, improving adversarial training efficiency and enhancing robust generalization ability.

[0034] S130. Based on the second sample label, perturbation step size, and perturbation boundary value, the third sample is perturbed to determine the second sample.

[0035] The perturbation boundary value is determined based on the category-level shared boundary value calculated in the previous training iteration, the sample-level residual value calculated in the previous training iteration, and the residual adjustment coefficient. The training iteration is a pre-set model training iteration. If the current iteration is the initial training iteration, the sample-level residual value is a preset value.

[0036] In this embodiment, the second sample label serves as a supervisory signal to guide the generation of subsequent adversarial examples. This sample label also indicates the target category that the third sample should correctly identify.

[0037] The perturbation step size is the amount of perturbation added to the samples in a single iteration. This step size controls the step size of the parameter updates. It should be noted that the perturbation step size can be a preset hyperparameter or can be adaptively adjusted during the training of the spiking neural network model.

[0038] The perturbation boundary value is the upper limit of the allowed total perturbation for a single sample. This boundary value constrains the difference norm between the second sample and the original sample. It should be noted that the perturbation boundary value is determined by the class-level shared boundary value calculated in the previous training iteration, the sample-level residual value calculated in the previous training iteration, and the residual adjustment coefficient.

[0039] The second sample is a new sample obtained after one perturbation process. This sample is an adversarial sample generated in the current training iteration.

[0040] The category-level shared boundary value is a perturbation boundary benchmark shared by all third samples within the same category. This boundary value provides a perturbation constraint on the average level of the corresponding category, reflecting the overall robustness of the corresponding category.

[0041] The sample-level residual is the deviation of an individual sample from the class average robustness. This parameter adapts the perturbation boundary to the robustness of a single sample, compensating for the inadequacy of the class boundary.

[0042] The residual adjustment parameter is a hyperparameter that controls the contribution of sample-level residuals to the perturbation boundary value. This parameter is a pre-set fixed value used to balance class-shared information with individual bias.

[0043] The training iteration count is the total number of training iterations for the spiking neural network model. This number of iterations is a pre-set hyperparameter. It should be noted that each training iteration can traverse the original dataset once. Training terminates after the preset number of training iterations has been completed.

[0044] The initial training iteration is the first iteration after training begins. It should be noted that in the initial training iteration, the sample residuals do not have the calculation results from the previous training iteration; therefore, the sample residuals can be set to a preset value. For example, if the preset value is 0, then the sample residuals in the initial training iteration will be 0.

[0045] Specifically, in each training iteration, for the selected third sample, based on the corresponding second sample label, the preset perturbation step size, and the perturbation boundary value, the third sample is perturbed using a multi-iteration method to generate adversarial examples, which are denoted as the second sample of the current training iteration. ; in, The second sample is obtained after k+1 training iterations from the i-th third sample. This is the second sample obtained after k training iterations from the i-th third sample, where i is the sample index and k is the current training iteration. For projection operators, a vector is projected onto a plane. Centered on, with radius as norm sphere Above, ensure that the disturbance does not exceed the boundary; As the third sample, This represents the effective perturbation boundary for the third sample i. The perturbation step size is the distance moved along the gradient direction in each training iteration. This parameter can be preset. The sign function is used to sign each component of the gradient vector; The loss function measures the difference between the model's predictions and the true sample labels. This is the predicted output of the spiking neural network model for the current second sample; For the second sample label, This is the gradient of the loss function with respect to the current adversarial example.

[0046] It should be noted that, The region of perturbation constraints is the region containing all conditions that satisfy the condition. The set consisting of points. Among them, Let x be the generated second sample, and x be the corresponding third sample. For the perturbation boundary value of sample class c, It is a p-norm.

[0047] The perturbation boundary value used to generate the second sample is determined by the category-level shared boundary value, the sample-level residual value, and the residual adjustment coefficient calculated in the previous training iteration: ; in, Let i be the perturbation boundary value of the third sample i, where i is the sample index. For category-level shared boundary values, Let k be the label of the second sample corresponding to the third sample i, and k be the residual adjustment coefficient. This represents the sample-level residual value corresponding to the third sample i.

[0048] Based on the above formula, the perturbation boundary actually used by each third sample i when generating the second sample is calculated. That is, based on the category-level shared boundary value, plus the sample-level residual value adjusted by the residual adjustment coefficient, the perturbation boundary not only retains the overall robustness of the category, but also compensates for individual differences through the sample-level residual. At the same time, it realizes the dynamic evolution of the boundary by using the historical information of the previous iteration round, thereby guiding the efficient generation of adversarial examples.

[0049] To fully illustrate the process of determining perturbation boundary values, the methods for determining category-level shared boundary values ​​and sample-level residual values ​​are further elaborated. Optionally, category-level shared boundary values ​​and sample-level residual values ​​are determined based on the following methods: Based on the sample category corresponding to the second sample label, obtain all third samples for each sample category; for each sample category, determine the initial perturbation scale at the sample level based on all third samples of that sample category to output the category-level shared boundary value of that category; determine the sample-level residual value based on the average perturbation distance of the third sample, the second sample corresponding to the third sample, and the sample category corresponding to the second sample label.

[0050] In this embodiment, the sample category is the category indicated by the second sample label. This category is the semantic category to which the sample corresponding to the second sample label belongs. The initial perturbation scale at the sample level is the initial perturbation intensity that can be tolerated for a single third sample. This parameter is calculated from the features of the third sample itself.

[0051] The mean perturbation distance is the statistical average of the perturbation distances between all third samples and their corresponding second samples within the same sample category. This statistical average is used to calculate the sample-level residuals.

[0052] Specifically, after determining all third samples and their corresponding second sample labels, the category-level shared boundary values ​​and sample-level residual values ​​are calculated.

[0053] First, based on the sample category indicated by the second sample label, all third samples are grouped by sample category to obtain the third sample set corresponding to each sample category. Further, for each sample category, all third samples corresponding to that category are obtained. The sample-level initial perturbation scale for each third sample is calculated based on its normalized norm. The expected value of the sample-level initial perturbation scale for each third sample is then calculated to obtain the category-level shared boundary value. ; in, Let E be the category-level shared boundary value for sample category c, where c is the category index and E is the expectation, which is the arithmetic mean of the initial perturbation scales at the sample level for all third samples. Let x be the set of samples that follow the sample class c. The third sample set is of sample class c. The initial perturbation scale at the sample level corresponds to the third sample x.

[0054] It should be noted that the category-level shared boundary value of sample category c calculated based on the above formula is the category-level shared boundary value of sample category c at the beginning of training, i.e., the initial training iteration round. In subsequent training iteration rounds, the category-level shared boundary value will be updated adaptively.

[0055] Furthermore, the process of obtaining the sample-level residual values ​​is described. In the initial training iterations, the sample-level residual values ​​are preset values. In the second training iteration, the sample-level residual values ​​are determined as follows: ; in, Let i be the sample residual value of the third sample i, where i is the sample index. To measure the norm of vector distance, For the third sample with sample index i, For the second sample with sample index i, For sample categories The average disturbance distance, The sample category is the second sample label corresponding to sample index i.

[0056] The above formula can be understood as calculating the actual perturbation distance of each third sample and its corresponding second sample, and obtaining the average perturbation distance of the sample class to which the third sample belongs. Subtracting the average perturbation distance of its sample class from the actual perturbation distance determines the sample-level residual value corresponding to the third sample.

[0057] The actual perturbation distance is calculated and determined by each third sample and its corresponding second sample. Optionally, for all third samples, the actual perturbation distance corresponding to the third sample is determined based on the following method: Based on the third sample and the second sample corresponding to the third sample, the actual disturbance distance corresponding to the third sample is determined.

[0058] In this embodiment, the actual perturbation distance is the degree of difference between a single third sample and its corresponding second sample. This perturbation distance reflects the intensity of the perturbation that the spiking neural network model actually adds to the third sample.

[0059] Specifically, for each third sample, its corresponding second sample is obtained, and the difference between the two is calculated using a preset p-norm. The calculated result is the actual perturbation distance corresponding to the third sample. The advantage of calculating the actual perturbation distance is that by aggregating the actual perturbation distances of all third samples within the same sample category, the average robustness level and dispersion of that sample category can be further statistically analyzed, thereby supporting adaptive updates of category-level shared boundary values.

[0060] It should be noted that the sample-level residual value calculated based on the above formula is determined during the second training iteration. In subsequent training iterations, the sample-level residual value can be adaptively updated based on the sample-level residual value determined in the second training iteration.

[0061] The average perturbation distance used to calculate the sample-level residuals needs to be determined based on the data calculated in the previous training iteration. The method for determining the average perturbation distance is described below. Optionally, the average perturbation distance is determined based on the following method: For each sample category, the average perturbation distance corresponding to the sample category is determined based on the third sample and the second sample corresponding to the third sample.

[0062] Specifically, multiple sample categories are determined based on the second sample labels, and the third sample set and the corresponding second sample set are obtained for each sample category. For each sample category, all third samples of that category and the corresponding second sample for each third sample are determined. Further, for each third sample, the actual perturbation distance between it and the corresponding second sample is calculated using a specified norm, and the arithmetic mean of the actual perturbation distances of all third samples under that sample category is calculated. The result is the average perturbation distance for that sample category. ; in, Let c be the average perturbation distance of sample class c, where c is the sample class index. Let c be the sample set, which includes the third sample set and the second sample set. As the third sample, The second sample is the counterpart to the third sample x. This is used to measure the vector distance norm.

[0063] The advantage of calculating the average perturbation distance is that it intuitively reflects the average robustness of the category, providing an objective benchmark for determining the subsequent perturbation boundary values.

[0064] It should be noted that, to dynamically update the optimal perturbation boundary value, in each training iteration, the average perturbation distance for that sample class can be updated based on the classification success rate of that sample class under the second sample condition. Optionally, the average perturbation distance is determined based on the following method: The average perturbation distance is determined based on the average perturbation distance of the previous training iteration, the preset first adjustment coefficient, the preset second adjustment coefficient, the average decay factor corresponding to the sample category, and the classification success rate corresponding to the sample category; where the classification success rate is the proportion of the second sample of the sample category that is correctly classified.

[0065] In this embodiment, the average perturbation distance of the previous training iteration is the average perturbation distance of the sample class after the completion of the previous training iteration. A preset first adjustment coefficient is a hyperparameter that influences the average perturbation distance. This preset first adjustment coefficient is used to determine the magnitude of the increase in average perturbation distance when the classification success rate is high. A preset second adjustment coefficient is a hyperparameter that controls the influence of average perturbation distance and is used to determine the magnitude of the decrease in average perturbation distance. It should be noted that the preset first and preset second adjustment coefficients can be preset.

[0066] The average decay factor for a sample category is the arithmetic mean of the decay factors for all samples within that category. This arithmetic mean is used to adjust the update amplitude so that the change in the average perturbation distance matches the inherent characteristics of the samples.

[0067] The classification success rate is the proportion of samples correctly classified after all second samples are input into the spiking neural network model for each sample category. Optionally, a higher classification success rate indicates that the current perturbation boundary value is conservative and lacks perturbation; a lower classification success rate indicates that the perturbation is too large.

[0068] Specifically, in each training iteration, the average perturbation distance of the current training iteration is determined based on the average perturbation distance of the previous training iteration, the preset first adjustment coefficient, the preset second adjustment coefficient, the average decay factor of all second samples in that sample class, and the classification success rate of that sample class. ; in, This represents the average perturbation distance corresponding to the updated sample class c. The updated average perturbation distance is used in the (t+1)th training iteration. Let be the average perturbation distance corresponding to sample class c in the t-th training iteration, where t is the training iteration. The first adjustment coefficient is preset. To improve classification success rate, This is the average decay factor corresponding to sample category c, which is the arithmetic mean of the decay factors of individual samples. This is the preset second adjustment coefficient.

[0069] The average perturbation distance is updated based on the above formula so that the average perturbation distance dynamically follows the robustness changes of the spiking neural network model, providing an accurate benchmark for subsequent class-level shared boundary adjustments.

[0070] Furthermore, to enable the next training iteration to generate more suitable adversarial examples (second samples) with more accurate perturbation constraints, thereby improving the robustness of the spiking neural network model, the perturbation boundary values ​​can be updated and adjusted. This update and adjustment includes at least the category-level shared boundary values ​​and the sample-level residual values. The adjustment process of the perturbation boundary values ​​is described in detail below. Optionally, based on the second sample label, perturbation step size, and perturbation boundary values, the third sample is perturbed. After determining the second sample, the method further includes: The second sample is input into the current spiking neural network model, causing the current spiking neural network model to output the classification result of the second sample. For each sample category, the category-level shared boundary value is adjusted based on the classification results of all second samples in that category and the labels of all second samples, thus determining the adjusted category-level shared boundary value. The adjusted sample-level residual value is determined based on the predefined residual smoothing coefficient, the sample-level residual value of the previous training iteration, the average perturbation distance corresponding to the sample category, and the actual perturbation distance. The actual perturbation distance is the perturbation norm between the third sample and the second sample corresponding to the third sample. The adjusted perturbation boundary value is determined based on the adjusted category-level shared boundary value, the adjusted sample-level residual value, and the residual adjustment coefficient. The adjusted category-level shared boundary value is used as the category-level shared boundary value for the next training iteration, the adjusted sample-level residual value is used as the sample-level residual value for the next training iteration, and the adjusted perturbation boundary value is used to generate the second sample for the next training iteration.

[0071] In this embodiment, the classification result of the second sample is the predicted sample category output by the current spiking neural network model after the second sample is input into it. This sample category is calculated by the forward propagation of the current spiking neural network. The adjusted category-level shared boundary value is a perturbation boundary benchmark shared by all samples of a certain sample category. This boundary benchmark is the category-level shared boundary value for the next training iteration, used to calculate the new perturbation boundary value.

[0072] The adjusted sample-level residual value describes the deviation between a single sample and the average robustness of the sample class. This deviation can be used as the sample-level residual value for the next training iteration to calculate new perturbation boundary values.

[0073] The actual perturbation distance is the perturbation norm between the third sample and the second sample corresponding to the third sample. The perturbation norm is the norm that measures the magnitude of the perturbation vector.

[0074] The residual smoothing coefficient is a hyperparameter that controls the weighting of historical and current observations when updating sample-level residual values. This hyperparameter ranges from 0 to 1. It is used to smooth out random fluctuations in a single batch, making the residual evolution smoother.

[0075] The residual adjustment coefficient is a hyperparameter that controls the contribution of sample-level residuals to the effective perturbation boundary. This hyperparameter is used to balance class-shared information with individual bias, preventing sample-level residuals from excessively influencing the boundary.

[0076] The adjusted perturbation boundary value is the effective perturbation boundary for generating the second sample in the next training iteration. This perturbation boundary is obtained by combining the adjusted class-level shared boundary and the adjusted sample-level residual. After adjusting the perturbation boundary value, it can serve as a constraint when processing perturbations in the next training iteration, ensuring that the perturbation of the generated second sample does not exceed the range corresponding to the perturbation boundary value. The next training iteration can be understood as the next round of training after the boundary adjustment is completed in the current training iteration.

[0077] Specifically, after generating the second sample in the current training iteration, the second sample is input into the current spiking neural network model. The current spiking neural network analyzes and processes the second sample and outputs the classification result. After obtaining the classification result for each second sample, the perturbation boundary value is updated and adjusted. Since the perturbation boundary value is obtained by combining the category-level shared boundary value and the sample-level residual value, the category-level shared boundary value and the sample-level residual value are adjusted first. That is, the classification result and corresponding label of all second samples in each sample category are obtained, and the category-level shared boundary value is adjusted based on the above data. Further, the current sample-level residual value is adjusted based on the residual smoothing coefficient, the sample-level residual value of the previous training iteration, the average perturbation distance corresponding to the sample category, and the actual perturbation distance. Finally, based on the adjusted category-level shared boundary value, the adjusted sample-level residual value, and the residual adjustment coefficient, the perturbation boundary value for the next training iteration is determined. This ensures that the perturbation boundary is robustly updated in real time based on the spiking neural network model, avoiding underfitting or overfitting caused by static boundaries.

[0078] For each sample category, the category-level shared boundary value can be adjusted based on the classification results and corresponding labels of all second samples in that sample category.

[0079] It should be noted that when comparing the classification result of each second sample under the same sample category with the corresponding second sample label, different comparison results will lead to different adjustments to the category-level shared boundary.

[0080] When the classification results of all second samples within a given sample category are consistent with their corresponding labels, the category-level shared boundary value is adjusted as follows. Optionally, the category-level shared boundary value is adjusted based on the classification results of all second samples within the given sample category and their corresponding labels. The adjusted category-level shared boundary value includes: If the classification result of all second samples is consistent with the label of the second sample, the category-level shared boundary value of the next training iteration is determined based on the preset decay factor, preset adjustment factor, preset global update step size coefficient, intra-category perturbation uncertainty value, sample-level initial perturbation scale, and category-level perturbation boundary value of the current training iteration. The intra-category perturbation uncertainty value is used to characterize the discreteness of the robustness distribution of category samples.

[0081] In this embodiment, the consistency between the second sample classification result and the second sample label can be understood as follows: for all second samples of a certain category, the predicted sample category output by the model is equal to its corresponding sample label. For example, if the second sample classification result is "vehicle" and the corresponding sample category in the second sample label is "vehicle", it indicates that the second sample classification result of this second sample is consistent with the corresponding second sample label.

[0082] The preset decay factor is a hyperparameter used to reduce the expansion amplitude. This hyperparameter is dynamically calculated in advance based on the normalization norm of the second sample. The advantage of setting a preset decay factor is that it prevents excessive boundary expansion from causing perturbation distortion, thus ensuring that the expansion step size is adapted to the sample itself.

[0083] The preset adjustment factor is a scaling factor used to control the expansion step size. This factor reflects the difficulty of generating the current second sample and is related to the strength of the loss gradient.

[0084] The preset global update step size coefficient is the learning rate that controls the maximum magnitude of a single expansion of the category-level shared boundary values. This parameter is a pre-set hyperparameter used to uniformly adjust the degree of category-level shared boundary value updates.

[0085] The intra-category perturbation uncertainty is a parameter reflecting the dispersion of robustness among category samples. This parameter can be calculated based on the average perturbation distance corresponding to the samples. It can be understood that a larger intra-category perturbation uncertainty indicates more significant differences in robustness among samples, weakening the incremental expansion of category-level shared boundary values; a smaller intra-category perturbation uncertainty indicates more suppressed robustness among samples, enhancing the incremental expansion of category-level shared boundary values.

[0086] The initial perturbation scale at the sample level is the perturbation intensity calculated based on the third sample. The class perturbation boundary value for the current training iteration is the class-shared boundary value used in this training iteration. This boundary value is obtained after adjustment from the previous training iteration.

[0087] Specifically, obtain all second samples for all sample categories. For each sample category, obtain all second samples in that category, their corresponding classification results, and their corresponding labels. When the classification results and labels of all samples in that category are consistent, the category-level perturbation boundary value is determined based on the current training iteration's category-level perturbation boundary value, combined with a preset global update step size coefficient, intra-category perturbation uncertainty value, sample-level initial perturbation scale, preset decay factor, and preset adjustment factor. The specific adjustment method for the category-level perturbation boundary value in the next training iteration is as follows: ; in, Shared boundary values ​​at the class level for the next training iteration. Here, t represents the category-level shared boundary value for the current training iteration. To preset the global update step size coefficient, This represents the in-class perturbation uncertainty value for the current training iteration. The initial perturbation scale is at the sample level. As a preset attenuation factor, This is a preset adjustment factor.

[0088] The above formula determines the category-level shared boundary value for the next training iteration when all second samples of the sample category are still correctly classified. This allows for adaptive updating of the category-level shared boundary value, reducing reliance on manual parameter tuning.

[0089] In adjusting the category-level shared boundary values, a category perturbation uncertainty value is introduced to represent the robustness distribution of samples corresponding to that category. If the robustness distribution of the sample category is relatively dispersed, the larger the category perturbation uncertainty value, the more it weakens the incremental expansion of the category perturbation boundary value, ensuring that the boundary update matches the intrinsic statistical characteristics of the category. Next, the method for determining the intra-category perturbation uncertainty value is described in detail. Optionally, the intra-category perturbation uncertainty value is determined based on the following method: Based on the average perturbation distance, the actual perturbation distance, and the number of samples in each sample category, the variance of the perturbation distance corresponding to each sample category is determined; based on the variance of the perturbation distance, the perturbation uncertainty value within each category is determined.

[0090] In this embodiment, the number of samples in a sample category is the total number of third samples in that sample category. This can be understood as obtaining all sample categories in the second sample label, determining the set of third samples contained in each sample category, and then obtaining the total number of third samples in each sample category as the number of samples corresponding to that sample category.

[0091] The variance of perturbation distance is a second-order statistic that measures the dispersion of the actual perturbation distance between all third samples and their corresponding second samples in a sample class relative to the mean perturbation distance. This second-order statistic is used to quantify the sample robustness differences within that sample class.

[0092] Specifically, for each sample category, the actual perturbation distances of all third samples within that category are obtained, and their arithmetic mean is calculated to obtain the average perturbation distance for that sample category. Further, based on the squared deviation between the actual perturbation distance and the average perturbation distance of each third sample, this deviation is summed over all third samples and divided by the number of samples in that category to obtain the variance of the perturbation distance. This variance quantifies the robustness dispersion within the sample category, guides adaptive boundary updates, and enhances the efficiency and stability of adversarial training. ; in, Let be the perturbation distance variance for sample class c, where c is the class index. Let x be the set of all third samples corresponding to sample category c, and let x be a third sample in the set of third samples. To measure the norm of vector distance, The second sample corresponding to the third sample x. The average perturbation distance for sample class c. The number of third samples in sample category c.

[0093] Furthermore, after obtaining the perturbation distance variance of sample class c... Subsequently, intra-category disturbance uncertainty This is obtained by taking the square root of the variance, which effectively suppresses fluctuations caused by single-batch sampling.

[0094] It should be noted that, considering the distinct phases of adversarial training, to avoid the impact of random fluctuations in intra-class perturbation uncertainty on the overall training process, an exponential smoothing update method can be used for the intra-class perturbation uncertainty value: ; in, Let be the perturbation uncertainty value within the category after the update in the t-th training iteration. Let c be the in-class perturbation uncertainty value after the (t-1)th training iteration, where c is the sample class index and t is the current training iteration. A preset smoothing coefficient is used to determine the weight of the intra-class perturbation uncertainty value in the previous training iteration. This represents the in-class perturbation uncertainty value from the previous training iteration. The original intra-class perturbation uncertainty value is calculated in the t-th training iteration.

[0095] The above formula integrates the intra-class perturbation uncertainty value from the previous training iteration with the original intra-class perturbation uncertainty value directly calculated in the current training iteration by applying a preset smoothing coefficient, thus obtaining the updated intra-class perturbation uncertainty value for the current training iteration.

[0096] This can be understood as follows: In the current training iteration, the variance of the perturbation distance corresponding to each sample class is first determined based on the average perturbation distance, the actual perturbation distance, and the number of samples in each class. Then, based on this variance, the intra-class perturbation uncertainty value is determined as the original intra-class perturbation uncertainty value for the current training iteration. Further, based on the intra-class perturbation uncertainty value from the previous training iteration and the original intra-class perturbation uncertainty value for the current iteration, the updated intra-class perturbation uncertainty value for the current iteration is determined. The advantage of introducing exponential smoothing updates is that it effectively suppresses abrupt changes in uncertainty caused by individual abnormal samples or phased gradient changes, thereby ensuring the continuity and stability of the perturbation boundary updates.

[0097] When the classification results of all second samples within a given sample category are inconsistent with their corresponding labels, the category-level shared boundary value is adjusted as follows. Optionally, the category-level shared boundary value is adjusted based on the classification results of all second samples within the given sample category and their corresponding labels, determining the adjusted category-level shared boundary value, including: If the classification result of the second sample corresponding to the second sample is inconsistent with the label of the second sample, determine the target second sample, the average perturbation distance, the intra-class perturbation uncertainty value, the preset balance coefficient, and the class-level shared boundary value of the current training iteration round, and determine the adjusted class-level shared boundary value. The target second sample is the second sample whose classification result is inconsistent with the label of the second sample.

[0098] In this embodiment, the inconsistency between the classification result of the second sample and the label of the second sample can be understood as the fact that, for the second sample, the predicted classification result output by the spiking neural network model is not equal to its corresponding sample label.

[0099] The target second sample is any second sample in the sample category whose classification result is inconsistent with the second sample label.

[0100] The preset balance coefficient is a hyperparameter that controls the weighting between the average perturbation distance and the class-level shared boundary value from the previous training iteration during contraction. This parameter determines whether the updated class-level shared boundary value is biased towards the current average perturbation distance or retains the class-level shared boundary value from the previous training iteration.

[0101] The adjusted category-level shared boundary value is the new category-level shared boundary value obtained after shrinkage update. This boundary value is used in the next training iteration.

[0102] Specifically, when the classification result of at least one second sample in a sample category is inconsistent with the corresponding second sample label, the category-shared boundary value is adjusted as follows: First, the second sample in that sample category whose classification result is inconsistent with the corresponding second sample label is identified as the target second sample. Further, the average perturbation distance and intra-category perturbation uncertainty value of that sample category are obtained. Then, the category-level shared boundary value and the average perturbation distance of the current training iteration are weighted and fused using a preset balance coefficient to obtain the adjusted category-level shared boundary value: ; in, The adjusted category-level shared boundary values ​​are used in the (t+1)th training iteration. Let c be the class-level shared boundary value for the current training iteration (i.e., the t-th training iteration), where c is the sample class index and t is the current training iteration. To preset the balance coefficient, The average perturbation distance for sample class c. This represents the in-class perturbation uncertainty value for the current training iteration.

[0103] Based on the above formula, the category-level shared boundary value is updated. The adjusted category-level shared boundary value is determined by weighted averaging of the category-level shared boundary value of the previous training iteration and the average perturbation distance after suppression of intra-category perturbation uncertainty value. This prevents the boundary from dropping sharply due to abnormal samples in a single training iteration, thereby maintaining the stability of the training process.

[0104] After explaining the update process of the perturbation boundary values, the training process of the spiking neural network model is further described in detail. Optionally, the model parameters in the spiking neural network model are determined based on the following method: Obtain multiple third samples and their corresponding multiple second samples; input the multiple third samples and multiple second samples into a spiking neural network model to obtain the third sample classification result of the third sample and the second sample classification result of the second sample; adjust the model parameters in the spiking neural network model according to the third sample classification result, the second sample classification result, the second sample label and the preset loss function; when the loss function converges, the obtained spiking neural network model is used as the trained spiking neural network model.

[0105] In this embodiment, the third sample is a clean sample from the original dataset that has not undergone any perturbation. This sample can be based on the original images collected by a drone.

[0106] The third sample classification result is the predicted category output by the current spiking neural network model after the third sample is input. This category is obtained through forward propagation of the current spiking neural network model. The second sample label is the true category of the third sample pre-labeled in the original dataset.

[0107] The preset loss function is a function used to measure the difference between the classification results predicted by the spiking neural network model and the actual sample labels.

[0108] Model parameters are the learnable parameters in a spiking neural network model. These parameters can include synaptic connection weights between layers, etc. It should be noted that the model parameters can be calculated using the backpropagation algorithm to obtain the gradient values ​​of the loss function with respect to the model parameters, and then the optimizer can be used to update the model parameters, thus obtaining a practically usable spiking neural network model.

[0109] Loss function convergence can be understood as follows: during training, the loss function value no longer decreases significantly with increasing iterations, or the change is less than a preset threshold for multiple consecutive iterations; this is considered a convergence state. A trained spiking neural network model is the set of model parameters saved after the loss function has converged; at this point, the model is considered a trained spiking neural network model.

[0110] Specifically, multiple third samples and corresponding second samples are obtained for training. These third samples and their corresponding second samples are then input into the current spiking neural network model. Through forward propagation and time-cumulative decoding, the third classification result of the third sample and the second classification result of the second sample are obtained, respectively. Further, a clean sample loss is calculated based on the third classification result and the second sample label, and an adversarial sample loss is calculated based on the second classification result and the second sample label. The weighted sum of these two losses serves as a preset loss function. The gradient of the loss function with respect to the model parameters is calculated using the direction propagation algorithm, and the parameters are iteratively updated using an optimizer. This process is repeated until the loss function converges. The saved model parameters at this point represent the trained spiking neural network model. This approach achieves both accurate identification of clean samples and robustness against adversarial perturbations within a certain intensity, making it suitable for deployment in practical UAV target recognition tasks.

[0111] It should be noted that the second sample can be obtained by perturbing the third sample based on dynamically updated perturbation boundary values. See [link / reference] Figure 2This paper provides an overall description of the dynamic update process of the perturbation boundary value. First, in the current training iteration, the current perturbation boundary value is determined based on the existing class-level shared boundary value and sample-level residual value. Further, using this perturbation boundary value as a constraint, a multi-step iterative gradient perturbation method is employed to perturb the third sample, generating the second sample.

[0112] The generated second sample is input into a spiking neural network model, undergoes forward propagation at multiple discrete time steps, and the outputs at each time step are accumulated or averaged to obtain the classification result of the second sample. The classification result of the second sample is compared with the corresponding second sample label, and the classification success rate of all second samples under that sample category is calculated. At the same time, the actual perturbation distance between each second sample and its corresponding third sample is calculated.

[0113] For each sample category, the sample information of all samples in that category is summarized. This sample information includes at least the classification success rate, average perturbation distance (average perturbation magnitude), and perturbation variance (perturbation distance variance). Based on this sample information, the category-level shared boundary values ​​are adaptively expanded or shrunk. Simultaneously, the sample-level residual values ​​are updated using exponential smoothing, utilizing the actual perturbation distance, the average perturbation distance, and the sample-level residual values ​​from the previous training iteration.

[0114] Finally, the updated category-level shared boundary values ​​are recombined with the sample-level residual values ​​to obtain the perturbation boundary values ​​for the next training iteration.

[0115] The technical solution provided by this invention involves inputting multiple first samples into a current spiking neural network model, causing the current spiking neural network model to output the classification result of the first samples; selecting multiple first samples whose classification results are consistent with the first sample labels as third samples, and using the first sample classification results as the corresponding second sample labels; and performing perturbation processing on the third samples based on the second sample labels, perturbation step size, and perturbation boundary values ​​to determine the second samples, thereby accelerating the adaptation process of the spiking neural network model to adversarial perturbations and improving the efficiency of adversarial training.

[0116] Figure 3 This is a flowchart illustrating a UAV target recognition method based on pulse category perception boundary evolution, provided by an embodiment of the present invention. This embodiment is applicable to UAV target recognition. The method can be executed by a UAV target recognition device based on pulse category perception boundary evolution, which can be implemented in hardware and / or software. This UAV target recognition device based on pulse category perception boundary evolution can be configured in a computing device. Detailed implementation can be found in the description of this embodiment. Technical features that are the same as or similar to those in the foregoing embodiments will not be repeated here.

[0117] like Figure 3 As shown, the method includes: S210. Acquire the image to be detected; wherein the image to be detected is an image containing the target to be detected, collected by a UAV.

[0118] In this embodiment, the image to be detected is the original input image for target recognition. This image is the data object processed by the trained spiking neural network during the inference phase. The image to be detected can be an image acquired in real-time by an imaging sensor mounted on the UAV during flight, or a historical image read from a storage device.

[0119] The image to be detected may contain at least one target to be detected. The target to be detected can be a specific object in the image that needs to be identified. For example, the target to be detected can be an object observed by the drone during flight, and the object includes at least other drones, birds, ground vehicles, people, and buildings.

[0120] Specifically, a raw image containing the target to be detected is acquired in real time and / or loaded offline using a drone. This image can then be input into a spiking neural network model to achieve end-to-end target category recognition, enabling automatic end-to-end identification and improving recognition efficiency.

[0121] S220. The image to be detected is input into a pre-trained spiking neural network model for analysis and processing to output the UAV target category of the target to be detected. The spiking neural network model is trained based on perturbation boundary values ​​created and updated based on training iterations. The perturbation boundary values ​​include category-level shared boundary values ​​and sample-level residual values. The category-level shared boundary values ​​are used to provide a baseline perturbation constraint for each sample category, and the sample-level residual values ​​are used to perform sample-level correction on the baseline perturbation constraint. The perturbation boundary values ​​are used to constrain the upper limit of the perturbation amplitude when multiple adversarial samples are generated. The category-level shared boundary values ​​and sample-level residual values ​​are adaptively adjusted based on the classification results of multiple adversarial samples. The multiple adversarial samples are multiple second samples, which are generated after perturbation processing of the first sample. The first sample is the original sample.

[0122] In this embodiment, the pre-trained spiking neural network model is an adversarial spiking neural network model trained based on dynamically updated perturbation boundary values. This spiking neural network model can be directly used for UAV target recognition tasks.

[0123] It should be noted that the perturbation boundary values ​​can be dynamically updated during the training of the spiking neural network model. These perturbation boundary values ​​constrain the upper limit of the perturbation amplitude when generating multiple adversarial examples. The upper limit of the perturbation amplitude is the maximum allowable amount of modification when generating adversarial examples (secondary examples) to prevent excessive distortion of the adversarial examples.

[0124] The perturbation boundary value can be obtained based on the category-level shared boundary value and the sample-level residual value. The category-level shared boundary value is used to constrain the baseline perturbation for each sample category. The baseline perturbation constraint can be understood as providing a uniform upper limit for the perturbation of the category-level shared boundary value. This upper limit can serve as a reference point when generating adversarial examples (secondary examples) for all samples in that sample category, ensuring the stability of training and category consistency.

[0125] Sample-level residuals are used to perform sample-level corrections to the baseline perturbation constraints. Sample-level correction can be understood as individualizing the baseline perturbation constraints using sample-level residuals. The adjusted perturbation constraints can accommodate the robustness differences of each sample.

[0126] After the image to be detected is input into the spiking neural network model, the output is the drone target category of the detected target. The drone target category is the category label output by the spiking neural network model after identifying the target in the input image. It should be noted that the specific definition of this target category may depend on the actual application task. It is not specifically limited here. For example, the target category may include, but is not limited to, drones, ground vehicles, people, buildings, and birds.

[0127] Specifically, the spiking neural network model is obtained through adversarial training based on dynamically updated perturbation boundary values. During the inference phase, images acquired in real-time and / or received offline by the UAV are used as the images to be detected and input into the pre-trained spiking neural network model for processing. After forward propagation and time-cumulative decoding, the spiking neural network model outputs the predicted UAV target category. By introducing adversarial examples based on dynamic perturbation boundaries during training, the spiking neural network model learns to resist interference from input perturbations, enabling it to maintain stable and accurate classification results when faced with noise interference in the actually acquired images to be detected during the inference phase. This improves the model's robustness and enhances the accuracy of UAV target category recognition.

[0128] The technical solution provided by this invention solves the problem of poor stability in drone target recognition in dynamic scenes by acquiring a target image containing the target collected by a drone, and inputting the target image into a pre-trained spiking neural network model for analysis and processing to output the drone target category of the target. This improves the accuracy of drone target category recognition.

[0129] Figure 4 This diagram illustrates the training process of a UAV target recognition method based on pulse category perception boundary evolution, as provided in an embodiment of the present invention. Based on the above embodiment, an optional example is provided, which can be used for UAV target recognition scenarios.

[0130] like Figure 4 As shown, firstly, the first sample (training sample) used to train the spiking neural network model and the corresponding first sample label (label) are input into the preprocessing module. The preprocessing module preprocesses the samples and divides them into batches for subsequent iterative training.

[0131] Furthermore, a spiking neural network model is constructed. The processed first sample is input into the spiking neural network model, and forward propagation is performed over multiple discrete time steps. At each time step, the neuron updates its membrane potential based on the input pulse and triggers pulse firing. After all time steps are completed, the output pulses from each time step are accumulated or averaged to obtain the first classification result.

[0132] Furthermore, the first classification result is compared with the corresponding first sample label, and the first sample whose first classification result matches the first sample label is selected. The first sample is then used as the third sample (clean sample), and the corresponding first sample label is used as the second sample label. This ensures that the subsequent generation of the second sample is only based on the knowledge that the spiking neural network model has already mastered, thereby improving training efficiency.

[0133] Furthermore, the third sample is perturbed based on the perturbation boundary value of the current training iteration to generate the second sample. It should be noted that the perturbation boundary value is obtained by combining the category-level shared boundary value and the sample-level residual value. Based on the different sample categories determined by the second sample label, the third classification result of the third sample under adversarial conditions and the corresponding actual perturbation distance (perturbation amplitude information) are statistically analyzed for each sample category.

[0134] Based on the third classification result, the second classification result, and the perturbation amplitude information, the category-level shared boundary values ​​are adaptively expanded or shrunk. Simultaneously, using the actual perturbation distance and the average perturbation distance, the sample-level residual values ​​are updated exponentially to construct the perturbation boundary values ​​required for the next training iteration.

[0135] Finally, based on multiple third-sample (clean) samples, multiple second-sample samples, and the labels of the second-sample samples, the third-sample loss and adversarial-sample loss are calculated. The loss function is then combined with the backpropagation algorithm to calculate the gradient of the loss with respect to the model parameters of the spiking neural network model, and the model parameters are updated through the optimizer. The above steps are repeated, updating the perturbation boundary values ​​and model parameters in each training iteration. Training stops when the joint loss function converges, and the trained spiking neural network model is output.

[0136] Figure 5 This is a schematic diagram of a UAV target recognition device based on pulse category perception boundary evolution, provided as an embodiment of the present invention. Figure 5As shown, the UAV target recognition device based on pulse category perception boundary evolution includes: a target image acquisition module 310 and a target category output module 320.

[0137] The system includes a target image acquisition module 310, which acquires a target image; the target image is an image containing the target object collected by a UAV; a target category output module 320, which inputs the target image into a pre-trained spiking neural network model for analysis and processing, and outputs the UAV target category of the target object; the spiking neural network model is trained based on perturbation boundary values ​​created and updated based on training iterations, the perturbation boundary values ​​include category-level shared boundary values ​​and sample-level residual values, the category-level shared boundary values ​​are used to provide a baseline perturbation constraint for each sample category, and the sample-level residual values ​​are used to perform sample-level correction on the baseline perturbation constraint; the perturbation boundary values ​​are used to constrain the upper limit of the perturbation amplitude when multiple adversarial samples are generated, and the category-level shared boundary values ​​and the sample-level residual values ​​are adaptively adjusted based on the classification results of the multiple adversarial samples, the multiple adversarial samples are multiple second samples, the second samples are generated after perturbation processing of the first samples, and the first samples are the original samples.

[0138] This invention addresses the problem of poor stability in drone target recognition in dynamic scenarios by acquiring a target image from a drone and inputting it into a pre-trained spiking neural network model for analysis and processing to output the drone target category of the target. This improves the accuracy of drone target category recognition.

[0139] Based on the above embodiments, for the current training iteration, the second sample is generated in the following manner: The first classification result output module is used to input the plurality of first samples into the current spiking neural network model so that the current spiking neural network model outputs the classification result of the first sample. The sample label determination module is used to select multiple first samples whose first sample classification results are consistent with the first sample label as third samples, and use the first sample classification results as the corresponding second sample labels. The second sample determination module is used to perturb the third sample based on the second sample label, perturbation step size, and perturbation boundary value to determine the second sample; wherein, the perturbation boundary value is determined based on the category-level shared boundary value calculated in the previous training iteration, the sample-level residual value calculated in the previous training iteration, and the residual adjustment coefficient, and the training iteration is a pre-set model training iteration, and if the current iteration is the initial training iteration, the sample-level residual value is a preset value.

[0140] Based on the above embodiments, the category-level shared boundary value and the sample-level residual value are determined in the following manner: The category-level shared boundary value output module is used to obtain all third samples of each sample category based on the sample category corresponding to the second sample label; for each sample category, the initial perturbation scale at the sample level is determined based on all third samples of that sample category, so as to output the category-level shared boundary value of that category; The sample-level residual value determination module is used to determine the sample-level residual value based on the average perturbation distance of the third sample, the second sample corresponding to the third sample, and the sample category corresponding to the label of the second sample.

[0141] Based on the above embodiments, after perturbing the third sample according to the second sample label, perturbation step size, and perturbation boundary value, the method further includes: The second classification result output module is used to input the second sample into the current spiking neural network model so that the current spiking neural network model outputs the classification result of the second sample. The category-level shared boundary value adjustment module is used to adjust the category-level shared boundary value for each sample category based on the classification results of the second samples corresponding to all second samples in that sample category and the second sample labels corresponding to all second samples, and to determine the adjusted category-level shared boundary value. The sample-level residual adjustment module is used to determine the adjusted sample-level residual value based on the predefined residual smoothing coefficient, the sample-level residual value of the previous training iteration, the average perturbation distance corresponding to the sample category, and the actual perturbation distance; wherein, the actual perturbation distance is the perturbation norm between the third sample and the second sample corresponding to the third sample; The perturbation boundary value adjustment module is used to determine the adjusted perturbation boundary value based on the adjusted category-level shared boundary value, the adjusted sample-level residual value, and the residual adjustment coefficient; wherein, the adjusted category-level shared boundary value is used as the category-level shared boundary value for the next training iteration, the adjusted sample-level residual value is used as the sample-level residual value for the next training iteration, and the adjusted perturbation boundary value is used to generate the second sample for the next training iteration.

[0142] Based on the above embodiments, the category-level shared boundary value adjustment module includes: The category-level shared boundary value determination unit is used to determine the category-level shared boundary value of the next training iteration based on a preset decay factor, a preset adjustment factor, a preset global update step size coefficient, an intra-category perturbation uncertainty value, the sample-level initial perturbation scale, and the category-level perturbation boundary value of the current training iteration when the classification result of the second sample corresponding to all the second samples is consistent with the label of the second sample. The intra-category perturbation uncertainty value is used to characterize the discreteness of the robustness distribution of the category samples.

[0143] Based on the above embodiments, the category-level shared boundary value adjustment module includes: The category-level shared boundary value determination unit is used to determine the target second sample, the average perturbation distance, the intra-category perturbation uncertainty value, the preset balance coefficient, and the category-level shared boundary value of the current training iteration when the classification result of the second sample corresponding to the second sample is inconsistent with the label of the second sample. The adjusted category-level shared boundary value is then determined, wherein the target second sample is the second sample whose classification result is inconsistent with the label of the second sample.

[0144] Based on the above embodiments, the average disturbance distance is determined in the following manner: The average perturbation distance determination module is used to determine the average perturbation distance corresponding to the sample category for each sample category based on the third sample and the second sample corresponding to the third sample.

[0145] Based on the above embodiments, the intra-category perturbation uncertainty value is determined in the following manner: The in-category perturbation uncertainty value determination module is used to determine the perturbation distance variance corresponding to the sample category based on the average perturbation distance, the actual perturbation distance, and the number of samples in the sample category; and to determine the in-category perturbation uncertainty value based on the perturbation distance variance.

[0146] Based on the above embodiments, for all third samples, the actual disturbance distance corresponding to the third sample is determined in the following manner: The actual disturbance distance determination module is used to determine the actual disturbance distance corresponding to the third sample based on the third sample and the second sample corresponding to the third sample.

[0147] Based on the above embodiments, the model parameters in the spiking neural network model are determined in the following manner: A sample acquisition module is used to acquire the plurality of third samples and the corresponding plurality of second samples; The classification result acquisition module is used to input the plurality of third samples and the plurality of second samples into the spiking neural network model to obtain the third sample classification result of the third sample and the second sample classification result of the second sample; The model parameter adjustment module is used to adjust the model parameters in the spiking neural network model based on the classification result of the third sample, the classification result of the second sample, the label of the second sample, and the preset loss function. The model determination module is used to take the spiking neural network model obtained when the loss function converges as the trained spiking neural network model.

[0148] The UAV target recognition device based on pulse category perception boundary evolution provided in the embodiments of the present invention can execute the UAV target recognition method based on pulse category perception boundary evolution provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0149] Figure 6 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0150] like Figure 6As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0151] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0152] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a UAV target recognition method based on pulse category-aware boundary evolution.

[0153] In some embodiments, the UAV target recognition method based on pulse category-aware boundary evolution can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the UAV target recognition method based on pulse category-aware boundary evolution described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the UAV target recognition method based on pulse category-aware boundary evolution by any other suitable means (e.g., by means of firmware).

[0154] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0155] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0156] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0157] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0158] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0159] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0160] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0161] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A UAV target recognition method based on pulse category perception boundary evolution, characterized in that, The method includes: Acquire the image to be detected; wherein the image to be detected is an image containing the target to be detected, acquired by a UAV. The image to be detected is input into a pre-trained spiking neural network model for analysis and processing to output the UAV target category of the target. The spiking neural network model is trained based on perturbation boundary values ​​created and updated based on training iterations. These perturbation boundary values ​​include category-level shared boundary values ​​and sample-level residual values. The category-level shared boundary values ​​provide a baseline perturbation constraint for each sample category, and the sample-level residual values ​​are used to perform sample-level corrections to the baseline perturbation constraint. The perturbation boundary values ​​constrain the upper limit of the perturbation amplitude when multiple adversarial examples are generated. The category-level shared boundary values ​​and the sample-level residual values ​​are adaptively adjusted based on the classification results of the multiple adversarial examples. The multiple adversarial examples are multiple second samples, which are generated after perturbating a first sample. The first sample is the original sample.

2. The method according to claim 1, characterized in that, For the current training iteration, the second sample is generated based on the following method: The plurality of first samples are input into the current spiking neural network model so that the current spiking neural network model outputs the classification result of the first samples; Select multiple first samples whose classification results match the first sample label as third samples, and use the first sample classification results as the corresponding second sample labels; Based on the second sample label, the perturbation step size, and the perturbation boundary value, the third sample is perturbed to determine the second sample; wherein, the perturbation boundary value is determined based on the category-level shared boundary value calculated in the previous training iteration, the sample-level residual value calculated in the previous training iteration, and the residual adjustment coefficient, and the training iteration is a pre-set model training iteration; if the current iteration is the initial training iteration, the sample-level residual value is a preset value.

3. The method according to claim 2, characterized in that, The category-level shared boundary value and the sample-level residual value are determined based on the following method: Based on the sample category corresponding to the second sample label, obtain all third samples for each sample category; for each sample category, determine the sample-level initial perturbation scale based on all third samples of that sample category, so as to output the category-level shared boundary value of that category; The sample-level residual value is determined based on the average perturbation distance of the third sample, the second sample corresponding to the third sample, and the sample category corresponding to the label of the second sample.

4. The method according to claim 2, characterized in that, Based on the second sample label, the perturbation step size, and the perturbation boundary value, the third sample is perturbed to determine the second sample. The method then further includes: The second sample is input into the current spiking neural network model so that the current spiking neural network model outputs the classification result of the second sample. For each sample category, the category-level shared boundary value is adjusted based on the classification results of the second samples corresponding to all second samples in that sample category and the labels of the second samples corresponding to all second samples, and the adjusted category-level shared boundary value is determined. The adjusted sample-level residual value is determined based on the predefined residual smoothing coefficient, the sample-level residual value of the previous training iteration, the average perturbation distance corresponding to the sample category, and the actual perturbation distance; wherein, the actual perturbation distance is the perturbation norm between the third sample and the second sample corresponding to the third sample; Based on the adjusted category-level shared boundary value, the adjusted sample-level residual value, and the residual adjustment coefficient, an adjusted perturbation boundary value is determined; wherein, the adjusted category-level shared boundary value is used as the category-level shared boundary value for the next training iteration, the adjusted sample-level residual value is used as the sample-level residual value for the next training iteration, and the adjusted perturbation boundary value is used to generate the second sample for the next training iteration.

5. The method according to claim 4, characterized in that, The classification results of the second samples corresponding to all second samples under the sample category and the labels of the second samples corresponding to all second samples are used to adjust the category-level shared boundary value, and the adjusted category-level shared boundary value is determined, including: If the classification result of the second sample corresponding to all the second samples is consistent with the label of the second sample, the category-level shared boundary value of the next training iteration is determined based on the preset decay factor, preset adjustment factor, preset global update step size coefficient, intra-category perturbation uncertainty value, the sample-level initial perturbation scale, and the category-level perturbation boundary value of the current training iteration. The intra-category perturbation uncertainty value is used to characterize the discreteness of the robustness distribution of the category samples.

6. The method according to claim 4, characterized in that, The classification results of the second samples corresponding to all second samples under the sample category and the labels of the second samples corresponding to all second samples are used to adjust the category-level shared boundary value, and the adjusted category-level shared boundary value is determined, including: If the classification result of the second sample corresponding to the second sample is inconsistent with the label of the second sample, determine the target second sample, the average perturbation distance, the intra-category perturbation uncertainty value, the preset balance coefficient, and the category-level shared boundary value of the current training iteration round, and determine the adjusted category-level shared boundary value, wherein the target second sample is the second sample whose classification result is inconsistent with the label of the second sample.

7. The method according to claim 3, characterized in that, The average disturbance distance is determined based on the following method: For each sample category, the average perturbation distance corresponding to the sample category is determined based on the third sample and the second sample corresponding to the third sample.

8. The method according to claim 5 or 6, characterized in that, The in-category disturbance uncertainty value is determined based on the following method: Based on the average perturbation distance, the actual perturbation distance, and the number of samples in the sample category, the perturbation distance variance corresponding to the sample category is determined; based on the perturbation distance variance, the perturbation uncertainty value within the category is determined.

9. The method according to claim 8, characterized in that, For all third samples, the actual perturbation distance corresponding to the third sample is determined based on the following method: Based on the third sample and the second sample corresponding to the third sample, the actual disturbance distance corresponding to the third sample is determined.

10. The method according to claim 2, characterized in that, The model parameters in the spiking neural network model are determined based on the following method: Obtain the plurality of third samples and the corresponding plurality of second samples; The plurality of third samples and the plurality of second samples are input into the spiking neural network model to obtain the third sample classification result of the third sample and the second sample classification result of the second sample; Based on the classification results of the third sample, the classification results of the second sample, the label of the second sample, and the preset loss function, the model parameters in the spiking neural network model are adjusted. When the loss function converges, the resulting spiking neural network model is taken as the trained spiking neural network model.

Images