Small sample self-learning accurate identification method based on distillation knowledge migration

Abstract

The invention discloses a small sample self-learning accurate identification method based on distillation knowledge migration. The method comprises the following steps: S1, extracting deep semantic features of a source domain and shallow features of a small number of samples of a target domain, and calculating a mapping matrix; s2, calculating an entropy difference distillation excitation function based on the initial alignment features; s3, executing domain knowledge distillation and generating staged distillation representation; s4, constructing a composite fitness function and initializing a parameter population; s5, performing iterative optimization by adopting a variable step size dynamic feedback compression strategy; s6, loading the optimal parameters and performing coupling alignment with the historical distillation representation; and S7, performing combined fine adjustment on the distillation weight and the model parameters through self-learning feedback. According to the method, through adaptive knowledge distillation and dynamic optimization feedback closed loop, high-precision and adaptive identification under extremely few labeled samples is realized, the generalization ability of the model is remarkably improved, and overfitting is effectively inhibited.

Description

Technical Field

[0001] This invention relates to the field of pattern recognition technology, and in particular to a few-sample self-learning accurate recognition method based on distillation knowledge transfer. Background Technology

[0002] Few-shot learning and accurate recognition are important research directions in the field of artificial intelligence, aiming to achieve high-precision classification or regression with a very small amount of labeled data. Traditional few-shot recognition methods mainly rely on data augmentation, metric learning, or meta-learning techniques. Data augmentation expands the sample set through methods such as rotation, pruning, and noise injection, but it is still difficult to achieve sufficient diversity in scenarios where labeled samples are extremely scarce. Metric learning clusters similar samples and separates dissimilar samples by designing distance metrics or prototype networks, but it often relies on a large amount of auxiliary data for new categories. Meta-learning improves the model's ability to quickly adapt to new tasks by "learning to learn," and although it can alleviate the few-shot problem to some extent, meta-models have high training costs and are easily affected by differences in task distribution.

[0003] In recent years, domain adaptation and knowledge distillation have become key technical approaches to improve the performance of few-sample recognition. Domain adaptation methods reduce inter-domain differences by aligning the feature distributions of the source and target domains, often employing strategies such as adversarial training, maximum mean difference, or normalization reshaping. Knowledge distillation enhances the generalization ability of student models by passing soft labels or intermediate features from the teacher model to the student model. However, most existing domain adaptation and knowledge distillation methods use static or manually set weight combinations, failing to dynamically adjust the distillation force based on the distribution of new samples in the target domain and the model's learning state. This results in the risk of overfitting even in scenarios with very few labeled samples.

[0004] Furthermore, commonly used optimization strategies such as gradient descent and grid search struggle to balance global exploration with local convergence speed. Traditional hyperparameter tuning methods often require multiple trials, consuming significant computational resources and lacking sensitivity to small-sample problems, failing to respond promptly to changes in model error that necessitate adjustments to the search direction. Therefore, constructing an optimization strategy that dynamically feeds back model error and distillation weights while balancing global exploration and local utilization remains a significant challenge in the process of accurate identification through small-sample self-learning.

[0005] In summary, existing technologies still have significant shortcomings in areas such as data scarcity, adaptive model adjustment, and dynamic optimization. There is an urgent need for a comprehensive method that integrates adaptive domain knowledge distillation and efficient dynamic feedback optimization to improve the stability and accuracy of recognition models in small sample scenarios.

[0006] Therefore, how to provide a small-sample self-learning accurate identification method based on distillation knowledge transfer is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] One objective of this invention is to propose a small-sample self-learning accurate identification method based on distillation knowledge transfer. This invention has the technical advantages of dynamic feedback distillation weight and parameter optimization, adaptive balance exploration and utilization, and high-precision generalization identification under very few sample conditions.

[0008] A few-sample self-learning accurate identification method based on distillation knowledge transfer according to an embodiment of the present invention includes the following steps: S1. Extract the deep semantic features of the source domain samples and the shallow semantic features of a small number of labeled samples in the target domain, and calculate the feature mapping matrix between the two to obtain the initial alignment feature representation of the target domain. S2. Based on the initial alignment feature representation of the target domain, calculate the information entropy of the source domain teacher model and the target domain student model in the predicted output, and construct a distillation activation function with entropy difference as the core to obtain the dynamic distillation control weight. S3. Perform the domain knowledge distillation process according to the dynamic distillation control weights to obtain the staged distillation representation; S4. Construct an initial parameter population based on the staged distillation representation to obtain the initial search state of the Secretary Bird optimization algorithm; S5. In the iterative process of the Secretary Bird optimization algorithm, based on the previous round of recognition error and distillation weight change rate, a variable step size dynamic feedback compression strategy is introduced to adjust the ratio between global exploration and local utilization, and the candidate parameter set is iteratively optimized to obtain the optimal model parameter set. S6. Load the optimal model parameter set into the student model and call the callback mechanism to couple and align the current model recognition state with the historical stage distillation representation to generate the current round coupling correction representation; S7. Use the current round coupling correction representation to identify the newly added samples in the target domain, and input the identification results into the distillation activation function to complete the joint fine-tuning of the dynamic distillation control weights and the optimal model parameter set, so as to obtain the final stable target domain adaptive identification model.

[0009] Optionally, S1 specifically includes: S11. Semantic feature maps under different receptive fields are obtained by sequentially performing multi-layer convolution operations on the source domain samples, which correspond to global, regional and local semantic representations respectively. The same channel dimension standardization processing is applied to each layer of semantic feature map to obtain the source domain multi-layer standardized deep semantic feature tensor sequence. S12. Extract the first few layers of shallow semantic feature maps from a small number of labeled samples in the target domain, and perform channel expansion operation independently on each channel to increase the semantic expression capacity. Apply the channel perturbation adjustment process, and introduce micro-perturbation without changing the main direction of semantic distribution to generate the target domain channel-granular perturbation semantic feature tensor. S13. Map the source domain multi-level standardized deep semantic feature tensor sequence to the target domain channel-granular perturbation semantic feature tensor one by one according to the channel order, construct the semantic difference between each source-target channel pair, and form a set of cross-space semantic residual vectors. S14. Normalize each vector in the cross-space semantic residual vector set, and construct a bidirectional scoring function by the difference measure between the original vector and its dual vector to obtain the normalized difference balance matrix. S15. The normalized difference balance matrix is ​​used as the fusion weight input to the channel coupling fusion mechanism. Based on the perturbation response characteristics and semantic residual information of each channel in the target domain, channel-level weighted compensation and residual back-injection operations are performed, and the initial alignment feature representation of the target domain is finally output.

[0010] Optionally, S2 specifically includes: S21. Input the initial alignment feature representation of the target domain into the source domain teacher model and the target domain student model respectively, and obtain the predicted output probability distribution tensor on the same class set; S22. Calculate the information entropy of the prediction output probability distribution tensors of the source domain teacher model and the target domain student model, and construct the first entropy mapping vector and the second entropy mapping vector, which correspond to the information change paths from teacher to student and from student to teacher, respectively. S23. Pair the first entropy mapping vector and the second entropy mapping vector element by element to obtain the entropy difference cross vector set; S24. Calculate the category difference masking coefficient for each vector in the entropy difference cross vector set according to the category index i. : ; in, For the information entropy difference of category i, The prediction confidence level for category i. The entropy difference threshold. The confidence threshold. , To smooth the power factor, the category difference masking coefficient is used. Apply the action to the corresponding elements of the entropy difference cross vector to generate a clean entropy difference vector; S25. Perform a nonlinear perturbation injection operation on the clean entropy difference vector. The perturbation amplitude is controlled by the rate of change of confidence for each class. After the perturbation, normalization is applied to construct the distillation perturbation response vector. S26. Based on the $i$-th component of the distillation perturbation response vector Categorical difference occlusion coefficient Constructing the distillation excitation function : ; in, The total number of categories, , For the coupled power factor, The perturbation scaling factor is based on Generate a normalized dynamic distillation control weight vector; S27. The dynamic distillation control weight vector is used as an information transmission regulation factor between the source domain teacher model and the target domain student model, and is applied to the distillation path to output the dynamic distillation control weight.

[0011] Optionally, S3 specifically includes: S31. Input the dynamic distillation control weight vector into the distillation loss construction path, and perform weighted Kullback-Leibler divergence calculation on the source domain teacher model output probability distribution tensor and the target domain student model output probability distribution tensor according to the category index to obtain the distillation loss tensor based on category difference modulation. S32. After applying a temperature scaling factor to the distillation loss tensor according to the sample dimension, perform a non-equal weighted linear combination with the cross-entropy supervision loss tensor generated by the student model in the target domain to construct a joint loss function, where the combined weights of each loss term are adaptively generated by the sample confidence. S33. Using the joint loss function as the optimization objective, perform a forward and backward propagation process on all trainable parameter matrices in the student model, and use the momentum correction gradient update strategy to calculate the update value of the student model identification parameters in this round, forming the identification parameter increment tensor. S34. Divide the incremental tensor of the identification parameters into multiple groups of hierarchical incremental sub-tensors according to the network structure hierarchy, and flatten each sub-tensor into a fixed-dimensional vector after applying L2 normalization, mean centering and channel compression operations, respectively forming a hierarchical local feature embedding representation sequence. S35. Input the hierarchical local feature embedding sequence into the multi-head self-attention encoding structure, fuse the parameter change trends of different levels, and obtain the current cross-level local representation embedding vector. S36. The cross-level local representation embedding vectors of each historical round are concatenated with the current round embedding vector in chronological order to form a distillation embedding trajectory sequence. Position encoding vectors are introduced to identify the temporal features of each round, and finally a stage distillation representation tensor is generated. S37. Apply channel-weighted pooling and projection mapping to the stage distillation representation tensor and output a fixed-length embedding vector as the current stage distillation representation.

[0012] Optionally, S4 specifically includes: S41. On the target domain validation samples, calculate the recognition accuracy of the student model in the prediction task, the generalization error index based on unconfidence samples, and the entropy-driven loss value driven by information entropy, and construct the dual difference tensor based on the numerical differences between the pairs of the three types of sub-targets. S42. Feed the dual difference tensor into the normalization function and softmax activation module to obtain the sub-target response weight vector, and construct a composite fitness function by combining the three basic indices: ; in, Represents the composite fitness function. This represents the recognition accuracy of the student model on the validation samples in the target domain. This represents the generalization error estimate of the student model in the target domain based on confidence fluctuations. This represents the average information entropy driving the loss value of the student model along the distillation path. , , This represents the sub-objective response weight coefficient obtained from the objective duality difference. , , This represents the power-law adjustment factor corresponding to each of the three target sub-items, used to adjust the nonlinear sensitivity of each sub-item; S43. Divide the current round's stage distillation representation into two dimensions: time and channel, to obtain a multi-scale distillation representation sub-tensor sequence. Each sub-tensor is used to reflect the semantic information of distillation behavior at different scales. S44. Input each distillation tensor into the perturbation generation function to construct the perturbation control kernel at the corresponding scale. Each perturbation control kernel records the response amplitude, directionality and distribution state of the distillation behavior at the corresponding scale. S45. Apply asymmetric perturbation paths to the parameter prototype set one by one based on the perturbation control check to generate a set of parameter individuals with heterogeneous perturbation structures. S46. Load each parameter individual in the parameter individual set into the student model to perform the test prediction task, and calculate its performance evaluation score by combining the composite fitness function to construct the initial search state of the secretary bird optimization algorithm.

[0013] Optionally, S44 specifically includes: S441. For each distillation sub-tensor, calculate the normalized correlation coefficient between the feature maps of adjacent channels according to the channel dimension, and select several highly correlated channel combinations as the main direction semantic slice based on the correlation coefficient sorting. S442. For each main direction semantic slice, calculate the amplitude difference of adjacent pixels along its horizontal and vertical directions respectively, and sum the absolute values ​​of the amplitude differences to generate perturbation distribution maps in the horizontal and vertical directions. S443. Calculate the first and third order statistical characteristic values ​​of the disturbance distribution map in the spatial domain, and normalize the statistical characteristic values ​​of each direction to form a direction weight vector. S444. Map the direction weight vector to the main direction semantic slice according to the corresponding main direction, and perform bit-by-bit scaling on the elements in each slice to generate a direction guidance perturbation slice. S445. The perturbation slices in each direction are spliced ​​together in a preset scale sequence, and scale adjustment factors that decrease or increase are applied according to the position of the slices in the sequence to obtain a multi-scale perturbation control kernel with multi-scale response characteristics. S446. Perform interval scaling and upper and lower limit pruning on the multi-scale disturbance control kernel according to the channel dimension, map all channels to a unified amplitude range, and generate the final disturbance control kernel.

[0014] Optionally, S45 specifically includes: S451. Based on the disturbance control kernel, extract the prototype vector of each parameter in time order, and calculate the rate of change of prediction error of each parameter in the most recent several iterations. Use the ratio of error change to the error of the previous iteration to represent the parameter sensitivity. S452. Arrange the parameter sensitivity in descending order and divide it into high sensitivity segment, medium sensitivity segment and low sensitivity segment based on the preset quantile threshold. Each sensitivity segment corresponds to a set of disturbance amplitude range and direction offset coefficient. S453. For each sensitive segment, according to the corresponding perturbation amplitude range and directional offset coefficient, different step increments are set for the positive and negative directions respectively, and an intra-segment perturbation offset sequence is generated. S454. The perturbation offset sequences of each sensitive segment are superimposed on the corresponding parameter positions of the prototype vector in order of sensitivity from high to low to form the first set of individual parameters, and the maximum perturbation amplitude and step size increment of each parameter are recorded. S455. Calculate the behavioral differences of the first set of individual parameters under the fitness function in the previous round, and adjust the perturbation amplitude range and step size increment of each sensitive segment according to the positive and negative feedback of the behavioral differences to generate an optimized perturbation configuration. S456. Regenerate the perturbation offset sequence with optimized perturbation configuration, and apply it to the prototype vector in the same way as in step S454 to form a second batch of structurally heterogeneous parameter individual sets. S457. The first and second batches of parameter individual sets and their corresponding perturbation configuration data are summarized as the final parameter individual set with a multi-level asymmetric perturbation structure.

[0015] Optionally, S5 specifically includes: S51. At the beginning of each iteration of the Secretary Bird optimization algorithm, the student model's per-sample recognition error on the target domain verification samples is collected, and the error distribution of each parameter vector in the candidate parameter set in the previous round and the current round is divided into percentiles to form high error cluster, medium error cluster and low error cluster. At the same time, the absolute change amplitude of each element of the dynamic distillation weight vector is calculated and the corresponding mean is obtained as the weight fluctuation intensity. S52. Based on the sample proportions and weight fluctuation intensity of high-error clusters, medium-error clusters and low-error clusters, determine the corresponding global exploration step size compression coefficient and local utilization step size amplification coefficient, and generate a multi-stage step size configuration sequence. S53. For each parameter vector in the candidate parameter set, according to the multi-stage step size configuration sequence of its error cluster, construct a global exploration offset vector and a local utilization offset vector in turn. The global exploration offset vector is formed by weighting the difference between the parameter vector and the best vector in the same cluster, and the local utilization offset vector is generated by scaling the gradient sensitivity of the parameter vector in the historical iteration. The two offset vectors are superimposed on the original parameter vector step by step to form a new generation of candidate parameter set. S54. Load the new generation of candidate parameters into the student model one by one, calculate the global error convergence rate and local model diversity index corresponding to each parameter vector, and dynamically adjust the combined weight of the two according to the iteration round to generate the final fitness score. S55. Based on the fitness score and the similarity threshold between parameter vectors, perform similarity clustering, and select the parameter vector with the highest fitness in each cluster to form the next round of candidate set. S56. When the number of clusters is lower than the preset cluster threshold, the error convergence rate gain in the most recent rounds is lower than the preset gain threshold, or the maximum number of iterations is reached, the iteration is terminated, and all parameter vectors in the current candidate set are taken as the optimal model parameter set.

[0016] Optionally, S6 specifically includes: S61. Load the optimal model parameter set into the student model, fix the model weights except for the output layer, and set the model to callback prediction mode. S62. For the target domain validation samples, extract the multi-scale feature maps and the final prediction probability vector of the student model at a specified depth, and apply spatial pyramid pooling to each feature map to unify the feature resolution, thus forming the current model recognition state. S63. From the stored historical stage distillation representation sequence, apply exponential decay weighting to the representations of the most recent rounds according to the round index and then sum them to generate an aggregated historical representation tensor. S64. For the multi-scale feature map and aggregated history representation tensor of the current model, calculate the in-channel covariance at each scale and in each channel and normalize it to obtain the channel alignment coefficient matrix. S65. Multiply the channel alignment coefficient matrix with the corresponding multi-scale feature map channel by channel, and merge the weighted features of all scales in the order of scale hierarchy using an interpolation fusion strategy to obtain the coupled fused feature map. S66. Perform linear projection on the coupled fusion feature map according to the channel dimension and perform layer normalization according to the historical variance of each channel, and output a fixed-length current round coupled correction representation.

[0017] Optionally, S7 specifically includes: S71. Combine the current round of coupled correction representation with the multi-scale feature representation of newly added samples in the target domain dimension by dimension to construct the semantic alignment tensor of coupled samples, and calculate the alignment strength vector between each newly added sample and the coupled representation to obtain the coupling sample affinity result. S72. Input the coupling sample affinity result and the predicted probability distribution of the new sample into the distillation activation function in sequence, extract the initial weight bias of the function on each category channel, and generate multi-level distillation weight modulation coefficients. S73. Combining the multi-stage distillation weight modulation coefficients with the historical update trajectories of each parameter in the optimal model parameter set, calculate the incremental modulation vector of each parameter, and apply the incremental modulation vector to the optimal model parameter set to obtain the preliminary rearranged model parameter set. S74. Load the initially rearranged set of model parameters into the student model, and perform cross-distillation fine-tuning training with the newly added samples in the target domain in batches. After each fine-tuning, collect the fluctuation of the affinity of the coupled samples and the change of the distillation weight in real time to dynamically adjust the fine-tuning learning rate and the distillation weight allocation, and form the set of model parameters after fine-tuning. S75. Based on the stability index of the fine-tuned model parameter set on the time series curve of the affinity of the coupled samples, compare it with the preset stability threshold. When the stability index meets the threshold, determine to terminate the fine-tuning and output the candidate stable model parameter set. S76. Load the candidate stable model parameter set into the student model and perform batch coupling consistency verification on unknown samples in the target domain. When the coupling activation consistency reaches the preset standard, the target domain adaptive recognition model is finally generated and output.

[0018] The beneficial effects of this invention are: (1) This invention extracts deep semantic features of the source domain and shallow semantic features of the target domain and constructs a multi-layer feature mapping matrix. Combined with a dynamic distillation and weight adjustment mechanism based on information entropy difference, it achieves accurate domain alignment under conditions of very few labeled samples, effectively improves the generalization ability of the model and avoids overfitting.

[0019] (2) This invention defines a composite fitness function that covers recognition accuracy, generalization error and distillation loss, adopts the secretary bird optimization algorithm and introduces a variable step size dynamic feedback compression strategy based on error change and distillation weight fluctuation, thereby achieving an adaptive balance between global exploration and local utilization, which significantly improves the efficiency of model parameter search and convergence speed.

[0020] (3) In the iterative correction stage, this invention couples and aligns the current round multi-scale feature map with the historical stage distillation representation according to the channel covariance, and adopts spatial pyramid pooling and channel weighted fusion, which breaks through the limitation of poor model adaptability to new samples in the prior art and realizes the cross-round stability improvement of the recognition model.

[0021] (4) This invention utilizes the current round coupling correction representation to jointly fine-tune the newly added samples in the target domain, and uses the real-time collected coupling affinity and distillation weight changes to dynamically adjust the fine-tuning learning rate and weight allocation, thereby completing the synchronous optimization of distillation weight and model parameters, thus effectively improving the accuracy and robustness of the target domain adaptive recognition model. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0023] Figure 1 This is a schematic diagram of the overall process of a small-sample self-learning accurate identification method based on distillation knowledge transfer proposed in this invention; Figure 2 This is a schematic diagram of the dynamic distillation control weight generation and initial parameter population construction in a small-sample self-learning accurate identification method based on distillation knowledge transfer proposed in this invention. Figure 3 This is a schematic diagram of the iterative feedback and coupling correction process of the Secretary Bird optimization algorithm in the small-sample self-learning accurate recognition method based on distillation knowledge transfer proposed in this invention. Detailed Implementation

[0024] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0025] refer to Figure 1 A few-sample self-learning accurate identification method based on distillation knowledge transfer includes the following steps: S1. Extract the deep semantic features of the source domain samples and the shallow semantic features of a small number of labeled samples in the target domain, and calculate the feature mapping matrix between the two to obtain the initial alignment feature representation of the target domain. S2. Based on the initial alignment feature representation of the target domain, calculate the information entropy of the source domain teacher model and the target domain student model in the predicted output, and construct a distillation activation function with entropy difference as the core to obtain the dynamic distillation control weight. S3. Perform the domain knowledge distillation process according to the dynamic distillation control weights to obtain the staged distillation representation; S4. Construct an initial parameter population based on the staged distillation representation to obtain the initial search state of the Secretary Bird optimization algorithm; S5. In the iterative process of the Secretary Bird optimization algorithm, based on the previous round of recognition error and distillation weight change rate, a variable step size dynamic feedback compression strategy is introduced to adjust the ratio between global exploration and local utilization, and the candidate parameter set is iteratively optimized to obtain the optimal model parameter set. S6. Load the optimal model parameter set into the student model and call the callback mechanism to couple and align the current model recognition state with the historical stage distillation representation to generate the current round coupling correction representation; S7. Use the current round coupling correction representation to identify the newly added samples in the target domain, and input the identification results into the distillation activation function to complete the joint fine-tuning of the dynamic distillation control weights and the optimal model parameter set, so as to obtain the final stable target domain adaptive identification model.

[0026] By extracting deep semantics from the source domain and shallow semantics from the target domain and establishing multi-layer feature mapping, and dynamically adjusting distillation weights based on information entropy difference, distillation representations are generated in stages and an initial parameter population is constructed. A variable step-size dynamic feedback compression strategy is used for parameter search, and coupled correction representations are used for multi-scale alignment and joint distillation fine-tuning of new samples. This achieves adaptive training and model fine-tuning under conditions of very few labeled samples, effectively improving the generalization ability of the recognition model and reducing the risk of overfitting.

[0027] In this embodiment, S1 specifically includes: S11. Semantic feature maps under different receptive fields are obtained by sequentially performing multi-layer convolution operations on the source domain samples, which correspond to global, regional and local semantic representations respectively. The same channel dimension standardization processing is applied to each layer of semantic feature map to obtain the source domain multi-layer standardized deep semantic feature tensor sequence. S12. Extract the first few layers of shallow semantic feature maps from a small number of labeled samples in the target domain, and perform channel expansion operation independently on each channel to increase the semantic expression capacity. Apply the channel perturbation adjustment process, and introduce micro-perturbation without changing the main direction of semantic distribution to generate the target domain channel-granular perturbation semantic feature tensor. S13. Map the source domain multi-level standardized deep semantic feature tensor sequence to the target domain channel-granular perturbation semantic feature tensor one by one according to the channel order, construct the semantic difference between each source-target channel pair, and form a set of cross-space semantic residual vectors. S14. Normalize each vector in the cross-space semantic residual vector set, and construct a bidirectional scoring function by the difference measure between the original vector and its dual vector to obtain the normalized difference balance matrix. S15. The normalized difference balance matrix is ​​used as the fusion weight input to the channel coupling fusion mechanism. Based on the perturbation response characteristics and semantic residual information of each channel in the target domain, channel-level weighted compensation and residual back-injection operations are performed, and the initial alignment feature representation of the target domain is finally output.

[0028] By performing multi-layer convolution and channel normalization on source domain samples and channel expansion and micro-perturbation on target domain samples, a cross-space semantic residual mapping and normalized difference balance matrix are constructed. Combined with channel coupling fusion and residual feedback compensation, fine-grained channel-level feature alignment is achieved, which effectively improves the initial alignment accuracy between the source and target domains and reduces the recognition error caused by distribution differences.

[0029] In this embodiment, S2 specifically includes: S21. Input the initial alignment feature representation of the target domain into the source domain teacher model and the target domain student model respectively, and obtain the predicted output probability distribution tensor on the same class set; S22. Calculate the information entropy of the prediction output probability distribution tensors of the source domain teacher model and the target domain student model, and construct the first entropy mapping vector and the second entropy mapping vector, which correspond to the information change paths from teacher to student and from student to teacher, respectively. S23. Pair the first entropy mapping vector and the second entropy mapping vector element by element to obtain the entropy difference cross vector set; S24. Calculate the category difference masking coefficient for each vector in the entropy difference cross vector set according to the category index i. : ; in, For the information entropy difference of category i, The prediction confidence level for category i. The entropy difference threshold. The confidence threshold. , To smooth the power factor, the category difference masking coefficient is used. Apply the action to the corresponding elements of the entropy difference cross vector to generate a clean entropy difference vector; S25. Perform a nonlinear perturbation injection operation on the clean entropy difference vector. The perturbation amplitude is controlled by the rate of change of confidence for each class. After the perturbation, normalization is applied to construct the distillation perturbation response vector. S26. Based on the $i$-th component of the distillation perturbation response vector Categorical difference occlusion coefficient Constructing the distillation excitation function : ; in, The total number of categories, , For the coupled power factor, The perturbation scaling factor is based on Generate a normalized dynamic distillation control weight vector; S27. The dynamic distillation control weight vector is used as an information transmission regulation factor between the source domain teacher model and the target domain student model, and is applied to the distillation path to output the dynamic distillation control weight.

[0030] By constructing bidirectional entropy mapping vectors between teachers and students and between students and teachers, and combining the masking coefficient based on category confidence and information entropy difference, nonlinear perturbation injection and multi-scale compression processing are applied to the entropy difference vector to generate dynamic distillation control weights. This achieves refined weight allocation at the category and channel levels, effectively improving the targeting and stability of the knowledge distillation process and optimizing the student model's adaptive learning ability for a small number of samples in the target domain.

[0031] In this embodiment, S3 specifically includes: S31. Input the dynamic distillation control weight vector into the distillation loss construction path, and perform weighted Kullback-Leibler divergence calculation on the source domain teacher model output probability distribution tensor and the target domain student model output probability distribution tensor according to the category index to obtain the distillation loss tensor based on category difference modulation. S32. After applying a temperature scaling factor to the distillation loss tensor according to the sample dimension, perform a non-equal weighted linear combination with the cross-entropy supervision loss tensor generated by the student model in the target domain to construct a joint loss function, where the combined weights of each loss term are adaptively generated by the sample confidence. S33. Using the joint loss function as the optimization objective, perform a forward and backward propagation process on all trainable parameter matrices in the student model, and use the momentum correction gradient update strategy to calculate the update value of the student model identification parameters in this round, forming the identification parameter increment tensor. S34. Divide the incremental tensor of the identification parameters into multiple groups of hierarchical incremental sub-tensors according to the network structure hierarchy, and flatten each sub-tensor into a fixed-dimensional vector after applying L2 normalization, mean centering and channel compression operations, respectively forming a hierarchical local feature embedding representation sequence. S35. Input the hierarchical local feature embedding sequence into the multi-head self-attention encoding structure, fuse the parameter change trends of different levels, and obtain the current cross-level local representation embedding vector. S36. The cross-level local representation embedding vectors of each historical round are concatenated with the current round embedding vector in chronological order to form a distillation embedding trajectory sequence. Position encoding vectors are introduced to identify the temporal features of each round, and finally a stage distillation representation tensor is generated. S37. Apply channel-weighted pooling and projection mapping to the stage distillation representation tensor and output a fixed-length embedding vector as the current stage distillation representation.

[0032] By constructing a distillation loss tensor modulated by category difference, and updating the student model's recognition parameters by combining weight-based reconstruction and grouped hierarchical increment, and by introducing local feature embedding and global encoding structures, distillation errors at different scales are fused to form a distillation local saliency embedding vector. This effectively enhances the student model's multi-scale response to distillation stage error information and improves the expressive integrity and temporal consistency of the stage distillation representation.

[0033] In this embodiment, S4 specifically includes: S41. On the target domain validation samples, calculate the recognition accuracy of the student model in the prediction task, the generalization error index based on unconfidence samples, and the entropy-driven loss value driven by information entropy, and construct the dual difference tensor based on the numerical differences between the pairs of the three types of sub-targets. S42. Feed the dual difference tensor into the normalization function and softmax activation module to obtain the sub-target response weight vector, and construct a composite fitness function by combining the three basic indices: ; in, Represents the composite fitness function. This represents the recognition accuracy of the student model on the validation samples in the target domain. This represents the generalization error estimate of the student model in the target domain based on confidence fluctuations. This represents the average information entropy driving the loss value of the student model along the distillation path. , , This represents the sub-objective response weight coefficient obtained from the objective duality difference. , , This represents the power-law adjustment factor corresponding to each of the three target sub-items, used to adjust the nonlinear sensitivity of each sub-item; S43. Divide the current round's stage distillation representation into two dimensions: time and channel, to obtain a multi-scale distillation representation sub-tensor sequence. Each sub-tensor is used to reflect the semantic information of distillation behavior at different scales. S44. Input each distillation tensor into the perturbation generation function to construct the perturbation control kernel at the corresponding scale. Each perturbation control kernel records the response amplitude, directionality and distribution state of the distillation behavior at the corresponding scale. S45. Apply asymmetric perturbation paths to the parameter prototype set one by one based on the perturbation control check to generate a set of parameter individuals with heterogeneous perturbation structures. S46. Load each parameter individual in the parameter individual set into the student model to perform the test prediction task, and calculate its performance evaluation score by combining the composite fitness function to construct the initial search state of the secretary bird optimization algorithm.

[0034] By constructing a complex fitness function based on multi-index fusion, the recognition accuracy, label volatility, and distillation path entropy loss are weighted and combined. At the same time, even difference tensors, multi-scale segmentation, and perturbation evaluation mechanisms are introduced to quantify the strength of the execution direction, behavior fit, and distribution perturbation of the staged distillation representation. This effectively improves the discriminativeness and representativeness of the initial fitness distribution construction of the parameter population and enhances the early search efficiency and convergence path control capability of the Secretary Bird optimization algorithm in high-dimensional space.

[0035] In this embodiment, S44 specifically includes: S441. For each distillation sub-tensor, calculate the normalized correlation coefficient between the feature maps of adjacent channels according to the channel dimension, and select several highly correlated channel combinations as the main direction semantic slice based on the correlation coefficient sorting. S442. For each main direction semantic slice, calculate the amplitude difference of adjacent pixels along its horizontal and vertical directions respectively, and sum the absolute values ​​of the amplitude differences to generate perturbation distribution maps in the horizontal and vertical directions. S443. Calculate the first and third order statistical characteristic values ​​of the disturbance distribution map in the spatial domain, and normalize the statistical characteristic values ​​of each direction to form a direction weight vector. S444. Map the direction weight vector to the main direction semantic slice according to the corresponding main direction, and perform bit-by-bit scaling on the elements in each slice to generate a direction guidance perturbation slice. S445. The perturbation slices in each direction are spliced ​​together in a preset scale sequence, and scale adjustment factors that decrease or increase are applied according to the position of the slices in the sequence to obtain a multi-scale perturbation control kernel with multi-scale response characteristics. S446. Perform interval scaling and upper and lower limit pruning on the multi-scale disturbance control kernel according to the channel dimension, map all channels to a unified amplitude range, and generate the final disturbance control kernel.

[0036] By constructing a normalized correlation coefficient matrix for the distillation tensor along the channel dimension, extracting feature slices along the main direction, and calculating gradient differences and perturbation distribution intensity along the main direction, and combining this with direction-guided feature sequences to achieve multi-layer perturbation statistical discrimination, the fine-grained adjustment of perturbation amplitude and direction is effectively realized. This enhances the ability of the local perturbation mechanism in the distillation representation to distinguish abnormal channel structures and unstable regions, thereby strengthening the adaptive compression capability of the perturbation path.

[0037] In this embodiment, S45 specifically includes: S451. Based on the disturbance control kernel, extract the prototype vector of each parameter in time order, and calculate the rate of change of prediction error of each parameter in the most recent several iterations. Use the ratio of error change to the error of the previous iteration to represent the parameter sensitivity. S452. Arrange the parameter sensitivity in descending order and divide it into high sensitivity segment, medium sensitivity segment and low sensitivity segment based on the preset quantile threshold. Each sensitivity segment corresponds to a set of disturbance amplitude range and direction offset coefficient. S453. For each sensitive segment, according to the corresponding perturbation amplitude range and directional offset coefficient, different step increments are set for the positive and negative directions respectively, and an intra-segment perturbation offset sequence is generated. S454. The perturbation offset sequences of each sensitive segment are superimposed on the corresponding parameter positions of the prototype vector in order of sensitivity from high to low to form the first set of individual parameters, and the maximum perturbation amplitude and step size increment of each parameter are recorded. S455. Calculate the behavioral differences of the first set of individual parameters under the fitness function in the previous round, and adjust the perturbation amplitude range and step size increment of each sensitive segment according to the positive and negative feedback of the behavioral differences to generate an optimized perturbation configuration. S456. Regenerate the perturbation offset sequence with optimized perturbation configuration, and apply it to the prototype vector in the same way as in step S454 to form a second batch of structurally heterogeneous parameter individual sets. S457. The first and second batches of parameter individual sets and their corresponding perturbation configuration data are summarized as the final parameter individual set with a multi-level asymmetric perturbation structure.

[0038] By sorting the prototype vectors of parameters according to the rate of change of error and dividing them into sensitivity intervals, and constructing an asymmetric disturbance path in combination with the directional bias coefficient, and iteratively updating the disturbance strategy based on the disturbance response feedback, a dynamic disturbance control configuration generated in multiple rounds is finally formed. This allows the model parameters to better adapt to changes in the identification error structure during fine-tuning, effectively improving the directionality, discriminability, and closed-loop accuracy of individual parameter disturbances and strategy optimization.

[0039] In this embodiment, S5 specifically includes: S51. At the beginning of each iteration of the Secretary Bird optimization algorithm, the student model's per-sample recognition error on the target domain verification samples is collected, and the error distribution of each parameter vector in the candidate parameter set in the previous round and the current round is divided into percentiles to form high error cluster, medium error cluster and low error cluster. At the same time, the absolute change amplitude of each element of the dynamic distillation weight vector is calculated and the corresponding mean is obtained as the weight fluctuation intensity. S52. Based on the sample proportions and weight fluctuation intensity of high-error clusters, medium-error clusters and low-error clusters, determine the corresponding global exploration step size compression coefficient and local utilization step size amplification coefficient, and generate a multi-stage step size configuration sequence. S53. For each parameter vector in the candidate parameter set, according to the multi-stage step size configuration sequence of its error cluster, construct a global exploration offset vector and a local utilization offset vector in turn. The global exploration offset vector is formed by weighting the difference between the parameter vector and the best vector in the same cluster, and the local utilization offset vector is generated by scaling the gradient sensitivity of the parameter vector in the historical iteration. The two offset vectors are superimposed on the original parameter vector step by step to form a new generation of candidate parameter set. S54. Load the new generation of candidate parameters into the student model one by one, calculate the global error convergence rate and local model diversity index corresponding to each parameter vector, and dynamically adjust the combined weight of the two according to the iteration round to generate the final fitness score. S55. Based on the fitness score and the similarity threshold between parameter vectors, perform similarity clustering, and select the parameter vector with the highest fitness in each cluster to form the next round of candidate set. S56. When the number of clusters is lower than the preset cluster threshold, the error convergence rate gain in the most recent rounds is lower than the preset gain threshold, or the maximum number of iterations is reached, the iteration is terminated, and all parameter vectors in the current candidate set are taken as the optimal model parameter set.

[0040] By introducing a variable step size adjustment mechanism jointly driven by the error distribution ratio and the dynamic distillation control amplitude, a sequence of global exploration ratio and local utilization ratio factors is constructed. Combined with error level classification and historical perturbation trajectory, the hierarchical perturbation and screening optimization of candidate parameter vectors are completed, which significantly improves the search accuracy of the Secretary Bird optimization algorithm in different error regions and enhances the parameter convergence stability and dynamic adaptive adjustment capability of the model in the small sample recognition process.

[0041] In this embodiment, S6 specifically includes: S61. Load the optimal model parameter set into the student model, fix the model weights except for the output layer, and set the model to callback prediction mode. S62. For the target domain validation samples, extract the multi-scale feature maps and the final prediction probability vector of the student model at a specified depth, and apply spatial pyramid pooling to each feature map to unify the feature resolution, thus forming the current model recognition state. S63. From the stored historical stage distillation representation sequence, apply exponential decay weighting to the representations of the most recent rounds according to the round index and then sum them to generate an aggregated historical representation tensor. S64. For the multi-scale feature map and aggregated history representation tensor of the current model, calculate the in-channel covariance at each scale and in each channel and normalize it to obtain the channel alignment coefficient matrix. S65. Multiply the channel alignment coefficient matrix with the corresponding multi-scale feature map channel by channel, and merge the weighted features of all scales in the order of scale hierarchy using an interpolation fusion strategy to obtain the coupled fused feature map. S66. Perform linear projection on the coupled fusion feature map according to the channel dimension and perform layer normalization according to the historical variance of each channel, and output a fixed-length current round coupled correction representation.

[0042] By loading the optimal set of model parameters into the student model and freezing the output layer to enhance model stability, and by constructing a historical representation tensor in combination with historical distillation features, and by performing multi-scale fusion processing on the predicted probability and deep features to generate a channel-aligned distillation matrix, and further by performing channel-by-channel backtracking correction through the difference between the correction matrix and the historical alignment map, the representation capability of the current round-coupled correction representation in terms of temporal consistency and channel structure adaptability is effectively improved.

[0043] In this embodiment, S7 specifically includes: S71. Combine the current round of coupled correction representation with the multi-scale feature representation of newly added samples in the target domain dimension by dimension to construct the semantic alignment tensor of coupled samples, and calculate the alignment strength vector between each newly added sample and the coupled representation to obtain the coupling sample affinity result. S72. Input the coupling sample affinity result and the predicted probability distribution of the new sample into the distillation activation function in sequence, extract the initial weight bias of the function on each category channel, and generate multi-level distillation weight modulation coefficients. S73. Combining the multi-stage distillation weight modulation coefficients with the historical update trajectories of each parameter in the optimal model parameter set, calculate the incremental modulation vector of each parameter, and apply the incremental modulation vector to the optimal model parameter set to obtain the preliminary rearranged model parameter set. S74. Load the initially rearranged set of model parameters into the student model, and perform cross-distillation fine-tuning training with the newly added samples in the target domain in batches. After each fine-tuning, collect the fluctuation of the affinity of the coupled samples and the change of the distillation weight in real time to dynamically adjust the fine-tuning learning rate and the distillation weight allocation, and form the set of model parameters after fine-tuning. S75. Based on the stability index of the fine-tuned model parameter set on the time series curve of the affinity of the coupled samples, compare it with the preset stability threshold. When the stability index meets the threshold, determine to terminate the fine-tuning and output the candidate stable model parameter set. S76. Load the candidate stable model parameter set into the student model and perform batch coupling consistency verification on unknown samples in the target domain. When the coupling activation consistency reaches the preset standard, the target domain adaptive recognition model is finally generated and output.

[0044] By semantically aligning and fusing the current round coupling correction representation with the features of newly added samples in the target domain, and combining the affinity results to drive the distillation activation function to generate multi-level distillation weight modulation coefficients, and guiding the dynamic fine-tuning of the optimal model parameter set, a gradual adaptation and feedback optimization to new samples is achieved. Simultaneously, by identifying stability indicators to control the termination time of model parameter updates, the model's recognition consistency and adaptive stability on new data are improved, enhancing the closed-loop regulation capability of the overall distillation optimization process.

[0045] Example 1: To verify the feasibility of this invention in practice, it was applied to the classification and recognition task of ten newly collected images of endangered animals at a wildlife conservation station. This dataset provided only ten manually labeled images for each animal, totaling one hundred images, to test the model's recognition performance under extremely limited labeled sample conditions. Traditional convolutional neural networks often suffer from severe overfitting in this scenario due to insufficient samples: after training a conventional five-layer convolutional network for fifty epochs, the average classification accuracy on the validation set was only 62.3%, and the confidence score for the highest category fluctuated significantly when testing with new samples, resulting in an error rate exceeding 38% in actual deployment, failing to meet the conservation station's requirements for automated and accurate recognition.

[0046] In practical applications, the original images are first uniformly preprocessed, including adjusting the image size to 224×224 pixels, normalizing the RGB three channels to the [0,1] range, and performing random horizontal mirroring and brightness fine-tuning augmentation operations on each image. Then, the method of this invention is used to extract multi-layer standardized deep semantic features from the source domain, and dynamic distillation weights are constructed based on entropy difference to perform domain knowledge distillation on the student model—this model structure is consistent with the traditional five-layer convolutional network, with each layer having 64, 128, 256, 512, and 512 kernels, respectively, an initial learning rate of 0.001, and updates using the Adam optimizer. After fifty rounds of distillation iterations, the model parameters are iteratively optimized using the Secretary Bird optimization algorithm, with a population size of 100 individuals and fifty rounds of optimization iterations. The error and the rate of change of distillation weights are used as inputs to a variable step-size dynamic feedback compression strategy to generate global and local perturbation components, ultimately determining the optimal set of model parameters.

[0047] The optimal model parameters are then loaded into the student model, and a callback mechanism is invoked to couple and align the current model's multi-scale feature map with the historical stage distillation representation, generating a coupled correction representation for accurate identification of newly added samples in the target domain. In tests on twenty newly added images, by inputting the coupled correction representation and prediction results into the distillation activation function, and fine-tuning the dynamic distillation weights and model parameters together for two rounds, the model achieved an average recognition accuracy of 84.7% on the newly added samples, an improvement of 22.4 percentage points compared to traditional methods; the average prediction confidence remained stable above 0.89, and the confidence variance decreased from 0.15 to 0.04.

[0048] The table below compares the classification results and key metrics of the method of this invention with those of traditional convolutional neural networks for some newly added samples: Table 1 Comparison of Classification Performance of Newly Added Samples

[0049] As shown in the table, the prediction accuracy of this invention on five typical new samples decreased from the average misclassification rate of traditional CNNs (approximately 40%) to 6%, while the mean prediction confidence increased from 0.61 to 0.88, indicating that the method of this invention significantly enhances the stability and confidence level of identification for new samples. Further analysis shows that on the entire new sample set, the precision, recall, and F1 score of the method of this invention reached 0.85, 0.83, and 0.84, respectively, representing an overall improvement of over 25% compared to traditional methods.

[0050] The results of this embodiment demonstrate that by introducing adaptive domain knowledge distillation and the dynamic feedback loop of the secretary bird optimization algorithm, this invention effectively improves the generalization accuracy and recognition confidence of the model in environments with very few labeled samples, and exhibits excellent stability and robustness in tests with newly added samples. It provides a practical technical solution for application scenarios such as wildlife conservation stations that have high requirements for small sample recognition.

[0051] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A few-sample self-learning accurate identification method based on distillation knowledge transfer, characterized in that, Includes the following steps: S1. Extract the deep semantic features of the source domain samples and the shallow semantic features of a small number of labeled samples in the target domain, and calculate the feature mapping matrix between the two to obtain the initial alignment feature representation of the target domain. S2. Based on the initial alignment feature representation of the target domain, calculate the information entropy of the source domain teacher model and the target domain student model in the predicted output, and construct a distillation activation function with entropy difference as the core to obtain the dynamic distillation control weight. S3. Perform the domain knowledge distillation process according to the dynamic distillation control weights to obtain the staged distillation representation; S4. Construct an initial parameter population based on the staged distillation representation to obtain the initial search state of the Secretary Bird optimization algorithm; S5. In the iterative process of the Secretary Bird optimization algorithm, based on the previous round of recognition error and distillation weight change rate, a variable step size dynamic feedback compression strategy is introduced to adjust the ratio between global exploration and local utilization, and the candidate parameter set is iteratively optimized to obtain the optimal model parameter set. S6. Load the optimal model parameter set into the student model and call the callback mechanism to couple and align the current model recognition state with the historical stage distillation representation to generate the current round coupling correction representation; S7. Use the current round coupling correction representation to identify the newly added samples in the target domain, and input the identification results into the distillation activation function to complete the joint fine-tuning of the dynamic distillation control weights and the optimal model parameter set, so as to obtain the final stable target domain adaptive identification model.

2. The method for accurate identification based on distillation knowledge transfer using few-sample self-learning, as described in claim 1, is characterized in that... S1 specifically includes: S11. Semantic feature maps under different receptive fields are obtained by sequentially performing multi-layer convolution operations on the source domain samples, which correspond to global, regional and local semantic representations respectively. The same channel dimension standardization processing is applied to each layer of semantic feature map to obtain the source domain multi-layer standardized deep semantic feature tensor sequence. S12. Extract the first few layers of shallow semantic feature maps from a small number of labeled samples in the target domain, and perform channel expansion operation independently on each channel to increase the semantic expression capacity. Apply the channel perturbation adjustment process, and introduce micro-perturbation without changing the main direction of semantic distribution to generate the target domain channel-granular perturbation semantic feature tensor. S13. Map the source domain multi-level standardized deep semantic feature tensor sequence to the target domain channel-granular perturbation semantic feature tensor one by one according to the channel order, construct the semantic difference between each source-target channel pair, and form a set of cross-space semantic residual vectors. S14. Normalize each vector in the cross-space semantic residual vector set, and construct a bidirectional scoring function by the difference measure between the original vector and its dual vector to obtain the normalized difference balance matrix. S15. The normalized difference balance matrix is ​​used as the fusion weight input to the channel coupling fusion mechanism. Based on the perturbation response characteristics and semantic residual information of each channel in the target domain, channel-level weighted compensation and residual back-injection operations are performed, and the initial alignment feature representation of the target domain is finally output.

3. The few-sample self-learning accurate identification method based on distillation knowledge transfer according to claim 1, characterized in that, S2 specifically includes: S21. Input the initial alignment feature representation of the target domain into the source domain teacher model and the target domain student model respectively, and obtain the predicted output probability distribution tensor on the same class set; S22. Calculate the information entropy of the prediction output probability distribution tensors of the source domain teacher model and the target domain student model, and construct the first entropy mapping vector and the second entropy mapping vector, which correspond to the information change paths from teacher to student and from student to teacher, respectively. S23. Pair the first entropy mapping vector and the second entropy mapping vector element by element and find the difference to obtain the entropy difference cross vector set; S24. Calculate the category difference masking coefficient for each vector in the entropy difference cross vector set according to the category index i. : ； in, For the information entropy difference of category i, The prediction confidence level for category i. The entropy difference threshold. The confidence threshold. , To smooth the power factor, the category difference masking coefficient is used. Apply the same action to the corresponding elements of the entropy difference cross vector to generate a clean entropy difference vector; S25. Perform a nonlinear perturbation injection operation on the clean entropy difference vector. The perturbation amplitude is controlled by the rate of change of confidence for each class. After the perturbation, normalization is applied to construct the distillation perturbation response vector. S26. Based on the $i$-th component of the distillation perturbation response vector Categorical difference occlusion coefficient Constructing the distillation excitation function : ； in, The total number of categories, , For the coupled power factor, The perturbation scaling factor is based on Generate a normalized dynamic distillation control weight vector; S27. The dynamic distillation control weight vector is used as an information transmission regulation factor between the source domain teacher model and the target domain student model, and is applied to the distillation path to output the dynamic distillation control weight.

4. The few-sample self-learning accurate identification method based on distillation knowledge transfer according to claim 1, characterized in that, S3 specifically includes: S31. Input the dynamic distillation control weight vector into the distillation loss construction path, and perform weighted Kullback-Leibler divergence calculation on the source domain teacher model output probability distribution tensor and the target domain student model output probability distribution tensor according to the category index to obtain the distillation loss tensor based on category difference modulation. S32. After applying a temperature scaling factor to the distillation loss tensor according to the sample dimension, perform a non-equal weighted linear combination with the cross-entropy supervision loss tensor generated by the student model in the target domain to construct a joint loss function, where the combined weights of each loss term are adaptively generated by the sample confidence. S33. Using the joint loss function as the optimization objective, perform a forward and backward propagation process on all trainable parameter matrices in the student model, and use the momentum correction gradient update strategy to calculate the update value of the student model identification parameters in this round, forming the identification parameter increment tensor. S34. Divide the incremental tensor of the identification parameters into multiple groups of hierarchical incremental sub-tensors according to the network structure hierarchy, and flatten each sub-tensor into a fixed-dimensional vector after applying L2 normalization, mean centering and channel compression operations, respectively forming a hierarchical local feature embedding representation sequence. S35. Input the hierarchical local feature embedding sequence into the multi-head self-attention encoding structure, fuse the parameter change trends of different levels, and obtain the current cross-level local representation embedding vector. S36. The cross-level local representation embedding vectors of each historical round are concatenated with the current round embedding vector in chronological order to form a distillation embedding trajectory sequence. Position encoding vectors are introduced to identify the temporal features of each round, and finally a stage distillation representation tensor is generated. S37. Apply channel-weighted pooling and projection mapping to the stage distillation representation tensor and output a fixed-length embedding vector as the current stage distillation representation.

5. The method for accurate identification based on distillation knowledge transfer using few-sample self-learning, as described in claim 1, is characterized in that... S4 specifically includes: S41. On the target domain validation samples, calculate the recognition accuracy of the student model in the prediction task, the generalization error index based on unconfidence samples, and the entropy-driven loss value driven by information entropy, and construct the dual difference tensor based on the numerical differences between the pairs of the three types of sub-targets. S42. Feed the dual difference tensor into the normalization function and softmax activation module to obtain the sub-target response weight vector, and construct a composite fitness function by combining the three basic indices: ； in, Represents the composite fitness function. This represents the recognition accuracy of the student model on the validation samples in the target domain. This represents the generalization error estimate of the student model in the target domain based on confidence fluctuations. This represents the average information entropy driving the loss value of the student model along the distillation path. , , This represents the sub-objective response weight coefficient obtained from the objective duality difference. , , This represents the power-law adjustment factor corresponding to each of the three target sub-items, used to adjust the nonlinear sensitivity of each sub-item; S43. Divide the current round's stage distillation representation into two dimensions: time and channel, to obtain a multi-scale distillation representation sub-tensor sequence. Each sub-tensor is used to reflect the semantic information of distillation behavior at different scales. S44. Input each distillation tensor into the perturbation generation function to construct the perturbation control kernel at the corresponding scale. Each perturbation control kernel records the response amplitude, directionality and distribution state of the distillation behavior at the corresponding scale. S45. Apply asymmetric perturbation paths to the parameter prototype set one by one based on the perturbation control check to generate a set of parameter individuals with heterogeneous perturbation structures. S46. Load each parameter individual in the parameter individual set into the student model to perform the test prediction task, and calculate its performance evaluation score by combining the composite fitness function to construct the initial search state of the secretary bird optimization algorithm.

6. The few-sample self-learning accurate identification method based on distillation knowledge transfer according to claim 5, characterized in that, S44 specifically includes: S441. For each distillation sub-tensor, calculate the normalized correlation coefficient between the feature maps of adjacent channels according to the channel dimension, and select several highly correlated channel combinations as the main direction semantic slice based on the correlation coefficient sorting. S442. For each main direction semantic slice, calculate the amplitude difference of adjacent pixels along its horizontal and vertical directions respectively, and sum the absolute values ​​of the amplitude differences to generate perturbation distribution maps in the horizontal and vertical directions. S443. Calculate the first and third order statistical eigenvalues ​​of the disturbance distribution map in the spatial domain, and normalize the statistical eigenvalues ​​in each direction to form a direction weight vector. S444. Map the direction weight vector to the main direction semantic slice according to the corresponding main direction, and perform bit-by-bit scaling on the elements in each slice to generate a direction guidance perturbation slice. S445. The perturbation slices in each direction are spliced ​​together in a preset scale sequence, and scale adjustment factors that decrease or increase are applied according to the position of the slices in the sequence to obtain a multi-scale perturbation control kernel with multi-scale response characteristics. S446. Perform interval scaling and upper and lower limit pruning on the multi-scale disturbance control kernel according to the channel dimension, map all channels to a unified amplitude range, and generate the final disturbance control kernel.

7. The method for accurate identification based on distillation knowledge transfer using small-sample self-learning, as described in claim 5, is characterized in that... Specifically, S45 includes: S451. Based on the disturbance control kernel, extract the prototype vector of each parameter in time order, and calculate the rate of change of prediction error of each parameter in the most recent several iterations. Use the ratio of error change to the error of the previous iteration to represent the parameter sensitivity. S452. Arrange the parameter sensitivity in descending order and divide it into high sensitivity segment, medium sensitivity segment and low sensitivity segment based on the preset quantile threshold. Each sensitivity segment corresponds to a set of disturbance amplitude range and direction offset coefficient. S453. For each sensitive segment, according to the corresponding perturbation amplitude range and directional offset coefficient, different step increments are set for the positive and negative directions respectively, and an intra-segment perturbation offset sequence is generated. S454. The perturbation offset sequences of each sensitive segment are superimposed on the corresponding parameter positions of the prototype vector in order of sensitivity from high to low to form the first set of individual parameters, and the maximum perturbation amplitude and step size increment of each parameter are recorded. S455. Calculate the behavioral differences of the first set of individual parameters under the fitness function in the previous round, and adjust the perturbation amplitude range and step size increment of each sensitive segment according to the positive and negative feedback of the behavioral differences to generate an optimized perturbation configuration. S456. Regenerate the perturbation offset sequence with optimized perturbation configuration, and apply it to the prototype vector in the same way as in step S454 to form a second batch of structurally heterogeneous parameter individual sets. S457. The first and second batches of parameter individual sets and their corresponding perturbation configuration data are summarized as the final parameter individual set with a multi-level asymmetric perturbation structure.

8. The few-sample self-learning accurate identification method based on distillation knowledge transfer according to claim 1, characterized in that, S5 specifically includes: S51. At the beginning of each iteration of the Secretary Bird optimization algorithm, the student model's per-sample recognition error on the target domain verification samples is collected, and the error distribution of each parameter vector in the candidate parameter set in the previous round and the current round is divided into percentiles to form high error cluster, medium error cluster and low error cluster. At the same time, the absolute change amplitude of each element of the dynamic distillation weight vector is calculated and the corresponding mean is obtained as the weight fluctuation intensity. S52. Based on the sample proportions and weight fluctuation intensity of high-error clusters, medium-error clusters and low-error clusters, determine the corresponding global exploration step size compression coefficient and local utilization step size amplification coefficient, and generate a multi-stage step size configuration sequence. S53. For each parameter vector in the candidate parameter set, according to the multi-stage step size configuration sequence of its error cluster, construct a global exploration offset vector and a local utilization offset vector in turn. The global exploration offset vector is formed by weighting the difference between the parameter vector and the best vector in the same cluster, and the local utilization offset vector is generated by scaling the gradient sensitivity of the parameter vector in the historical iteration. The two offset vectors are superimposed on the original parameter vector step by step to form a new generation of candidate parameter set. S54. Load the new generation of candidate parameters into the student model one by one, calculate the global error convergence rate and local model diversity index corresponding to each parameter vector, and dynamically adjust the combined weight of the two according to the iteration round to generate the final fitness score. S55. Based on the fitness score and the similarity threshold between parameter vectors, perform similarity clustering, and select the parameter vector with the highest fitness in each cluster to form the next round of candidate set. S56. When the number of clusters is lower than the preset cluster threshold, the error convergence rate gain in the most recent rounds is lower than the preset gain threshold, or the maximum number of iterations is reached, the iteration is terminated, and all parameter vectors in the current candidate set are taken as the optimal model parameter set.

9. The few-sample self-learning accurate identification method based on distillation knowledge transfer according to claim 1, characterized in that, S6 specifically includes: S61. Load the optimal model parameter set into the student model, fix the model weights except for the output layer, and set the model to callback prediction mode. S62. For the target domain validation samples, extract the multi-scale feature maps and the final prediction probability vector of the student model at a specified depth, and apply spatial pyramid pooling to each feature map to unify the feature resolution, thus forming the current model recognition state. S63. From the stored historical stage distillation representation sequence, apply exponential decay weighting to the representations of the most recent rounds according to the round index and then sum them to generate an aggregated historical representation tensor. S64. For the multi-scale feature map and aggregated history representation tensor of the current model, calculate the in-channel covariance at each scale and in each channel and normalize it to obtain the channel alignment coefficient matrix. S65. Multiply the channel alignment coefficient matrix with the corresponding multi-scale feature map channel by channel, and merge the weighted features of all scales in the order of scale hierarchy using an interpolation fusion strategy to obtain the coupled fused feature map. S66. Perform linear projection on the coupled fusion feature map according to the channel dimension and perform layer normalization according to the historical variance of each channel, and output a fixed-length current round coupled correction representation.

10. The few-sample self-learning accurate identification method based on distillation knowledge transfer according to claim 1, characterized in that, Specifically, S7 includes: S71. Combine the current round of coupled correction representation with the multi-scale feature representation of newly added samples in the target domain dimension by dimension to construct the semantic alignment tensor of coupled samples, and calculate the alignment strength vector between each newly added sample and the coupled representation to obtain the coupling sample affinity result. S72. Input the coupling sample affinity result and the predicted probability distribution of the new sample into the distillation activation function in sequence, extract the initial weight bias of the function on each category channel, and generate multi-level distillation weight modulation coefficients. S73. Combining the multi-stage distillation weight modulation coefficients with the historical update trajectories of each parameter in the optimal model parameter set, calculate the incremental modulation vector of each parameter, and apply the incremental modulation vector to the optimal model parameter set to obtain the preliminary rearranged model parameter set. S74. Load the initially rearranged set of model parameters into the student model, and perform cross-distillation fine-tuning training with the newly added samples in the target domain in batches. After each fine-tuning, collect the fluctuation of the affinity of the coupled samples and the change of the distillation weight in real time to dynamically adjust the fine-tuning learning rate and the distillation weight allocation, and form the set of model parameters after fine-tuning. S75. Based on the stability index of the fine-tuned model parameter set on the time series curve of the affinity of the coupled samples, compare it with the preset stability threshold. When the stability index meets the threshold, determine to terminate the fine-tuning and output the candidate stable model parameter set. S76. Load the candidate stable model parameter set into the student model and perform batch coupling consistency verification on unknown samples in the target domain. When the coupling activation consistency reaches the preset standard, the target domain adaptive recognition model is finally generated and output.

Images