Prototype network-based mirror feature enhancement training method and device

Abstract

This invention discloses a mirror feature enhancement training method and apparatus based on a prototype network, relating to the field of intelligent signal feature analysis and recognition technology. The method includes: splitting the end-to-end deep learning model to be enhanced into a generalized encoding layer, a generalized decoding layer, and the original loss function; constructing a mirror prototype network training framework with left and right branches, where the left branch has the same structure as the original model, and the right branch consists of an isomorphic encoder and a distance calculation layer; optimizing the framework through a combination loss function; and completing feature enhancement training using an alternating fixed branch weight strategy. This invention does not require significant modifications to the original model, allows the model to acquire more feature data, and the framework is pluggable to adapt to automated training frameworks, is compatible with multiple models, has strong adaptability, and can be extended to other small-sample recognition scenarios for data processing.

Description

Technical Field

[0001] This invention relates to the field of intelligent signal analysis and recognition technology, specifically to a mirror feature enhancement training method and apparatus based on a prototype network. Background Technology

[0002] In the field of intelligent analysis and recognition of communication signal features, the industry generally uses various end-to-end deep learning models to complete signal recognition tasks. The working principle of these models is essentially to map signal data from the original space to a high-dimensional space, and then transform it to a low-dimensional space through linear mapping, and finally directly output the classification result to which the data belongs.

[0003] In practical applications, communication signal data often has problems such as multiple categories, long data points, and extremely unbalanced sample size. During training, the sample categories of large sample sizes and long data will excessively affect the model's weight adjustment, resulting in a significant reduction in the model's recognition performance of the categories of small sample data, and the recognition accuracy is difficult to meet the needs of practical applications.

[0004] Deep learning models for analyzing and recognizing specific communication signals are often designed with specialized network structures by technical professionals. Improving small-sample recognition performance by modifying the model's structure would incur extremely high R&D costs and would be difficult to adapt to different signal recognition scenarios. In addition to modifying the model structure, related technologies can also improve the training quality or number of training iterations for small-sample data through data augmentation and resampling to enhance the model's recognition performance on small-sample data. However, these methods only optimize at the data level and do not address the fundamental need for model feature extraction, resulting in limited optimization effects and the same problem of poor scenario adaptability.

[0005] Therefore, it is urgent to find technical means to preserve the original model structure without making significant changes to the model, while effectively improving the small sample recognition performance of the end-to-end model in the field of intelligent analysis of communication signals, and to have engineering and plug-in implementation capabilities, in order to solve the technical problem of low recognition accuracy caused by extreme sample imbalance. Summary of the Invention

[0006] In view of the above problems, the present invention provides a mirror feature enhancement training method and device based on prototype networks. It utilizes the advantages of prototype networks in small sample recognition to improve the performance of the original model, while being compatible with various end-to-end intelligent signal recognition models. It can be easily deployed in an engineering and plug-in manner, automating the performance enhancement of intelligent models in small sample scenarios, reducing model modifications while allowing the model to acquire more feature data, and improving the model's adaptability and suitability.

[0007] In a first aspect, embodiments of the present invention provide a mirror feature enhancement training method based on a prototype network, comprising the following steps: S1. Model splitting step: The end-to-end deep learning model M to be enhanced is split into a generalized encoding layer, a generalized decoding layer and the original loss function; wherein, the generalized encoding layer is the model layer in model M that realizes the mapping of data from the original space to the high-dimensional semantic space, and its connection structure remains unchanged; the generalized decoding layer is the model layer in model M that realizes the mapping of high-dimensional semantic features to the classification results, and its connection structure remains unchanged. S2. Framework Construction Steps: Construct a mirror prototype network training framework, which includes a left-side original network branch and a right-side prototype network branch with consistent structures. The left-side original network branch includes an encoder and a decoder connected in sequence, whose structures are consistent with the generalized encoding layer and generalized decoding layer in the model splitting step, respectively. The right-side prototype network branch includes an encoder and a distance calculation layer. The structure of the encoder is consistent with the generalized encoding layer in the model splitting step. The distance calculation layer is used to calculate the distance between the features output by the encoder of this branch and various prototypes to obtain the classification results. The mirror prototype network training framework is optimized using a combined loss function, which is expressed as: LOSS0 + LOSS1 - LOSS2; where LOSS0 is the classification loss of the original left network branch, calculated in the same way as the original loss function in step S1; LOSS1 is the distance loss, defined as the Euclidean distance between the sample features and their respective class prototypes; and LOSS2 is the similarity loss, defined as the Euclidean distance between the high-dimensional information output by the encoder of the original left network branch and the high-dimensional information output by the encoder of the prototype right network branch. Before performing distance calculation, the distance calculation layer of the right prototype network branch first divides the input data into support sets and query sets and calculates class prototypes, and then completes the distance calculation and outputs the classification results based on the class prototypes. S3, Feature Enhancement Training Step, which employs an alternating training strategy, includes: S31. Fix the model weights of the right prototype network branch and train the model weights of the remaining parts of the framework until convergence. S32. Fix the model weights of the original network branch on the left, and train the model weights of the rest of the framework until convergence. S33. After alternately repeating steps S31 and S32 for the preset number of rounds, release all weight fixation and train the entire framework until convergence; the preset number of rounds is an empirical value K, which is determined based on the actual convergence effect of the model training. S34. Repeat steps S31 to S33 until the model performance meets the early stopping condition and training is complete.

[0008] Furthermore, the structure definition of the distance calculation layer can be flexibly adjusted according to actual data conditions and the actual needs of model feature extraction, so as to adapt to different communication signal data characteristics and recognition requirements.

[0009] Furthermore, the method for calculating the distance between sample features and their class prototype in LOSS1 can be replaced by Manhattan distance calculation or Mahalanobis distance calculation, which can be flexibly selected according to the distance calculation requirements of the actual scenario.

[0010] Furthermore, the mirror prototype network training framework can be instantiated as a plugin to adapt to automated model training frameworks, enabling model adjustment and performance enhancement in various micro-model training application scenarios.

[0011] Furthermore, the end-to-end deep learning model M to be enhanced can be replaced by a traditional training model, a hierarchical network, or a YOLO network. The training method of the present invention has good adaptability to the above models and can effectively enhance the small sample features of various models.

[0012] The mirror feature enhancement training method based on prototype networks of the present invention can be extended to other data processing fields, in addition to application in communication signal feature processing scenarios, to sample imbalance and small sample recognition scenarios.

[0013] Secondly, this application provides a mirror feature enhancement training device based on a prototype network; the device includes a model splitting module, a framework construction module, and a feature enhancement training module, and the modules work together to complete the feature enhancement training of the model to be enhanced, thereby improving the model's small sample recognition performance; Model splitting module: used to split the end-to-end deep learning model M to be enhanced into a generalized encoding layer, a generalized decoding layer and the original loss function; wherein, the generalized encoding layer is the model layer in model M that realizes the mapping of data from the original space to the high-dimensional semantic space, and its connection structure remains unchanged; the generalized decoding layer is the model layer in model M that realizes the mapping of high-dimensional semantic features to the classification results, and its connection structure remains unchanged. Framework building module: Used to build a mirror prototype network training framework, which includes a left original network branch and a right prototype network branch with consistent structure; the left original network branch includes an encoder and a decoder connected in sequence, whose structures are consistent with the generalized encoding layer and generalized decoding layer obtained by the model splitting module, respectively; the right prototype network branch includes an encoder and a distance calculation layer, the structure of which is consistent with the generalized encoding layer obtained by the model splitting module, and the distance calculation layer is used to calculate the distance between the features output by the encoder of this branch and various prototypes to obtain the classification results; The framework construction module is also used to define a combined loss function for the training framework of the mirror prototype network. The combined loss function is expressed as: LOSS0 + LOSS1 - LOSS2; where LOSS0 is the classification loss of the original network branch on the left, and its calculation method is consistent with the original loss function obtained by the model splitting module; LOSS1 is the distance loss, defined as the Euclidean distance between the sample features and their respective class prototypes; LOSS2 is the similarity loss, defined as the Euclidean distance between the high-dimensional information output by the encoder of the original network branch on the left and the high-dimensional information output by the encoder of the prototype network branch on the right. The distance calculation layer in the framework construction module first divides the input data into support sets and query sets and calculates class prototypes before performing distance calculation. Then, it completes the distance calculation and outputs the classification results based on the class prototypes. Feature enhancement training module: used to perform feature enhancement training on the mirror prototype network training framework using an alternating training strategy. The training process of the feature enhancement training module includes: S31. Fix the model weights of the right prototype network branch and train the model weights of the remaining parts of the framework until convergence. S32. Fix the model weights of the original network branch on the left, and train the model weights of the rest of the framework until convergence. S33. After alternately repeating steps S31 and S32 for the preset number of rounds, release all weight fixation and train the entire framework until convergence; the preset number of rounds is an empirical value K, which is determined based on the actual convergence effect of the model training. S34. Repeat steps S31 to S33 until the model performance meets the early stopping condition and training is complete.

[0014] Furthermore, the distance calculation layer structure definition in the framework construction module can be adjusted according to the actual data conditions and the actual needs of model feature extraction; the distance calculation method in LOSS1 can be replaced by Manhattan distance calculation method or Mahalanobis distance calculation method.

[0015] Furthermore, the mirror prototype network training framework built by the framework building module can be instantiated as a plugin to adapt to the model training automation framework.

[0016] Furthermore, the model to be enhanced processed by the model splitting module can be replaced with a traditional training model, a hierarchical network, or a YOLO network, in addition to the end-to-end deep learning model M.

[0017] The mirror feature enhancement training method and apparatus based on a prototype network provided in this invention have the following beneficial effects: 1. This application does not require modification of the core structure of the original end-to-end model. It only achieves feature enhancement by splitting the model and building a mirror prototype network training framework, which greatly reduces the technical cost of model modification. The mirror prototype network modifies the decoding layer into a distance calculation layer. During training, it causes the overall weight of the right prototype network branch to shift from the left original network branch. By calculating and optimizing the combined loss function, under the premise of consistent encoding structure, the left and right branches encode different data feature information. The feature extraction mode of the model is controlled by the result, so that the model can obtain more effective feature data in small sample scenarios, improve the recognition accuracy, reduce model modification, and obtain more feature data.

[0018] 2. The distance calculation layer structure in this application can be flexibly adjusted according to actual data conditions and feature extraction requirements. At the same time, the distance calculation method in the loss function can be replaced with Manhattan distance, Mahalanobis distance, etc., to adapt to different communication signal data scenarios. The model to be enhanced is not limited to end-to-end deep learning models, but can also be replaced with traditional training models, hierarchical networks, YOLO networks, etc., with a wide range of applications and better adaptability and suitability.

[0019] 3. The mirror prototype network training framework in this application can be instantiated as a plugin, which can perfectly adapt to the model training automation framework. Without additional customized development, it can quickly realize model adjustment and performance enhancement in various small model training application scenarios, realize the automation of performance enhancement of intelligent models in small sample scenarios, and improve the efficiency of engineering applications.

[0020] 4. This application adopts an alternating fixed weight training strategy. First, different branches of the framework are trained separately, and then the whole framework is trained and optimized. Through multiple alternating convergence and early stopping mechanisms, the model can fully learn the features of small sample data during the training process, effectively solving the problem that large sample categories have an excessive influence on the model weight adjustment, greatly improving the model's recognition effect on small sample data, and the training process has good convergence and stable model performance.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] The invention will now be described in more detail with reference to embodiments and the accompanying drawings.

[0023] Figure 1 The diagram illustrates the workflow of a prototype network-based mirror feature enhancement training method proposed in one embodiment of the present invention. Figure 2A schematic diagram of an exemplary mirror prototype network training framework proposed in one embodiment of the present invention is shown. Figure 3 The diagram shows a structural block diagram of a prototype network-based mirror feature enhancement training device proposed in one embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0025] This embodiment provides a mirror feature enhancement training method based on a prototype network, applied to the scenario of extremely imbalanced small sample size in intelligent analysis and recognition of communication signal features. The model to be enhanced is an end-to-end deep learning model M in the field of communication signal recognition. The specific implementation steps are as follows: The recognition scenario is constructed, and the learning model under few-shot conditions is split into three parts according to rules. Assuming the existing scenario is an existing end-to-end deep learning model M for few-shot recognition, M is split into three parts according to the generalized encoding layer, generalized decoding layer, and loss function: the generalized encoding layer E_LAYERS consists of those model layers in M ​​that map the data from the original space to the high-dimensional semantic space, and their connection structure remains unchanged; the generalized decoding layer D_LAYERS consists of those model layers in M ​​that map the high-dimensional semantic features to the output classification result, and their connection structure also remains unchanged; the loss function LOSS remains the original loss function of M.

[0026] See Figure 1 As shown, it includes S1, model splitting step: splitting the end-to-end deep learning model M to be enhanced into a generalized encoding layer, a generalized decoding layer and the original loss function; wherein, the generalized encoding layer is the model layer in model M that realizes the mapping of data from the original space to the high-dimensional semantic space, and its connection structure remains unchanged; the generalized decoding layer is the model layer in model M that realizes the mapping of high-dimensional semantic features to the classification result, and its connection structure remains unchanged. S2. Framework Construction Steps: Construct a mirror prototype network training framework, which includes a left-side original network branch and a right-side prototype network branch with consistent structures. The left-side original network branch includes an encoder and a decoder connected in sequence, with structures consistent with the generalized encoding layer and generalized decoding layer in the model splitting step, respectively. The right-side prototype network branch includes an encoder and a distance calculation layer. The encoder has the same structure as the generalized encoding layer in the model splitting step. The distance calculation layer is used to calculate the distance between the features output by the encoder of this branch and various prototypes to obtain the classification results.

[0027] See Figure 2 As shown, the mirror prototype network framework structure for model training is constructed as follows: The original network on the left has E_LAYER0 and D_LAYER0 with the same structure as E_LAYER and D_LAYER. E_LAYER0 calculates the high-dimensional information 0, and D_LAYER0 decodes it to output the original network classification result 0. Its data flow is consistent with the original network training process. The E_LAYER1 on the right has the same structure as E_LAYER. Euclid_LAYER calculates the distance between the high-dimensional information 1 output by E_LAYER1 and the class prototype and gives the distance classification result 1. Its data flow conforms to the standard prototype network model, that is, the input data is divided into support set and query set and the prototype is calculated to give the distance classification result. The overall loss function generated by the framework consists of three parts: LOSS0 is the original model loss function, which is consistent with the LOSS calculation method; LOSS1 is the distance loss function, which is defined as the Euclidean distance between the sample and the class prototype; LOSS2 is the similarity loss function, which is defined as the Euclidean distance between the high-dimensional information 0 and the high-dimensional information 1; the final loss function FINAL_LOSS is LOSS0, LOSS1, and -1. The sum of LOSS2.

[0028] Finally, new model feature enhancement training is conducted. The new model training consists of four steps: 1. Fix the weights of the mirror prototype network part of the model, and train the weights of the other parts of the model until convergence; 2. Fix some weights of the original model network and train the other parts of the model weights until convergence; 3. Repeat steps 1 and 2 until each converges K times (K is an empirical value), then stop fixing the model weights and train until convergence; 4. Repeat steps 1, 2, and 3 until the final model performance meets the early stopping condition, and complete the model training.

[0029] In practical applications, this application considers that because the mirror prototype network modifies the decoding layer to an Euclidean distance encoding layer, the overall weights of the right mirror prototype network shift from the left original network during training. By calculating and optimizing the differences in the outputs of each layer of the model, such as the encoding layer mapping information and the decoding classification result information, the network structures on both sides are guaranteed to encode different data features while maintaining a consistent encoding structure. This allows for result-oriented control of the model's feature extraction mode with minimal modifications to the model structure, achieving reduced model modifications and the acquisition of more data features. Furthermore, the structural definition of this model layer can be flexibly modified according to actual conditions to adapt to different model feature extraction needs under real-world data conditions and effectively control them. It can also be easily instantiated as a plugin, facilitating model adjustment and enhancement in various small-scale model training application scenarios while being compatible with automated model training frameworks.

[0030] It should be noted that the distance calculation method of the distance calculation layer in this invention can be replaced with any reasonable distance calculation method such as Manhattan distance or Mahalanobis distance as needed; the empirical value K can be flexibly adjusted according to different model types and data scales, and the convergence effect of actual training is the basis for judgment. In addition to being applied to the field of intelligent analysis and recognition of communication signal features, the method and apparatus of this invention can also be extended to other data processing fields such as image recognition, speech recognition, and text classification, to identify imbalanced small sample scenarios. The plug-in form of the mirror prototype network training framework can be adapted to different model training automation frameworks for development, enabling cross-platform deployment.

[0031] See Figure 3 , Figure 3 This application provides a mirror feature enhancement training device 300 based on a prototype network. The device includes a model splitting module 310, a framework construction module 320, and a feature enhancement training module 330. Each module can be implemented based on computer program code and deployed on a model training server. Its specific functions are as follows: The model splitting module 310 is used to split the end-to-end deep learning model M to be enhanced into a generalized encoding layer, a generalized decoding layer, and the original loss function. The generalized encoding layer is the model layer in model M that maps data from the original space to the high-dimensional semantic space, and its connection structure remains unchanged. The generalized decoding layer is the model layer in model M that maps high-dimensional semantic features to classification results, and its connection structure remains unchanged.

[0032] For example, the system receives the communication signal recognition model to be enhanced (end-to-end deep learning model / YOLO network / traditional training model, etc.), performs model splitting according to the rules of generalized coding layer, generalized decoding layer, and original loss function, and keeps the connection structure of generalized coding layer and generalized decoding layer unchanged, and synchronizes the splitting results to the framework construction module.

[0033] The framework building module 320 is used to build a mirror prototype network training framework, which includes a left original network branch and a right prototype network branch with consistent structure. The left original network branch includes an encoder and a decoder connected in sequence, whose structures are consistent with the generalized encoding layer and generalized decoding layer in the model splitting step, respectively. The right prototype network branch includes an encoder and a distance calculation layer. The encoder has the same structure as the generalized encoding layer in the model splitting step. The distance calculation layer is used to calculate the distance between the features output by the encoder of this branch and the prototypes of various types to obtain the classification results.

[0034] For example, based on the splitting results of the model splitting module, a mirror prototype network training framework is constructed, generating a left original network branch and a right prototype network branch with consistent structure. The combined loss function is defined and the relevant parameters of the distance calculation layer are configured (including distance calculation method, support set / query set partitioning rules, etc.). At the same time, the framework can be instantiated as a plugin as needed to adapt to external model training automation frameworks, and the constructed framework is synchronized to the feature enhancement training module.

[0035] The feature enhancement training module 330 can be used to load the mirror prototype network training framework of the framework building module, execute the training process according to the alternating training strategy, set the empirical value K and early stopping conditions, and automatically complete steps such as weight fixing, alternating convergence, and overall training. During the training process, the model performance is monitored in real time, and the training is automatically stopped when the early stopping conditions are met, and the enhanced model that has been trained is output.

[0036] The device in this embodiment can automate the training of small sample feature enhancement for communication signal recognition models. Each module is independent yet works in concert, and the module parameters can be flexibly adjusted according to actual scenario requirements to adapt to different models and data scenarios.

[0037] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for mirror feature enhancement training based on a prototype network, characterized by The method includes: S1. Model splitting step: The end-to-end deep learning model M to be enhanced is split into a generalized encoding layer, a generalized decoding layer and the original loss function; wherein, the generalized encoding layer is the model layer in model M that realizes the mapping of data from the original space to the high-dimensional semantic space, and its connection structure remains unchanged; the generalized decoding layer is the model layer in model M that realizes the mapping of high-dimensional semantic features to the classification results, and its connection structure remains unchanged. S2. Framework Construction Steps: Construct a mirror prototype network training framework, which includes a left-side original network branch and a right-side prototype network branch with consistent structures. The left-side original network branch includes an encoder and a decoder connected in sequence, whose structures are consistent with the generalized encoding layer and generalized decoding layer in the model splitting step, respectively. The right-side prototype network branch includes an encoder and a distance calculation layer. The structure of the encoder is consistent with the generalized encoding layer in the model splitting step. The distance calculation layer is used to calculate the distance between the features output by the encoder of this branch and various prototypes to obtain classification results.

2. The mirror feature enhancement training method based on a prototype network according to claim 1, characterized in that, In the framework construction step, the mirror prototype network training framework is optimized by combining loss functions, the expression of which is: LOSS0+LOSS1-LOSS2; Wherein, LOSS0 is the classification loss of the original network branch on the left, and its calculation method is the same as that of the original loss function in step S1; LOSS1 is the distance loss, defined as the Euclidean distance between a sample feature and its class prototype. LOSS2 is a similarity loss, defined as the Euclidean distance between the high-dimensional information output by the encoder of the original left network branch and the high-dimensional information output by the encoder of the prototype right network branch.

3. A mirror feature enhancement training method based on a prototype network according to claim 1 or 2, characterized in that, The method further includes: S3, a feature enhancement training step, which employs an alternating training strategy, including: S31. Fix the model weights of the right prototype network branch and train the model weights of the remaining parts of the framework until convergence. S32. Fix the model weights of the original network branch on the left, and train the model weights of the rest of the framework until convergence. S33. After alternately repeating steps S31 and S32 for the preset number of rounds, release all weight fixation and train the entire framework until convergence. S34. Repeat steps S31 to S34 until the model performance meets the early stopping condition and training is complete.

4. The mirror feature enhancement training method based on a prototype network according to claim 1, characterized in that, Before performing distance calculation, the distance calculation layer of the right prototype network branch first divides the input data into support sets and query sets and calculates class prototypes, and then completes the distance calculation and outputs the classification results based on the class prototypes.

5. The mirror feature enhancement training method based on a prototype network according to claim 3, characterized in that, The preset number of rounds mentioned in step S33 is an empirical value K, which is determined based on the actual convergence effect of model training.

6. The mirror feature enhancement training method based on a prototype network according to claim 2, characterized in that, The structure definition of the distance calculation layer is adjusted according to the actual data conditions and the actual needs of model feature extraction.

7. The mirror feature enhancement training method based on a prototype network according to claim 2, characterized in that, The method for calculating the distance between sample features and their class prototype in LOSS1 can be replaced by the Manhattan distance calculation method or the Mahalanobis distance calculation method.

8. The mirror feature enhancement training method based on a prototype network according to claim 1, characterized in that, The mirror prototype network training framework includes instantiated plugins to adapt to automated model training frameworks.

9. The mirror feature enhancement training method based on a prototype network according to claim 2, characterized in that, The end-to-end deep learning model M to be enhanced can be replaced by a traditional training model, a hierarchical network, or a YOLO network.

10. A mirror feature enhancement training device based on a prototype network, characterized in that, The device includes: The model splitting module is used to split the end-to-end deep learning model M to be enhanced into a generalized encoding layer, a generalized decoding layer, and the original loss function; wherein, the generalized encoding layer is the model layer in model M that realizes the mapping of data from the original space to the high-dimensional semantic space, and its connection structure remains unchanged; the generalized decoding layer is the model layer in model M that realizes the mapping of high-dimensional semantic features to the classification results, and its connection structure remains unchanged. The framework building module is used to construct a mirror prototype network training framework, which includes a left-side original network branch and a right-side prototype network branch with consistent structures. The left-side original network branch includes an encoder and a decoder connected in sequence, whose structures are consistent with the generalized encoding layer and generalized decoding layer in the model splitting step, respectively. The right-side prototype network branch includes an encoder and a distance calculation layer. The encoder has the same structure as the generalized encoding layer in the model splitting step. The distance calculation layer is used to calculate the distance between the features output by the encoder of this branch and various prototypes to obtain the classification results.

Images