Feature extraction model optimization method and device, and electronic equipment

Abstract

Embodiments of the present application relate to a feature extraction model optimization method and device, and electronic equipment. The method comprises: obtaining a plurality of teacher network models; extracting a backbone network in each teacher network model; using each backbone network to extract a first high-dimensional image feature in a training image; fusing a plurality of first high-dimensional image features of the training image to obtain a fused feature; reducing the dimension of the fused feature to obtain a reduced feature; inputting the training image into a student network model to obtain a second high-dimensional image feature of the training image; and optimizing the student network model according to the reduced feature and the second high-dimensional image feature. The student network model optimized by the present application can improve the prediction speed of the model without increasing the inference cost.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a feature extraction model optimization method, device and electronic device. Background Art

[0002] With the development and advancement of artificial intelligence (AI), feature extraction model optimization techniques have enabled tracking, matching, and identification of target individuals across time and space, and are widely used in all aspects of social life. Feature extraction model optimization has been a research hotspot in computer vision in recent years. Currently, a common approach involves using deep learning models to learn relationships between human bodies to extract features and conduct searches. Extracting features from multiple models and then combining them for search can significantly improve model accuracy. However, using multiple models for prediction results in significant additional resource consumption. This is especially true when there are many models, as the corresponding model predictions require a high computational load. This not only reduces model prediction speed but also increases inference costs. Summary of the Invention

[0003] The present application provides a feature extraction model optimization method, device and electronic device to solve the technical problems in the prior art that multi-model feature extraction results in a large amount of calculation, reduces the model prediction rate, and increases the inference cost.

[0004] In a first aspect, the present application provides a feature extraction model optimization method, the method comprising:

[0005] Obtain multiple teacher network models;

[0006] Extract the backbone network from each teacher network model;

[0007] Utilize each backbone network to extract the first high-dimensional image features in the training image;

[0008] Fusing multiple first high-dimensional image features of the training image to obtain a fused feature;

[0009] Reduce the dimension of the fused features to obtain the reduced dimension features;

[0010] Input the training image into the student network model to obtain the second highest dimensional image features of the training image;

[0011] The student network model is optimized based on the dimensionality reduction features and the second high-dimensional image features.

[0012] In an optional embodiment, after extracting the first high-dimensional image features from the training image using each backbone network, the method further includes:

[0013] Normalization processing is performed on the first high-dimensional image feature so that the normalized high-dimensional image features of the same training image can be subsequently fused to obtain fused features.

[0014] In a second aspect, the present application provides a multi-feature extraction model optimization device, which includes:

[0015] A processing module for obtaining multiple teacher network models;

[0016] An extraction module is used to extract the backbone network in each teacher network model; using each backbone network, the first high-dimensional image features in the training image are extracted respectively;

[0017] A fusion module, configured to fuse multiple first high-dimensional image features of a training image to obtain a fusion feature;

[0018] Dimensionality reduction module, used to reduce the dimension of fusion features and obtain dimensionality reduction features;

[0019] The processing module is further used to input the training image into the student network model to obtain the second high-dimensional image features of the training image;

[0020] The optimization module optimizes the student network model based on the dimensionality reduction features and the second high-dimensional image features.

[0021] In an optional embodiment, the processing module is further configured to perform normalization processing on the first high-dimensional image features so as to subsequently fuse the normalized high-dimensional image features of the same training image to obtain fused features.

[0022] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0023] Memory for storing computer programs;

[0024] The processor is used to implement the steps of the feature extraction model optimization method of any embodiment of the first aspect when executing the program stored in the memory.

[0025] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the feature extraction model optimization method as in any embodiment of the first aspect are implemented.

[0026] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0027] The method provided in the embodiment of the present application obtains multiple teacher network models, then extracts the backbone network from each teacher network model. Using each backbone network, the first high-dimensional image features in the training image are extracted separately. The multiple first high-dimensional image features corresponding to the same training image are fused to obtain a fused feature. The fused feature integrates the output results of multiple teacher network models and is more comprehensive than the output results of a single teacher network model. The fused feature is a "super-high-dimensional" image feature that is k times the dimension of the first high-dimensional image feature. In order to subsequently reduce the amount of computation, it is necessary to perform dimensionality reduction processing on it. This is used to compress the data, eliminate redundancy, and extract the main components of the information. The training image is input into the student network model, and the second high-dimensional image features corresponding to the training image are obtained. The student network model is then optimized and trained based on the dimensionality reduction features and the second high-dimensional image features. Ultimately, the performance of the student network model can approach or even reach the effect of the integration of multiple teacher network models. However, the computational complexity and parameter count of a single student network model are of the order of magnitude of a single network model. The prediction speed of its model is close to the prediction speed of the integration of multiple teacher network models, but its inference cost is the inference cost of a single network model. Therefore, the student network model optimized in this application can improve the model prediction speed without increasing the inference cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic flow chart of a feature extraction model optimization method provided by an embodiment of the present invention;

[0029] Figure 2 A schematic diagram of the overall extraction process of image feature vectors from a human body base database in the traditional human weight recognition method provided by the present invention;

[0030] Figure 3 A schematic flow chart of another feature extraction model optimization method provided by an embodiment of the present invention;

[0031] Figure 4 A schematic flow chart of another feature extraction model optimization method provided by an embodiment of the present invention;

[0032] Figure 5 A flow chart of a method provided by the present invention for updating parameters in the network model by cross-selecting different loss functions according to the number of iterations of the current training;

[0033] Figure 6 A schematic diagram of the overall process of feature fusion using normalized high-dimensional image features provided by the present invention;

[0034] Figure 7 A schematic flow chart of another feature extraction model optimization method provided by an embodiment of the present invention;

[0035] Figure 8 A schematic flow chart of a method for calculating a loss function using the second high-dimensional image feature provided by the present invention and the dimensionality reduction feature, and completing parameter updates in the student network model;

[0036] Figure 9 A schematic diagram of the structure of a multi-feature extraction model optimization device provided by an embodiment of the present invention;

[0037] Figure 10 A schematic structural diagram of an electronic device is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0039] To facilitate understanding of the embodiments of the present invention, specific embodiments will be further explained below with reference to the accompanying drawings. The embodiments do not limit the embodiments of the present invention.

[0040] In response to the technical problems mentioned in the background technology, the present application embodiment provides a feature extraction model optimization method, see Figure 1 As shown, Figure 1 A flow chart of a feature extraction model optimization method provided by an embodiment of the present invention.

[0041] Before introducing the feature extraction model optimization method provided by the embodiment of the present invention, the application scenario of the feature extraction model is first introduced.

[0042] In a specific example, this can be applied to scenarios such as human weight recognition. Currently, common human weight recognition techniques use detection algorithms such as Yolo to detect human bodies from multiple frames of original video files, cropping and saving them to form a human body database. A feature extraction network is then used to extract feature vectors from all images in the cropped human body database. Each feature vector is represented as a multi-digit floating-point number and measured using metrics such as Euclidean distance or cosine distance. The database image with the closest feature vector to the search image is then recommended.

[0043] See Figure 2 As shown, Figure 2 The figure shows the overall extraction process of the image feature vector from the human body base database in the traditional human body weight recognition method. Specifically, it includes:

[0044] Step 210: The camera captures the original image.

[0045] Step 220: Perform human body detection and positioning on the original image.

[0046] Step 230: perform operations such as cropping on the image after human body detection and positioning, and then store it in a database.

[0047] Step 240: Use a feature extraction network to extract feature vectors from all images in the cropped human body base library and save them.

[0048] Among them, the feature vector of each image is represented by a multi-bit floating-point number.

[0049] When a new image requires human weight recognition, steps 210 and 220 are also performed. Then, operations such as cropping are performed on the image after human body detection and positioning. A feature extraction network is then used to extract feature vectors from the cropped human image. The Euclidean distance or cosine similarity is then calculated between the newly extracted feature vectors and the feature vectors of each image in the database. Based on the Euclidean distance or cosine similarity, the database image closest to the current image is recommended from the database.

[0050] Throughout the entire process, a deep learning model is used to learn the relationship between the human body to extract features and search for the closest image. As described in the background art, the method of extracting features through multiple models in deep learning and then splicing and searching can greatly improve the accuracy of the model, but using multiple models for prediction will result in a large amount of additional resource consumption. However, through the method of the embodiment of the present application, it is possible to optimize the feature model to reduce the amount of calculation and the magnitude of the number of parameters, while improving the model prediction speed without increasing the reasoning cost. After the method of the embodiment of the present application optimizes the feature extraction model, and after the optimized feature extraction model is applied to the application scenario of human weight recognition, it will be more convenient and quick to provide the demander with a base library image that is closer to the image to be searched in the base library.

[0051] Of course, the feature extraction model provided in the embodiments of the present invention can be applied not only to optimizing the feature extraction model used in human weight recognition technology but also to optimizing other feature extraction models, such as those used in face recognition technology. Alternatively, the feature extraction model can be optimized for other application scenarios. The specific application scenarios can be determined based on actual circumstances and are not limited here.

[0052] The following describes in detail a method for optimizing a feature extraction model provided by an embodiment of the present invention. The method comprises the following steps:

[0053] Step 110: Acquire multiple teacher network models.

[0054] Specifically, the teacher network model is a network model obtained after training for implementing image classification or image recognition, such as a neural network model.

[0055] Step 120: extract the backbone network in each teacher network model.

[0056] Specifically, each teacher network model includes at least a feature extraction layer and a classification layer (fully connected layer).

[0057] In this embodiment, the fully connected layer is removed from each teacher network model, and the remaining network model is defined as the backbone network. The reason for removing the fully connected layer is that it is mainly used to classify sample data based on image features. It does not participate in image feature extraction and is not very useful for this embodiment. In addition, the execution process consumes system resources and prolongs calculation time. Therefore, the fully connected layer is removed and only the backbone network is retained.

[0058] Similarly, the student network model described below also removes the fully connected layers in specific applications, retaining only the feature extraction network. This is because the human body class in the application scenario generally does not appear in the training samples, so only feature similarity comparison is required to determine the target image. This part will not be discussed in detail below.

[0059] In an optional specific example, a classic residual network can be selected as the backbone network.

[0060] Step 130: Utilize each backbone network to extract the first high-dimensional image features in the training image.

[0061] Specifically, after feature extraction of the training image, the features may be, for example, 216-dimensional vector features or 512-dimensional vector features. The specific dimensions of the vector features are not limited here.

[0062] Step 140: fuse multiple first high-dimensional image features of the training image to obtain a fused feature.

[0063] Specifically, considering that each teacher network model extracts a first high-dimensional image feature as described in step 130 for the same image, multiple teacher network models will generate multiple first high-dimensional image features. Fusion of the different first high-dimensional image features output by different teacher network models, i.e., integrating the outputs of different teacher network models, can ensure the comprehensiveness of the fused features.

[0064] Step 150: perform dimension reduction on the fused features to obtain dimension-reduced features.

[0065] In a specific example, assuming there are 100 teacher network models and the first high-dimensional image feature is a 512-dimensional feature vector, after fusing these 100 first high-dimensional image features, the resulting fused feature is a 512-dimensional feature vector multiplied by 100 dimensions, or a 51200-dimensional feature vector. Because this higher-dimensional feature vector inevitably leads to a significant increase in the final computational effort and parameters, which in turn reduces the model's prediction speed and increases in inference costs. Therefore, it is necessary to reduce the dimensionality of the fused features to obtain reduced-dimensionality features.

[0066] A specific dimensionality reduction method can be principal component analysis (PCA). PCA is one of the most important dimensionality reduction methods. It is widely used in data compression, redundancy elimination, and noise reduction. It uses an orthogonal transformation to linearly transform the observed values ​​of a series of potentially correlated variables, projecting them into a series of linearly uncorrelated values.

[0067] In this embodiment, the 51200-dimensional splicing fusion feature in the example listed above is reduced to 512 dimensions by principal component analysis and saved for use in the subsequent method steps described below. The specific execution steps of the principal component analysis method are referred to in the prior art and will not be described in detail here.

[0068] Step 160: Input the training image into the student network model to obtain the second high-dimensional image features of the training image.

[0069] Step 170: Optimize the student network model based on the dimensionality reduction features and the second high-dimensional image features.

[0070] Specifically, the student network is used to imitate the teacher network so that the performance of the student network can approach or even reach the performance of the teacher network, which is the idea of ​​supervised learning method.

[0071] Therefore, it is also necessary to input the training image into the student network model to obtain the second highest dimensional image features of the training image.

[0072] In an optional example, the student network model is any one of the multiple backbone network models mentioned above.

[0073] The dimensionality reduction features are used as the features of the teacher model, and the backbone network model is used as the student network model to distill the features of the teacher model.

[0074] That is, the dimensionality reduction features are compared with the second high-dimensional image features, and the difference between the two is used as the reverse input to optimize the student network model.

[0075] The resulting student network model has roughly the same computational complexity and parameter size as a single network model. However, its performance is significantly better than that of a single network model, as it approaches the ensemble of multiple teacher network models. Furthermore, because the computational complexity is significantly reduced compared to multiple teacher network models, prediction speed is not reduced, nor is inference cost increased. In other words, while model accuracy is improved, prediction speed and inference cost are not reduced.

[0076] The feature extraction model optimization method provided by an embodiment of the present invention obtains multiple teacher network models, then extracts the backbone network from each teacher network model. Using each backbone network, it extracts first-dimensional image features from training images. Multiple first-dimensional image features corresponding to the same training image are then fused to obtain a fused feature. This fused feature integrates the outputs of multiple teacher network models and is more comprehensive than the output of a single teacher network model. The fused feature is a "super-high-dimensional" image feature that is k times the dimension of the first-dimensional image feature. To reduce computational complexity, it undergoes dimensionality reduction. This compresses the data, eliminates redundancy, and extracts the key components of the information. The training image is input into a student network model to obtain second-dimensional image features corresponding to the training image. The student network model is then optimized and trained based on the dimensionality reduction features and the second-dimensional image features. Ultimately, the performance of the student network model approaches or even reaches the performance of an integrated model of multiple teacher network models. However, the computational complexity and parameter count of a single student network model are comparable to those of a single network model. While the prediction speed of this model approaches that of an integrated model of multiple teacher network models, the inference cost is that of a single network model. Therefore, the student network model optimized in this application can improve the model prediction speed without increasing the inference cost.

[0077] The present invention also provides another feature extraction model optimization method, see Figure 3 As shown, the same or similar contents as those in the above embodiment will not be repeated here. This embodiment focuses on explaining how to obtain multiple teacher network models.

[0078] See below for details, including:

[0079] Step 310: After obtaining the sample data, divide the sample data into multiple groups.

[0080] In step 320 , each of the multiple groups of sample data is used as a validation set, and the remaining sample data are used as training sets to form multiple training subsets.

[0081] In step 330, a network model is trained using each training subset, and multiple teacher network models are finally obtained.

[0082] Specifically, in the deep learning modeling process, it's common practice to divide data into training and test sets. The test set is independent of training data and is used solely for final model evaluation. During training, overfitting often occurs, meaning the model fits the training data well but fails to predict data outside the training set. Using test data to adjust model parameters at this point is equivalent to knowing only part of the test data during training, which can affect the accuracy of the final evaluation results.

[0083] A common practice is to set aside a portion of the training data as validation data to evaluate the model's training effectiveness. Validation data is taken from the training data but does not participate in training. This allows for a relatively objective assessment of the model's fit to data outside the training set. Cross-validation, also known as cyclic validation, is a common method for evaluating models on validation data. Specifically, the original data is divided into k groups, each subset is used as a validation set, and the remaining k-1 subsets are used as training sets. For example, if there are five data sets in total, when the first set is used as the validation set, the other four sets are used as training sets to train a network model, resulting in a teacher network model.

[0084] Similarly, for example, the second set of data from the five sets of data is used as the validation set, and the other four sets of data are used as the training set to train another network model and obtain another teacher network model. And so on, five teacher network models can be obtained.

[0085] Optionally, in another embodiment of the present invention, in order to increase the expressive power of the teacher network model, when training a certain network model and obtaining the teacher network model, the following operations may also be performed. Figure 4 As shown, the method steps include:

[0086] Step 410: When training the network model using the training data in the training subset, determine the number of iterations of the current training.

[0087] Step 420: Cross-select different loss functions to update the parameters in the network model according to the number of iterations of the current training.

[0088] That is, when the network model is trained using one set of training data from the current training subset, each iteration counts as one. If the network model is trained using k-1 sets of training data, the number of iterations can be determined to be k-1. Before each iteration, the number of iterations can be determined in advance. For example, if the current iteration is the fifth, the number of adoptions for the network model can be updated based on the loss function selected for the fifth iteration.

[0089] In a specific example, different loss functions can be selected to iteratively update the parameters in the network model according to whether the number of iterations is odd or even. Figure 5 shown. Figure 5 The article provides an example of how to update the parameters of a network model by cross-selecting different loss functions based on the number of iterations. The examples include the triplet loss function, the cross entropy loss function, and the Arcface loss function.

[0090] When the number of iterations is odd, you can choose the Arcface loss function and the cross entropy loss function to update the parameters in the network model. Because these two loss functions have no requirements on sample data, you can randomly select n images for training.

[0091] When the number of iterations is even, the triplet loss function and the cross entropy loss function are selected to update the parameters in the network model.

[0092] Given that the triplet loss function has certain sample data requirements, we need to select P individuals from the training set, each of whom selects m images as input sample data to calculate the loss. The network parameters are then updated based on these two losses. The specific process of calculating the loss and updating the network parameters can be implemented using existing technologies and will not be detailed here.

[0093] It should be noted that in this embodiment, both odd and even number of iterations are used. In actual applications, one or more loss functions can be selected. Moreover, the mechanism for selecting the cross-loss function is not limited to odd or even number of iterations, but can also consider multiples of the number of iterations, such as multiples of 2. When the number of iterations is 2, some of the loss functions are selected to update the parameters of the network model. When the number of iterations is 4, other loss functions are selected to update the parameters of the network model, etc. The selection of pictures from the training set can be made according to the requirements of the loss function. The above only lists limited examples to illustrate the above technical solution of the present invention, and does not mean that there are only the limited methods listed above. Other implementable methods are still within the scope of the embodiments of the present application.

[0094] The purpose of selecting the loss function to calculate the loss value and updating the parameters of the network model is to increase the model's expressiveness to ensure that the final teacher network performance is better.

[0095] Optionally, in another embodiment of the present invention, considering that there may be some differences between the first high-dimensional image features output by multiple teacher network models, in order to remove these differences, the method may further include:

[0096] Normalization processing is performed on the first high-dimensional image features so that the normalized high-dimensional image features of the same training image can be subsequently fused to obtain fused features.

[0097] The specific normalization formula is as follows:

[0098]

[0099] Where x is the dimension of the first high-dimensional image feature.

[0100] After normalization, the high-dimensional image features are fused. For details of the overall process, see Figure 6 As shown, including:

[0101] Human body cropped images are input into the backbone network respectively, and the first high-dimensional image features output by the backbone network are L2 normalized. The high-dimensional image features after L2 normalization are then fused.

[0102] L2 normalization can ensure that all first-dimensional image features are expressed in a unified and more standardized manner.

[0103] In another optional embodiment, based on any of the above embodiments, the embodiment of the present invention further provides another feature extraction model optimization method, in which how to optimize the student network model based on the dimensionality reduction feature and the second high-dimensional image feature will be described in detail. Figure 7 As shown, the method includes:

[0104] Step 710, determining the loss between the dimensionality reduction feature and the second high-dimensional image feature;

[0105] Step 720: Optimize the student network model using the loss.

[0106] Specifically, by calculating the loss between the dimensionality reduction features and the second-highest-dimensional image features, such as Euclidean distance or cosine similarity, and iteratively optimizing the parameters in the student network model based on the loss, the computational complexity and parameter size of the student network model can be made roughly the same as those of a single network model. However, its performance is significantly better than that of a single network model because it is close to the effect of integrating multiple teacher network models. Moreover, because the computational complexity is significantly reduced compared to multiple teacher network models, the prediction speed will naturally not decrease, nor will the inference cost increase. In other words, while improving the model's accuracy, the model's prediction speed will not decrease, and the inference cost will not increase.

[0107] Figure 8A flow chart of an exemplary method for extracting second high-dimensional image features through a student network model, calculating a loss function with dimensionality reduction features, and completing parameter updates in the student network model is shown.

[0108] Specifically, the method involves taking a fixed-size human image and inputting it into the student network model to extract features and obtain the second-highest-dimensional image features. Loss is calculated between the dimensionality-reduced features and the second-highest-dimensional features, and the loss is used to update the parameters of the student network model.

[0109] The above are several method embodiments of the multi-feature extraction model optimization provided by this application. The following describes other embodiments of the multi-feature extraction model optimization provided by this application. Please refer to the following for details.

[0110] Figure 9 A schematic structural diagram of a multi-feature extraction model optimization device provided in an embodiment of the present invention includes: a processing module 901, an extraction module 902, a fusion module 903, a dimensionality reduction module 904, and an optimization module 905.

[0111] Processing module 901, used to obtain multiple teacher network models;

[0112] Extraction module 902, used to extract the backbone network in each teacher network model; using each backbone network, respectively extract the first high-dimensional image features in the training image;

[0113] A fusion module 903 is configured to fuse multiple first high-dimensional image features of a training image to obtain a fused feature;

[0114] A dimension reduction module 904 is used to reduce the dimension of the fused features to obtain reduced-dimensionality features;

[0115] The processing module 901 is further configured to input the training image into the student network model to obtain a second high-dimensional image feature of the training image;

[0116] The optimization module 905 optimizes the student network model according to the dimensionality reduction features and the second high-dimensional image features.

[0117] Optionally, the processing module 901 is specifically configured to:

[0118] After obtaining the sample data, the sample data is divided into multiple groups;

[0119] Each set of sample data in multiple groups of sample data is used as a validation set, and the remaining sample data is used as a training set to form multiple training subsets;

[0120] Each training subset is used to train a network model, and finally multiple teacher network models are obtained.

[0121] Optionally, the processing module 901 is specifically configured to:

[0122] When training the network model using the training data in the training subset, determine the number of iterations of the current training;

[0123] According to the number of iterations of the current training, different loss functions are cross-selected to update the parameters in the network model, wherein each selected loss function includes at least one.

[0124] Optionally, the extraction module 902 is specifically used to remove the fully connected layer of the teacher network model from the teacher network model to obtain the backbone network.

[0125] Optionally, the optimization module 905 is specifically configured to determine a loss between the dimensionality reduction feature and the second high-dimensional image feature;

[0126] The student network model is optimized using loss.

[0127] Optionally, the processing module 901 is further configured to perform normalization processing on the first high-dimensional image features, so as to subsequently fuse the normalized high-dimensional image features of the same training image to obtain fused features.

[0128] The functions performed by each component in the multi-feature extraction model optimization device provided by an embodiment of the present invention have been described in detail in any of the above method embodiments, and therefore will not be repeated here.

[0129] An embodiment of the present invention provides a multi-feature extraction model optimization device. Multiple teacher network models are obtained, and then a backbone network is extracted from each teacher network model. Each backbone network is used to extract first high-dimensional image features from training images. Multiple first high-dimensional image features corresponding to the same training image are then fused to obtain a fused feature. This fused feature integrates the outputs of multiple teacher network models and is more comprehensive than the output of a single teacher network model. The fused feature is a "super-high-dimensional" image feature that is k times the dimension of the first high-dimensional image feature. To reduce computational complexity, dimensionality reduction is required. This compresses the data, eliminates redundancy, and extracts the main components of the information. The training image is input into a student network model to obtain second high-dimensional image features corresponding to the training image. The student network model is then optimized and trained based on the dimensionality reduction features and the second high-dimensional image features. Ultimately, the performance of the student network model approaches or even reaches the performance of an integrated model of multiple teacher network models. However, the computational complexity and parameter count of a single student network model are comparable to those of a single network model. While the prediction speed of this model approaches that of an integrated model of multiple teacher network models, the inference cost is that of a single network model. Therefore, the student network model optimized in this application can improve the model prediction speed without increasing the inference cost.

[0130] like Figure 10 As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0131] Memory 113, for storing computer programs;

[0132] In one embodiment of the present application, the processor 111 is configured to execute a program stored in the memory 113 to implement the feature extraction model optimization method provided by any of the aforementioned method embodiments, including:

[0133] Obtain multiple teacher network models;

[0134] Extract the backbone network from each teacher network model;

[0135] Utilize each backbone network to extract the first high-dimensional image features in the training image;

[0136] Fusing multiple first high-dimensional image features of the training image to obtain a fused feature;

[0137] Reduce the dimension of the fused features to obtain the reduced dimension features;

[0138] Input the training image into the student network model to obtain the second highest dimensional image features of the training image;

[0139] The student network model is optimized based on the dimensionality reduction features and the second high-dimensional image features.

[0140] Optionally, after obtaining the sample data, the sample data is divided into multiple groups;

[0141] Each set of sample data in multiple groups of sample data is used as a validation set, and the remaining sample data is used as a training set to form multiple training subsets;

[0142] Each training subset is used to train a network model, and finally multiple teacher network models are obtained.

[0143] Optionally, when training the network model using the training data in the training subset, determining the number of iterations of the current training;

[0144] According to the number of iterations of the current training, different loss functions are cross-selected to update the parameters in the network model, wherein each selected loss function includes at least one.

[0145] Optionally, remove the fully connected layer of the teacher network model from the teacher network model to obtain the backbone network.

[0146] Optionally, determining a loss between the dimensionality-reduced features and the second higher-dimensional image features;

[0147] The student network model is optimized using loss.

[0148] Optionally, normalization processing is performed on the first high-dimensional image features so that the normalized high-dimensional image features of the same training image can be subsequently fused to obtain fused features.

[0149] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the feature extraction model optimization method provided in any of the aforementioned method embodiments are implemented.

[0150] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device that includes the element.

[0151] The foregoing is merely a detailed description of the present invention, intended to enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein, but is to be construed in the widest manner consistent with the principles and novel features claimed herein.

Claims

1. A feature extraction model optimization method, characterized in that: The method comprises: Obtain multiple teacher network models; Extracting a backbone network from each of the teacher network models; Utilizing each of the backbone networks, respectively extracting first high-dimensional image features from the training image; fusing the plurality of first high-dimensional image features of the training image to obtain a fused feature; Performing dimensionality reduction on the fused features to obtain dimensionality reduction features; Inputting the training image into a student network model to obtain a second high-dimensional image feature of the training image, wherein the student network model is any one backbone network among the plurality of backbone networks; determining a loss between the dimensionality-reduced feature and the second high-dimensional image feature; The student network model is optimized using the loss.

2. The method according to claim 1, characterized in that The obtaining of multiple teacher network models specifically includes: After obtaining the sample data, dividing the sample data into multiple groups; Each set of sample data in the multiple sets of sample data is used as a validation set, and the remaining sample data is used as a training set to form multiple training subsets; Each training subset is used to train a network model, and finally a plurality of teacher network models are obtained.

3. The method according to claim 2, characterized in that When each training subset is used to train a network model, the method further comprises: determining the number of iterations of the current training when the network model is trained using the training data in the training subset; According to the number of iterations of the current training, different loss functions are cross-selected to update the parameters in the network model, wherein the loss function selected each time includes at least one.

4. The method according to any one of claims 1 to 3, characterized in that The extracting of the backbone network from each teacher network model specifically includes: From the teacher network model, remove the fully connected layer of the teacher network model to obtain the backbone network.

5. A multi-feature extraction model optimization device, characterized in that: The device comprises: A processing module for obtaining multiple teacher network models; An extraction module is used to extract the backbone network in each of the teacher network models; and to respectively extract the first high-dimensional image features in the training image using each of the backbone networks; a fusion module, configured to fuse the plurality of first high-dimensional image features of the training image to obtain a fused feature; A dimensionality reduction module, used to reduce the dimensionality of the fused features to obtain reduced dimensionality features; The processing module is further configured to input the training image into a student network model to obtain a second high-dimensional image feature of the training image, wherein the student network model is any one of the plurality of backbone networks; An optimization module determines a loss between the dimensionality reduction feature and the second high-dimensional image feature; and optimizes the student network model using the loss.

6. The device according to claim 5, characterized in that The processing module is specifically used to: After obtaining the sample data, dividing the sample data into multiple groups; Each set of sample data in the multiple sets of sample data is used as a validation set, and the remaining sample data is used as a training set to form multiple training subsets; Each training subset is used to train a network model, and finally a plurality of teacher network models are obtained.

7. The device according to claim 5 or 6, characterized in that The processing module is specifically used to: When training the network model using the training data in the training subset, determining the number of iterations of the current training; According to the number of iterations of the current training, different loss functions are cross-selected to update the parameters in the network model, wherein the loss function selected each time includes at least one.

8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; The processor is configured to implement the steps of the feature extraction model optimization method according to any one of claims 1 to 4 when executing the program stored in the memory.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the feature extraction model optimization method according to any one of claims 1 to 4 are implemented.

Images