Classification method and device based on parameter deentanglement and readable storage medium

Abstract

The invention discloses a parameter deentanglement-based classification method and device and a readable storage medium, and belongs to the field of computer vision and deep learning. The method comprises the steps of obtaining an image data set; preprocessing the image data set to obtain a standardized image data set; dividing the standardized image data set into a training set and a test set; importing the training set into a pre-constructed skeleton network structure based on parameter deentanglement diversified representation for training to obtain an updated classification model; evaluating the updated classification model by using the test set to obtain a classification model with optimal performance; and carrying out real-time classification processing on the image data based on the classification model with the optimal performance to obtain a classification result. According to the method, the feature representation capability of the model is improved by utilizing the classification model based on parameter deentanglement diversified representation, so that the accuracy of the model on a classification task is remarkably improved, and higher classification accuracy is realized.

Description

Technical Field

[0001] The present invention relates to a classification method, device and readable storage medium based on parameter disentanglement, and belongs to the field of computer vision and deep learning. Background Art

[0002] Convolutional neural networks (CNNs) are deep learning models that are mainly used to process image and video data. Since AlexNet made a breakthrough in the ImageNet competition in 2012, CNNs have developed rapidly and have been applied to tasks such as image classification, object detection, and semantic segmentation. In addition, CNNs have also achieved remarkable results in the fields of medical imaging, autonomous driving, natural language processing, and artistic creation, greatly promoting the advancement of artificial intelligence technology.

[0003] Multi-branch operation is a common method in CNNs design, which can be traced back to AlexNet. This method reduces the number of parameters and GPU limitations by deploying two convolution branches in parallel in the network and calculating them on two GPUs respectively. The success of multi-branch operation has enabled researchers to develop grouped lightweight attention modules such as SKNet and MixNet. In addition, neural network architecture search (NAS) is a technique for finding efficient network structures by reducing the search space and using effective search methods. Parameter disentanglement technology optimizes the efficiency of binary networks by treating learnable weights as linear combinations of multiple scaled binary parameters.

[0004] Although existing technologies can improve model performance, they also have some problems. Multi-branch operations and NAS require a lot of computing resources, especially the search process of NAS is time-consuming and computationally complex. Increasing the depth and width of the network can improve performance, but it will also increase latency during inference. Introducing more branches to utilize additional features can improve model performance, but it also brings more learnable parameters and increases computational overhead. Summary of the invention

[0005] The purpose of the present invention is to provide a classification method, device and readable storage medium based on parameter disentanglement. The present invention utilizes a classification model based on diversified representation of parameter disentanglement to improve the feature representation ability of the model, thereby significantly improving the accuracy of the model in land cover classification tasks and achieving higher classification accuracy.

[0006] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0007] A classification method based on parameter disentanglement diversified representation, comprising:

[0008] Get image dataset;

[0009] Preprocessing the image data set to obtain a standardized image data set;

[0010] Divide the standardized image dataset into training and testing sets;

[0011] Import the training set into the pre-built skeleton network structure based on parameter disentanglement diversified representation for training to obtain an updated classification model;

[0012] Use the test set to evaluate the updated classification model and obtain the classification model with the best performance;

[0013] In practical applications, real-time classification processing of image data is performed based on the classification model with the best performance to obtain classification results.

[0014] Optionally, preprocessing the image dataset to obtain a standardized image dataset;

[0015] Randomly cropping the image data set to obtain a number of cropped images;

[0016] Randomly scale several cropped images to obtain several images scaled in equal proportions;

[0017] According to the nvidia.dali toolkit, several images scaled in equal proportions are decoded, cropped, resized, and normalized in turn to obtain a standardized image dataset.

[0018] Optional, pre-built methods for skeleton network structures based on parameter disentangled diverse representations include:

[0019] Select the basic network structure;

[0020] The convolutional layer of the basic network structure is improved by adopting the diversified representation method of parameter disentanglement, and a skeleton network structure based on the diversified representation of parameter disentanglement is obtained.

[0021] Optionally, the skeleton network structure based on parameter disentanglement diversified representation includes: a PDDR-A network module and a PDDR-B network module;

[0022] The PDDR-A network module is composed of four branches, including two standard convolutions, one group convolution and one skeleton convolution. The four branches add together the learned weights, so that the skeleton network structure performs weighted summation of the features learned by multiple branches to generate a more representative feature representation.

[0023] The PDDR-B network module is composed of four branches and a refinement module. The four branches are the same as the four branches of the PDDR-A network module. The PDDR-B module uses dynamically learned weight parameters. To adjust the output weight of each branch.

[0024] Optionally, the expression of the standard convolution is:

[0025] (1)

[0026] In the formula, is the output feature map of the standard convolution, is the convolution kernel weight, is the input feature map, is the bias of the convolution kernel weight, and * represents the convolution operation.

[0027] The expression of group convolution is:

[0028] (2)

[0029] In the formula, is the output feature map of the group convolution, is the number of groups, For the The convolution kernel of the group, For the The input feature map of the group, For splicing operation, The group number for group convolution.

[0030] The expression of skeleton convolution is:

[0031] (3)

[0032] In the formula, is the output feature map of the skeleton convolution, is the vertical kernel, is the horizontal kernel, and They are respectively the clipping functions;

[0033] The PDDR-B module uses dynamically learned weight parameters The expression to adjust the output weight of each branch is:

[0034] W (4)

[0035] In the formula, is the convolution kernel weight, is the weight parameter for dynamic learning, is the total number of branches, is the branch number.

[0036] Optionally, the training set is imported into a pre-built skeleton network structure based on parameter disentangled diversified representation for training, and the method of obtaining an updated classification model includes:

[0037] The training set imports a skeleton network structure based on parameter disentanglement diversified representation, performs standard convolution feature extraction, group convolution feature extraction and skeleton convolution feature extraction through the PDDR-A network module and the PDDR-B network module, respectively, and obtains the image features of standard convolution, group convolution and skeleton convolution of the PDDR-A network module and the PDDR-B network module respectively;

[0038] The PDDR-A network module and the PDDR-B network module respectively accumulate the image features of the standard convolution, the group convolution and the skeleton convolution to obtain the image feature data accumulated by the PDDR-A network module and the image feature data accumulated by the PDDR-B network module;

[0039] The image feature data accumulated by the PDDR-A network module and the image feature data accumulated by the PDDR-B network module are input into multiple convolutional layers, pooling layers, and fully connected layers for further feature extraction, and finally the prediction value is output through the Softmax layer;

[0040] The pre-stored true value and predicted value are calculated based on the cross entropy function to obtain the loss value of the difference between the true value and the predicted value;

[0041] The Adam optimization algorithm is used to derive the loss value to obtain the gradient of the model parameters;

[0042] Update the parameter weights of the skeleton network structure according to the gradient of the model parameters to obtain an updated classification model;

[0043] Among them, when the network model based on diversified representation of parameter disentanglement is updated to be relatively stable according to a preset number of training times or loss value, the training of the network model based on diversified representation of parameter disentanglement is stopped to obtain an updated network model based on diversified representation of parameter disentanglement.

[0044] Optionally, the updated classification model is evaluated using the test set. Methods for obtaining the classification model with the best performance include:

[0045] Predict the test set based on the updated network model based on the diverse representation of parameter disentanglement, and obtain the prediction results for each sample category in the test set;

[0046] Substitute the true value and prediction result of each sample category into the accuracy, recall and F1-score calculations to obtain the evaluation indicators of the model;

[0047] The model with higher evaluation index than other models is selected as the classification model with the best performance.

[0048] A classification device based on parameter disentanglement diversified representation, comprising:

[0049] Data acquisition module, update module, evaluation module and application module

[0050] Wherein, the data acquisition module is used to obtain an image data set; preprocess the image data set to obtain a standardized image data set;

[0051] The updating module is used to divide the standardized image data set into a training set and a test set; import the training set into a pre-built skeleton network structure based on parameter disentanglement diversified representation for training to obtain an updated classification model;

[0052] The evaluation module is used to evaluate the updated classification model using the test set to obtain the classification model with the best performance;

[0053] The application module is used to perform real-time classification processing of image data based on the classification model with the best performance in practical applications to obtain classification results.

[0054] A computer-readable storage medium stores a computer program / instruction thereon, which, when executed by a processor, implements the steps of the above-mentioned classification method.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] (1) The present invention utilizes a classification model based on parameter disentanglement and diversified representation to improve the feature representation capability of the model, thereby significantly improving the accuracy of the model in land cover classification tasks and achieving higher classification accuracy.

[0057] (2) During the model inference process, the multi-branch structure is accumulated by utilizing the characteristics of convolutional layer calculations, avoiding complex calculations and memory usage, further improving the inference efficiency while shortening the calculation time and balancing the calculation load. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 Shown is a flow chart of a classification method based on parameter disentanglement of the present invention;

[0059] Figure 2 Shown is the image data after cropping of the present invention;

[0060] Figure 3 FIG. 4 is a schematic diagram showing a combination of a standard convolution and a standard convolution of the present invention;

[0061] Figure 4 FIG. 4 is a schematic diagram showing a combination of a standard convolution and a group convolution of the present invention;

[0062] Figure 5It is a schematic diagram showing the combination of the standard convolution and the skeleton convolution of the present invention;

[0063] Figure 6 Shown is a schematic diagram of the structure of PDDR-A without refinement modules of the present invention;

[0064] Figure 7 Shown is a schematic diagram of the structure of a PDDR-B with a refinement module of the present invention;

[0065] Figure 8 Shown is a flow chart of a land cover classification method based on parameter disentanglement diversified representation of the present invention;

[0066] Fig. 9 Shown is a comparison chart of the classification accuracy and computational complexity on the ImageNet dataset of the present invention;

[0067] Fig.10 Shown is a comparison chart of detection accuracy and computational complexity on the MSCOCO dataset of the present invention;

[0068] Fig.11 Shown is a comparison chart of the classification accuracy and computational complexity of the present invention on various lightweight Vision Transformer models. DETAILED DESCRIPTION

[0069] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0070] Embodiment 1;

[0071] This embodiment provides a classification method based on parameter disentanglement and diversified representation, such as Figure 1 Shown include:

[0072] Step 1: Get the image dataset;

[0073] Step 2: preprocessing the image data set to obtain a standardized image data set;

[0074] Step 3: Divide the standardized image dataset into training set and test set;

[0075] Step 4: Import the training set into the pre-built skeleton network structure based on parameter disentanglement diversified representation for training to obtain an updated classification model;

[0076] Step 5: Use the test set to evaluate the updated classification model and obtain the classification model with the best performance;

[0077] Step 6: In practical applications, perform real-time classification processing of image data based on the classification model with the best performance to obtain classification results.

[0078] Optionally, preprocessing the image dataset to obtain a standardized image dataset;

[0079] Randomly cropping the image data set to obtain a number of cropped images;

[0080] Randomly scale several cropped images to obtain several images scaled in equal proportions;

[0081] According to the nvidia.dali toolkit, several images scaled in equal proportions are decoded, cropped, resized, and normalized in turn to obtain a standardized image dataset.

[0082] Optional, pre-built methods for skeleton network structures based on parameter disentangled diverse representations include:

[0083] Select the basic network structure;

[0084] The convolutional layer of the basic network structure is improved by adopting the diversified representation method of parameter disentanglement, and a skeleton network structure based on the diversified representation of parameter disentanglement is obtained.

[0085] Optional, such as Figures 3 to 7 The skeleton network structure based on parameter disentanglement diversified representation shown includes: a PDDR-A network module and a PDDR-B network module;

[0086] Among them, Figure 6 The PDDR-A network module shown is composed of four branches, including two standard convolutions, one group convolution and one skeleton convolution; the four branches add together the learned weights, so that the skeleton network structure performs weighted summation of the features learned by multiple branches to generate a more representative feature representation; Figures 3 to 5 The diagram shows a schematic diagram of the structure in which each branch is combined with each other;

[0087] like Figure 7 The PDDR-B network module shown is composed of four branches and a refinement module. The four branches are the same as the four branches of the PDDR-A network module. The PDDR-B module uses dynamically learned weight parameters. To adjust the output weight of each branch.

[0088] The biggest difference between PDDR-B and PDDR-A in this embodiment is that there is an additional refinement module. The function of the refinement module is to adaptively adjust the weight of each branch according to the input data. PDDR-A can only accumulate the four branches in a ratio of 1:1:1:1. PDDR-B can use the refinement module to adaptively adjust the distribution ratio of the four branches to achieve a more refined feature representation. During the reasoning process, PDDR-A can directly accumulate the weights of the four branches to become the same structure as before the PDDR module is added, without any increase in parameters and calculations during reasoning. PDDR-B will retain the weights of the four branches, and obtain the distribution ratio according to the refinement module, and accumulate the four branches in proportion. Therefore, compared with the original model, PDDR-B only adds a negligible amount of parameters and calculations brought by the refinement module, which is about 0.08M. Compared with the SE module and the AC module, our PDDR module brings better performance with less parameters. Figure 8 As shown in the figure, after a simple processing, the data set is input into the PDDR structure for feature extraction operation to obtain a feature image containing richer information after being processed by the PDDR module. This process is called parameter disentanglement for diverse representations (PDDR) of the data set.

[0089] Optionally, the expression of the standard convolution is:

[0090] (1)

[0091] In the formula, is the output feature map of the standard convolution, is the convolution kernel weight, is the input feature map, is the bias of the convolution kernel weight, and * represents the convolution operation.

[0092] The expression of group convolution is:

[0093] (2)

[0094] In the formula, is the output feature map of the group convolution, is the number of groups, For the The convolution kernel of the group, For the The input feature map of the group, For splicing operation, The group number for group convolution.

[0095] The expression of skeleton convolution is:

[0096] (3)

[0097] In the formula, is the output feature map of the skeleton convolution, is the vertical kernel, is the horizontal kernel, and They are respectively the clipping functions;

[0098] The PDDR-B module uses dynamically learned weight parameters The expression to adjust the output weight of each branch is:

[0099] W (4)

[0100] In the formula, is the convolution kernel weight, is the weight parameter for dynamic learning, is the total number of branches, The branch number.

[0101] This part corresponds to the model inference process. The structure of the model during inference is not completely consistent with the model structure during training. During training, the four convolution branches of the PDDR module are processed in parallel, and the output is the accumulation of the outputs of the four convolution branches. During inference, the model will accumulate the weight parameters of the four parallel convolution branches during training to obtain a merged branch, and this structure is completely consistent with the structure before adding the PDDR module.

[0102] Optionally, the training set is imported into a pre-built skeleton network structure based on parameter disentangled diversified representation for training, and the method of obtaining an updated classification model includes:

[0103] The training set imports a skeleton network structure based on parameter disentanglement diversified representation, performs standard convolution feature extraction, group convolution feature extraction and skeleton convolution feature extraction through the PDDR-A network module and the PDDR-B network module, respectively, and obtains the image features of standard convolution, group convolution and skeleton convolution of the PDDR-A network module and the PDDR-B network module respectively;

[0104] The PDDR-A network module and the PDDR-B network module respectively accumulate the image features of the standard convolution, the group convolution and the skeleton convolution to obtain the image feature data accumulated by the PDDR-A network module and the image feature data accumulated by the PDDR-B network module;

[0105] The image feature data accumulated by the PDDR-A network module and the image feature data accumulated by the PDDR-B network module are input into multiple convolutional layers, pooling layers, and fully connected layers for further feature extraction, and finally the prediction value is output through the Softmax layer;

[0106] The pre-stored true value and predicted value are calculated based on the cross entropy function to obtain the loss value of the difference between the true value and the predicted value;

[0107] The Adam optimization algorithm is used to derive the loss value to obtain the gradient of the model parameters;

[0108] Update the parameter weights of the skeleton network structure according to the gradient of the model parameters to obtain an updated classification model;

[0109] Among them, when the network model based on diversified representation of parameter disentanglement is updated to be relatively stable according to a preset number of training times or loss value, the training of the network model based on diversified representation of parameter disentanglement is stopped to obtain an updated network model based on diversified representation of parameter disentanglement.

[0110] In this embodiment, the predicted land cover class value based on each satellite image pixel block is compared with the pre-stored true value, and the loss value is obtained based on the cross entropy loss function:

[0111] First, the predicted value is softmaxed to convert it into class probabilities. We use the cross entropy loss function to calculate the difference between the predicted and true class. The cross entropy measures how much the probability predicted by the model differs from the true class. The greater the difference, the greater the loss.

[0112] The loss value is derived according to the stochastic gradient optimization algorithm to obtain the model optimization parameters, including:

[0113] Parameters are optimized by minimizing the loss function. The loss function measures the error between the model's prediction and the true label. In order to know how to adjust the model parameters to reduce the value of the loss function, we need to calculate the derivative of the loss function with respect to the model parameters, i.e., the gradient. With the gradient, we can use the gradient to update the model parameters to reduce the value of the loss function.

[0114] The parameter weights of the skeleton network structure are updated according to the gradient of the model parameters, and the expression of the updated classification model is obtained as follows:

[0115] (5)

[0116] are the parameters of the current model, are the parameters of the updated model, is the gradient of the parameter θ, that is, the partial derivative of the loss function with respect to the parameter. η is the learning rate, which determines the size of each update.

[0117] Optionally, the updated classification model is evaluated using the test set. Methods for obtaining the classification model with the best performance include:

[0118] Predict the test set based on the updated network model based on the diverse representation of parameter disentanglement, and obtain the prediction results for each sample category in the test set;

[0119] Substitute the true value and prediction result of each sample category into the accuracy, recall and F1-score calculations to obtain the evaluation indicators of the model;

[0120] The model with higher evaluation index than other models is selected as the classification model with the best performance.

[0121] Example 2

[0122] This embodiment provides a classification device based on parameter disentanglement diversified representation, including:

[0123] Data acquisition module, update module, evaluation module and application module

[0124] Wherein, the data acquisition module is used to obtain an image data set; preprocess the image data set to obtain a standardized image data set;

[0125] The updating module is used to divide the standardized image data set into a training set and a test set; import the training set into a pre-built skeleton network structure based on parameter disentanglement diversified representation for training to obtain an updated classification model;

[0126] The evaluation module is used to evaluate the updated classification model using the test set to obtain the classification model with the best performance;

[0127] The application module is used to perform real-time classification processing of image data based on the classification model with the best performance in practical applications to obtain classification results.

[0128] Example 3

[0129] This embodiment provides a computer-readable storage medium, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the following steps of the classification method are implemented:

[0130] Step 1: Get the image dataset;

[0131] Step 2: preprocessing the image data set to obtain a standardized image data set;

[0132] Step 3: Divide the standardized image dataset into training set and test set;

[0133] Step 4: Import the training set into the pre-built skeleton network structure based on parameter disentanglement diversified representation for training to obtain an updated classification model;

[0134] Step 5: Use the test set to evaluate the updated classification model and obtain the classification model with the best performance;

[0135] Step 6: Perform real-time classification processing on the image data based on the classification model with the best performance to obtain the classification results.

[0136] Example 4

[0137] This embodiment provides a classification method for land cover based on a classification model with diverse representations of parameter disentanglement. The method effectively improves the accuracy and reasoning efficiency of land cover classification through the steps of satellite image data acquisition, preprocessing, model training, feature representation, model reasoning, and result statistics, including:

[0138] Step 1: Obtain a satellite image dataset. Obtain a high-resolution satellite image dataset from a public remote sensing image database, named "satellite_dataset". The dataset contains images of multiple geographic areas, covering various surface types such as farmland, forest, grassland, water, and buildings.

[0139] Step 2: Create training and test data sets based on satellite image data. Perform preprocessing operations on the obtained "satellite_dataset" data set, including the following steps:

[0140] Step 2.1: Image cropping and labeling: Crop each image, crop the original image into image blocks of uniform size (e.g. 224x224 pixels), and label them according to the actual land cover type. Each cropped block is assigned a label according to its land cover type, such as farmland (label_1), forest (label_2), grassland (label_3), water body (label_4), building (label_5), etc. The output result is the cropped image dataset satellite_dataset_cropped.

[0141] Step 2.2: Dataset division: Divide the cropped and labeled dataset: satellite_dataset_cropped into the training dataset train_dataset and the test dataset test_dataset in proportion (e.g. 80% training set, 20% test set).

[0142] Step 3: Use NVIDIA DALI tools to preprocess the data set to facilitate distributed training. Use NVIDIA DALI (Data Loading Library) to perform data enhancement and standardization on the training data set train_dataset to improve the training efficiency and generalization ability of the model, and further:

[0143] Step 3.1: Data augmentation: Perform data augmentation operations on train_dataset, including random cropping, rotation, color jittering, and normalization. The result of data augmentation is the preprocessed training dataset train_dataset_augmented.

[0144] Step 3.2: Data distribution: Distribute the train_dataset_augmented dataset to multiple GPU devices to ensure that the training data received by each GPU is independent and evenly distributed. The output result is the distributed training dataset distributed_train_dataset.

[0145] Step 4: Pass the processed training data set into the training model. Pass distributed_train_dataset into the deep learning model for training. The model uses a deep convolutional neural network based on the ResNet structure and introduces the parameter disentangled diversified representation module (PDDR).

[0146] Step 5: Perform parameter disentangled diversified representation (PDDR) on the dataset to obtain high-dimensional feature data. In order to solve the problem of insufficient classification accuracy of the traditional ResNet model in land cover classification, a parameter disentangled diversified representation module (PDDR) is designed. The module has two variants, namely PDDR-A and PDDR-B, and further:

[0147] Step 5.1: PDDR-A diversified representation: Pass distributed_train_dataset into the PDDR-A module, which splits the standard convolution layer into four branches: two original standard convolutions, one group convolution and one skeleton convolution. The outputs of the four branches are , the final output feature expression is:

[0148] (6)

[0149] in, is the output feature of the standard convolution, is the output feature of the group convolution, is the output feature of the skeleton convolution.

[0150] Step 5.2: PDDR-B dynamic weight representation: Based on PDDR-A, the PDDR-B module uses dynamically learned weight parameters To adjust the output weight of each branch. The expression of the output feature is:

[0151] (7)

[0152] in, is the weight parameter learned during training, It is The output features of each branch.

[0153] In this embodiment, the four outputs are accumulated, and then the conventional feature extraction operation is performed on these data. This use of a diverse branch structure can improve the feature representation capability of the model parameters and improve the classification accuracy.

[0154] The output data obtained by the above two variants are collectively referred to as feature_data.

[0155] Step 6: Continue to extract features from the obtained feature data to obtain the predicted value of each classification category. Input the feature data feature_data into multiple convolutional layers, pooling layers, and fully connected layers for further feature extraction. Finally, the predicted value of each category is obtained through the Softmax layer. The predicted value is recorded as .

[0156] Step 7: Compare the predicted value with the true value and calculate the loss using the cross entropy function. Use the cross entropy loss function to calculate the category predicted by the model. and the true category The difference between them gives the loss value loss:

[0157] (8)

[0158] Among them, each sample The loss is its true label and the probability predicted by the model The smaller the cross entropy loss, the closer the model's prediction result is to the true value, and the better the model's classification performance is.

[0159] This embodiment continues to extract features from the obtained data, and obtains the predicted value of each classification category including:

[0160] The feature data obtained by the parameter disentanglement diversified representation module will be further extracted through multiple convolutional layers, pooling layers and fully connected layers of the model. These features contain a variety of information about the surface cover, such as texture, shape, lighting and seasonal changes. In the last layer of the model, the fully connected layer maps the extracted features to specific land cover categories. This process is based on the probability distribution of each category corresponding to the output of each image pixel or block in the satellite image dataset, reflecting the model's ability to distinguish complex and diverse surface features. The Softmax activation function is applied to the output of the fully connected layer to convert the output value of each classification category into a probability value, and the predicted probability of the land cover type corresponding to each satellite image pixel or block is obtained.

[0161] Cross Entropy is selected as the loss function to measure the difference between the land cover category predicted from satellite image data and the true labeled value. This loss function is particularly effective for multi-category classification problems and can accurately reflect the performance of the model when dealing with complex surface features.

[0162] For each image sample in the satellite image dataset, the land cover category predicted by the model is compared with the true labeled value and the cross entropy loss is calculated. Then the sample losses of the entire dataset are summed and averaged to evaluate the overall classification effect of the model on the satellite image dataset.

[0163] Step 8: Use stochastic gradient optimization to derive the loss and optimize the model parameters. Use the Adam optimization algorithm to derive the loss value loss and update the model parameters to minimize the loss function. Get the updated model parameters updated_model_params.

[0164] This example uses Adam (Adaptive Moment Estimation) as the optimization algorithm. Aiming at the complexity of the satellite image data set, the model parameters are updated by deriving the loss function to minimize the loss function based on the satellite image data. The Adam algorithm adaptively adjusts the learning rate of each parameter and is particularly suitable for complex data sets containing multiple surface features and environmental variables.

[0165] The model's parameter weights are gradually updated based on the gradient results calculated from satellite image data. The Adam algorithm dynamically adjusts the learning rate to ensure that the model's performance gradually improves when dealing with complex land cover types.

[0166] Step 9: Update the model parameters to make the predicted value closer to the true value. Through multiple iterations of distributed_train_dataset training, the model parameters updated_model_params are continuously updated. In each iteration, the predicted value of the model With the true value The error is continuously reduced. The optimized model parameters optimized_model_params are obtained.

[0167] This embodiment repeatedly performs loss calculation, gradient derivation and parameter update by performing multiple iterative training on the satellite image dataset. After each iteration, the parameters of the model are gradually optimized to make the predicted land cover type closer to the true value, thereby improving the classification accuracy on the satellite image dataset. To prevent the model from overfitting on the satellite image data, an early stopping strategy is introduced. This strategy monitors the changes in the validation set loss. When the validation loss no longer decreases or starts to increase, the training is stopped in advance to ensure the generalization ability of the model.

[0168] Step 10: Complete model training and obtain the weight parameters of the training model. After training, save the optimized model weight parameters as a file model_weights.pth for use in actual reasoning tasks.

[0169] After training, this embodiment saves the optimized model parameter weights in a file format suitable for industrial applications (such as HDF5, Checkpoint, etc.) to facilitate reasoning and application in actual land cover classification tasks. By using satellite image data on an independent validation set or test set for evaluation, it is confirmed whether the classification performance of the model meets the requirements of industrial applications, and the evaluation indicators (such as accuracy, recall, F1-score, etc.) are recorded to ensure the applicability of the model in real scenarios.

[0170] Step 11: Import the obtained model weight parameters into the inference model and pass in the test data. Load the trained model weight parameters model_weights.pth into the inference model. To improve the inference efficiency, multiple branches of the parameter disentanglement diversified representation module are fused into a standard convolution. Further:

[0171] Step 11.1: Fuse convolution operations: Fuse multiple convolution operations in PDDR-A and PDDR-B into one standard convolution

[0172] Step 11.2: Test data input: pass test_dataset into the fused inference model and calculate the predicted category of each test sample .

[0173] In the process of model inference, in order to improve the inference efficiency of the model, the present invention merges multiple branches of the parameter disentanglement diversified representation module into a standard convolution, that is, the amount of calculation and the amount of parameters are consistent with the original ResNet model. The theoretical basis for this is that the convolution operation can be regarded as the numerical accumulation of multiple convolution operations, which can be referred to in formulas (1), (4) and (5).

[0174] (8)

[0175] in, It is the output feature map obtained by accumulating multiple convolution operations. is the weight of each convolution, Indicates the number of convolutions that need to be accumulated, represents the convolution kernel weight of each convolution, represents the bias of each convolution, Represents the input feature map. This can improve the efficiency of inference without losing the model weight parameters.

[0176] In this step, the two variants PDDR-A and PDDR-B have slightly different inference models. The inference model of PDDR-A is completely consistent with the original ResNet model in terms of network structure, without any increase in parameters and computation. In order to retain the weight of each branch adaptively obtained according to the input, PDDR-B has about 0.08M more parameters than PDDR-A, but this is negligible compared to the number of parameters of the model itself.

[0177] Step 12: Obtain land cover statistics. The statistical results of each category in the , calculate the coverage of each surface type, such as the proportion of farmland, forest, grassland, water body and building. Finally, the land cover classification statistical results land_cover_stats are obtained.

[0178] Compared with the prior art, the performance of the model of the present invention is significantly improved. The model using PDDR performs well in the ImageNet classification task and the MS COCO object detection task, significantly improving the classification accuracy and detection precision of the model. Figures 9 and 10 It can be seen from Tables 1 and 2 that PDDR has a significant improvement on the accuracy.

[0179] The computational efficiency of the model of the present invention is high. Through the parameter disentanglement technology, the demand for computing resources is reduced and the reasoning efficiency of the model is improved. See Table 1 and Table 2. The computational amount and parameter amount of the model have not been increased in detail.

[0180] like Fig.11As shown in Table 3, the model of the present invention is highly versatile. PDDR is not only applicable to convolutional neural networks, but can also be extended to other models such as Vision Transformer and MobileNet, showing a wide range of application prospects. Compared with other methods, it can achieve higher accuracy without significantly increasing network parameters.

[0181] In summary, the present invention utilizes a classification model based on parameter disentanglement and diversified representation to improve the model's feature representation capability, which can significantly improve the model's accuracy in land cover classification tasks, thereby achieving a higher classification accuracy. In the model reasoning process, by utilizing the characteristics of convolutional layer calculations, the multi-branch structure is accumulated, avoiding complex calculations and memory usage, further improving reasoning efficiency while shortening calculation time and balancing calculation load.

[0182] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0183] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0184] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0185] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0186] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A classification method based on parameter disentanglement and diversified representation, characterized in that: include: Get image dataset; Preprocessing the image data set to obtain a standardized image data set; Divide the standardized image dataset into training and testing sets; Import the training set into the pre-built skeleton network structure based on parameter disentanglement diversified representation for training to obtain an updated classification model; Use the test set to evaluate the updated classification model and obtain the classification model with the best performance; Based on the classification model with the best performance, real-time classification processing of image data is performed to obtain classification results.

2. The classification method based on parameter disentanglement and diversified representation according to claim 1 is characterized in that: Preprocessing the image data set to obtain a standardized image data set; Randomly cropping the image data set to obtain a number of cropped images; Randomly scale several cropped images to obtain several images scaled in equal proportions; According to the nvidia.dali toolkit, several images scaled in equal proportions are decoded, cropped, resized, and normalized in turn to obtain a standardized image dataset.

3. The classification method based on parameter disentanglement and diversified representation according to claim 1 is characterized in that: Pre-built methods for skeleton network structures based on parameter disentangled diverse representations include: Select the basic network structure; The convolutional layer of the basic network structure is improved by adopting the diversified representation method of parameter disentanglement, and a skeleton network structure based on the diversified representation of parameter disentanglement is obtained.

4. The classification method based on parameter disentanglement and diversified representation according to claim 3 is characterized in that: The skeleton network structure based on parameter disentanglement diversified representation includes: a PDDR-A network module and a PDDR-B network module; The PDDR-A network module is composed of four branches, including two standard convolutions, one group convolution and one skeleton convolution. The four branches add together the learned weights, so that the skeleton network structure performs weighted summation of the features learned by multiple branches to generate a more representative feature representation. The PDDR-B network module is composed of four branches and a refinement module. The four branches are the same as the four branches of the PDDR-A network module. The PDDR-B module uses dynamically learned weight parameters. To adjust the output weight of each branch.

5. The classification method based on parameter disentanglement and diversified representation according to claim 1 is characterized in that: The expression of the standard convolution is: (1) In the formula, is the output feature map of the standard convolution, is the convolution kernel weight, is the input feature map, is the bias of the convolution kernel weight, and * represents the convolution operation; The expression of group convolution is: (2) In the formula, is the output feature map of the group convolution, is the number of groups, For the The convolution kernel of the group, For the The input feature map of the group, For splicing operation, The group number for the group convolution; The expression of skeleton convolution is: (3) In the formula, is the output feature map of the skeleton convolution, is the vertical kernel, is the horizontal kernel, and They are respectively the clipping functions; The PDDR-B module uses dynamically learned weight parameters The expression to adjust the output weight of each branch is: W (4) In the formula, is the convolution kernel weight, is the weight parameter for dynamic learning, is the total number of branches, The branch number.

6. The classification method based on parameter disentanglement and diversified representation according to claim 5 is characterized in that: The method of importing the training set into the pre-built skeleton network structure based on parameter disentangled diversified representation for training to obtain an updated classification model includes: The training set imports a skeleton network structure based on parameter disentanglement diversified representation, performs standard convolution feature extraction, group convolution feature extraction and skeleton convolution feature extraction through the PDDR-A network module and the PDDR-B network module, respectively, and obtains the image features of standard convolution, group convolution and skeleton convolution of the PDDR-A network module and the PDDR-B network module respectively; The PDDR-A network module and the PDDR-B network module respectively accumulate the image features of the standard convolution, the group convolution and the skeleton convolution to obtain the image feature data accumulated by the PDDR-A network module and the image feature data accumulated by the PDDR-B network module; The image feature data accumulated by the PDDR-A network module and the image feature data accumulated by the PDDR-B network module are input into multiple convolutional layers, pooling layers, and fully connected layers for further feature extraction, and finally the prediction value is output through the Softmax layer; The pre-stored true value and predicted value are calculated based on the cross entropy function to obtain the loss value of the difference between the true value and the predicted value; The Adam optimization algorithm is used to derive the loss value to obtain the gradient of the model parameters; The parameter weights of the skeleton network structure are updated according to the gradient of the model parameters to obtain an updated classification model.

7. The classification method based on parameter disentanglement and diversified representation according to claim 6 is characterized in that: The parameter weights of the skeleton network structure are updated according to the gradient of the model parameters to obtain an updated classification model, which also includes: when the network model of the diversified representation based on parameter disentanglement is updated to be relatively stable according to a preset number of training times or loss value, the training of the network model of the diversified representation based on parameter disentanglement is stopped to obtain an updated network model of the diversified representation based on parameter disentanglement.

8. The classification method based on parameter disentanglement and diversified representation according to claim 1 is characterized in that: The updated classification model is evaluated using the test set. Methods for obtaining the classification model with the best performance include: The test set is predicted based on the updated network model based on the diverse representation of parameter disentanglement to obtain the prediction results for each sample category in the test set; Substitute the true value and prediction result of each sample category into the accuracy, recall and F1-score calculations to obtain the evaluation indicators of the model; The model with higher evaluation index than other models is selected as the classification model with the best performance.

9. A classification device based on parameter disentanglement diversified representation, characterized in that: include: Data acquisition module, update module, evaluation module and application module Wherein, the data acquisition module is used to obtain an image data set; Preprocessing the image data set to obtain a standardized image data set; The updating module is used to divide the standardized image data set into a training set and a test set; import the training set into a pre-built skeleton network structure based on parameter disentanglement diversified representation for training to obtain an updated classification model; The evaluation module is used to evaluate the updated classification model using the test set to obtain the classification model with the best performance; The application module is used to perform real-time classification processing of image data based on the classification model with the best performance in practical applications to obtain classification results.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program implements the steps of the classification method described in any one of claims 1-8.

Images