Generalizable classification method, system and device based on domain-invariant category features

Abstract

The present invention discloses a generalizable classification method, system and device based on domain-invariant category features, belonging to the field of computer vision processing. The method of the present invention first collects images from multiple environments for a classification task of a preset category to construct a source domain image data set, and after sampling and preprocessing the source domain image data set, constructs multiple batch sample sets for training, each preprocessed image corresponds to a real category label, and then trains an image classification model on the batch sample set, and finally inputs the image to be classified into the trained image classification model, and outputs the classification result of the image category. The present invention can find stable features that are domain-invariant and category-specific from the input features, and predict categories based on this, which can effectively improve the generalization ability of the image classification model.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision processing, and in particular relates to a generalizable classification method, system and device based on domain-invariant category features. Background Art

[0002] The core problem of domain generalization is how to make the model learn features with generalization ability from different source domains so as to achieve good performance in the target domain. The data in the target domain is invisible during the training phase. Therefore, domain generalization is different from domain adaptation. In domain adaptation, part of the information in the target domain may have been given during training. However, domain generalization has no access to the target domain information at all.

[0003] There are three key challenges in domain generalization: (1) Domain shift. Differences between domains may be reflected in feature distribution, background information, image style, etc. These changes often affect the prediction accuracy of the model. (2) Overfitting the source domain: The model may overfit to specific features in the source domain, which are not applicable to the target domain, resulting in generalization failure. (3) Relevance bias: The model may rely on "shortcut" features (such as background, color, etc.) that are relevant to the category but not relevant to the domain. These features are invalid or misleading in the target domain.

[0004] The methods to solve domain generalization mainly include: invariant feature learning, data enhancement and meta-learning methods. Among them, invariant feature learning aims to find invariant features shared across domains, so that the model is less sensitive to domain changes and improves the generalization ability in new domains. Common methods include invariant risk minimization, which enhances generalization ability by making the model learn the same feature representation on different domains. Another common method is adversarial training, which introduces a domain discriminator to force the features learned by the model to be unable to distinguish between different domains, thereby ensuring that the model extracts features that are not related to the domain. Data enhancement simulates the data distribution of different domains by performing various transformations on the training data (such as image rotation, flipping, color perturbation, etc.), so that the model can see more possible situations during training, thereby improving the generalization ability. The meta-learning method trains by simulating inter-domain changes, that is, dividing the source domain data into multiple "pseudo-domains" and continuously optimizing the model to adapt to changes in different domains.

[0005] Domain generalization is to enable machine learning models to maintain efficient prediction capabilities when crossing domain differences. Researchers use technologies such as invariant feature learning, data enhancement, and meta-learning to try to solve challenges such as domain shift and overfitting to improve the generalization performance of the model. The study of domain generalization is of great significance to the application of machine learning models in reality, especially for tasks with dynamically changing distributions or diverse domains. Summary of the invention

[0006] The purpose of the present invention is to solve the problems existing in the prior art and to provide a generalizable classification method, system and device based on domain-invariant category features.

[0007] The inventive concept of the present invention is: in order to reduce the model's inability to make effective predictions in unseen target domains by learning domain-related features or category-shared features during the training phase, the present invention proposes a new generalizable classification method. The image classification model searches for domain-independent and category-specific features among the features of images in multiple source domains and categories, and uses them as a reliable and stable classification basis for prediction, so as to improve the generalization performance of the image classification model in unseen target domains.

[0008] In order to achieve the above-mentioned invention object, the present invention specifically adopts the following technical solutions:

[0009] In a first aspect, the present invention provides a generalizable classification method based on domain-invariant category features, which comprises the following steps:

[0010] S1: Obtain a number of images from different environments to construct an original image dataset. Different images from the same environment in the original image dataset constitute a source domain image dataset. Sample images in each source domain image dataset, process the sampled images, and construct multiple batch sample sets for training from the processed images. Each processed image corresponds to a true category label.

[0011] S2: Input the image to be classified in the target domain into the trained image classification model, and output the classification result of the image category; wherein the image classification model includes an encoder and a classifier;

[0012] In the training process of the image classification model, the encoder encodes the images in the batch sample set and outputs mixed features. After the mixed features are subtracted from the acquired source domain features, the domain-invariant category features are obtained and stored in the category feature queue for calculating the similarity, and the similarity calculation result is used as the weighted fusion coefficient of the true category label to obtain the category soft label of each category feature, and the cross entropy loss between the category soft label of each category feature and the true category label is used as the category invariance loss. The mixed features are input into the classifier, and the classifier outputs the predicted category probability. The cross entropy loss between the predicted category probability and the true category label of each image in the batch sample set is calculated. Two images belonging to the same category in the batch sample set are taken as pairs of images of the same category, and the similarity between the category features of the pairs of images of the same category is calculated. The domain invariance loss is constructed from the similarity calculation result, and the weighted sum of the category invariance loss, the cross entropy loss and the domain invariance loss is used as the total loss. The image classification model parameters are updated based on minimizing the total loss, and the training is iterated continuously until a preset number of iterations is reached and the image classification model converges to obtain a trained image classification model.

[0013] Based on the above scheme, each step can be implemented in the following preferred specific manner.

[0014] As a preferred embodiment of the above-mentioned first aspect, in step S1, in K source domain image data sets, M different images are sampled to form a sampling data set, each image in the sampling data set is randomly center cropped, and then the cropped image is enhanced with a preset probability, the enhanced image is converted into a tensor format, and the format-converted image is regularized, the true category label corresponding to the regularized image is re-encoded into an index value, the regularized image and its corresponding index value are combined into a sample pair to generate a batch sample set for training; wherein K represents the number of environmental sources; M represents the number of images in the sampling data set.

[0015] As a preferred embodiment of the first aspect, in step S2, the training process of the image classification model is specifically as follows:

[0016] S21: In the current iteration round, the batch sample set is divided into K sub-data sets according to the environmental source, and the K sub-data sets are batch-input into the encoder updated in the previous iteration round, and features are extracted for each image in the K sub-data sets. The features extracted by the encoder are used as mixed features, and all the mixed features of each sub-data set constitute the encoding feature set corresponding to each sub-data set;

[0017] S22: Obtain the updated source domain features of the previous iteration round corresponding to each environmental source, calculate the similarity between the source domain features of the same environmental source and each mixed feature in the corresponding encoding feature set, and construct the domain loss corresponding to each source domain feature based on the similarity calculation result, and optimize the source domain features of the current iteration round corresponding to each source domain feature based on minimizing the domain loss;

[0018] S23: For each set of encoded features, subtract the mixed features therein from the source domain features corresponding to the current iteration round to separate the domain-invariant category features;

[0019] S24: obtaining the current category feature queue, calculating the similarity between each category feature and each category feature in the category feature queue in turn, using the similarity calculation result as the weighted fusion coefficient of the true category label, performing weighted summation on the true category labels of each category feature in the category feature queue, and obtaining the category soft label of each category feature in S23;

[0020] S25: Calculate the cross entropy loss between the category soft label of each category feature corresponding to the batch sample set and the true category label of each image therein as the category invariance loss, input the mixed features extracted by the S21 encoder into the classifier updated in the previous iteration round, and the classifier outputs the predicted category probability, and calculates the cross entropy loss between the predicted category probability and the true category label of each image in the batch sample set. Take two images belonging to the same category in the batch sample set as pairs of images of the same category, calculate the similarity between the category features of the pairs of images of the same category, and construct the domain invariance loss from the similarity calculation results. The weighted sum of the category invariance loss, the cross entropy loss and the domain invariance loss is used as the total loss. Update the image classification model parameters based on minimizing the total loss, and continuously iterate the training until the preset iteration rounds are reached and the image classification model converges to obtain a trained image classification model.

[0021] Furthermore, in step S22, the domain loss The functional form is:

[0022]

[0023] in, Indicates the encoder The first Mixed features; Indicates the number of updates after the last iteration. Source domain features; Represents similarity calculation.

[0024] As a preferred embodiment of the first aspect above, the category features extracted in step S23 and their respective corresponding true category labels are formed into corresponding data pairs and pushed into the category feature queue, and the category features and true category labels first stored in the category feature queue are popped out to obtain a new category feature queue, and then when the next iteration round starts, the new category feature queue is used as the current category feature queue in S24 for similarity calculation.

[0025] As a preferred embodiment of the first aspect, in step S25, the domain invariance loss The functional form is:

[0026]

[0027] in, Indicates that the batch sample set belongs to the category The number of images; Represents the number of categories of images in the source domain image dataset; Indicates that the batch sample set belongs to the category No. Category features of images; Indicates that the batch sample set belongs to the category No. The category features of the images.

[0028] As a preferred embodiment of the first aspect, in step S2, ResNet50 is used as the image classification model.

[0029] In a second aspect, the present invention provides a generalizable classification system based on domain-invariant category features, comprising:

[0030] The data acquisition module is used to acquire a number of images from different environments to construct an original image dataset, and different images from the same environment in the original image dataset constitute a source domain image dataset. Images are sampled in each source domain image dataset, and the sampled images are processed. Multiple batch sample sets for training are constructed from the processed images, and each processed image corresponds to a real category label.

[0031] A result acquisition module, used to input the target domain image to be classified into the trained image classification model, and output the classification result of the image category; wherein the image classification model includes an encoder and a classifier;

[0032] In the training process of the image classification model, the encoder encodes the images in the batch sample set and outputs mixed features. After the mixed features are subtracted from the acquired source domain features, the domain-invariant category features are obtained and stored in the category feature queue for calculating the similarity, and the similarity calculation result is used as the weighted fusion coefficient of the true category label to obtain the category soft label of each category feature, and the cross entropy loss between the category soft label of each category feature and the true category label is used as the category invariance loss. The mixed features are input into the classifier, and the classifier outputs the predicted category probability. The cross entropy loss between the predicted category probability and the true category label of each image in the batch sample set is calculated. Two images belonging to the same category in the batch sample set are taken as pairs of images of the same category, and the similarity between the category features of the pairs of images of the same category is calculated. The domain invariance loss is constructed from the similarity calculation result, and the weighted sum of the category invariance loss, the cross entropy loss and the domain invariance loss is used as the total loss. The image classification model parameters are updated based on minimizing the total loss, and the training is iterated continuously until a preset number of iterations is reached and the image classification model converges to obtain a trained image classification model.

[0033] In a third aspect, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement a generalizable classification method based on domain-invariant category features as described in any one of the schemes of the first aspect above.

[0034] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the generalizable classification method based on domain-invariant category features as described in any of the schemes of the first aspect above is implemented.

[0035] In a fifth aspect, the present invention provides a computer electronic device comprising a memory and a processor;

[0036] The memory is used to store computer programs;

[0037] The processor is used to implement the generalizable classification method based on domain-invariant category features as described in any solution of the first aspect when executing the computer program.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention proposes a generalizable classification method based on domain-invariant category features for training with limited specific style images, which can find key feature classification in the task of classifying unseen new style images and improve the generalization performance of the image classification model. The method of the present invention makes full use of limited training images from the perspective of practical application. Through contrast learning and invariant representation learning methods, the classification performance of simple image classification models for unseen style images is significantly improved. Based on the present invention, while relying only on images with limited styles, images of other diverse styles can be stably predicted, which expands the application scenarios of image classification models in the real world and improves performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of the method of the present invention;

[0041] Figure 2 The image classification model architecture and flow chart provided for the embodiments of the present invention;

[0042] Figure 3 It is a system block diagram of the present invention. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0044] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for the purpose of distinguishing descriptions, and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0045] In order to solve the problems existing in the prior art, an embodiment of the present invention provides a method for finding domain-invariant and class-unique features for generalized classification, which is implemented based on invariant representation learning. The present invention can be trained based on limited data, and find domain-invariant and class-unique features from multiple source domains and multiple categories for classification, so as to improve the generalization ability of the image classification model.

[0046] like Figure 1As shown, in a preferred implementation of the present invention, the above generalizable classification method based on domain-invariant category features includes the following steps S1 to S2. The specific implementation process is described in detail below.

[0047] S1: Obtain a number of images from different environments to construct an original image dataset. Different images from the same environment in the original image dataset constitute a source domain image dataset. Sample images in each source domain image dataset, process the sampled images, and construct multiple batch sample sets for training from the processed images. Each processed image corresponds to a true category label.

[0048] It should be noted that, in the present invention, the specific process of the above step S1 is as follows: In the source domain image dataset, we sample Different images form a sampling data set, randomly crop each image in the sampling data set, then enhance the cropped image with a preset probability, convert the enhanced image into a tensor format, and perform regularization on the format-converted image, and re-encode the true category label corresponding to the regularized image into an index value , the regularized image and its corresponding index value are combined into a sample pair to generate a batch sample set for training .in, Represents a batch sample set The Sample pairs, Represents a batch sample set The images, Represents a batch sample set The The index value of the image; , Indicates the number of environmental sources; represents the number of images in the sampled dataset; Represents the number of categories of images in the source domain image dataset.

[0049] In step S1 of the embodiment of the present invention, when performing random center cropping on the image in the sampled data set, the size of the image is first determined, and an image of uniform size is cropped from the center point. When performing image enhancement on the randomly center cropped image, the image is randomly horizontally flipped with a probability of 50%, and the brightness, contrast, saturation and hue of the image are changed with a probability of 50%.

[0050] S2: Input the image to be classified in the target domain into the trained image classification model, and output the classification result of the image category; wherein the image classification model includes an encoder and a classifier.

[0051] In the training process of the image classification model, the encoder encodes the images in the batch sample set and outputs mixed features. After the mixed features are subtracted from the acquired source domain features, the domain-invariant category features are obtained and stored in the category feature queue for calculating the similarity, and the similarity calculation result is used as the weighted fusion coefficient of the true category label to obtain the category soft label of each category feature, and the cross entropy loss between the category soft label of each category feature and the true category label is used as the category invariance loss. The mixed features are input into the classifier, and the classifier outputs the predicted category probability. The cross entropy loss between the predicted category probability and the true category label of each image in the batch sample set is calculated. Two images belonging to the same category in the batch sample set are taken as pairs of images of the same category, and the similarity between the category features of the pairs of images of the same category is calculated. The domain invariance loss is constructed from the similarity calculation result, and the weighted sum of the category invariance loss, the cross entropy loss and the domain invariance loss is used as the total loss. The image classification model parameters are updated based on minimizing the total loss, and the training is iterated continuously until a preset number of iterations is reached and the image classification model converges to obtain a trained image classification model.

[0052] It should be noted that, in the present invention, the specific network structure of the image classification model is not limited, as long as it contains an encoder and a classifier. In this embodiment, ResNet50 is specifically used as the image classification model, and its implementation method belongs to the prior art and will not be repeated here.

[0053] It should be noted that, in the present invention, Figure 2 As shown in the figure, the training process of the image classification model is as follows:

[0054] S21: In the current iteration, batch samples are collected according to the environment source. Divide into sub-datasets, The sub-data sets are batch-entered into the encoder updated in the previous iteration. Feature extraction is performed on each image in the sub-datasets, and the features extracted by the encoder are used as mixed features. All mixed features of each sub-dataset constitute the encoding feature set corresponding to each sub-dataset.

[0055] In this embodiment, Sub-datasets All mixed features constitute the The encoding feature set corresponding to the sub-data set .in, Respectively represent the samples from the first The first sample pairs; Respectively represent the encoder The first A mixed feature.

[0056] S22: Obtain the updated source domain features of the previous iteration round corresponding to each environmental source, calculate the similarity between the source domain features of the same environmental source and each mixed feature in the corresponding encoding feature set, and construct the domain loss corresponding to each source domain feature based on the similarity calculation result, and optimize the source domain features of the current iteration round corresponding to each of them based on minimizing the domain loss.

[0057] In this embodiment, A random noise is used as the initialized source domain feature. In each subsequent iteration round, the source domain features updated in the previous iteration round need to be obtained and updated.

[0058] In this embodiment, all source domain features after the previous iteration round update can be expressed as , and then calculate the first updated value after the previous iteration Source domain features And the corresponding mixed features in the encoding feature set The similarity between Mixed features For example, it is updated with the last iteration round. Source domain features Similarity The calculation method is as follows:

[0059]

[0060] in, is a hyperparameter, is a vector and Dimensions; Represents matrix transpose; Indicates normalization.

[0061] In this way, for each source domain feature of the environment source, after calculating the similarity with the corresponding mixed feature, we can get The similarity calculation results are then added together to form the domain loss corresponding to each source domain feature to optimize each source domain feature. Source domain features For example, the corresponding domain loss It can be expressed as:

[0062]

[0063] S23: For each set of encoded features, subtract the mixed features therein from the source domain features corresponding to the current iteration round to separate the domain-invariant category features.

[0064] It should be noted that in the present invention, for each image input in the batch sample set, the feature extracted by the encoder is a mixture of source domain features and category features. Therefore, the present invention subtracts the corresponding source domain feature from each mixed feature to separate the domain-invariant and category-related features, namely, the category features.

[0065] In this embodiment, The first Mixed features For example, the first Source domain features After subtraction, the corresponding category features are obtained , Marked The category number to which the category feature belongs.

[0066] S24: Obtain the current category feature queue, calculate the similarity between each category feature and each category feature in the category feature queue in turn, use the similarity calculation result as the weighted fusion coefficient of the true category label, perform weighted summation on the true category labels of each category feature in the category feature queue, and obtain the category soft label of each category feature in S23.

[0067] In this embodiment, after all sample pairs in the batch sample set undergo step S23, a category feature set can be obtained. , Indicates the corresponding In order to calculate the similarity with multiple previously learned category features, the present invention constructs a category feature queue , to temporarily store multiple learned Category features and its corresponding true category label In the initial stage, the category feature queue is used The random noise with the same scale as the category feature is used as the initialized category feature. In this embodiment, the number of category features stored in the category feature queue is A positive integer multiple of .

[0068] Next, for each category feature obtained in S23, the similarity between it and each category feature in the category feature queue is calculated in turn, and then a size of The similarity matrix Then, the similarity in the similarity matrix is ​​used as the weighted fusion coefficient of the real category label of the one-hot code to calculate the category soft label of each category feature. Category features For example, the similarity is calculated with each category feature in the category feature queue, and the first Category features Category soft label :

[0069]

[0070] in, and Respectively represent the first Category features and their corresponding true category labels; express and The similarity between .

[0071] It should be noted that, in the present invention, the category features extracted in step S23 are And their corresponding true category labels Form corresponding data pairs The category feature queue is pushed in, and the category feature and the true category label stored first are popped out to obtain a new category feature queue. Then, when the next iteration round starts, the new category feature queue is used as the current category feature queue in S24 for similarity calculation.

[0072] S25: Calculate the cross entropy loss between the category soft label of each category feature corresponding to the batch sample set and the true category label of each image in it as the category invariance loss :

[0073]

[0074] S26: Input the mixed features extracted by the S21 encoder into the classifier updated in the previous iteration round, and the classifier outputs the predicted category probability. Calculate the cross entropy loss between the predicted category probability and the true category label of each image in the batch sample set :

[0075]

[0076] in, Represents the corresponding first The class probability of the image; represents the classifier, Indicates the encoder.

[0077] S27: Take two images of the same category in the batch sample set as pairs of images of the same category, calculate the similarity between the category features of the pairs of images of the same category, and construct the domain invariance loss based on the similarity calculation results :

[0078]

[0079] in, Indicates that the batch sample set belongs to the category The number of images; Represents the number of categories of images in the source domain image dataset; Indicates that the batch sample set belongs to the category No. Category features of images; Indicates that the batch sample set belongs to the category No. The category features of the images.

[0080] By optimizing the domain invariance loss, the similarity of features of the same category from different source domains is pulled closer to strengthen the independence of classification features from categories.

[0081] S28: Loss due to category invariance , cross entropy loss And domain invariance loss The weighted sum is the total loss :

[0082]

[0083] in, and Both represent weighting parameters.

[0084] S29: updating the image classification model parameters based on minimizing the total loss, and continuously iterating the training until a preset number of iterations is reached and the image classification model converges, thereby obtaining a trained image classification model.

[0085] It should be noted that, in the present invention, the domain invariance loss is optimized to ensure that the separated features are domain-independent. Specifically, the similarity between domain-invariant and category-related features extracted from images of the same category from different source domains is optimized, and the similarity is maximized to improve the domain invariance of the extracted features; the cross entropy loss between the category probability predicted by the classifier output and the true category label of each image is optimized to improve the similarity between similar features and exclude the similarity between non-similar features, that is, to strengthen the category specificity of the extracted features; the category invariance loss between the category soft label of the category feature and the true category label of each image is optimized to strengthen the representativeness of the extracted features for the current category, reduce the similarity between similar features, and ensure that the separated features have sufficient category specificity to highlight the category uniqueness of the extracted features of each category.

[0086] The generalizable classification method based on domain-invariant category features in the above embodiment is applied to a specific data set for classification testing. The specific steps are as described in S1 to S2, which will not be repeated here, and the specific parameters and technical effects are mainly shown.

[0087] Example

[0088] This embodiment follows the implementation process of the aforementioned S1~S2 steps, firstly obtains an image data set collected from the real world, and classifies it according to style and category. Use these images as training data to train the image classification model proposed in the present invention, and test the accuracy on an unseen target domain. For quantitative indicators, this embodiment uses ResNet50 as the image classification model, randomly sets seeds to train the image classification model three times and test its performance, and calculates the average accuracy. Four generalized data sets of PACS, OfficeHome, TerraIncognita and DomainNet were selected for multi-domain generalization ability testing. The comparison results of the image classification model of the present invention and other models are shown in Table 1. The implementation methods of the ERM model, CORAL model, SagNet model, CB-CORAL model and RDM model in Table 1 belong to the prior art and will not be repeated.

[0089] Table 1. Multi-domain generalization evaluation table

[0090]

[0091] It should also be noted that the generalizable classification method based on domain-invariant category features in the above embodiment can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a generalizable classification system based on domain-invariant category features corresponding to the generalizable classification method based on domain-invariant category features provided in the above embodiment, such as Figure 3As shown, it includes:

[0092] The data acquisition module is used to acquire a number of images from different environments to construct an original image dataset, and different images from the same environment in the original image dataset constitute a source domain image dataset. Images are sampled in each source domain image dataset, and the sampled images are processed. Multiple batch sample sets for training are constructed from the processed images, and each processed image corresponds to a real category label.

[0093] A result acquisition module, used to input the target domain image to be classified into the trained image classification model, and output the classification result of the image category; wherein the image classification model includes an encoder and a classifier;

[0094] In the training process of the image classification model, the encoder encodes the images in the batch sample set and outputs mixed features. After the mixed features are subtracted from the acquired source domain features, the domain-invariant category features are obtained and stored in the category feature queue for calculating the similarity, and the similarity calculation result is used as the weighted fusion coefficient of the true category label to obtain the category soft label of each category feature, and the cross entropy loss between the category soft label of each category feature and the true category label is used as the category invariance loss. The mixed features are input into the classifier, and the classifier outputs the predicted category probability. The cross entropy loss between the predicted category probability and the true category label of each image in the batch sample set is calculated. Two images belonging to the same category in the batch sample set are taken as pairs of images of the same category, and the similarity between the category features of the pairs of images of the same category is calculated. The domain invariance loss is constructed from the similarity calculation result, and the weighted sum of the category invariance loss, the cross entropy loss and the domain invariance loss is used as the total loss. The image classification model parameters are updated based on minimizing the total loss, and the training is iterated continuously until a preset number of iterations is reached and the image classification model converges to obtain a trained image classification model.

[0095] It is understandable that the generalizable classification method based on domain-invariant category features described in S1-S2 above can be substantially implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the generalizable classification method based on domain-invariant category features provided in the above embodiment, which includes a computer program / instruction. When the computer program / instruction is executed by a processor, the generalizable classification method based on domain-invariant category features described in the above embodiment can be implemented.

[0096] Similarly, based on the same inventive concept, another preferred embodiment of the present invention further provides a computer electronic device corresponding to the generalizable classification method based on domain-invariant category features provided in the above embodiment, which includes a memory and a processor;

[0097] The memory is used to store computer programs;

[0098] The processor is used to implement the generalizable classification method based on domain-invariant category features in the above embodiment when executing the computer program.

[0099] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0100] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the generalizable classification method based on domain-invariant category features provided in the above embodiment, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the generalizable classification method based on domain-invariant category features in the above embodiment can be implemented.

[0101] It is understandable that the above storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. The storage medium may also be a U disk, a mobile hard disk, a magnetic disk or an optical disk, etc., which can store program codes.

[0102] It is understandable that the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0103] It should also be noted that those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the various embodiments provided in this application, the division of steps or modules in the system and method is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or steps can be combined or integrated together, and a module or step can also be split.

[0104] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A generalizable classification method based on domain-invariant category features, characterized in that: The following steps are involved: S1: Obtain a number of images from different environments to construct an original image dataset. Different images from the same environment in the original image dataset constitute a source domain image dataset. Sample images in each source domain image dataset, process the sampled images, and construct multiple batch sample sets for training from the processed images. Each processed image corresponds to a true category label. S2: Input the image to be classified in the target domain into the trained image classification model, and output the classification result of the image category; wherein the image classification model includes an encoder and a classifier; In the training process of the image classification model, the encoder encodes the images in the batch sample set and outputs mixed features. After the mixed features are subtracted from the acquired source domain features, the domain-invariant category features are obtained and stored in the category feature queue for calculating the similarity, and the similarity calculation result is used as the weighted fusion coefficient of the true category label to obtain the category soft label of each category feature, and the cross entropy loss between the category soft label of each category feature and the true category label is used as the category invariance loss. The mixed features are input into the classifier, and the classifier outputs the predicted category probability. The cross entropy loss between the predicted category probability and the true category label of each image in the batch sample set is calculated. Two images belonging to the same category in the batch sample set are taken as pairs of images of the same category, and the similarity between the category features of the pairs of images of the same category is calculated. The domain invariance loss is constructed from the similarity calculation result, and the weighted sum of the category invariance loss, the cross entropy loss and the domain invariance loss is used as the total loss. The image classification model parameters are updated based on minimizing the total loss, and the training is iterated continuously until a preset number of iterations is reached and the image classification model converges to obtain a trained image classification model.

2. A generalizable classification method based on domain-invariant category features as claimed in claim 1, characterized in that: In step S1, in K source domain image data sets, M different images are sampled to form a sampling data set, each image in the sampling data set is randomly center cropped, and then the cropped image is enhanced with a preset probability, the enhanced image is converted into a tensor format, and the format-converted image is regularized, the true category label corresponding to the regularized image is re-encoded into an index value, the regularized image and its corresponding index value are combined into a sample pair, and a batch sample set for training is generated; wherein K represents the number of environmental sources; M represents the number of images in the sampling data set.

3. A generalizable classification method based on domain-invariant category features as claimed in claim 1, characterized in that: In step S2, the training process of the image classification model is specifically as follows: S21: In the current iteration round, the batch sample set is divided into K sub-data sets according to the environmental source, and the K sub-data sets are batch-input into the encoder updated in the previous iteration round, and features are extracted for each image in the K sub-data sets. The features extracted by the encoder are used as mixed features, and all the mixed features of each sub-data set constitute the encoding feature set corresponding to each sub-data set; S22: Obtain the updated source domain features of the previous iteration round corresponding to each environmental source, calculate the similarity between the source domain features of the same environmental source and each mixed feature in the corresponding encoding feature set, and construct the domain loss corresponding to each source domain feature based on the similarity calculation result, and optimize the source domain features of the current iteration round corresponding to each source domain feature based on minimizing the domain loss; S23: For each set of encoded features, subtract the mixed features therein from the source domain features corresponding to the current iteration round to separate the domain-invariant category features; S24: obtaining the current category feature queue, calculating the similarity between each category feature and each category feature in the category feature queue in turn, using the similarity calculation result as the weighted fusion coefficient of the true category label, performing weighted summation on the true category labels of each category feature in the category feature queue, and obtaining the category soft label of each category feature in S23; S25: Calculate the cross entropy loss between the category soft label of each category feature corresponding to the batch sample set and the true category label of each image therein as the category invariance loss, input the mixed features extracted by the S21 encoder into the classifier updated in the previous iteration round, and the classifier outputs the predicted category probability, and calculates the cross entropy loss between the predicted category probability and the true category label of each image in the batch sample set. Take two images belonging to the same category in the batch sample set as pairs of images of the same category, calculate the similarity between the category features of the pairs of images of the same category, and construct the domain invariance loss from the similarity calculation results. The weighted sum of the category invariance loss, the cross entropy loss and the domain invariance loss is used as the total loss. Update the image classification model parameters based on minimizing the total loss, and continuously iterate the training until the preset iteration rounds are reached and the image classification model converges to obtain a trained image classification model.

4. A generalizable classification method based on domain-invariant category features as claimed in claim 3, characterized in that: The category features extracted in step S23 and their respective corresponding true category labels are formed into corresponding data pairs and pushed into the category feature queue, and the category features and true category labels first stored in the category feature queue are popped out to obtain a new category feature queue. Then, when the next iteration round starts, the new category feature queue is used as the current category feature queue in S24 for similarity calculation.

5. A generalizable classification method based on domain-invariant category features as claimed in claim 1, characterized in that: In step S25, the domain invariance loss The functional form is: ； in, Indicates that the batch sample set belongs to the category The number of images; Represents the number of categories of images in the source domain image dataset; Indicates that the batch sample set belongs to the category No. Category features of images; Indicates that the batch sample set belongs to the category No. The category features of the images.

6. A generalizable classification method based on domain-invariant category features as claimed in claim 1, characterized in that: In step S2, ResNet50 is used as the image classification model.

7. A generalizable classification system based on domain-invariant category features, characterized in that: include: The data acquisition module is used to acquire a number of images from different environments to construct an original image dataset, and different images from the same environment in the original image dataset constitute a source domain image dataset. Images are sampled in each source domain image dataset, and the sampled images are processed. Multiple batch sample sets for training are constructed from the processed images, and each processed image corresponds to a real category label. A result acquisition module, used to input the target domain image to be classified into the trained image classification model, and output the classification result of the image category; wherein the image classification model includes an encoder and a classifier; In the training process of the image classification model, the encoder encodes the images in the batch sample set and outputs mixed features. After the mixed features are subtracted from the acquired source domain features, the domain-invariant category features are obtained and stored in the category feature queue for calculating the similarity, and the similarity calculation result is used as the weighted fusion coefficient of the true category label to obtain the category soft label of each category feature, and the cross entropy loss between the category soft label of each category feature and the true category label is used as the category invariance loss. The mixed features are input into the classifier, and the classifier outputs the predicted category probability. The cross entropy loss between the predicted category probability and the true category label of each image in the batch sample set is calculated. Two images belonging to the same category in the batch sample set are taken as pairs of images of the same category, and the similarity between the category features of the pairs of images of the same category is calculated. The domain invariance loss is constructed from the similarity calculation result, and the weighted sum of the category invariance loss, the cross entropy loss and the domain invariance loss is used as the total loss. The image classification model parameters are updated based on minimizing the total loss, and the training is iterated continuously until a preset number of iterations is reached and the image classification model converges to obtain a trained image classification model.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, it can implement the generalizable classification method based on domain-invariant category features as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the generalizable classification method based on domain-invariant category features as described in any one of claims 1 to 6 is implemented.

10. A computer electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the generalizable classification method based on domain-invariant category features as described in any one of claims 1 to 6 when executing the computer program.

Images