A model training method, a target classification method, an apparatus, a device, and a medium

Abstract

The application discloses a model training method, a target classification method, an apparatus, a device and a medium. The model training method comprises: obtaining a real domain unbalanced data set and a virtual domain balanced data set; generating a real domain augmented balanced data set according to the real domain unbalanced data set and the virtual domain balanced data set; training a model based on the real domain unbalanced data set, the virtual domain balanced data set, the real domain augmented balanced data set and a loss function; and the loss function comprises a domain self-adaptive loss function and a supervised loss function. The scheme uses a class confusion matrix to enhance tail class samples of the real domain unbalanced data set, improves the diversity of the tail class features, solves the domain shift problem caused by the inconsistency of the class distribution between the real domain and the virtual domain, realizes cross-domain self-adaptive learning of virtual and real sample mixing, enables the model to effectively learn the diversity of the sample features in the balanced virtual domain and migrate to the unbalanced real domain, and improves the generalization and robustness of the model in the unbalanced real domain.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model training method, a target classification method, an apparatus, a device, and a medium. Background Technology

[0002] Object classification models can be used to enable UAVs to visually perceive maritime targets (ships). Currently, the training of object classification models is mainly based on real-world sample data with imbalanced class distributions, which limits the classification accuracy of these models. This is because, compared to the large standard datasets used in basic theoretical research, the number and quality of imbalanced class distribution samples collected in real-world scenarios are significantly different. Therefore, it is necessary to consider expanding the existing samples to increase the diversity of the training dataset, thereby improving the model's performance.

[0003] In some cases, virtual simulation environments (i.e., virtual scenarios) can be used for the learning and validation of intelligent unmanned systems. Examples include OpenAI's open-source multi-agent hide-and-seek environment, Airsim's unmanned vehicle and drone simulations, and ML-Agents-based multi-type intelligent agent training simulation environments. Virtual simulation environments generally possess sufficiently powerful physics engines to effectively simulate the environment. Furthermore, they can construct diverse instances for each category through custom modeling, achieving a balanced distribution of targets across different categories within the virtual scenario and increasing the feature diversity of each target type. To address this, one could consider training the model using sample data from both the real-world and virtual domains collected in the real-world scenario. However, a key issue to consider is that the features of samples collected from the real-world scenario often exhibit distributional differences compared to those from the virtual scenario. This domain shift in the sample data features between different scenarios can lead to model learning confusion. Even if the samples in the virtual domain are relatively balanced and diverse, the model learned from virtual data may still not demonstrate the same excellent learning performance in the real-world domain.

[0004] In summary, how to solve the problem of effective cross-domain learning of models under mixed multi-domain sample conditions, enabling models to effectively learn the diversity of sample features in a balanced virtual domain and transfer them to an imbalanced real domain, thereby improving model performance through mixed virtual and real data learning, is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a model training method, target classification method, device, equipment, and medium to achieve cross-domain adaptive learning with mixed virtual and real samples, thereby improving the generalization and robustness of the target classification model.

[0006] In a first aspect, embodiments of this application provide a method for training a target classification model, including: Obtain the imbalanced dataset in the real domain and the balanced dataset in the virtual domain; A real-domain augmented equilibrium dataset is generated based on the real-domain imbalanced dataset and the virtual-domain balanced dataset. The real-domain augmented equilibrium dataset is obtained by augmenting the tail class sample data in the real-domain imbalanced dataset using the statistics of the real-domain imbalanced dataset and the virtual-domain balanced dataset, as well as the class confusion matrix of the real domain. The target classification model is trained based on the virtual domain balanced dataset, the real domain augmented balanced dataset, and the loss function; the loss function includes a domain adaptive loss function and a supervised loss function, the domain adaptive loss function is used to measure the difference in feature distribution between sample data in the real domain and the virtual domain, and the supervised loss function is used to measure the target classification accuracy of the target classification model.

[0007] Secondly, embodiments of this application also provide a target classification method, including: Obtain visual data of the target; The visual data is input into the target classification model to obtain the classification result of the target; The target classification model is trained based on the target classification model training method described in the first aspect.

[0008] Thirdly, embodiments of this application also provide a target classification model training apparatus, comprising: The acquisition module is used to acquire imbalanced datasets in the real domain and balanced datasets in the virtual domain. The generation module is used to generate a real-domain augmented equilibrium dataset based on the real-domain imbalanced dataset and the virtual-domain balanced dataset. The real-domain augmented equilibrium dataset is obtained by augmenting the tail class sample data in the real-domain imbalanced dataset using the statistics of the real-domain imbalanced dataset and the virtual-domain balanced dataset, as well as the class confusion matrix of the real domain. The training module is used to train a target classification model based on the virtual domain balanced dataset, the real domain augmented balanced dataset, and a loss function. The loss function includes a domain adaptive loss function and a supervised loss function. The domain adaptive loss function is used to measure the difference in feature distribution between sample data in the real domain and the virtual domain, and the supervised loss function is used to measure the target classification accuracy of the target classification model.

[0009] Fourthly, embodiments of this application provide an electronic device, including: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the target classification model training method as described in the first aspect.

[0010] Fifthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the target classification model training method as described in the first aspect.

[0011] This application provides a model training method, a target classification method, an apparatus, a device, and a medium. The target classification model training method includes: acquiring a real-domain imbalanced dataset and a virtual-domain balanced dataset; generating a real-domain augmented balanced dataset based on the real-domain imbalanced dataset and the virtual-domain balanced dataset, wherein the real-domain augmented balanced dataset is obtained by augmenting the tail class sample data in the real-domain imbalanced dataset using statistics from the real-domain imbalanced dataset and the virtual-domain balanced dataset, as well as the class confusion matrix of the real-domain dataset; training a target classification model based on the virtual-domain balanced dataset, the real-domain augmented balanced dataset, and a loss function; the loss function includes a domain adaptive loss function and a supervised loss function, wherein the domain adaptive loss function is used to measure the difference in feature distribution between the sample data in the real-domain and virtual-domain, and the supervised loss function is used to measure the target classification accuracy of the target classification model. The above technical solution uses a class confusion matrix to enhance the tail class samples of imbalanced datasets in the real domain, improves the diversity of tail class features, solves the domain shift problem caused by the inconsistency between the class distribution in the real domain and the virtual domain, realizes cross-domain adaptive learning of mixed virtual and real samples, enables the model to effectively learn the diversity of sample features in the balanced virtual domain and transfer them to the imbalanced real domain, and improves the generalization and robustness of the model in the imbalanced real domain. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0013] Figure 1 A flowchart illustrating a target classification model training method provided in this application embodiment; Figure 2 A flowchart of a target classification method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a target classification model training device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a target classification device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0014] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present application, not the entire structure.

[0015] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0016] It should be noted that the concepts of "first" and "second" mentioned in the embodiments of this application are only used to distinguish different devices, modules, units or other objects, and are not used to limit the order of functions performed by these devices, modules, units or other objects or their interdependencies.

[0017] It should also be noted that, unless otherwise specified, "multiple" in the embodiments of this application refers to at least two.

[0018] Furthermore, the embodiments and features described in this application may be combined with each other, unless otherwise specified.

[0019] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0020] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the relevant content of the solution.

[0021] Figure 1This flowchart illustrates a target classification model training method provided in this application embodiment. This embodiment is applicable to training target classification models, where the target classification model can refer to a deep learning model or machine learning model used to identify targets or determine target types. Taking maritime target classification as an example, the target classification model can be a model used to identify ship types. Exemplarily, the target classification model mainly refers to a neural network model, such as a feedforward neural network (FNN), convolutional neural network (CNN), recurrent neural network (RNN), graph neural network (GNN), or Transformer model, etc. This target classification model training method can be executed by a target classification model training device, which can be implemented through software and / or hardware and integrated into an electronic device. The electronic device includes, but is not limited to, devices with data processing capabilities such as computers, smartphones, or servers.

[0022] like Figure 1 As shown, the method specifically includes the following steps: S110. Obtain the imbalanced dataset in the real domain and the balanced dataset in the virtual domain; For example, a balanced virtual domain dataset can refer to a virtual domain training dataset with a balanced class distribution obtained by collecting data based on a constructed virtual simulation scenario, represented as: Imbalanced datasets in the real domain can refer to real-domain training datasets with imbalanced class distributions, denoted as... .

[0023] S120. Generate a real-domain augmented equilibrium dataset based on the real-domain imbalanced dataset and the virtual-domain balanced dataset, wherein the real-domain augmented equilibrium dataset is obtained by augmenting the tail class sample data in the real-domain imbalanced dataset using the statistics of the real-domain imbalanced dataset and the virtual-domain balanced dataset, as well as the class confusion matrix of the real domain.

[0024] In this embodiment, the training dataset used to train the target classification model includes, in addition to the imbalanced real-domain dataset and the balanced virtual-domain dataset, a real-domain augmented balanced dataset. The real-domain augmented balanced dataset can be understood as a real-domain training dataset with a balanced class distribution obtained by expanding or enhancing the sample data in the imbalanced real-domain dataset. In this process, the focus is on expanding or enhancing the sample data of the tail classes (classes with fewer samples and lower frequency of occurrence) in the imbalanced real-domain dataset, generating more sample data for them. This ensures that the distribution of the number of samples of the same class is consistent between the real-domain and virtual-domain datasets, improving the feature diversity of the tail classes in the imbalanced real-domain dataset and achieving the goal of balanced class distribution. For example, if the number of samples of a certain class in the imbalanced real-domain dataset is less than a specified number, or the proportion of samples of a certain class in the imbalanced real-domain dataset is less than a specified percentage, then that class is a tail class; or, for example, the classes with the fewest samples in the imbalanced real-domain dataset can be designated as tail classes. Optionally, in an imbalanced dataset in the real domain, the categories with more samples and higher frequency of occurrence are designated as non-tailed classes. These non-tailed classes can be further subdivided into head classes (i.e., categories with a high number of samples and high frequency of occurrence) and middle classes (i.e., categories with a moderate number of samples and moderate frequency of occurrence) based on the number of samples or frequency of occurrence.

[0025] When expanding or enhancing the model, the distribution characteristics of imbalanced datasets in the real domain and balanced datasets in the virtual domain can be combined. This allows the model to effectively learn the diverse distribution of sample features in the balanced virtual domain and transfer them to the imbalanced domain. The distribution characteristics of the training dataset are mainly reflected by statistics such as mean and covariance. Furthermore, for real-domain sample data, the class confusion matrix analysis of the real-domain domain can be used to determine the probability of identifying a target of one class as another. This can guide the expansion or enhancement process, resulting in a more balanced distribution of the expanded or enhanced sample data. This addresses the domain shift problem caused by the inconsistency between the class distributions of the real and virtual domains, providing more accurate and higher-quality features for model training.

[0026] S130. Train a target classification model based on the virtual domain balanced dataset, the real domain augmented balanced dataset, and a loss function; the loss function includes a domain adaptive loss function and a supervised loss function, the domain adaptive loss function is used to measure the difference in feature distribution between sample data in the real domain and the virtual domain, and the supervised loss function is used to measure the target classification accuracy of the target classification model.

[0027] In this embodiment, both the real-domain augmented balanced dataset (containing the real-domain imbalanced dataset and the expanded sample data) and the virtual-domain balanced dataset are used as the training dataset, and cross-domain adaptive learning is employed for training. The loss function consists of at least two parts: a domain adaptive loss function and a supervised loss function. The domain adaptive loss function guides the alignment of features between the virtual and real domains, ensuring a more consistent class distribution and avoiding domain shift issues. The supervised loss function guarantees the accuracy of the target classification model. Combining both improves the overall performance of the target classification model. During training, variable weights can be set for the domain adaptive loss function and the supervised loss function to emphasize training at different performance levels at different stages.

[0028] The target classification model training method provided in this application embodiment is based on a training dataset enhanced by the hybrid feature of virtual and real classes. It adopts a cross-domain adaptive learning approach, which can solve the domain offset problem caused by the introduction of a balanced virtual domain, realize the alignment between domain features, solve the domain offset problem caused by the inconsistency between the class distribution of the real domain and the virtual domain, enhance the generalization and robustness of the target classification model in the imbalanced real domain, and improve the overall performance of the target classification model.

[0029] In one embodiment, generating a real-domain augmented equilibrium dataset based on a real-domain imbalanced dataset and the virtual-domain balanced dataset includes: For each batch of sample data in the real-domain imbalanced dataset and the virtual-domain balanced dataset: S1210. Using the feature extraction network in the target classification model, obtain the hidden layer features of each category in the real domain and each category in the virtual domain for the current batch. Determine the global statistics of each category accumulated at the current time based on the hidden layer features of each category in the real domain and each category in the virtual domain. The global statistics include the global mean and the global covariance.

[0030] In this embodiment, the hidden layer feature enhancement method is used to enhance the tail class sample data in the real domain, so that the distribution of the number of similar samples in the real domain and the virtual domain is consistent.

[0031] In object classification models, the feature extraction network is responsible for automatically learning and extracting multi-level semantic features from the input image. Its core component is the hidden layer. These hidden layers, through operations such as convolution, pooling, and non-linear activation, progressively transform the original pixels into high-level feature representations suitable for classification. The feature extraction network can be a Convolutional Neural Network (CNN) or a Transformer model, etc. For example, a real-world imbalanced dataset ( ) and virtual domain balanced dataset ( The sample data in both datasets can be processed in batches, with each batch obtained from both datasets separately. One sample data, The batch size is used for each optimization or iteration process. and Using a batch of sample data as input, a feature extraction network is used to extract hidden features from the real domain and the virtual domain respectively. For example, for the first sample data in the current batch... One sample data, defined From A category label obtained from sampling is Sample data, From A category label obtained from sampling is Sample data, among which, , yes The sample data in this batch contains sample images of the target. yes The sample data in this batch contains sample images of the target. and The features are processed by the feature extraction network of the target classification model (this processing can be represented as a function). The hidden features of various categories in the real domain are obtained accordingly. Hidden features of various categories of virtual domains Hidden features can include features such as edges, textures, colors, shapes, corners, patterns, and object structures contained in the input sample data, which can provide a basis for target classification.

[0032] Based on the latent features of each category in the real domain and the virtual domain, the statistical measures of the latent features of each category in the real and virtual domains for the current batch can be determined. For example, the time corresponding to the current batch is the current time. For the current batch, hidden features of all categories were obtained; for any category... c It can calculate the category in the current batch of two domains. c The mean of the hidden layer features and the category in the current batch of the real domain. c Covariance of hidden layer features (Its form is a matrix, namely the covariance matrix). and Can be used as the current moment Or the statistics for the current batch; in addition, at the current moment Previously, one or more batches of hidden features may have been acquired, and for each batch, hidden features for all categories were acquired. Based on this, and using the hidden features for each category acquired in the current batch, the global mean of the accumulated hidden features for each category in both domains over all time steps (i.e., all batches processed from the first batch, the second batch, up to the current batch) can be updated. Global covariance of the real domain ,in, , C The total number of categories for the target. For example, up to the previous time step. (i.e., up to the previous batch), the global mean of the two domains is The global covariance of the real domain is Then for the current moment Alternatively, for the current batch, the mean and covariance can be recalculated based on the hidden features of all categories in the two domains obtained in previous batches, plus the newly obtained hidden features of all categories in the two domains in the current batch, to obtain the result up to the current time step. Or the global mean of the two domains in the current batch. Global covariance of the real domain .

[0033] S1220. Determine the class confusion matrix of the real domain, and obtain the corrected statistics of the real domain for each type of sample data based on the class confusion matrix. The corrected statistics include the corrected mean and the corrected covariance.

[0034] For example, based on Calculate the real-domain confusion matrix from all sample data. In the class confusion matrix, rows represent the true class labels, and columns represent the predicted classes output by the target classification model when predicting the sample data. Line 1 Column elements This can represent the true category label as In the sample data, those identified as categories The number of samples in the true class label is The proportion of the total number of samples, that is, the true category label. The probability that the sample data is misclassified. Specifically, it can be expressed as: ,in, Indicates an indicator function, when the condition is met. Output 1 if true, output 0 if false. Represents the hidden features of the input Perform category prediction, Indicates category All sample data, "" indicates a logical AND operation. In this formula, the numerator represents the category label of the sample data. (Right now The sample data was identified as category (Right now The number of samples, that is, for Each sample data point is evaluated sequentially, and for each sample data point that satisfies... and ,but If the value is 1, then the cumulative number of samples meeting this condition is incremented by 1. The final cumulative value is the category label of the sample data. The sample data was identified as a category The number of samples; then divide by The entire formula represents the category label as The proportion of sample data, i.e., the category label is The probability that the sample data is misclassified.

[0035] Based on this, the global mean and global covariance of the latent features of various types of sample data in the real domain can be corrected using the confusion matrix to obtain the corrected mean. and corrected covariance ,in, The corrected identifier indicates that the corresponding statistic is the corrected value. Specifically, the number of tail class samples is relatively small, lacking sufficient data support when calculating global statistics for them. However, by utilizing the inter-class misclassification information provided by the confusion matrix, the representation of the tail class in the feature space can be adjusted to alleviate the problem of insufficient feature learning caused by the long-tail distribution. For example, for the tail class... global mean According to the tail class In the corresponding row of the confusion matrix, the tail class can be... of Alternatively, the model can be shifted towards other classes that are frequently misclassified (i.e., non-tail classes, which could be head classes and / or middle classes), or the feature distributions of other classes can be learned and transferred to the sample data of tail classes; for tail classes global covariance If the tail class Widespread misclassification in the confusion matrix indicates dispersed features, which can be corrected. By shrinking the features, or calculating the intra-class variance of other classes and transferring it to the tail class, the features of the tail class can be expanded, thereby indirectly correcting its global covariance.

[0036] S1230. Generate augmented equilibrium sample data of the current batch based on the correction statistic.

[0037] In this embodiment, by using the corrected statistics, the features of the tail class can be expanded to generate richer and more balanced tail class sample data. This data is then merged with other class sample data, resulting in a real-domain augmented balanced sample data distribution that satisfies the distribution characteristics of the real-domain tail class sample data after correction based on the class confusion matrix. This results in higher quality and a more balanced distribution. Based on this, the real-domain augmented balanced sample data from each batch constitutes the real-domain augmented balanced sample dataset.

[0038] Building upon this, the use of hidden layer feature enhancement methods to augment tail class samples in the real domain can improve the diversity of tail class features in the real domain, making the distribution of the number of similar samples in the real and virtual domains consistent. This solves the problem of domain adaptive learning difficulties caused by the inconsistent distribution of categories in the real and virtual domains, thereby achieving effective virtual data-assisted model learning. It realizes information sharing between hidden layer features of different categories in the same domain and different categories in the same domain, enhances the generalization of the model to extract features from real data, and achieves better performance under the test conditions of balanced categories in real data.

[0039] In one embodiment, the correction statistics for the real-world domain of each category are obtained based on the class confusion matrix and the global statistics of each category, including: for sample data of each category in the current batch, based on the class confusion matrix, calculating the weighted average of the global statistics of the sample data of all other categories except the stated category, to obtain the correction statistics for the real-world domain of each category; wherein, the weight corresponding to any other category except the stated category is: the element in the class confusion matrix corresponding to the stated category and the other categories, the element representing the probability that the sample data of the stated category is misclassified as the other category.

[0040] In this embodiment, a class confusion matrix that is updated during the training process is maintained to dynamically measure the feature similarity between different classes. Based on the feature similarity between classes, the global mean and global covariance of each class are corrected. In this way, the sampling space for tail class feature enhancement is expanded by using the corrected mean and covariance, thereby improving the diversity of tail class enhancement features.

[0041] For example, it can be based on the class confusion matrix according to the following formula. The corrected means for each category were calculated. and corrected covariance : ; ;in, Indicates the first Category 1 (the first) Each category is excluding the category. The global mean of the sample data (excluding other categories). Indicates the first The global covariance of the sample data for each class For category The corresponding weights represent the categories. Misclassified The probability of.

[0042] In one embodiment, generating augmented equilibrium sample data of the current batch of real-domain data based on the correction statistic includes: adjusting the feature distribution of the tail class sample data of the current batch based on the correction statistic to obtain the augmented features corresponding to the tail class sample data; and using the union of the latent features of each class of real-domain data and the augmented features as the equilibrium latent features of the real-domain data to obtain augmented equilibrium sample data of the current batch of real-domain data.

[0043] In this embodiment, based on the corrected mean and corrected covariance Features of tail class samples of this category in the real domain Perform hidden layer feature enhancement to generate augmented features The augmented feature can be represented as: ; in, This represents randomly sampling a new feature from a Gaussian distribution with the original features as the mean and the class covariance as the variance. Indicates random from A tail-class sample feature obtained from sampling, It is the mean offset term, i.e., the hyperparameter that controls the intensity of the offset. It is a hyperparameter that controls the amplification factor of covariance. This indicates the number of samples from the head class in a training batch. This represents the number of samples in the tail class. It can be understood that by calculating the number of missing samples in the tail class using the above formula, that same number of samples are added until the distribution of categories in the real domain within that batch is equal to the number of head classes, thus aligning with the balanced distribution in the virtual domain. This allows for [the following process / action]. Perform hidden layer feature enhancement.

[0044] Define the hidden features of each category in the real domain after equilibrium as follows: The enhanced tail class features, combined with the original real-domain tail class features, can be used to update the feature mean of the real-domain tail class. Based on this, virtual domain data features can be transferred to the real-domain during domain adaptive learning, further increasing the sampling space for real-domain tail class sample feature enhancement. By performing class-balanced augmentation on the imbalanced real-domain data at the feature level, the sampling space for tail class feature enhancement can be expanded. The corrected mean and corrected covariance reflect the balanced distribution of the virtual domain training data, and the augmented features generated accordingly also have a balanced distribution, achieving consistency in the distribution of class sample numbers between the real-domain and virtual-domain training data. This reduces the difficulty of domain adaptive learning and enables effective domain adaptive learning with mixed virtual and real data.

[0045] When introducing balanced virtual domain data to address the class imbalance problem of long-tail distribution in the real domain, the model extracts data features from both the real and virtual domains during the learning process. Considering that after transforming the semantic features of tail-class samples, it is desirable that the corresponding image content still belongs to the tail category in the real domain, and considering that maritime targets are mostly various types of ships with certain similarities (e.g., tail-class container ships and head-class bulk carriers look very similar when unloaded), effectively utilizing the sample diversity of the head class to enhance the tail-class samples can significantly improve the model's accuracy in recognizing the tail class. Therefore, to achieve semantic consistency in the hidden layer feature enhancement process and effectively utilize the feature diversity of other similar categories, a hidden layer feature enhancement method that shares feature information between classes can be introduced.

[0046] The following is an example of augmenting tail class sample data.

[0047] First, the feature extraction network for extracting image features is used... This indicates that the weights of the feature extraction network are... The first in the imbalanced dataset in the real domain The sample data for each batch are represented as follows: , , For batch size, , C The total number of target categories, Representing sample data The hidden layer features extracted by the feature extraction network, among which The first hidden layer feature represents the... Each element. In order to... Perform semantic enhancement to generate augmented semantic features The mean and covariance of each feature in the current batch are dynamically calculated using the following formula: ; ; ; in, Indicate category At the moment of model learning The global mean of the hidden layer features. Indicate category At any moment The global covariance; Indicates at time The training sample data belongs to the category The number of samples, Indicates time included The accumulated belonging to the category The number of training samples, Indicates at time The training sample data belongs to time step The mean of the hidden layer features calculated after feature extraction from (or corresponding batches) of samples. Indicates at time (or the corresponding batch) of training sample data belonging to time step The covariance of the hidden layer features is calculated after feature extraction from the samples.

[0048] In general, any feature The semantic enhancement can be represented as: ;in, Represents class covariance, Representation of features Category tags, The hyperparameter used to control the degree of semantic enhancement, i.e., controlling the covariance amplification factor, is crucial because the model's ability to distinguish between different classes of samples increases gradually with training. The covariance calculated in the early stages of model training cannot accurately reflect the differences in inter-class features. Therefore, it can be... Set as , For model hyperparameters, This represents the number of training iterations for the model. However, the feature enhancement process described above did not consider the potential class imbalance problem in the data. Taking a long-tailed distribution as an example of class imbalance, during model training... An update was performed due to the scarcity of sample data for the tail class. Updates are very sparse. Therefore The information contained in the tail class is limited, so despite data augmentation, the diversity of tail class samples remains very limited. To introduce feature diversity from other classes under the same task, a class confusion matrix that is updated during training can be maintained. This allows for dynamic measurement of feature similarity between different classes, and the correction of the feature mean and covariance matrix of the tail class based on the feature similarity between classes, thereby expanding the sampling space for tail class feature enhancement and improving the diversity of tail class enhancement features.

[0049] Based on the aforementioned semantic enhancement methods, features from tail classes in the real-world domain are... Enhancement : ; ; ; in, Indicates the header class, Indicates the header class, Represents a batch in the real domain The number of samples in the head class of a sample dataset. Represents a batch in the real domain The number of tail class samples in a sample dataset is illustrated in this embodiment using the case of an imbalanced dataset in the real domain that includes both head and tail classes.

[0050] In one embodiment, the domain-adaptive loss function is used to represent the weighted sum of the losses of sample data of each category, wherein the loss of sample data of any category is the feature distance of the sample data of that category between the real domain and the virtual domain; the weight corresponding to the category is determined according to the mean of the hidden layer features after equalization of the current batch, and the weight corresponding to the category is inversely proportional to the number of sample data of that category.

[0051] In this embodiment, based on the corrected mean obtained by updating the global mean of the real domain, adaptive weights can be calculated using the number of samples of different categories in the real and virtual domains, thus obtaining the domain adaptive loss function of the target classification model: ; .in, , Representing the hidden features of various categories in the real domain after equilibrium. Hidden features of various types of virtual domains Features in This indicates sampling from the global feature mean set across all domains and all categories. This indicates that a distance function between arbitrary feature vectors, weighted by the inverse of the class samples, has been added. The adaptive weighting coefficients depend on three variables: the current class. The domain where the current sample is located And the target domain to be adapted This coefficient directly determines that the model only cares about "pulling virtual domain features toward the tail class of the real domain", and not about "pulling real domain features toward the virtual domain". This means that for the current sample, we traverse the domain other than the domain it contains. This represents the value after excluding the domain and category of the current sample itself from the entire set of all (domain, category) combinations. This represents the distance between the features of the current sample and the class center of "a certain erroneous domain (e.g., a virtual domain) and a certain erroneous category (e.g., target B)". This represents the similarity between the current sample feature and the class center of the same class in the other domain. A larger denominator is better (representing closer similarity across domains). A smaller numerator is better (representing distance between similar features across domains / classes). The overall adaptive loss function of this domain can be used to represent how, for a certain sample feature of the current input, it is forced to be extremely close to the class center of "same class as itself" in another domain, while being extremely far away from the class center of "all other domain-class combinations", in order to bring similar features between the real domain and the virtual domain closer.

[0052] By adding supervised learning loss to the domain adaptive loss, the final loss function can be defined as: ,in, To monitor learning loss, , Represents all hidden layer features Corresponding category tags, For domain adaptive learning weights, Representing hidden layer features Category tags, This represents the cross-entropy loss in binary classification. Representation domain classifier.

[0053] In this embodiment, considering that the method of training the model based solely on imbalanced data from the real-world domain still lags significantly behind its learning performance under balanced class distribution, this may be limited by insufficient feature diversity due to the small number of tail class samples in a single domain. However, with technological advancements, the construction of virtual simulation environments has become relatively simple and cost-effective. Therefore, introducing virtual domain data with a balanced distribution of samples from different classes to enhance the feature diversity of tail class samples in the real-world domain data is feasible. But when the training data shifts from a single domain to multiple domains, it is necessary to address the domain shift problem between features from different domains in domain adaptation research. General domain adaptation classification methods achieve domain transfer learning by training a domain classifier, but such constraints are relatively weak, and the effective transfer of virtual domain features to the corresponding tail classes in the real-world domain cannot be guaranteed during domain transfer. Therefore, a domain adaptation learning model can be introduced to achieve better domain adaptation classification learning by optimizing feature domain alignment constraints.

[0054] The following is an example of performing domain adaptive learning.

[0055] Taking a multi-domain mixed multi-class classification task as an example, the category set is defined as follows: The set of domain tags is Define the training dataset as , ,in Indicates sample Category tags, Indicates sample The domain tag. Defined as a sample Feature representation, The weights of the feature extraction network are defined as follows: the final classification prediction output is defined as... Further Defined as the training set for each domain, defined For training set The number of samples, additionally defined Representation domain Medium category All sample data were processed by a feature extraction network. The set of hidden layer features after feature extraction.

[0056] A complete multi-domain, multi-class correspondence dataset can be represented as: Therefore, given any distance function... Then, the function for calculating the transferability between categories of any two domains. It can be defined as: ,in, It is a domain Medium category The first-order mean of the sample features. It represents the expected value in mathematics, that is, the average.

[0057] To more intuitively represent the transferability between different classes in each domain, a transferability matrix is ​​applied. ,in , It is a domain Medium category The first-order mean of the sample features. The node values ​​in the matrix represent the transferability between two classes of two domains. Therefore, the transferability between domains and classes can be summarized as follows: Transferability of the same category between different domains: ; Transferability between different categories within the same domain: ; Transferability between different categories in different domains: .

[0058] To achieve more detailed feature domain alignment, it is first necessary to ensure effective transfer of data of the same category between different domains. Therefore, for a learnable model... The value should be gradually increased. However, for different categories within the same domain, the transferability should be kept sufficiently small so that different categories can be effectively distinguished within a single domain. The value of should gradually decrease during model learning. Finally, there's the transferability between different categories across different domains. Since features inherently differ between domains, this difference can lead to a certain similarity between a category in one domain and other categories in another domain—an undesirable situation. Therefore... The size should also be gradually reduced during the model learning process to ensure that the model has effective discriminative ability for different classes in different domains. Therefore, ( , , To a certain extent, it represents the model's learning objective. Optimizing the model based on the idea of ​​gradient backpropagation can... + - This serves as the objective function to guide model learning. To quantitatively represent transferability, the relationship between distance and transferability is defined as exponential decay, i.e. Therefore, for any The normalized likelihood of samples of the same class across different domains can be expressed as: ; Among them, the denominator is excluding The domain and categories The sum of the portability of all other domains and classes.

[0059] Therefore, to make features from different domains within the same class as close as possible and features from different classes as far apart as possible, the loss of each sample can be... Defined as the average negative log-likelihood of the same domain and class: .

[0060] Furthermore, considering the problem of imbalanced distribution of training data classes, a proposal is put forward in... Based on this, weights based on the inverse of the class sample are added to samples of different categories to achieve a certain balance in learning. The loss after adding weights can be expressed as: ; .

[0061] In this embodiment, the focus during model training is on performance in the real-world domain, without excessive optimization for model performance in virtual scenarios. Due to the imbalanced class distribution in real-world data, the number of tail-class samples in the real-world domain is inevitably significantly less than the corresponding tail-class samples in the virtual domain. This causes the model to overemphasize its performance in the virtual domain during training, hindering effective improvement. Therefore, to achieve effective performance improvement in the real-world domain, this method proposes a loss function for adaptive learning of the model domain using a mixture of real-world and virtual domain data. Adaptive weighting is performed across different domains, with the weights defined as follows: ,in, This indicates that adjustments are made to the total loss from the domain. To Domain Transfer learning categories c The purpose of assigning weights is to give larger weights to categories with fewer samples. Representation domain Medium category The total number of training samples, This indicates the number of domains, which is 2 in this article, including both the physical domain and the virtual domain. This represents a regularization term used to prevent certain weights from becoming too small, causing the corresponding samples to be ignored during training. It can represent selecting all relevant weights. The minimum value among all field pairs ( , All categories of Take the minimum value.

[0062] The weighted model domain adaptive loss function is defined as follows: .

[0063] Based on this, the model will pay more attention to generalization in the real domain during the training process, realizing effective hybrid learning of the model based on real domain and virtual domain data, and improving performance in a realistic balanced testing environment.

[0064] In one embodiment, the loss function is a weighted sum of the supervised loss function and the domain adaptive loss function; during training, the weights corresponding to the domain adaptive loss function gradually decrease.

[0065] For example, the domain adaptive training loss of a complete supervised learning model can be defined as: ,in, These are hyperparameters used to control adaptive learning of the model domain. They can be appropriately reduced when the difference in feature distribution between the two domains is small. This enables effective model training.

[0066] The following is an example of the training process of an imbalanced classification model based on the enhancement of mixed features between virtual and real classes.

[0067] The inputs to the training process include: the real-domain training set. Virtual domain training set Number of model training iterations Batch size is Learning rate The model weights after random initialization; The output is the trained model weights; the training algorithm is as follows: for do From real-domain training set and virtual domain training set Instance-Balanced Sampling is used to obtain and ; Using the model's feature extraction network right and After feature extraction, the following results are obtained and ; based on and Calculate the mean of the hidden features of the real domain and virtual domain classes in the current training batch. Covariance of various features in the real domain and update the global mean. and global covariance ; right After classifying all training data, calculate the real-domain confusion matrix. The elements in the matrix are ; Calculate the corrected mean of tail class sample data in the real domain based on the class confusion matrix. and corrected covariance ; Features from tail class samples Perform hidden layer feature enhancement to generate augmented features : Then and Merging to obtain ; based on Update the mean of global class features in the real domain ; Calculate adaptive weights ; Calculation according to Weighted model domain adaptive training loss: ; Domain-adaptive learning weights The training loss of the model is calculated after weighting: ; Based on learning rate We use stochastic gradient descent to optimize the network weights.

[0068] Figure 2 This is a flowchart illustrating a target classification method provided in an embodiment of this application. This embodiment is applicable to situations where a target classification model is used to identify targets. Specifically, the target classification method can be executed by a target classification model device, which can be implemented through software and / or hardware and integrated into an electronic device. The electronic device includes, but is not limited to, devices with data processing capabilities such as computers, smartphones, or servers.

[0069] like Figure 2 As shown, the target classification method includes: S210. Obtain visual data of the target; S220. Input the visual data into the target classification model to obtain the classification result of the target; The target classification model is trained based on the target classification model training method described in any of the above embodiments.

[0070] In this embodiment, the target classification model used in this method fully learns the diversity of sample features in the balanced virtual domain and transfers them to the imbalanced real domain. It exhibits good generalization and robustness in the imbalanced real domain, enabling accurate target classification. Here, "target" mainly refers to the object to be identified or classified. Taking maritime target classification as an example, the target can be an object on the water surface, such as a ship. Ship types can be civilian vessels like passenger or cargo ships, or engineering vessels, fishing vessels, or military vessels, etc.; the target can also be an underwater object, such as a submarine, shipwreck, rock, or waste. Furthermore, for land or air target classification tasks, the target can also be other land or air objects. This embodiment does not specifically limit the specific task scenario or target category.

[0071] Figure 3 A schematic diagram of a target classification model training device provided in this embodiment of the application. The target classification model training device provided in this embodiment includes: Module 310 is used to acquire the imbalanced dataset in the real domain and the balanced dataset in the virtual domain. The generation module 320 is used to generate a real-domain augmented equilibrium dataset based on the real-domain imbalanced dataset and the virtual-domain balanced dataset, wherein the real-domain augmented equilibrium dataset is obtained by augmenting the tail class sample data in the real-domain imbalanced dataset using the statistics of the real-domain imbalanced dataset and the virtual-domain balanced dataset and the class confusion matrix of the real domain. Training module 330 is used to train a target classification model based on the virtual domain balanced dataset, the real domain augmented balanced dataset, and a loss function; the loss function includes a domain adaptive loss function and a supervised loss function, the domain adaptive loss function is used to measure the difference in feature distribution between sample data in the real domain and the virtual domain, and the supervised loss function is used to measure the target classification accuracy of the target classification model.

[0072] This device uses a class confusion matrix to enhance the tail class samples of an imbalanced dataset in the real domain, thereby increasing the diversity of tail class features. This solves the domain shift problem caused by the inconsistency between the class distributions in the real and virtual domains, and enables cross-domain adaptive learning of mixed real and virtual samples. This allows the model to effectively learn the diversity of sample features in the balanced virtual domain and transfer them to the imbalanced real domain, thus improving the model's generalization and robustness in the imbalanced real domain.

[0073] Based on any of the above embodiments, the generation module 320 includes a feature extraction unit, a tail class statistics unit, and a correction unit: For each batch of sample data in the real-domain imbalanced dataset and the virtual-domain balanced dataset: The feature extraction unit is used to obtain the hidden layer features of each category in the real domain and each category in the virtual domain for the current batch through the feature extraction network in the target classification model, and to determine the global statistics of each category currently accumulated in the real domain and the virtual domain based on the hidden layer features of each category in the real domain and the hidden layer features of each category in the virtual domain. The global statistics include the global mean and the global covariance. The tail-class statistical unit is used to determine the class confusion matrix of the real domain and obtain the real domain correction statistics of each type of sample data based on the class confusion matrix, wherein the correction statistics include the correction mean and the correction covariance. The correction unit is used to generate real-domain augmented equilibrium sample data for the current batch based on the correction statistics; The real-domain augmented equilibrium sample data from each batch constitute the real-domain augmented equilibrium sample dataset.

[0074] Based on any of the above embodiments, the corrected statistics of the real domain of each category are obtained based on the class confusion matrix and the global statistics of each category. This includes, for the sample data of each category in the current batch, calculating the weighted average of the global statistics of the sample data of all other categories besides the stated category based on the class confusion matrix, to obtain the corrected statistics of the real domain of each category; wherein, the weight corresponding to any other category besides the stated category is: the element in the class confusion matrix corresponding to the stated category and the other categories, the element representing the probability that the sample data of the stated category is misclassified as the other category.

[0075] Based on any of the above embodiments, the correction unit is specifically used for: The feature distribution of the tail class sample data in the current batch is adjusted based on the corrected statistics to obtain the augmented features corresponding to the tail class sample data. The augmented and balanced sample data of the current batch of real-domain domains are obtained by taking the union of the hidden features of each category of the real-domain domain and the augmented features as the balanced hidden features of the real-domain domain.

[0076] Based on any of the above embodiments, the domain adaptive loss function is used to represent the weighted sum of the losses of sample data of each category, wherein the loss of sample data of any category is the feature distance of sample data of the category between the real domain and the virtual domain; the weight corresponding to the category is determined according to the mean of the hidden layer features after equalization of the current batch, and the weight corresponding to the category is inversely proportional to the number of sample data of the category.

[0077] Based on any of the above embodiments, the loss function is a weighted sum of the supervised loss function and the domain adaptive loss function; during training, the weights corresponding to the domain adaptive loss function gradually decrease.

[0078] Figure 4 A schematic diagram of a target classification device provided in an embodiment of this application. The target classification device provided in this embodiment includes: The acquisition module 410 is used to acquire visual data of the target; The classification module 420 is used to input the visual data into the target classification model to obtain the classification result of the target; wherein the target classification model is trained based on the target classification model training method described in any of the above embodiments.

[0079] The target classification model training device provided in this application embodiment can be used to execute the target classification model training method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0080] Figure 5A schematic diagram of an electronic device 10, which can be used to implement embodiments of this application, is shown. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, user equipment, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0081] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0082] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks and wireless networks.

[0083] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above.

[0084] In some embodiments, the methods described above can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the methods of any of the embodiments described above by any other suitable means (e.g., by means of firmware).

[0085] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0086] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0087] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0088] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device 10, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device 10. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0089] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0090] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0091] This application also provides a computer program product, including a computer program and / or instructions, which, when executed by a processor, implement the target classification model training method as described in any of the above embodiments.

[0092] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0093] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for training a target classification model, characterized in that, include: Obtain the imbalanced dataset in the real domain and the balanced dataset in the virtual domain; A real-domain augmented equilibrium dataset is generated based on the real-domain imbalanced dataset and the virtual-domain balanced dataset. The real-domain augmented equilibrium dataset is obtained by augmenting the tail class sample data in the real-domain imbalanced dataset using the statistics of the real-domain imbalanced dataset and the virtual-domain balanced dataset, as well as the class confusion matrix of the real domain. The target classification model is trained based on the virtual domain balanced dataset, the real domain augmented balanced dataset, and the loss function; the loss function includes a domain adaptive loss function and a supervised loss function, the domain adaptive loss function is used to measure the difference in feature distribution between sample data in the real domain and the virtual domain, and the supervised loss function is used to measure the target classification accuracy of the target classification model.

2. The method according to claim 1, characterized in that, A real-domain augmented equilibrium dataset is generated based on the real-domain imbalanced dataset and the virtual-domain balanced dataset, including: For each batch of sample data in the real-domain imbalanced dataset and the virtual-domain balanced dataset: The hidden features of each category in the real domain and the virtual domain in the current batch are obtained through the feature extraction network in the target classification model. Based on the hidden features of each category in the real domain and the virtual domain, the global statistics of each category currently accumulated in the real domain and the virtual domain are determined. The global statistics include the global mean and the global covariance. Determine the class confusion matrix of the real domain, and obtain the corrected statistics of each class of the real domain based on the class confusion matrix and the global statistics of each class, wherein the corrected statistics include the corrected mean and the corrected covariance; The current batch of augmented equilibrium sample data in the real domain is generated based on the corrected statistics. The real-domain augmented equilibrium sample data from each batch constitute the real-domain augmented equilibrium sample dataset.

3. The method according to claim 2, characterized in that, The process of obtaining corrected statistics for each category of the real-world domain based on the class confusion matrix and global statistics for each category includes: For each category of sample data in the current batch, based on the class confusion matrix, calculate the weighted average of the global statistics of all other categories of sample data except for the stated category, and obtain the corrected statistics of the real domain for each category; The weights corresponding to any other category besides the stated category are: the elements in the class confusion matrix corresponding to the stated category and the other categories, where each element represents the probability that sample data of the stated category is misclassified as the other category.

4. The method according to claim 2, characterized in that, Based on the corrected statistics, the current batch of real-domain augmented equilibrium sample data is generated, including: The feature distribution of the tail class sample data in the current batch is adjusted based on the corrected statistics to obtain the augmented features corresponding to the tail class sample data. The augmented and balanced sample data of the current batch of real-domain domains are obtained by taking the union of the hidden features of each category of the real-domain domain and the augmented features as the balanced hidden features of the real-domain domain.

5. The method according to claim 4, characterized in that, The domain adaptive loss function is used to represent the weighted sum of the losses of sample data of each category, wherein the loss of sample data of any category is the feature distance of the sample data of that category between the real domain and the virtual domain. The weight corresponding to the category is determined based on the mean of the balanced hidden layer features of the current batch, and the weight corresponding to the category is inversely proportional to the number of sample data of the category.

6. The method according to claim 1, characterized in that, The loss function is a weighted sum of the supervised loss function and the domain adaptive loss function; during training, the weights corresponding to the domain adaptive loss function gradually decrease.

7. A target classification method, characterized in that, include: Obtain visual data of the target; The visual data is input into the target classification model to obtain the classification result of the target; The target classification model is trained based on the target classification model training method as described in any one of claims 1-6.

8. A target classification model training device, characterized in that, include: The acquisition module is used to acquire imbalanced datasets in the real domain and balanced datasets in the virtual domain. The generation module is used to generate a real-domain augmented equilibrium dataset based on the real-domain imbalanced dataset and the virtual-domain balanced dataset. The real-domain augmented equilibrium dataset is obtained by augmenting the tail class sample data in the real-domain imbalanced dataset using the statistics of the real-domain imbalanced dataset and the virtual-domain balanced dataset, as well as the class confusion matrix of the real domain. The training module is used to train a target classification model based on the virtual domain balanced dataset, the real domain augmented balanced dataset, and a loss function. The loss function includes a domain adaptive loss function and a supervised loss function. The domain adaptive loss function is used to measure the difference in feature distribution between sample data in the real domain and the virtual domain, and the supervised loss function is used to measure the target classification accuracy of the target classification model.

9. An electronic device, characterized in that, include: At least one processor; A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the target classification model training method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the target classification model training method as described in any one of claims 1-7.

Images