Multi-granularity perception and prototype driven ultra-high-definition image domain adaptive identification method

Abstract

The invention belongs to the technical field of domain adaptation of ultra-high-definition images, and provides a multi-granularity perception and prototype driven ultra-high-definition image domain self-adaptive recognition method, which comprises the following steps of: firstly, acquiring a conventional image data set and an ultra-high-definition image data set across image domains, and preprocessing the conventional image data set and the ultra-high-definition image data set to obtain an ultra-high-definition image data set; obtaining a preprocessed batch training image data set; secondly, constructing and initializing a multi-granularity feature extractor and a feature classifier; then, based on the batch training image data set, training is carried out in combination with a multi-granularity feature extractor and a feature classifier to obtain an ultra-high-definition image unsupervised domain self-adaptive recognition model; and finally, inputting a to-be-recognized ultra-high-definition image into the ultra-high-definition image-based unsupervised domain adaptive recognition model for classification to obtain a recognition result of the ultra-high-definition image. According to the method, the problems that the fine-grained features of the ultra-high-definition image are ignored, the ultra-high-definition image field is difficult to mark, the number is small, and the computing resource consumption is large are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of domain adaptation of ultra-high-definition images, and in particular to a domain adaptive recognition method for ultra-high-definition images driven by multi-granularity perception and prototype. Background Art

[0002] With the rapid development of machine vision technology, ultra-high-definition imaging technology has begun to be applied to monitoring, reconnaissance and other fields. However, due to the difficulty in obtaining ultra-high-definition images and the scarcity of well-labeled data, it is difficult to directly use a small number of well-labeled ultra-high-definition images to train a neural network model with strong generalization ability. In addition, if unsupervised learning is directly used or a large number of ultra-high-definition images with possible noise are used for training, the prediction accuracy and robustness of the obtained deep neural network model are difficult to meet actual needs. Therefore, how to use unsupervised domain adaptation technology to transfer the supervised knowledge in today's mature conventional image recognition to the field of ultra-high-definition images, so as to make the ultra-high-definition image recognition model more generalizable and robust, is the key solution path to improve the performance of ultra-high-definition image recognition.

[0003] Existing ultra-high-definition image recognition technology directly uses deep models such as convolutional neural networks and Transformers to directly extract features from a small number of fully annotated ultra-high-definition images, and then inputs the image features into the classifier to obtain the final recognition result. Alternatively, ultra-high-definition images are directly downsampled to regular images, and then input into the deep neural network of regular images for learning and recognition.

[0004] In addition, some unsupervised domain adaptation has begun to be applied to this field. Among them, the source domain with labeled information corresponds to the field of conventional images, while the unlabeled domain corresponds to the field of ultra-high-definition images. Traditional unsupervised domain adaptation methods usually regard conventional images and ultra-high-definition images as belonging to different data distributions, and then eliminate the differences between sample pairs one by one based on the instance level, that is, domain shift. With the development of deep learning, more methods use adversarial-based techniques to address the impact of domain shift. They use ultra-high-definition images as adversarial sets for conventional images, striving to weaken the differences between the two at the pixel level. In addition, the self-supervised domain adaptation method directly classifies the ultra-high-definition images by directly using the pre-trained model of conventional images to obtain pseudo labels, and uses this as a supervisory signal for learning to achieve recognition of ultra-high-definition images.

[0005] Therefore, existing ultra-high-definition image recognition methods are still limited to mature conventional image recognition technologies or directly apply unsupervised domain adaptation method templates, ignoring the characteristics of ultra-high-definition images themselves. Generally speaking, these existing ultra-high-definition image recognition technologies have some common limitations:

[0006] (1) The huge amount of fine-grained information contained in ultra-high-definition images has not been well utilized and is even directly discarded by some existing methods.

[0007] (2) It ignores the fact that ultra-high-definition images are difficult to annotate and are in small quantities, and direct training is unlikely to achieve strong generalization recognition performance.

[0008] (3) Ultra-high-definition images are large in size, and both instance-based and adversarial deep methods face huge time and space computing consumption. Summary of the invention

[0009] The purpose of the present invention is to provide a multi-granularity perception and prototype-driven ultra-high-definition image domain adaptive recognition method, which can grasp the characteristics of ultra-high-definition images with rich details, sparse annotations, and large single-image volume, realize the full perception of the fine-grained features of ultra-high-definition images, and solve the problem of the fine-grained features of ultra-high-definition images being ignored; at the same time, a large number of labeled conventional pictures and a small number of unlabeled ultra-high-definition images are used to realize unsupervised domain adaptive recognition, and a lightweight prototype learning technology is used to transfer the supervised knowledge of conventional images to the ultra-high-definition image field, so as to realize more accurate ultra-high-definition image recognition, and solve the problems of difficult annotation, small number, and large consumption of computing resources in the ultra-high-definition image field.

[0010] The present invention solves the technical problem and adopts the following technical solution:

[0011] The multi-granularity perception and prototype-driven ultra-high-definition image domain adaptive recognition method includes the following steps:

[0012] Obtain a conventional image dataset and an ultra-high-definition image dataset across image domains, and preprocess the conventional image dataset and the ultra-high-definition image dataset to obtain a preprocessed batch training image dataset;

[0013] Build and initialize multi-granularity feature extractors and feature classifiers;

[0014] Based on the batch training image dataset, the unsupervised domain adaptive recognition model of ultra-high-definition images is obtained by combining multi-granularity feature extractor and feature classifier training;

[0015] The ultra-high-definition image to be recognized is input into the unsupervised domain adaptive recognition model based on ultra-high-definition images for classification, and the recognition result of the ultra-high-definition image is obtained.

[0016] As a further optimization, the method of obtaining a conventional image dataset and an ultra-high definition image dataset across image domains, and preprocessing the conventional image dataset and the ultra-high definition image dataset to obtain a preprocessed batch training image dataset includes the following steps:

[0017] Obtain a regular image dataset with accurate labels, which contains regular natural images of various scenes in production and life;

[0018] Obtain an unlabeled ultra-high-definition image dataset;

[0019] The above two image data sets are cleaned to filter out erroneous image data and noisy image data to obtain a regular image data set. and ultra-high-definition image datasets

[0020] The conventional image dataset and ultra-high-definition image datasets The training data is divided and regularized to obtain a batch training image dataset

[0021] As a further optimization, the construction and initialization of the multi-granularity feature extractor and feature classifier includes the following steps:

[0022] Build and initialize the Faster RCNN network framework with pre-trained parameters to obtain the target detection model

[0023] Build and initialize the ResNet-18 network framework with pre-trained parameters to obtain the backbone network model

[0024] Establish and randomly initialize a single-layer convolutional neural network framework to obtain a fine-grained attention network And establish a single-layer fully connected neural network framework to obtain the feature granularity attention network

[0025] Combining object detection models Backbone network model Fine-grained Attention Network and feature-granular attention network Get a multi-granularity feature extractor

[0026] A multi-layer fully connected neural network framework is established and randomly initialized to obtain the feature classifier f.

[0027] As a further optimization, the unsupervised domain adaptive recognition model for ultra-high-definition images is obtained based on the batch training image dataset, combined with the multi-granularity feature extractor and feature classifier training, including the following steps:

[0028] Batch training image dataset Input to the multi-granularity feature extractor In the above example, we extract multi-granular features of each batch of images.

[0029] The conventional image is used as the source domain, and the multi-granular features of the source domain are According to the image category information and K-Means clustering algorithm, the prototype center c of the source domain is calculated s ;

[0030] The ultra-high-definition image is used as the target domain, and the multi-granular features of the target domain are Through the K-Means clustering algorithm, the prototype center c of the target domain is clustered. t ;

[0031] Utilize multi-granular features of source domain images and the prototype center c of the source domain s , and use the multi-granularity features of the image in the target domain and the prototype center c of the target domain t , the cohesion loss in the domain is calculated in each field

[0032] Combined multi-granular features of source domain images and the prototype center c of the source domain s , and the multi-granularity features of the image in the target domain and the prototype center c of the target domain t , the inter-domain coupling loss is calculated between domains

[0033] The multi-granular features of the source domain image and multi-granularity features of images in the target domain Input into the feature classifier to get the feature classification offset loss

[0034] According to the cohesion loss in the domain Inter-domain coupling loss and feature classification bias loss Calculate the total loss of the entire model optimization

[0035] Calculate total loss The gradient of the whole model is back-propagated and parameter optimization is performed. When the total loss meets the convergence condition, the trained unsupervised domain adaptive recognition model of ultra-high-definition images is obtained.

[0036] As a further optimization, the calculation formula of the multi-granularity feature is:

[0037]

[0038]

[0039]

[0040] in, is a multi-granularity feature set of batch images, is a multi-granularity feature set of the source domain image, is a multi-granularity feature set of the target domain image, is a multi-granularity feature extractor, and are batch training datasets of source domain images and target domain images, respectively.

[0041] As a further optimization, the prototype center c of the source domain s The calculation formula is:

[0042]

[0043] The prototype center c of the target domain t The calculation formula is:

[0044]

[0045] Among them, c s represents the prototype center of the constructed source domain, Represents the cluster center obtained after K-Means clustering of source domain features. represents the category center of each category in the source domain, e represents a weight hyperparameter, which is used to weigh the proportion of cluster center and category center in the construction of prototype center, c t Represents the prototype center of the constructed target domain, using To construct a prototype center, Represents the cluster center obtained after K-Means clustering of target domain features.

[0046] As a further optimization, the domain cohesion loss The expression is:

[0047]

[0048] in, represents the cohesive loss function in the domain, n s Refers to the total amount of source domain image data, n t refers to the total amount of target domain image data, i, j are indicator variables for iteratively calculating the loss function for image data, CE refers to the cross entropy loss function, represents the i-th feature of the extracted source domain image, represents the extracted j-th target domain image feature, is a multi-granularity feature set of the source domain image, is a multi-granularity feature set of the target domain image, Represents the target domain features The index of the nearest cluster center of ;

[0049] The inter-domain coupling loss The expression is:

[0050]

[0051] in, represents the inter-domain coupling loss function, and entropy refers to the calculation of information entropy. Cross-domain prototype centers representing feature clusters, Representative source domain features and target domain features The most recent cross-domain prototype center c g The index of

[0052] The feature classification bias loss The expression is:

[0053]

[0054] in, represents the feature classification offset loss, represents the category label corresponding to the source domain feature, f represents the constructed feature classifier, It is a function representation for classifying features.

[0055] As a further optimization, the expression of the total loss is:

[0056]

[0057] in, represents the total loss function, They represent the cohesion loss function within the domain, the coupling loss function between domains, and the feature classification offset loss function respectively. α, β, and γ represent the weight values ​​of the corresponding function terms respectively.

[0058] As a further optimization, the total loss meets the convergence condition, which means that the total loss of 5 consecutive batches meets:

[0059]

[0060] in, is the total loss of the epth batch, is the total loss of the ep-1th batch, and |·| is the absolute value function.

[0061] The beneficial effects of the present invention are as follows: through the above-mentioned multi-granularity perception and prototype-driven ultra-high-definition image domain adaptive recognition method, ultra-high-definition image feature extraction that integrates local and global information can be achieved based on a multi-granularity attention perception mechanism, using image convolution operations, pre-trained image encoders, and attention mechanisms, giving the model the ability to comprehensively extract fine-grained features and overall features of ultra-high-definition images; and, the present invention also adopts an ultra-high-definition image recognition solution for unsupervised domain adaptation based on conventional images as the source domain, so that extensive supervised knowledge in conventional images can be transferred to the ultra-high-definition image field, thereby improving the performance and generalization of the model in ultra-high-definition image recognition; in addition, the present invention adopts prototype learning as the driving technology for domain adaptation, and uses the "one-to-many" paradigm of prototype center-common representation to achieve intra-domain cohesive learning and inter-domain coupled learning, thereby improving the model's discrimination and eliminating domain offset at a lower computational cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a multi-granularity sensing and prototype-driven ultra-high-definition image domain adaptive recognition method in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0064] Example

[0065] This embodiment provides a method for adaptive recognition of ultra-high-definition image domains driven by multi-granularity perception and prototype. Figure 1 , wherein the method comprises the following steps:

[0066] S1. Obtain a conventional image dataset and an ultra-high-definition image dataset across image domains, and preprocess the conventional image dataset and the ultra-high-definition image dataset to obtain a preprocessed batch training image dataset;

[0067] S2, build and initialize a multi-granularity feature extractor and feature classifier;

[0068] S3, based on the batch training image dataset, combined with multi-granularity feature extractor and feature classifier training to obtain the ultra-high-definition image unsupervised domain adaptive recognition model;

[0069] S4. Input the ultra-high-definition image to be identified into an unsupervised domain adaptive recognition model based on ultra-high-definition images for classification, and obtain a recognition result of the ultra-high-definition image.

[0070] In this embodiment, firstly, based on the multi-granularity attention perception mechanism, the image convolution operation, the pre-trained image encoder and the attention mechanism are used to realize the ultra-high-definition image feature extraction that integrates local and global information, and endow the model with the ability to comprehensively extract the fine-grained features and overall features of the ultra-high-definition image; secondly, by adopting an ultra-high-definition image recognition solution based on conventional images as the source domain, the extensive supervised knowledge in conventional images can be transferred to the ultra-high-definition image field, thereby improving the performance and generalization of the model in ultra-high-definition image recognition; then, prototype learning is used as the driving technology for domain adaptation, and the "one-to-many" paradigm of prototype center-common representation is used to realize intra-domain cohesive learning and inter-domain coupled learning, thereby improving the model discrimination and eliminating domain offset at a lower computational cost.

[0071] It should be pointed out that in order to provide training data support and basis for the subsequent implementation of the ultra-high-definition image unsupervised domain adaptive recognition technology, in step S1 of this embodiment, the conventional image dataset and the ultra-high-definition image dataset across image domains are obtained, and the conventional image dataset and the ultra-high-definition image dataset are preprocessed to obtain the preprocessed batch training image dataset, which may include the following steps:

[0072] S101, obtaining a conventional image dataset with accurate labels, where the conventional image dataset contains conventional natural images of various scenes of production and life;

[0073] S102, obtaining an unlabeled ultra-high-definition image dataset;

[0074] S103: Clean the two image data sets to filter out erroneous image data and noise image data to obtain a regular image data set. and ultra-high-definition image datasets

[0075] S104: conventional image data set and ultra-high-definition image datasets The training data is divided and regularized to obtain a batch training image dataset

[0076] It should be pointed out that by building a specific multi-granularity feature extractor and feature classifier, a model framework foundation is established for the subsequent multi-granularity feature perception and accurate recognition and classification of conventional images and ultra-high-definition images. In step S2 of this embodiment, the construction and initialization of the multi-granularity feature extractor and feature classifier may include the following steps:

[0077] S201. Establish and initialize the Faster RCNN network framework with pre-trained parameters to obtain the target detection model

[0078] S202. Establish and initialize the ResNet-18 network framework with pre-trained parameters to obtain the backbone network model

[0079] S203. Establish and randomly initialize a single-layer convolutional neural network framework to obtain a fine-grained attention network And establish a single-layer fully connected neural network framework to obtain the feature granularity attention network

[0080] S204. Combined target detection model Backbone network model Fine-grained Attention Network and feature-granular attention network Get a multi-granularity feature extractor

[0081] S205. Establish and randomly initialize a multi-layer fully connected neural network framework to obtain a feature classifier f.

[0082] In addition, in order to extract the fine-grained dimensional features and feature dimensional characteristics of the image through the constructed multi-granularity feature extractor from the acquired training image data, and fuse them into an image depth representation to fully explore every detail and feature of the image, this embodiment can promote the alignment of the conventional image domain to the ultra-high-definition image domain and the accurate recognition of ultra-high-definition images with less computational effort through the prototype learning-based supervision of the intra-domain cohesion loss and the inter-domain coupling loss, and improve the unsupervised domain adaptive recognition capability of ultra-high-definition images. Therefore, in step S3 of this embodiment, the ultra-high-definition image unsupervised domain adaptive recognition model obtained based on the batch training image data set combined with the multi-granularity feature extractor and the feature classifier training can include the following steps:

[0083] S301, batch training image dataset Input to the multi-granularity feature extractor In the above example, we extract multi-granular features of each batch of images.

[0084] S302, taking the conventional image as the source domain, and taking the source domain multi-granularity features According to the image category information and K-Means clustering algorithm, the prototype center c of the source domain is calculated s ;

[0085] S303, taking the ultra-high definition image as the target domain, and multi-granularity features of the target domain Through the K-Means clustering algorithm, the prototype center c of the target domain is clustered. t ;

[0086] S304: Using multi-granular features of source domain images and the prototype center c of the source domain s , and use the multi-granularity features of the image in the target domain and the prototype center c of the target domain t , the cohesion loss in the domain is calculated in each field

[0087] S305. Multi-granularity features of joint source domain images and the prototype center c of the source domain s , and the multi-granularity features of the image in the target domain and the prototype center c of the target domain t , the inter-domain coupling loss is calculated between domains

[0088] S306: multi-granularity features of the source domain image and multi-granularity features of images in the target domain Input into the feature classifier to get the feature classification offset loss

[0089] S307, based on the cohesion loss in the domain Inter-domain coupling loss and feature classification bias loss Calculate the total loss of the entire model optimization

[0090] S308. Calculate total loss The gradient of the whole model is back-propagated and parameter optimized;

[0091] S309, determine whether the loss meets the convergence condition. If so, obtain the trained ultra-high-definition image unsupervised domain adaptive recognition model and enter step S4; otherwise, return to step S301.

[0092] It should be pointed out that in order to extract the multi-granularity features of the image, fully perceive the fine-grained level and feature-level features of the image, and obtain the deep encoding of the image, so as to facilitate the calculation of the subsequent prototype center and the target loss function, in step S301 of this embodiment, the calculation formula of the multi-granularity features is:

[0093]

[0094]

[0095]

[0096] in, is a multi-granularity feature set of batch images, is a multi-granularity feature set of the source domain image, is a multi-granularity feature set of the target domain image, is a multi-granularity feature extractor, and are batch training datasets of source domain images and target domain images, respectively.

[0097] At the same time, in order to construct the prototype center of each category in the source domain and the target domain, and use it as a representative of the category and cluster feature distribution, and lay the foundation for the calculation of the cohesion loss and inter-domain coupling loss functions in the subsequent domains, in step S302 of this embodiment, the prototype center c of the source domain is s The calculation formula is:

[0098]

[0099] The prototype center c of the target domain t The calculation formula is:

[0100]

[0101] Among them, c s represents the prototype center of the constructed source domain, Represents the cluster center obtained after K-Means clustering of source domain features. represents the category center of each category in the source domain, e represents a weight hyperparameter, which is used to weigh the proportion of cluster center and category center in the construction of prototype center, c t Represents the prototype center of the constructed target domain. Since the target domain lacks label information, only To construct a prototype center, Represents the cluster center obtained after K-Means clustering of target domain features.

[0102] It should be pointed out that the present embodiment adopts the domain cohesion loss, which can use the prototype centers in the source domain and the target domain as anchor points, shorten the distance between the features of the same class (cluster) and the prototype center, and thus make the features of the same class (cluster) more cohesive, improve the discriminability of the features in the feature space, and establish a certain degree of discriminability basis for subsequent feature classification and ultra-high-definition image recognition. Therefore, in step S304 of the present embodiment, the domain cohesion loss The expression is:

[0103]

[0104] in, represents the cohesive loss function in the domain, n s Refers to the total amount of source domain image data, n trefers to the total amount of target domain image data, i, j are indicator variables for iteratively calculating the loss function for image data, CE refers to the cross entropy loss function, represents the i′th feature extracted from the source domain (conventional) image, represents the extracted j′th target domain (ultra-high definition) image features, is a multi-granularity feature set of the source domain image, is a multi-granularity feature set of the target domain image, Represents the target domain features The index of the nearest cluster center.

[0105] It should be pointed out that the inter-domain coupling loss is used in this embodiment, which can use the cross-domain prototype center as an anchor point to shorten the distance between the source domain (conventional image) feature cluster and the target domain (ultra-high-definition image) feature cluster in the feature space, eliminate the domain offset between the two image domains, and enhance the ability of the model to supervise knowledge transfer. Therefore, in step S305 of this embodiment, the inter-domain coupling loss The expression is:

[0106]

[0107] in, represents the inter-domain coupling loss function, Cross-domain prototype centers representing feature clusters, Representative source domain features and target domain features The most recent cross-domain prototype center c g The index of .

[0108] In addition, in order to perform category label supervised learning on conventional image features in the source domain, this embodiment uses a cross entropy loss constraint to force the category label of the source domain to constrain the distribution of source domain features, making them more discriminative, and able to propagate to the target domain as the inter-domain coupling deepens, thereby improving the discriminative power of ultra-high-definition image features. Therefore, in step S306 of this embodiment, the feature classification offset loss The expression is:

[0109]

[0110] in, represents the feature classification offset loss, f represents the constructed feature classifier, It is a function representation for classifying features.

[0111] It should be pointed out that this embodiment can combine the cohesion loss in the domain, the coupling loss between domains, and the feature classification offset loss function terms in a weighted fusion manner to achieve joint supervision of the objective function during model training, thereby promoting the model to achieve domain adaptation and accurate ultra-high-definition image recognition and classification. Therefore, in step S307 of this embodiment, the expression of the total loss is:

[0112]

[0113] in, represents the total loss function, They represent the cohesion loss function within the domain, the coupling loss function between domains, and the feature classification offset loss function respectively. α, β, and γ represent the weight values ​​of the corresponding function terms respectively.

[0114] Finally, in order to clarify the stopping criteria of the training of the ultra-high-definition image unsupervised domain adaptive recognition model, this embodiment enables the model to stop and save when it is in the best state and result. Therefore, in step S309 of this embodiment, the total loss satisfies the convergence condition, which means that the total loss of 5 consecutive batches satisfies:

[0115]

[0116] in, is the total loss of the epth batch, is the total loss of the ep-1th batch, and |·| is the absolute value function.

[0117] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-granularity perception and prototype-driven ultra-high-definition image domain adaptive recognition method, characterized in that: The steps include: Obtain a conventional image dataset and an ultra-high-definition image dataset across image domains, and preprocess the conventional image dataset and the ultra-high-definition image dataset to obtain a preprocessed batch training image dataset; Build and initialize multi-granularity feature extractors and feature classifiers; Based on the batch training image dataset, the unsupervised domain adaptive recognition model of ultra-high-definition images is obtained by combining multi-granularity feature extractors and feature classifiers. The ultra-high-definition image to be recognized is input into the unsupervised domain adaptive recognition model based on ultra-high-definition images for classification, and the recognition result of the ultra-high-definition image is obtained.

2. The multi-granularity perception and prototype-driven ultra-high-definition image domain adaptive recognition method according to claim 1 is characterized in that: The step of obtaining a conventional image dataset and an ultra-high definition image dataset across image domains, and preprocessing the conventional image dataset and the ultra-high definition image dataset to obtain a preprocessed batch training image dataset includes the following steps: Obtain a regular image dataset with accurate labels, which contains regular natural images of various scenes in production and life; Obtain an unlabeled ultra-high-definition image dataset; The above two image data sets are cleaned to filter out erroneous image data and noisy image data to obtain a regular image data set. and ultra-high-definition image datasets The conventional image dataset and ultra-high-definition image datasets The training data is divided and regularized to obtain a batch training image dataset 3. The multi-granularity perception and prototype-driven ultra-high-definition image domain adaptive recognition method according to claim 1 is characterized in that: The construction and initialization of the multi-granularity feature extractor and feature classifier includes the following steps: Build and initialize the Faster RCNN network framework with pre-trained parameters to obtain the target detection model Build and initialize the ResNet-18 network framework with pre-trained parameters to obtain the backbone network model Establish and randomly initialize a single-layer convolutional neural network framework to obtain a fine-grained attention network And establish a single-layer fully connected neural network framework to obtain the feature granularity attention network Combining object detection models Backbone network model Fine-grained Attention Network and feature-granular attention network Get a multi-granularity feature extractor A multi-layer fully connected neural network framework is established and randomly initialized to obtain the feature classifier f.

4. The multi-granularity perception and prototype-driven ultra-high-definition image domain adaptive recognition method according to claim 3 is characterized in that: The method of obtaining an ultra-high-definition image unsupervised domain adaptive recognition model based on a batch training image dataset and combining a multi-granularity feature extractor and a feature classifier for training comprises the following steps: Batch training image dataset Input to the multi-granularity feature extractor In the above example, we extract multi-granular features of each batch of images. The conventional image is used as the source domain, and the multi-granular features of the source domain are According to the image category information and K-Means clustering algorithm, the prototype center c of the source domain is calculated s ; The ultra-high-definition image is used as the target domain, and the multi-granular features of the target domain are Through the K-Means clustering algorithm, the prototype center c of the target domain is clustered. t ; Utilize multi-granular features of source domain images and the prototype center c of the source domain s , and use the multi-granularity features of the image in the target domain and the prototype center c of the target domain t , the cohesion loss in the domain is calculated in each field Combined multi-granular features of source domain images and the prototype center c of the source domain s , and the multi-granularity features of the image in the target domain and the prototype center c of the target domain t , the inter-domain coupling loss is calculated between domains The multi-granular features of the source domain image and multi-granularity features of images in the target domain Input into the feature classifier to get the feature classification offset loss According to the cohesion loss in the domain Inter-domain coupling loss and feature classification bias loss Calculate the total loss of the entire model optimization Calculate total loss The gradient of the whole model is back-propagated and parameter optimization is performed. When the total loss meets the convergence condition, the trained unsupervised domain adaptive recognition model of ultra-high-definition images is obtained.

5. The multi-granularity sensing and prototype-driven ultra-high-definition image domain adaptive recognition method according to claim 4 is characterized in that: The calculation formula of the multi-granularity feature is: in, is a multi-granularity feature set of batch images, is a multi-granularity feature set of the source domain image, is a multi-granularity feature set of the target domain image, is a multi-granularity feature extractor, and are batch training datasets of source domain images and target domain images, respectively.

6. The multi-granularity sensing and prototype-driven ultra-high-definition image domain adaptive recognition method according to claim 4 is characterized in that: The prototype center c of the source domain s The calculation formula is: The prototype center c of the target domain t The calculation formula is: Among them, c s represents the prototype center of the constructed source domain, Represents the cluster center obtained after K-Means clustering of source domain features. represents the category center of each category in the source domain, e represents a weight hyperparameter, which is used to weigh the proportion of cluster center and category center in the construction of prototype center, c t Represents the prototype center of the constructed target domain, using To construct a prototype center, Represents the cluster center obtained after K-Means clustering of target domain features.

7. The multi-granularity sensing and prototype-driven ultra-high-definition image domain adaptive recognition method according to claim 4 is characterized in that: The domain has a loss of cohesion The expression is: in, represents the cohesive loss function in the domain, n s Refers to the total amount of source domain image data, n t refers to the total amount of target domain image data, i, j are indicator variables for iteratively calculating the loss function for image data, CE refers to the cross entropy loss function, represents the i-th feature of the extracted source domain image, represents the extracted j-th target domain image feature, is a multi-granularity feature set of the source domain image, is a multi-granularity feature set of the target domain image, Represents the target domain features The index of the nearest cluster center of ; The inter-domain coupling loss The expression is: in, represents the inter-domain coupling loss function, and entropy refers to the calculation of information entropy. Cross-domain prototype centers representing feature clusters, Representative source domain features and target domain features The most recent cross-domain prototype center c g The index of The feature classification bias loss The expression is: in, represents the feature classification offset loss, represents the category label corresponding to the source domain feature, f represents the constructed feature classifier, It is a function representation for classifying features.

8. The multi-granularity sensing and prototype-driven ultra-high-definition image domain adaptive recognition method according to claim 4 is characterized in that: The total loss is expressed as: in, represents the total loss function, They represent the cohesion loss function within the domain, the coupling loss function between domains, and the feature classification offset loss function respectively. α, β, and γ represent the weight values ​​of the corresponding function terms respectively.

9. The multi-granularity sensing and prototype-driven ultra-high-definition image domain adaptive recognition method according to claim 4 is characterized in that: The total loss meets the convergence condition, which means that the total loss of 5 consecutive batches meets the following condition: in, is the total loss of the epth batch, is the total loss of the ep-1th batch, and |·| is the absolute value function.

Images