A domain-adaptive object detection method based on graph matching

Abstract

The present invention discloses a domain adaptive target detection method based on graph matching. After acquiring source domain and target domain data, the method of the present invention obtains global features through a feature extraction network and converts them into graph structure features. The method divides the network into a target detection branch and a domain adaptive branch. In the target detection branch, the method adopts a cross entropy loss function to obtain the target detection loss. In the domain adaptive branch, the method generates pseudo nodes based on the graph structure features to complete the missing categories, and performs graph matching on the completed graph structure features. The present invention introduces intra-class differences through the graph structure, and completes the missing categories in the graph structure features by generating pseudo nodes, thereby obtaining a better and more comprehensive feature expression, thereby improving the accuracy of domain adaptive target detection.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a domain adaptive target detection method based on graph matching. Background Art

[0002] Object detection in computer vision involves automatically detecting the location and category of objects in images or videos. Object detection is crucial in many scenarios, such as autonomous driving, intelligent transportation, and drone control. Traditional object detection algorithms typically assume that the test dataset (also known as the target domain) and the training dataset (also known as the source domain) follow the same general distribution. When the distribution of the training and test data is consistent, well-trained object detection models have been shown to achieve impressive performance. However, variations in image background, shooting angle, image quality, and even acquisition device parameter settings can lead to inconsistent data distributions. Deploying traditional object detection methods in new domains can lead to catastrophic performance degradation due to domain differences. Collecting and annotating large numbers of samples in new domains consumes enormous resources and manpower. To overcome this challenge, researchers have explored unsupervised domain adaptation methods to align the unlabeled target domain with the labeled source domain.

[0003] Unsupervised domain adaptation involves learning knowledge in a source domain and transferring it to an unlabeled target domain, enabling the model to achieve good performance in both the target and source domains. Researchers have proposed a variety of solutions to the problem of domain adaptation in object detection, including example-based methods, adversarial learning-based methods, and reconstruction-based methods. These methods not only improve model generalization performance but also effectively alleviate the overfitting problem in object detection tasks, possessing significant theoretical and practical value.

[0004] Mainstream domain adaptive object detection methods use pixel-by-pixel feature distribution alignment methods. These methods share a common characteristic: they model the category centers of the domains and minimize the distance between the category centers between the domains, thereby narrowing the domain gap between the source and target domains at the category level. However, these methods have some obvious problems. First, mainstream methods ignore the importance of intra-class differences and directly manually set category centers for alignment. Because the size and shape of the objects to be detected vary, intra-class variations also contain important information representing the conditional distribution of the class, such as scale and shape, which should also be aligned in domain adaptation. Second, mainstream methods suffer from semantic mismatch within the same training batch, that is, they only perform domain adaptation on categories that appear in both domains. Mainstream methods ignore the loss of semantic knowledge caused by missing categories, resulting in poor domain adaptation results. Summary of the Invention

[0005] The embodiments of the present invention provide a domain-adaptive object detection method based on graph matching, which is used to solve the problems existing in the prior art.

[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions.

[0007] A domain adaptive object detection method based on graph matching, comprising the following steps:

[0008] Step 1: Obtain two different image datasets, one of which is a labeled source domain image dataset and the other is an unlabeled target domain image dataset.

[0009] Step 2: Use a shared feature extractor to extract global features, train the discriminator between the global features of the source domain and the target domain to obtain the adversarial loss of the global features, and use the global features of the source domain for the target detection task to obtain the target detection loss. For the global features of the source domain, the model uses uniform sampling to collect pixels within the annotation box and uses them as samples of the foreground node. At the same time, pixels outside the annotation box are sampled as background samples. For the global features of the target domain, the model forward propagates the features of the target domain to the classification head of FCOS (Fully Convolutional One-Stage Object Detection) to obtain a pseudo score map The model will sample to satisfy The pixels of are taken as foreground nodes, and the low-score pixels outside the annotation box are sampled at the same time, that is, Pixels of the source and target domains are used as background samples. After sampling global features, the model performs nonlinear projection to obtain the original graph nodes, thereby transforming features from visual space to graph space. A graph node discriminator is trained on the graph structure features of the source and target domains, resulting in an adversarial loss for the graph node discriminator.

[0010] Step 3: Based on the source domain graph structure features and target domain graph structure features obtained in step 2, the model performs graph semantic completion. In order to generate pseudo nodes in the missing categories, the model defines a graph-guided memory bank with specific category knowledge, and the memory bank is updated using spectral clustering. The model calculates the standard deviation of the target domain node corresponding to the missing category ω, and obtains the vector of the approximate missing category distribution Then, the model loads the memory seed of the corresponding semantic category from the memory bank as the expectation Gaussian sampling and linear projection are then used to obtain pseudo nodes for missing categories, ultimately completing the semantics of the source domain's graph structural features. The target domain's graph structural features are completed in the same manner. For semantically complete graph structural features, the model establishes cross-graph interactions between source and target domain nodes to obtain an affinity matrix. Finally, the model performs bipartite graph matching between the source and target domain affinity matrices to obtain the graph matching loss.

[0011] Step 4: Add the two adversarial losses and target detection loss generated in step 2 to the graph matching loss generated in step 3 to obtain a total loss function as the final loss function; then input the training sets of the source domain and target domain data sets into the network model in batches for forward propagation, calculate the loss value, and then perform backpropagation to optimize the network model. Training is stopped when the multi-task learning loss value converges, and a trained deep learning network model is obtained.

[0012] Step 5: Obtain a test set image of the target domain; input the test set image into the trained deep learning model, obtain the target detection result of the image and calculate the average accuracy of the model on the test set.

[0013] Preferably, the structure used for converting visual space features to graphic space features in step 1 is: fully connected layer-layer normalization-ReLU activation-fully connected layer.

[0014] Preferably, the feature extraction network in step 2 is a convolutional neural network, and the detection head for target detection is a detection head of a one-stage target detector.

[0015] Preferably, the expression formula of the graph structure in step 3 is in, is a set of vertices, each node represents a different category of features. The set of edges formed by the vertices in .

[0016] Preferably, the graph matching loss in step 3 comprises three components. The first component is used to correctly match node pairs and enhances the accuracy of the best match. The second component evaluates the difference between node affinities and true values to suppress false activations. The third component introduces a quadratic constraint as a third term to minimize the structural differences between matched node pairs within a local neighborhood. The sum of these three components is the graph matching loss.

[0017] Preferably, during the training process of step 4, the network model is optimized using an adaptive moment estimation (Adam) optimization method.

[0018] It can be seen from the technical solutions provided by the above-mentioned embodiments of the present invention that the present invention discloses a domain adaptive target detection method based on graph matching. After the method of the present invention obtains the source domain and target domain data, it obtains global features through a feature extraction network and converts them into graph structure features. The method divides the network into a target detection branch and a domain adaptive branch. In the target detection branch, the method adopts a cross entropy loss function to obtain the target detection loss. In the domain adaptive branch, the method generates pseudo nodes based on the graph structure features to complete the missing categories, and performs graph matching on the completed graph structure features. The present invention introduces intra-class differences through the graph structure, and completes the missing categories in the graph structure features by generating pseudo nodes, thereby obtaining a better and more comprehensive feature expression, thereby improving the accuracy of domain adaptive target detection.

[0019] Additional aspects and advantages of the present invention will be set forth in part in the following description, will become apparent from the following description, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A processing flow chart of a domain adaptive object detection method based on graph matching provided by the present invention;

[0022] Figure 2 A schematic diagram of the structure of a deep learning network model for a domain-adaptive object detection method based on graph matching provided by the present invention, used to illustrate the processing process of the deep learning network model;

[0023] Figure 3 A schematic diagram of the structure of a deep learning network model for a domain-adaptive object detection method based on graph matching provided by the present invention, used to illustrate the processing process of graph semantic completion;

[0024] Figure 4 The present invention provides a domain adaptive target detection method based on graph matching based on four domain adaptive target detection datasets, namely: Cityscapes, Foggy Cityscapes, Sim10K, and KITTI. DETAILED DESCRIPTION

[0025] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limiting the present invention.

[0026] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the description of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or couplings. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.

[0027] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art in the art to which the present invention pertains. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and, unless defined as such herein, will not be interpreted in an idealized or overly formal sense.

[0028] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings, and each embodiment does not constitute a limitation on the embodiments of the present invention.

[0029] See also Figure 1 The present invention provides a domain adaptive target detection method based on graph matching, comprising the following steps:

[0030] S1 obtains two different image datasets, one of which is a labeled source domain image dataset and the other is an unlabeled target domain image dataset;

[0031] S2 is based on the global features of the source domain image dataset and the target domain image dataset extracted by the deep learning network model. It obtains the adversarial loss by training the discriminator and uses the global features of the source domain image dataset to perform the target detection task and obtain the target detection loss.

[0032] S3 converts the global features of the source domain image dataset and the target domain image dataset into graph structure features through nonlinear projection; then the semantics of the graph structure features of the source domain image dataset and the target domain image dataset are complemented through the memory bank; the bipartite graph matching operation is performed using the graph structure features of the source domain image dataset and the target domain image dataset after the complement operation to obtain the graph matching loss;

[0033] S4 adds the adversarial loss, target detection loss, and graph matching loss to obtain the total loss function; inputs the source domain image dataset and the target domain image dataset into the deep learning network model in batches for forward propagation, and calculates the loss value of the total loss function; and performs backpropagation on the deep learning network model through the loss value;

[0034] S5 repeats steps S1 to S4 until the loss value converges, and a trained deep learning network model is obtained;

[0035] S6 inputs the acquired test set of the target domain into the trained deep learning network model, obtains the target detection results of the target image, and calculates the average precision (mAP) of the model on the test set.

[0036] In the embodiment provided by the present invention, the specific execution process of each step is as follows.

[0037] See attached Figure 1 and 2 , the present invention has a training model stage and a testing model stage. In the training model stage, the present invention extracts global features of the source domain and the target domain through a convolutional neural network. The model is then divided into two branches for training. One branch performs the target detection task to obtain the target detection loss of the source domain. The other branch converts the global features of the source domain and the target domain into graph structure features, and performs graph matching on the basis of completing the categories where the graph structure features are missing. In the testing model stage, this method proposes this domain adaptive target detection method based on graph matching, and the trained model performs target recognition detection on the test input image of the target domain and calculates the average precision mAP.

[0038] The specific method is as follows:

[0039] Step 1: Obtain two different image datasets, one of which is the labeled source domain image dataset Cityscapes, and the other is the unlabeled target domain image dataset Foggy Cityscapes.

[0040] Step 2: Use the shared feature extractor ResNet-50 to extract global features, such as Figure 2As shown, the global feature discriminator is trained between the global features of the source domain and the target domain to obtain the adversarial loss of the global feature discriminator, and the global features of the source domain are used for the target detection task to obtain the target detection loss (Specifically, it can be calculated based on the existing algorithm.) The process of obtaining the adversarial loss of the global feature discriminator is as follows:

[0041]

[0042] Where, Refers to the global features extracted by the feature extraction network from the source domain, Refers to the global features extracted by the feature extraction network from the target domain, (u, v) refers to a certain position on the feature, D GA Refers to the global feature discriminator, which is used to identify whether the pixel on each feature is from the source domain or the target domain. z refers to the domain label.

[0043] The detection head in the object detection task is the detection head of FCOS (Fully Convolutional One-Stage Object Detection), which includes three branches: classification, detection, and center. For the global features of the source domain, the model uses uniform sampling to collect pixels within the annotation box and uses them as samples of foreground nodes. At the same time, pixels outside the annotation box are sampled as background samples. For the global features of the target domain, the model forward propagates the features of the target domain to the classification head of FCOS to obtain a pseudo score map. The model will sample to meet The pixels of are taken as foreground nodes, and the low-score pixels outside the annotation box are sampled at the same time, that is, Pixels are used as background samples. After sampling global features, the model uses a fully connected layer-layer normalization-ReLU activation-fully connected layer structure to perform nonlinear projection to obtain the original graph structure, achieving feature conversion from visual space to graph space. The number of nodes in the graph structure is 100. A graph node discriminator is then trained on the graph structure features of the source and target domains, resulting in an adversarial loss for the graph node discriminator. The process is as follows:

[0044]

[0045] in, Refers to the source domain node set, refers to the target domain node set, refers to the i-th source domain node, Refers to the i-th target domain node. The node discriminator (ND) consists of a gradient reversal layer, three stacked discriminant blocks f b(where each discriminant block is a FC-LayerNrom-ReLU structure, i.e. a fully connected layer-layer normalization-RELU activation function structure), and a domain classifier f dc , followed by the binary cross entropy (BCE) loss function.

[0046] Step 3: Based on the source domain graph structure features and target domain graph structure features obtained in step 2, the model performs graph semantic completion. In order to generate pseudo nodes in the missing categories, such as Figure 3 As shown in Figure 2, the model defines a graph-guided memory bank with category-specific knowledge, which is updated using spectral clustering. The model calculates the standard deviation of the target domain node corresponding to the missing category ω, and obtains the vector of the approximate missing category distribution Then, the model loads the memory seed of the corresponding semantic category from the memory bank as the expectation Gaussian sampling and linear projection are then used to obtain pseudo nodes of missing categories, and finally the semantics of the graph structure features of the source domain are completed. It can be expressed as,

[0047]

[0048] In the formula, s represents the source domain, t represents the target domain, Represents the graph structure of the source domain, represents a node in the source domain graph structure, Represents a pseudo node, represents a linear mapping, ω represents the missing category in the source or target domain, represents the standard deviation of the target domain nodes corresponding to the missing categories of the source domain, A vector representing the approximate missing category distribution.

[0049] The graph structure features of the target domain are completed in a similar way. The difference is that the graph structure is generated After that, we need to use the source domain node corresponding to the missing category ω As supervisory information, the pseudo nodes are further enhanced through a single layer of Dynamic MLP. The feature representation ability (such as Figure 3 As shown), it can be expressed as,

[0050]

[0051] In the formula, s represents the source domain, t represents the target domain, Represents the graph structure of the source domain, represents a node in the source domain graph structure, represents a pseudo node, DynamicMlp(·) represents a dynamic multi-layer perceptron structure, ω represents the missing category in the source domain or target domain, Indicates the source domain node corresponding to the missing category in the target domain, represents the standard deviation of the source domain nodes corresponding to the missing categories of the target domain, A vector representing the approximate missing category distribution.

[0052] For the semantically complete graph structure features, the model establishes cross-graph interactions between source and target domain nodes and obtains the affinity matrix, which can be expressed as:

[0053]

[0054] in, Indicates the connection operation, f p represents linear projection, f mlp represents a multilayer perceptron machine, Represents the source domain node, Represents the target domain node, is the node association matrix.

[0055] Finally, the model performs bipartite graph matching between the affinity matrices of the source domain and the target domain to obtain the graph matching loss. The graph matching loss consists of three parts of loss. The first part is used to correctly match node pairs and enhance the correctness of the best match. The second part evaluates the difference between the node affinity and the true value to suppress the case of false activation. The third part introduces a quadratic constraint as the third term to minimize the structural difference of the matched node pairs in the local neighborhood. The sum of the three parts of the loss is the graph matching loss, which is expressed as,

[0056]

[0057] In the formula, if and In the same category ω, then Y Π The value corresponding to the (i, j) position of Μ is 1, otherwise it is 0. aff represents the node affinity matrix, ⊙ represents the dot product, is the number of nodes in the graph, and Respectively represent the figure and The adjacency matrix of the structural information encoding. The graph matching loss consists of three parts, namely the right side of the equal sign. The first part is used to correctly match node pairs and enhance the correctness of the best match. The second part evaluates the difference between the node affinity and the true value to suppress the case of wrong activation. The third part introduces a quadratic constraint as the third term to minimize the structural difference of the matching node pairs in the local neighborhood. The sum of the three parts of the loss is the graph matching loss. In addition, Figure 3 middle, Refers to loading memory seeds with the same semantics as the missing category from the memory bank in the source domain—— Refers to loading memory seeds with the same semantics as the missing category from the memory bank in the target domain——

[0058] The representation formula of the graph structure in step 3 is in, is a set of vertices, each node represents a different category of features. The graph structure feature conversion process is as follows: After sampling global features, the model uses a fully connected layer-layer normalization-ReLU activation-fully connected layer structure to perform nonlinear projection to obtain the original graph structure, realizing the feature conversion from visual space to graph space, where the number of nodes in the graph structure is 100.

[0059] Step 4: Add the two adversarial losses and target detection loss generated in step 2 to the graph matching loss generated in step 3 to obtain a total loss function as the final loss function. Then, the training set of the source domain and target domain datasets is input into the network model in batches of 4 (4 for the source domain and 4 for the target domain) for forward propagation (the training set images are as follows: Figure 4 ), calculate the loss value, and then perform backpropagation to optimize the network model. The adversarial loss is gradient-reversed during backpropagation. This process is repeated until the multi-task learning loss value converges, and training is terminated to obtain the trained deep learning network model. The training parameters are set as follows: learning rate 0.0025, momentum 0.9, and weight decay 5×10 -4 , the short side size of the input image is 800 and the long side size is less than or equal to 1333.

[0060] Step 5: Obtain images from the target domain's test set, Foggy Cityscapes. These images are fed into the trained deep learning model to obtain object detection results for these images. The model's mean average precision (mAP) on the test set is calculated. This is the mean of the AP for each category, consistent with the evaluation metric used by traditional object detection algorithms. The experimental results are shown in Table 1.

[0061] Table 1 Comparison of the performance of domain adaptive object detection methods from Cityscapes as the source domain to Foggy Cityscapes as the target domain

[0062]

[0063]

[0064] Table 1

[0065] Table 1 shows a comparison with existing methods, demonstrating that the proposed method significantly improves the accuracy of domain adaptive object detection. The quantitative evaluation results in Table 1 demonstrate the effectiveness and robustness of the proposed method.

[0066] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0067] In summary, the present invention discloses a domain adaptive target detection method based on graph matching. After acquiring source domain and target domain data, the method of the present invention obtains global features through a feature extraction network and converts them into graph structure features. The method divides the network into a target detection branch and a domain adaptation branch. In the target detection branch, the method adopts a cross entropy loss function to obtain the target detection loss. In the domain adaptation branch, the method generates pseudo nodes based on the graph structure features to complete the missing categories, and performs graph matching on the completed graph structure features. The present invention introduces intra-class differences through the graph structure, and completes the missing categories in the graph structure features by generating pseudo nodes, thereby obtaining a better and more comprehensive feature expression, thereby improving the accuracy of domain adaptive target detection.

[0068] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0069] From the above description of the embodiments, it can be seen that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus the necessary general-purpose hardware platform. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0070] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without making any creative efforts.

[0071] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A domain adaptive object detection method based on graph matching, characterized in that: The following steps are involved: S1 obtains two different image datasets, one of which is a labeled source domain image dataset and the other is an unlabeled target domain image dataset; S2 is based on the global features of the source domain image dataset and the target domain image dataset extracted by the deep learning network model. The discriminator is trained to obtain the adversarial loss of the global features, and the global features of the source domain image dataset are used to perform the target detection task to obtain the target detection loss. In this step: The global features of the source domain image dataset and the target domain image dataset are extracted through the feature extractor ResNet-50; In the process of performing an object detection task using the global features of a source domain image dataset, the detection head of the detection task includes a classification branch, a detection branch, and a centerness branch; Pass-through Calculate the adversarial loss of the global feature discriminator; where, Refers to the global features extracted by the feature extraction network from the source domain, Refers to the global features extracted by the feature extraction network from the target domain, (u, v) refers to a certain position on the feature, D GA Refers to the global feature discriminator, which is used to identify whether the pixel on each feature is from the source domain or the target domain, and z refers to the domain label; S3 converts the global features of the source domain image dataset and the target domain image dataset into graph structural features through nonlinear projection operations; trains a graph node discriminator between the graph structural features of the source domain and the target domain to obtain the adversarial loss of the graph node discriminator; performs a semantic complement operation on the graph structural features of the source domain image dataset and the target domain image dataset through the memory bank; performs a bipartite graph matching operation using the graph structural features of the source domain image dataset and the target domain image dataset after the complement operation to obtain the graph matching loss; S4 adds the adversarial loss of global features, the adversarial loss of graph node discriminators, the target detection loss, and the graph matching loss to obtain the total loss function; the source domain image dataset and the target domain image dataset are input into the deep learning network model in batches for forward propagation, and the loss value of the total loss function is calculated; the deep learning network model is back-propagated through the loss value; S5 repeats steps S1 to S4 until the loss value converges, and a trained deep learning network model is obtained; S6 inputs the acquired test set of the target domain into the trained deep learning network model, obtains the target detection result of the target image, and calculates the average precision (mAP) of the model on the test set.

2. The method according to claim 1, characterized in that In step S3, the process of converting the global features of the source domain image dataset and the target domain image dataset into graph structure features through nonlinear projection includes: Based on the global features of the source domain image dataset and the target domain image dataset, a nonlinear projection operation is performed through the structure of the deep learning network model of fully connected layer-layer normalization-ReLU activation-fully connected layer to obtain the global features of the source domain image dataset and the target domain image dataset, which are converted into graph structure features; The graph structure features of the source domain image dataset and the target domain image dataset are expressed by Indicates; where, is a vertex set, each node represents a feature of a different category. In the graph structure of the graph structure features of the source domain image dataset and the target domain image dataset, the number of nodes is 100, and ε is The set of edges formed by the vertices in The process of training the graph node discriminator between the graph structure features of the source domain and the target domain to obtain the adversarial loss of the graph node discriminator is as follows Where, Refers to the source domain node set, refers to the target domain node set, refers to the i-th source domain node, Refers to the i-th target domain node. The graph node discriminator consists of a gradient reversal layer, three stacked discriminant blocks f b , and a domain classifier f dc , followed by the binary cross entropy loss function.

3. The method according to claim 2, characterized in that Step S3 further includes: Pass-through Completion operation is performed on the global features of the source domain image dataset; where s represents the source domain, t represents the target domain, Represents the graph structure of the source domain, represents a node in the source domain graph structure, represents a pseudo node in the source domain image, represents a linear mapping, ω represents the missing category in the source or target domain, represents the standard deviation of the target domain nodes corresponding to the missing categories of the source domain, A vector representing the approximate missing class distribution; Pass-through Completion operation is performed on the global features of the target domain image dataset; where s represents the source domain, t represents the target domain, Represents the graph structure of the source domain, represents a node in the source domain graph structure, represents the pseudo node in the target domain image, DynamicMlp(·) represents the dynamic multi-layer perceptron structure, ω represents the missing category in the source domain or target domain, Indicates the source domain node corresponding to the missing category in the target domain, represents the standard deviation of the source domain nodes corresponding to the missing categories of the target domain, A vector representing the approximate missing class distribution; Using the global features of the source domain image dataset and the target domain image dataset after the completion operation, the formula Establish cross-graph interactions between source and target domain nodes and calculate the affinity matrix; where Indicates the connection operation, f p represents linear projection, f mlp represents a multilayer perceptron machine, Represents the source domain node, Represents the target domain node, is the node association matrix; Pass-through Perform a bipartite graph matching operation between the affinity matrices of the source domain and the target domain to calculate the graph matching loss; where, if and In the same category ω, then Y Π The value corresponding to the (i, j) position is 1, otherwise it is 0, aff represents the node affinity matrix, ⊙ represents the dot product, is the number of nodes in the graph, and Respectively represent the figure and The adjacency matrix encodes the structural information of the node. The first part on the right side of the equal sign is used to correctly match node pairs, the second part is used to evaluate the difference between the node affinity and the true value, and the third part is to minimize the structural difference of the matched node pairs in the local neighborhood.

4. The method according to claim 1, wherein During step S4: Set up each batch to extract 4 image data from the source domain image dataset and the target domain image dataset, and input them into the deep learning network model for forward propagation.

5. The method according to claim 1, wherein In the process of backpropagating the deep learning network model through the loss value described in step S4, the adaptive moment estimation method is used to optimize the deep learning network model.

Images