Pixel-instance information migration-based domain adaptive rotating target detection method

Abstract

The invention discloses a domain adaptation rotating target detection method based on pixel-instance information migration, belongs to the technical field of target detection, and solves the problem that SAR target detection precision is limited due to lack of labeled training data. The visual content difference between a visible light domain and an SAR domain is relieved through self-adaptive synthesis of a middle domain image, then multi-scale and multi-direction features are extracted based on the middle domain image and an SAR image, rotation instance features are obtained on the basis, and discernability of domain invariant features is enhanced by decoupling the instance features; according to the technical scheme, the information distribution gap between the visible light domain and the SAR domain is made up by combining the effective migration of the visual content and the rotation instance; according to the method, visible light domain-SAR domain information distribution fitting can be effectively realized, the generalization of cross-domain target detection is enhanced, and the difference between the visible light domain and the SAR domain in information distribution is effectively made up.

Description

Technical Field

[0001] The invention belongs to the field of target detection technology and relates to a domain adaptive rotation target detection method based on pixel-instance information migration. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active microwave imaging sensor with strong penetration ability. It can perform long-distance observations in all weather conditions and at all times of the day. It has become an important means of earth observation. However, due to the special observation method and imaging mechanism, SAR target detection faces two major challenges. First, due to the special overhead viewing angle and long-distance shooting mode, SAR images have problems such as large target scale differences, dense arrangement and strong directionality. When multiple targets are densely arranged at an inclined angle, the traditional horizontal detection frame will have overlapping boundaries, resulting in excessive coverage of background areas or unexpected targets. Second, due to the imaging mechanism, SAR images are difficult to identify and interpret, resulting in most SAR images being unlabeled and unusable, and target detection performance often relies on a large amount of labeled data, which makes it difficult to improve the detection accuracy of unlabeled SAR targets.

[0003] Rotated target detection and unsupervised domain adaptation are effective solutions to the above two challenges. Specifically, rotated target detection uses a directional detection box to represent the rotating target, which provides more complete geometric position information compared to the horizontal detection box. In addition, the annotation cost of visible light images is much lower than that of SAR images. Unsupervised domain adaptation can transfer domain information from the visible light domain to the SAR domain, thereby alleviating the domain gap between the label-rich domain and the new unlabeled domain. Ideally, the model trained in the visible light domain can effectively detect unlabeled SAR targets. In order to significantly improve the degree of exploration and interpretation reliability of rotating SAR target information, rotated target detection and unsupervised domain adaptation can be combined, namely the domain adaptation rotated target detection method.

[0004] Since the detection accuracy of SAR rotating targets depends not only on the alignment of information distribution across domains, but also on the precise capture of the rotation angle, how to explore specific domain information and extract the rotating target angle information is the key point to achieve accurate and stable domain information migration. Summary of the invention

[0005] The technical solution of the present invention is used to solve the problem of limited SAR target detection accuracy due to lack of labeled training data.

[0006] The present invention solves the above technical problems through the following technical solutions:

[0007] A domain adaptation object detection method based on pixel-instance information migration includes the following steps:

[0008] Step 1: Extract the SAR image domain D t Inherent speckle noise and visible image domain D s The basic structural information in the middle domain D is obtained by fusing the two domains. i , alleviate the visual content differences between the visible light domain and the SAR domain;

[0009] Step 2: Extract the intermediate domain D i and SAR image domain D t Multi-scale and multi-directional features

[0010] Step 3: Get the rotation instance feature F i , and decouples instance features into domain-invariant features and domain-specific features, promoting the cross-domain recognition ability of rotated instances to enhance the generalization of cross-domain object detection.

[0011] Furthermore, the SAR image domain D is extracted separately as described in step 1. t Inherent speckle noise and visible image domain D s The basic structural information in the middle domain D is obtained by fusing the two domains. i The method is as follows:

[0012] Step 1.1: Input with N s A visible light image dataset with 10 samples and with N t SAR image dataset with 10 samples For visible light images Apply contrast-limited adaptive histogram equalization to obtain an enhanced visible light image through contrast enhancement. Improve visuals and highlight structural details of targets;

[0013] Step 1.2, design an adaptive speckle noise background extraction method to extract the speckle noise background of varying sizes introduced during the SAR imaging process;

[0014] Step 1.3: Combine enhanced visible light images and coherent speckle noise background image Synthetic intermediate domain D i .

[0015] Furthermore, the visible light image Apply contrast-limited adaptive histogram equalization to obtain an enhanced visible light image through contrast enhancement. The method is as follows:

[0016] Will Divide into 8×8 sub-blocks, and calculate the equalization results through the following transformations:

[0017]

[0018] Where U = 2 8 , f u is the conversion result of the pixel with value u, n u is the number of pixels with value u, and N is the total number of pixels in each sub-block;

[0019] To prevent excessive contrast amplification in smooth areas, which results in noise amplification, the resulting histogram is clipped at the contrast limit, and the clipped portion is evenly redistributed back into the histogram;

[0020] Each sub-block is divided into The corresponding positions in the image are combined to form an enhanced visible light image.

[0021] Furthermore, the adaptive coherent speckle noise background extraction method is designed to extract the coherent speckle noise background of varying sizes introduced during the SAR imaging process, as follows:

[0022] For SAR images Adaptive thresholding is used for binary processing to dynamically generate a masked image, that is, the pixel values ​​greater than the threshold δ are set to 255, and the remaining pixels are set to 0; for small speckle noise, δ is defined as the interference threshold η that can be perceived by the human eye to capture tiny imaging differences; for large speckle noise, the sorted pixel value γ is used as the threshold to balance the removal of foreground information and the retention of background information; the threshold δ for adaptive speckle noise background extraction is selected as follows:

[0023]

[0024] Among them, γ is the input The values ​​of the top 60% of the pixel values ​​are sorted, and η is set to 70 based on experience.

[0025] After obtaining the masked image, the contour information of the foreground target is used to characterize it. By defining these areas as distorted areas and repairing them using adjacent information, the foreground targets of the SAR image are removed to generate a coherent speckle noise background image.

[0026] Furthermore, the synthetic intermediate domain D i The method is as follows:

[0027] Weighted combination of j-th enhanced visible light image with a randomly selected coherent speckle noise background image Get the jth intermediate domain image The formula is as follows:

[0028]

[0029] Among them, ε1=ε2=0.5, gradually synthesizing the intermediate domain

[0030] Furthermore, the extraction of the intermediate domain D i and SAR image domain D t Multi-scale and multi-directional features The method is as follows:

[0031] Enter the middle field D i and SAR image domain, and generate multi-directional Gabor features F through Gabor convolutional network g , the formula is as follows:

[0032]

[0033] Among them, F is the convolution feature of the input image, O is the total number of directions, and C o is the Gabor directional filter in the o direction, which consists of the convolution filter C and the Gabor filter group G o (x, y), the formula is as follows:

[0034]

[0035]

[0036] in, represents the pixel-by-pixel product operation, σ=2π is the standard deviation of the Gaussian function, Represents the direction information of the filter, o=0,1,…,O is the direction index;

[0037] Extracting multi-scale spatial features of input images based on feature pyramid network s , integrating multi-scale spatial features F s and multi-directional Gabor features F g , generating multi-scale and multi-directional features The fusion formula is as follows:

[0038]

[0039] Among them, Conv is a 1×1 convolution kernel.

[0040] Furthermore, the rotation instance feature F is obtained as described in step 3 i , and decouple instance features into domain-invariant features and domain-specific features, and promote the cross-domain recognition ability of rotated instances to enhance the generalization of cross-domain object detection as follows:

[0041] Step 3.1: Extracting multi-scale and multi-directional features Obtain the image rotation instance feature F based on the RoIAlign module i , and then apply the common specific decomposition module to decompose the instance features into domain-invariant features and domain-specific features, that is, design the invariant classification compensation loss L invariant and the category-specific compensation loss L specific ;

[0042] In addition, for each image, an orthogonal regularizer is used to make the domain invariant classifier Γ i and domain-specific classifiers The softmax parameters are orthogonalized to improve numerical stability, and the orthogonality loss L ort The definition is as follows:

[0043]

[0044] Where Φ=[ω i ,ω s ],ω i and ω s They represent domain invariant classifiers Γ i and domain-specific classifiers Softmax parameter; I is the unit matrix, L = 2 represents the number of participating domains, namely visible light domain and SAR domain;

[0045] Classification compensation loss L csd Defined as:

[0046] L csd =α1L invariant +α2L specific +α3L ort

[0047] Among them, α1=0.25, α2=0.5 and α3=0.1 are hyperparameters to balance these compensation loss components.

[0048] Step 3.2: Calculate Gabor information adversarial loss L based on gradient reversal layer gabor and the global instance adversarial loss L ins , thereby improving the consistency of feature distribution in the visible light domain and the SAR domain. The formula is as follows:

[0049]

[0050] in, and Represents the multi-directional Gabor features and rotation instance features obtained from the visible light domain, and represents the multi-directional Gabor features and rotation instance features obtained from the SAR domain, D ga and D in Represents the corresponding domain discriminator.

[0051] The Gabor information is used against the loss L gabor and the global instance adversarial loss L ins The fusion obtains the adversarial loss, which is defined as follows:

[0052] L adv =β1L gabor +β2L ins

[0053] Among them, β1=0.1 and β2=0.15 are hyperparameters that balance these adversarial loss components;

[0054] The overall loss function calculation formula is as follows:

[0055] Loss = L cls +L reg +L csd +L adv

[0056] Among them, L cls and L reg represent classification loss and regression loss respectively.

[0057] Furthermore, the design remains unchanged and the classification compensation loss L invariant and the category-specific compensation loss L specific , as follows:

[0058]

[0059] Where E represents the cross entropy loss function, Γ i and for domain-invariant classifiers and domain-specific classifiers; and Represent the instance features in the visible light domain and the SAR domain respectively, y s and t It is the label in the visible light domain and the pseudo label information in the SAR domain.

[0060] An electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above-mentioned domain adaptation target detection method based on pixel-instance information migration, and the processor is configured to execute the program stored in the memory.

[0061] A storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the above-mentioned domain adaptation target detection method based on pixel-instance information migration are executed.

[0062] The advantages of the present invention are:

[0063] The technical solution of the present invention combines the effective migration of visual content and rotation instances to make up for the information distribution gap between the visible light domain and the SAR domain. First, the coherent speckle noise in the SAR image is mixed into the visible light image, and the visual content difference between the visible light domain and the SAR domain is alleviated by adaptively synthesizing the intermediate domain image. Then, multi-scale and multi-directional features are extracted based on the intermediate domain image and the SAR image, and the rotation instance features are obtained on this basis. The discriminability of the domain invariant features is enhanced by decoupling the instance features. The present invention can effectively realize the information distribution fitting of the visible light domain-SAR domain, enhance the generalization of cross-domain target detection, and effectively make up for the information distribution gap between the visible light domain and the SAR domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is a flow chart of a domain adaptation target detection method based on pixel-instance information migration of the present invention;

[0065] Figure 2 It is a schematic diagram of the framework of the domain-adaptive rotation target detection method based on pixel-instance information migration of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments:

[0068] Embodiment 1

[0069] See also Figure 1 and Figure 2 As shown in FIG. 1 , a domain adaptation target detection method based on pixel-instance information migration is shown in FIG. 1 . The specific steps are as follows:

[0070] Step 1: Extract the SAR image domain D t Inherent speckle noise and visible image domain D s The basic structural information in the middle domain D is obtained by fusing the two domains.i , alleviate the visual content differences between the visible light domain and the SAR domain, as follows:

[0071] Step 1.1: Input with N s A visible light image dataset with 10 samples and with N t SAR image dataset with 10 samples Where x, y, and b represent the image, label, and bounding box annotation, respectively. Contrast-limited adaptive histogram equalization (CLAHE) is applied to obtain an enhanced visible light image by contrast enhancement. Improve visuals and highlight structural details of targets.

[0072] Specifically, Divide into 8×8 sub-blocks, and calculate the equalization results through the following transformations:

[0073]

[0074] Where U = 2 8 , f u is the conversion result of the pixel with value u, n u is the number of pixels with value u, and N is the total number of pixels in each sub-block. In order to prevent the contrast of the smooth area from being over-amplified, resulting in noise amplification, the obtained histogram is clipped at the contrast limit, and the clipped part is evenly redistributed back to the histogram. The corresponding positions in the image are combined to form an enhanced visible light image.

[0075] Step 1.2: Design an adaptive speckle noise background extraction method to extract the speckle noise background of varying sizes introduced during SAR imaging. The details are as follows:

[0076] For SAR images Adaptive thresholding is used for binary processing to dynamically generate a masked image, that is, pixel values ​​greater than the threshold δ (representing foreground information) are set to 255, and the remaining pixels (representing background information) are set to 0. For small coherent speckle noise, δ is defined as the interference threshold η that can be perceived by the human eye to capture tiny imaging differences; for large coherent speckle noise, the sorted pixel value γ is used as the threshold to balance the removal of foreground information and the retention of background information.

[0077] The selection of the threshold δ for adaptive coherent speckle noise background extraction is as follows:

[0078]

[0079] Among them, γ is the input The values ​​of the top 60% of the pixel values ​​are sorted, and η is set to 70 based on experience.

[0080] After obtaining the masked image, the contour information of the foreground target is used to characterize the foreground target. These areas are defined as distorted areas and repaired using adjacent information to remove the SAR image. The foreground target generates a coherent speckle noise background image

[0081] Step 1.3: Weighted combination of j-th enhanced visible light image with a randomly selected coherent speckle noise background image Get the jth intermediate domain image The formula is as follows:

[0082]

[0083] Among them, ε1=ε2=0.5, gradually synthesizing the intermediate domain

[0084] Step 2: Extract the intermediate domain D i and SAR image domain D t Multi-scale and multi-directional features The details are as follows:

[0085] Enter the middle field D i and SAR image domain, and generate multi-directional Gabor features F through Gabor convolutional network g ,as follows:

[0086]

[0087] Where F is the convolution feature of the input image, and O=4 is the total number of directions used. o is the Gabor directional filter (GoF) in the o direction, which consists of the convolution filter C and the Gabor filter group G o (x, y), the formula is as follows:

[0088]

[0089] in, represents the pixel-by-pixel product operation, σ=2π is the standard deviation of the Gaussian function, Represents the directional information of the filter, o=0,1,…,O is the directional index.

[0090] Then, the multi-scale spatial features F of the input image are extracted based on the feature pyramid network (FPN) s , integrating multi-scale spatial features F s and multi-directional Gabor features F g, generating multi-scale and multi-directional features The fusion formula is as follows:

[0091]

[0092] Among them, Conv is a 1×1 convolution kernel.

[0093] Step 3: Get the rotation instance feature F i , and decouple instance features into domain-invariant features and domain-specific features, promoting the cross-domain recognition ability of rotated instances to enhance the generalization of cross-domain object detection, as follows:

[0094] Step 3.1: Extracting multi-scale and multi-directional features Obtain the image rotation instance feature F based on the RoIAlign module i , then apply the common specific decomposition (CSD) module to decompose the instance features into domain-invariant features and domain-specific features, that is, design the invariant classification compensation loss L invariant and the category-specific compensation loss L specific , defined as follows:

[0095]

[0096] Where E represents the cross entropy loss function, Γ i and for domain-invariant classifiers and domain-specific classifiers. and Represent the instance features in the visible light domain and the SAR domain respectively, y s and t The labels in the visible light domain and the pseudo-label information in the SAR domain are used to enhance the discriminability of domain-invariant features, which greatly enhances the cross-domain detection generalization of rotated instances.

[0097] In addition, for each image, an orthogonal regularizer is used to make the domain invariant classifier Γ i and domain-specific classifiers The softmax parameters are orthogonalized to improve numerical stability, and the orthogonality loss L ort The definition is as follows:

[0098]

[0099] Where Φ=[ω i ,ω s ],ω i and ω s They represent domain invariant classifiers Γ i and domain-specific classifiers The softmax parameter of . I is the unit matrix, and L = 2 represents the number of participating domains, namely the visible light domain and the SAR domain.

[0100] Classification compensation loss L csd It can be defined as:

[0101] L csd =α1L invariant +α2L specific +α3L ort

[0102] Among them, α1=0.25, α2=0.5 and α3=0.1 are hyperparameters to balance these compensation loss components.

[0103] Step 3.2: Calculate Gabor information adversarial loss L based on gradient reversal layer (GRL) gabor and the global instance adversarial loss L ins , thereby improving the consistency of feature distribution in the visible light domain and the SAR domain. The formula is as follows:

[0104]

[0105] in, and Represents the multi-directional Gabor features and rotation instance features obtained from the visible light domain, and represents the multi-directional Gabor features and rotation instance features obtained from the SAR domain, D ga and D in Represents the corresponding domain discriminator.

[0106] The Gabor information is used against the loss L gabor and the global instance adversarial loss L ins The fusion obtains the adversarial loss, which is defined as follows:

[0107] L adv =β1L gabor +β2L ins

[0108] Among them, β1=0.1 and β2=0.15 are hyperparameters that balance these adversarial loss components.

[0109] The overall loss function of the model is calculated as follows:

[0110] Loss = L cls +L reg +L csd +L adv

[0111] Among them, L cls and L regrepresent classification loss and regression loss respectively.

[0112] Experimental verification

[0113] The present invention uses the ResNet-101 architecture pre-trained on ImageNet as the basic feature extractor. The experiment is based on the PyTorch framework and trained using stochastic gradient descent (SGD), with a weight decay of 0.0001 and a momentum of 0.9. In the training phase, a total of 24 cycles are trained, the initial learning rate is set to 0.005, and divided by 10 at the 16th and 22nd cycles respectively. Each training batch contains two images, one from the intermediate domain and the other from the SAR image domain.

[0114] In order to verify the effectiveness of the present invention, the domain adaptation performance of the method of the present invention is evaluated on two benchmark datasets dedicated to domain adaptation rotation target detection. The two benchmark datasets consist of a visible light dataset (i.e., HRSC2016) and two SAR datasets (i.e., SSDD and RSDD-SAR). The experiment uses the classic rotation target detection models OrientedRCNN and Gliding Vertex as basic detection models, and compares the method of the present invention with the model trained only with visible light domain data (Scource-only). In addition, four commonly used evaluation criteria are used to evaluate the accuracy of rotation target detection, thereby measuring the domain adaptation performance between visible light and SAR domains, namely, precision, recall, average accuracy (AP) and F1 score.

[0115] The experimental results are shown in Tables 1 and 2. For the domain adaptation task of HRSC2016→SSDD, the AP values ​​of the proposed method are improved by 85.13% and 90.91% respectively compared with the source-only model. For the case of HRSC2016→RSDD-SAR, the detection accuracy of the model is relatively low, that is, the AP values ​​are only 9.09 and 12.95. This shows that there is a huge domain difference between the two datasets, and the proposed method greatly narrows this difference, with the AP values ​​increased to 19.69 and 20.31, with gains of 116.61% and 56.83%. This shows that for the case of unlabeled SAR images, the proposed method can significantly improve the accuracy of rotating SAR target detection.

[0116] Table 1 Results of the proposed method in the HRSC2016→SSDD domain adaptation task (%)

[0117]

[0118] Table 2 Results of the proposed method in HRSC2016→RSDD-SAR domain adaptation task (%)

[0119]

[0120]

[0121] An electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the domain adaptation target detection method based on pixel-instance information migration in Example 1, and the processor is configured to execute the program stored in the memory.

[0122] A storage medium stores a computer program, which, when executed by a processor, performs the steps of the domain adaptation target detection method based on pixel-instance information migration in embodiment 1.

[0123] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A domain adaptation object detection method based on pixel-instance information migration, characterized in that: The following steps are involved: Step 1: Extract the SAR image domain D t Inherent speckle noise and visible image domain D s The basic structural information in the middle domain D is obtained by fusing the two domains. i , alleviate the visual content differences between the visible light domain and the SAR domain; Step 2: Extract the intermediate domain D i and SAR image domain D t Multi-scale and multi-directional features Step 3: Get the rotation instance feature F i , and decouples instance features into domain-invariant features and domain-specific features, promoting the cross-domain recognition ability of rotated instances to enhance the generalization of cross-domain object detection.

2. The domain adaptation object detection method based on pixel-instance information migration according to claim 1, characterized in that: The SAR image domain D is extracted separately as described in step 1 t Inherent speckle noise in the visible light image domain D s The basic structural information in the middle domain D is obtained by fusing the two domains. i The method is as follows: Step 1.1: Input with N s A visible light image dataset with 10 samples and with N t SAR image dataset with 10 samples For visible light images Apply contrast-limited adaptive histogram equalization to obtain an enhanced visible light image through contrast enhancement. Improve visuals and highlight structural details of targets; Step 1.2, design an adaptive speckle noise background extraction method to extract the speckle noise background of varying sizes introduced during the SAR imaging process; Step 1.3: Combine enhanced visible light images and coherent speckle noise background image Synthetic intermediate domain D i .

3. The domain adaptation object detection method based on pixel-instance information migration according to claim 2, characterized in that: The visible light image Apply contrast-limited adaptive histogram equalization to obtain an enhanced visible light image through contrast enhancement. The method is as follows: Will Divide into 8×8 sub-blocks, and calculate the equalization results through the following transformations: Where U = 2 8 , f u is the conversion result of the pixel with value u, n u is the number of pixels with value u, and N is the total number of pixels in each sub-block; To prevent excessive contrast amplification in smooth areas, which results in noise amplification, the resulting histogram is clipped at the contrast limit, and the clipped portion is evenly redistributed back into the histogram; Each sub-block is divided into The corresponding positions in the image are combined to form an enhanced visible light image.

4. The domain adaptation object detection method based on pixel-instance information migration according to claim 3, characterized in that: The method for designing an adaptive coherent speckle noise background extraction is used to extract the coherent speckle noise background of varying sizes introduced during the SAR imaging process. Specifically, the method is as follows: For SAR images Adaptive thresholding is used for binary processing to dynamically generate a masked image, that is, the pixel values ​​greater than the threshold δ are set to 255, and the remaining pixels are set to 0; for small speckle noise, δ is defined as the interference threshold η that can be perceived by the human eye to capture tiny imaging differences; for large speckle noise, the sorted pixel value γ is used as the threshold to balance the removal of foreground information and the retention of background information; the threshold δ for adaptive speckle noise background extraction is selected as follows: Among them, γ is the input The values ​​of the top 60% of the pixel values ​​are ranked, and η is set to 70 based on experience; After obtaining the masked image, the contour information of the foreground target is used to characterize it. By defining these areas as distorted areas and repairing them using adjacent information, the foreground targets of the SAR image are removed to generate a coherent speckle noise background image.

5. The domain adaptation object detection method based on pixel-instance information migration according to claim 4, characterized in that: The synthetic intermediate domain D i The method is as follows: Weighted combination of j-th enhanced visible light image with a randomly selected coherent speckle noise background image Get the jth intermediate domain image The formula is as follows: Among them, ε1=ε2=0.5, gradually synthesizing the intermediate domain 6. The domain adaptation object detection method based on pixel-instance information migration according to claim 5, characterized in that: Extract the intermediate domain D as described in step 2 i and SAR image domain D t The multi-scale and multi-directional features F s g The method is as follows: Enter the middle field D i and SAR image domain, and generate multi-directional Gabor features F through Gabor convolutional network g , the formula is as follows: Among them, F is the convolution feature of the input image, O is the total number of directions, and C o is the Gabor directional filter in the o direction, which consists of the convolution filter C and the Gabor filter group G o (x, y), the formula is as follows: in, represents the pixel-by-pixel product operation, σ=2π is the standard deviation of the Gaussian function, Represents the direction information of the filter, o=0,1,…,O is the direction index; Extracting multi-scale spatial features of input images based on feature pyramid network s , integrating multi-scale spatial features F s and multi-directional Gabor features F g , generate multi-scale and multi-directional features F s g , the fusion formula is as follows: F s g =Conv(F s +F g ) Among them, Conv is a 1×1 convolution kernel.

7. The domain adaptation object detection method based on pixel-instance information migration according to claim 6, characterized in that: Obtain the rotation instance feature F as described in step 3 i , and decouple instance features into domain-invariant features and domain-specific features, and promote the cross-domain recognition ability of rotated instances to enhance the generalization of cross-domain object detection as follows: Step 3.1: Extract multi-scale and multi-directional features F s g , based on the RoIAlign module, obtain the image’s rotation instance feature F i , and then apply the common specific decomposition module to decompose the instance features into domain-invariant features and domain-specific features, that is, design the invariant classification compensation loss L invariant and the category-specific compensation loss L specific ; In addition, for each image, an orthogonal regularizer is used to make the domain invariant classifier Γ i and domain-specific classifiers The softmax parameters are orthogonalized to improve numerical stability, and the orthogonality loss L ort The definition is as follows: Where Φ=[ω i ,ω s ],ω i and ω s They represent domain invariant classifiers Γ i and domain-specific classifiers ; I is the unit matrix, L = 2 represents the number of participating domains, namely visible light domain and SAR domain; Classification compensation loss L csd Defined as: L csd =α1L invariant +α2L specific +α3L ort Among them, α1=0.25, α2=0.5 and α3=0.1 are hyperparameters that balance these compensation loss components; Step 3.2: Calculate Gabor information adversarial loss L based on gradient reversal layer gabor and the global instance adversarial loss L ins , thereby improving the consistency of feature distribution in the visible light domain and SAR domain. The formula is as follows: in, and F i s Represents the multi-directional Gabor features and rotation instance features obtained from the visible light domain, and F i t represents the multi-directional Gabor features and rotation instance features obtained from the SAR domain, D ga and D in represents the corresponding domain discriminator; The Gabor information is used against the loss L gabor and the global instance adversarial loss L ins The fusion obtains the adversarial loss, which is defined as follows: L adv =β1L gabor +β2L ins Among them, β1=0.1 and β2=0.15 are hyperparameters that balance these adversarial loss components; The overall loss function calculation formula is as follows: Loss=L cls +L reg +L csd +L adv Among them, L cls and L reg represent classification loss and regression loss respectively.

8. The domain adaptation object detection method based on pixel-instance information migration according to claim 7, characterized in that: The design remains unchanged and the classification compensation loss L invariant and the category-specific compensation loss L specific , as follows: Where E represents the cross entropy loss function, Γ i and are domain-invariant classifiers and domain-specific classifiers; F i s and F i t Represent the instance features in the visible light domain and the SAR domain respectively, y s and t It is the label in the visible light domain and the pseudo label information in the SAR domain.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the domain adaptation target detection method based on pixel-instance information migration as described in any one of claims 1 to 8, and the processor is configured to execute the program stored in the memory.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the domain adaptation target detection method based on pixel-instance information migration according to any one of claims 1 to 8 are performed.

Images