Method and apparatus for target detection of optical remote sensing images

Abstract

The application provides a method and device for target detection of optical remote sensing images, wherein the method comprises: acquiring a target image; the target image is an optical remote sensing image of a target domain which needs target detection; inputting the target image into a preset target detection model, and outputting a target detection result about the target image through the preset target detection model; the target detection result comprises information representing the position and category of the detected target in the target image; the preset target detection model is trained based on a preset training framework and at least one set of training data; each set of training data in the at least one set of training data comprises first type training data and second type training data; the first type training data comprises a source domain sample image and a corresponding sample true value label; and the second type training data comprises a target domain sample image. The preset target detection model has strong cross-domain target detection generalization capability, and can realize accurate target detection of the target image.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a method, apparatus, electronic device, non-transitory computer-readable storage medium, and computer program product for target detection in optical remote sensing images. Background Technology

[0002] With social development and technological progress, the application of optical remote sensing images is becoming increasingly widespread; among these applications, target detection is a challenging task. Target detection aims to simultaneously locate and classify instances (targets) within optical remote sensing images.

[0003] In the prior art, target detection is usually performed on the optical remote sensing image based on a pre-trained deep learning model. However, the pre-trained deep learning model has the problem of poor generalization ability. That is, when there is a neighborhood offset between the sample optical remote sensing image used to train the deep learning model and the target optical remote sensing image being processed, the accuracy of the prediction result of the deep learning model will decrease. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, non-transitory computer-readable storage medium, and computer program product for target detection in optical remote sensing images, in order to overcome the deficiencies in the prior art.

[0005] This invention provides a method for target detection in optical remote sensing images, comprising: Acquire the target image; wherein the target image is an optical remote sensing image of the target domain for which target detection is required; The target image is input into a preset target detection model, and the preset target detection model outputs a target detection result for the target image; wherein, the target detection result includes information characterizing the position and category of the detected target in the target image; The preset target detection model is trained based on a preset training framework and at least one set of training data. Each set of training data includes a first type of training data and a second type of training data. The first type of training data includes a source domain sample image and a corresponding sample ground truth label. The second type of training data includes a target domain sample image.

[0006] The present invention also provides an apparatus for target detection in optical remote sensing images, comprising: The image acquisition module is configured to acquire a target image; wherein the target image is a target domain optical remote sensing image that needs to be detected. The target detection execution module is configured to: input the target image into a preset target detection model, and output a target detection result for the target image through the preset target detection model; wherein, the target detection result includes information characterizing the position and category of the detected target in the target image; The preset target detection model is trained based on a preset training framework and at least one set of training data. Each set of training data includes a first type of training data and a second type of training data. The first type of training data includes a source domain sample image and a corresponding sample ground truth label. The second type of training data includes a target domain sample image.

[0007] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for target detection of optical remote sensing images as described above.

[0008] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for target detection of optical remote sensing images as described above.

[0009] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method for target detection of optical remote sensing images as described above.

[0010] As described above, the method for target detection in optical remote sensing images provided by this invention can achieve accurate target detection in target images by utilizing a preset target detection model. The preset training framework effectively enhances the cross-domain target detection generalization ability of the preset target detection model, while also making it robust and scalable. Specifically, on the one hand, a region-selective fusion strategy can be deployed within the preset training framework to selectively fuse source domain sample images and target domain sample images; this not only fully utilizes the advantages of source domain labels but also constructs a fused image that maximizes foreground object information based on the confidence level of pseudo-labels. On the other hand, based on the preset training framework, a discriminative network (including "multi-scale" unbiased adversarial units / modules) is also constructed during training, enabling the target detection model to focus on domain-independent features, thereby enhancing the learning of generalization ability. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0012] Figure 1 This is one of the flowcharts of a method for target detection in optical remote sensing images provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the preset training framework provided in the embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the biasless countermeasure unit provided in the embodiment of the present invention; Figure 4 This is a second schematic flowchart of a method for target detection in optical remote sensing images provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of the device for target detection in optical remote sensing images provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0014] Overview of the inventive concept: The inventors of this invention discovered that the problem with the prior art stems from the fact that deep learning-based models (e.g., high-performance depth detectors) are typically trained on labeled datasets. For optical remote sensing images containing dozens or even hundreds of objects per image, precise annotation is extremely difficult and requires significant time and effort, resulting in limited sample annotation data for optical remote sensing images. Consequently, models trained under these conditions exhibit poor generalization ability. Furthermore, differences in sensors, geographical regions, atmospheric conditions, ground sampling distances, and sensor shooting angles may exist during the acquisition of sample and target optical remote sensing images, further increasing the model's domain limitation and exacerbating the problem of decreased prediction accuracy for different scenarios.

[0015] To address the aforementioned technical problems, the inventors of this invention have discovered an unsupervised domain adaptation (UDA) method that minimizes the distance between a source domain rich in labeled data and a target domain lacking labels, without relying on ground truth labels in the target domain. This facilitates the transfer of learned knowledge from the source domain to the target domain and adjusts the distribution of the two domains in an unsupervised manner, enabling the model to effectively generalize to the target domain data without the need for additional annotations.

[0016] Specifically, the mean-based teacher model (hereinafter referred to as the "teacher model") effectively handles the problem of inconsistent data distribution. Firstly, the teacher model is identical to the student model. During the refinement process, subtle perturbations are applied to the unlabeled data. By prioritizing samples with high prediction probabilities, the student model reduces the variance of unlabeled target samples, thereby enhancing the overall robustness of the model. This characteristic makes it particularly effective in object detection tasks, as simultaneously predicting detailed bounding boxes and object categories makes the model susceptible to data fluctuations. Furthermore, the student model is updated via backpropagation, while the teacher model is updated using the exponential moving average (EMA) of the student weights. Therefore, the teacher network model can be viewed as the historical average of the student network, thus improving training stability.

[0017] Furthermore, a confidence-based region-selective mixing strategy can maximize the accuracy of foreground information content and allow for multiple utilization of source domain data labels. Incorporating a mixing phase into the domain adaptation process as a strategic means of constructing an auxiliary domain helps to blend data from two different domains into a unified domain and assists in the transfer of model knowledge from the source domain to the target domain. Because the source data labels are available, both the teacher and student models may exhibit a preference for the source domain. Specifically, refer to... Figure 2 The pseudo-labels generated by the teacher model for the target domain (optical remote sensing) image are fundamentally derived from the knowledge gained by the student model trained using the ground truth labels of samples from the source domain. Therefore, it is essential to mitigate the domain bias between the source and target domains; adversarial learning strategies can effectively address this issue by adjusting the distribution of the two domains. However, current adversarial learning methods involve adding domain classifiers and convolutional neural networks. While these methods aim to enable the model to learn more domain-invariant features and focus on the elements possessed by the object, single-scale domain features cannot achieve deeper extraction of domain information. More accurate and effective domain feature extraction is crucial to the effectiveness of adversarial learning.

[0018] In view of this, the present invention provides a scheme for target detection in optical remote sensing images as described in the following embodiments. Specifically, it introduces an effective region-selective fusion and multi-scale unbiased adversarial learning strategy under the mean teacher framework to better complete the target detection task in optical remote sensing images.

[0019] Example: The scheme for target detection in optical remote sensing images according to the present invention will now be described with reference to the accompanying drawings.

[0020] Figure 1 This is a schematic flowchart of a method for target detection in optical remote sensing images provided by an exemplary embodiment of the present invention. This embodiment can be applied to electronic devices (e.g., servers or cloud computing platforms), such as... Figure 1 As shown, the method for target detection in optical remote sensing images includes the following steps: S110, Obtain the target image.

[0021] The target image is a target domain optical remote sensing image that needs to be detected.

[0022] The present invention does not limit the method of acquisition. For example, an electronic device (e.g., a server or cloud computing platform) acting as the execution subject can communicate with the server storing the "target image" to obtain the "target image".

[0023] S120. Input the target image into a preset target detection model, and output the target detection result of the target image through the preset target detection model.

[0024] The target detection result includes information characterizing the location and category of the detected target in the target image.

[0025] The preset target detection model is trained based on a preset training framework and at least one set of training data. Each set of training data includes a first type of training data and a second type of training data. The first type of training data includes a source domain sample image and a corresponding sample ground truth label. The second type of training data includes a target domain sample image.

[0026] The source domain refers to the data environment or domain in which the model to be trained initially learns or trains. In this domain, the model accumulates a large amount of supervised or unsupervised training data and learns feature representations, patterns, or rules through this data. Data in the source domain is typically used to train the model to perform a specific task or predict a target. The target domain is the environment or domain in which the trained model is ultimately applied or its performance evaluated. Although the target domain may appear similar to the source domain, there are often some differences between them, which may include data distribution, feature space, task type, etc.

[0027] As described above, the method for target detection in optical remote sensing images provided by this invention can achieve accurate target detection in target images by utilizing a preset target detection model. The preset training framework effectively enhances the cross-domain target detection generalization ability of the preset target detection model, while also making it robust and scalable. Specifically, on the one hand, a region-selective fusion strategy can be deployed within the preset training framework to selectively fuse source domain sample images and target domain sample images; this not only fully utilizes the advantages of source domain labels but also constructs a fused image that maximizes foreground object information based on the confidence level of pseudo-labels. On the other hand, based on the preset training framework, a discriminative network (including "multi-scale" unbiased adversarial units / modules) is also constructed during training, enabling the target detection model to focus on domain-independent features, thereby enhancing the learning of generalization ability.

[0028] exist Figure 1 Based on the embodiments, as an optional implementation method, refer to Figure 2 The preset training framework includes a cascaded base detection network (i.e., Figure 2 The network structure corresponding to the "basic detection stage" shown), and the fusion network (i.e., Figure 2 The network structure corresponding to the "fusion stage" shown) and the discriminant network (i.e., Figure 2 The network structure corresponding to the "discrimination stage" is shown below.

[0029] As an optional example, the base detection network includes identical and parallel student models to be trained (i.e., Figure 2 The "student model" in the network structure corresponding to the "basic detection phase" shown above, and the teacher model to be trained (i.e., Figure 2 The "classroom model" in the network structure corresponding to the "basic detection stage" shown above, wherein the input end of the student model to be trained is used to receive the source domain sample images. Xs The input terminal of the teacher model to be trained is used to receive the target domain sample image. Xt .

[0030] As an optional example, the fusion network includes a confidence mask unit, a feature fusion unit, and a post-processing enhancement unit; wherein, the confidence mask unit is used to generate a confidence mask based on the confidence information of the detection region output by the teacher model to be trained; the feature fusion unit is used to generate a preliminary fusion map based on the confidence mask, the source domain prediction result output by the student model to be trained, and the target domain prediction result output by the teacher model to be trained. The post-processing enhancement unit is used to enhance the preliminary fused image to obtain a fused image. .

[0031] As described above, the feature fusion unit can execute a confidence-based region-selective fusion strategy to maximize the accuracy of foreground information content and achieve target object-level alignment between source domain sample images and target domain sample images. Specifically, it allows multiple use of the ground truth labels corresponding to the source domain sample images, employs a confidence-based region-selective fusion strategy for mixing, evaluates the status of foreground and background features of the regions, and adaptively mixes source domain sample images and target domain sample images.

[0032] The feature fusion unit generates a preliminary fusion map. It can be expressed using the following formula:

[0033] in, This represents the initial fusion diagram; Indicates the confidence level mask; Represents the target domain sample image; This represents the source domain sample image.

[0034] As an optional example, the discriminative network includes cascaded student models to be trained (i.e., Figure 2 The "discrimination stage" corresponds to the "student model" in the network structure shown, and the gradient reversal layer (i.e., Figure 2 The GRL shown) and the biasless adversarial unit (i.e., Figure 2 The "unbiased adversarial" layer is shown. The gradient inversion layer is used to modify the student model being trained (i.e., Figure 2 The sign of the output of the "student model" in the network structure corresponding to the "discrimination stage" is determined to facilitate the construction of the subsequent loss function.

[0035] As an optional example, the output of the unbiased adversarial unit serves as the first output of the preset training framework; the output of the student model to be trained in the basic detection network serves as the second output of the preset training framework.

[0036] exist Figure 1 , 2 Based on the embodiments, as an optional implementation, the student model to be trained is a YOLOv5-based neural network model. The YOLOv5-based neural network model includes a cascaded feature extraction backbone unit, a neck unit, and a head unit; wherein, the neck unit includes a feature pyramid sub-unit and a path aggregation sub-unit.

[0037] exist Figure 1 , 2 Based on the embodiments, as an optional implementation method, refer to Figure 3The unbiased adversarial unit includes a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, a fifth convolutional block, a sixth convolutional block, a seventh convolutional block, and a domain classifier.

[0038] Reference Figure 3 As an optional example, the first convolutional block (connected to feature channel F-1), the second convolutional block (connected to feature channel F-2), the third convolutional block (connected to feature channel F-3), and the fourth convolutional block (connected to feature channel F-4) are connected in parallel. The output of the first convolutional block and the output of the second convolutional block are concatenated via channels (i.e., Figure 3 The symbol shown is " After that, it is cascaded with the fifth convolutional block; the output of the fifth convolutional block and the output of the third convolutional block are concatenated through a channel and then cascaded with the sixth convolutional block; the output of the sixth convolutional block and the output of the fourth convolutional block are concatenated through a channel and then cascaded with the seventh convolutional block; the seventh convolutional block is cascaded with the domain classifier.

[0039] Reference Figure 3 As an optional example, the first convolutional block includes a cascaded 64-channel convolutional layer, a 128-channel downsampling layer, and a 32-channel convolutional layer; the second convolutional block includes a cascaded 128-channel convolutional layer and a 32-channel convolutional layer; the third convolutional block includes a cascaded 256-channel convolutional layer and a 64-channel convolutional layer; the fourth convolutional block includes a cascaded 512-channel convolutional layer, a 128-channel convolutional layer, and a 128-channel convolutional layer; the fifth convolutional block includes a cascaded 32-channel convolutional layer and a 128-channel downsampling layer; the sixth convolutional block includes a cascaded 64-channel convolutional layer and a 96-channel downsampling layer; and the seventh convolutional block includes a cascaded 32-channel convolutional layer and a 1-channel convolutional layer.

[0040] exist Figures 1 to 3 Based on the embodiments, as an optional implementation, the structure of the preset target detection model is the same as the structure of the student model to be trained.

[0041] Reference Figure 2 The student model used in the testing phase is the preset target detection model. That is, the student model to be trained is trained using the preset training framework and training data. After training, the network layer parameters of the student model to be trained are retained as parameters of the preset target detection model, thereby obtaining the preset target detection model.

[0042] exist Figures 1 to 3 Based on the embodiments, as an optional implementation method, refer to Figure 4 The training steps for training the preset target detection model based on the preset training framework include: S410. Obtain the at least one set of training data.

[0043] Each set of training data in the at least one set of training data includes a first type of training data and a second type of training data. The first type of training data includes a source domain sample image and the corresponding sample ground truth label, and the second type of training data includes a target domain sample image.

[0044] S420. Input the source domain sample image and the target domain sample image in each group of training data into the preset training framework. The student model to be trained in the basic detection network outputs the corresponding source domain prediction result, the teacher model to be trained in the basic detection network outputs the corresponding target domain prediction result, and the discriminant network outputs the corresponding fusion prediction result.

[0045] The fusion prediction result includes a first prediction result of the fused image in the source domain and a second prediction result in the target domain.

[0046] It should be noted that the above prediction results may include the target objects identified in the sample images and the category probability of the target objects, which can be formed into a visual label as a whole.

[0047] S430. Using a preset comprehensive loss function, based on the sample ground truth labels in each group of training data and the corresponding source domain prediction results, target domain prediction results, and fusion prediction results, obtain the function value of the comprehensive loss function.

[0048] As an alternative example, S430 can be implemented as follows: 1) Based on the ground truth labels of the samples in each set of training data and the corresponding source domain prediction results, target domain prediction results, fused images and fused prediction results, calculate the function values ​​of the source domain data supervision loss function, the region selection consistency loss function and the unbiased adversarial discrimination loss function respectively.

[0049] Optionally, based on the ground truth labels of samples in each set of training data and the corresponding source domain prediction results, the function value of the source domain data supervision loss function is calculated, including: The function value of the source domain data supervision loss function is calculated using the following formula (1); (1) in, This represents the source domain prediction result in each set of training data; Indicates the true label of each sample in each training data set; This represents the loss function used to perform supervised loss calculation using the source domain prediction results and the sample ground truth labels.

[0050] Optionally, based on the ground truth labels of the samples, the target domain prediction results, the fused image, and the fused prediction results in each set of training data, the function value of the region selection consistency loss function is calculated, including: The function value of the region selection consistency loss function is calculated using the following formulas (2) to (3); (2) (3) in, This represents the bounding box loss function for the target domain portion of the fused image; This represents the bounding box loss function for the source domain portion of the fused image; This indicates the prediction result for the target domain; This represents the true value label of each sample in the training data set, corresponding to the label of the source domain portion in the fusion domain; This represents the prediction result of the fused image in the target domain; This represents the prediction result of the fused image in the source domain; This represents the student model to be trained; This refers to the fused image.

[0051] Optionally, based on the ground truth labels of the samples in each set of training data and the corresponding fused image, the function value of the unbiased adversarial discriminative loss function is calculated, including: The function value of the unbiased adversarial discriminant loss function is calculated using the following formulas (4) to (5); (4) (5) in, This indicates the number of rows and columns of the image blocks divided in the fused image; Indicates the first i In the fused image located at The domain label of the image patch at the location, where, when When the image patch at the location corresponds to the source domain The value is 1, when When the image patch at the location corresponds to the target domain The value is 0; This represents the coordinates of the position of an image block, determined based on the number of rows and columns of the image blocks divided in the fused image; No. i In the fused image located at The predicted class probability of the image patch at the location; This represents the feature extraction backbone unit in the discriminant network; express Feature extractor in; Indicates the first i A fused image.

[0052] 2) Using preset weights, the function values ​​of the source domain data supervision loss function, the region selection consistency loss function, and the unbiased adversarial discrimination loss function are weighted and fused to obtain the function value of the comprehensive loss function.

[0053] Optionally, the value of the comprehensive loss function can be calculated using the following formula (6); (6) in, The weighting coefficients of the region selection consistency loss function are represented. The weight coefficients represent the unbiased adversarial discriminant loss function.

[0054] S440. Based on the function value of the comprehensive loss function, train the preset training framework until the preset training completion condition is met, and obtain the preset target detection model from the student model to be trained.

[0055] As an optional example, S440 can be implemented as follows: I1) The function value of the comprehensive loss function is back-transmitted to the student model to be trained; I2) The student model to be trained adjusts the first network parameters of each network layer according to the function value of the comprehensive loss function; I3) The teacher model to be trained adjusts the second network parameters of each network layer according to the first network parameters based on the exponential moving average strategy; I4) The steps of obtaining the function value of the comprehensive loss function to adjusting the second network parameters of each network layer are executed iteratively until the preset training completion conditions are met.

[0056] In step I2), the student model to be trained is updated through backpropagation. In step I3), the teacher model to be trained is updated by weighting the student models using an exponential moving average (EMA). Therefore, the teacher model to be trained can be considered as the historical average of the student models to be trained; that is, it can be expressed by the following formula:

[0057] in, This represents the first network parameter; This represents the second network parameter; This represents the exponential decay coefficient. Preferably, The value is set to 0.99 or 0.999 to ensure that the weighted model has some memory of earlier models.

[0058] As described above, the method for target detection in optical remote sensing images provided by this invention can achieve accurate target detection in target images by utilizing a preset target detection model. The preset training framework effectively enhances the cross-domain target detection generalization ability of the preset target detection model, while also making it robust and scalable. Specifically, on the one hand, a region-selective fusion strategy can be deployed within the preset training framework to selectively fuse source domain sample images and target domain sample images; this not only fully utilizes the advantages of source domain labels but also constructs a fused image that maximizes foreground object information based on the confidence level of pseudo-labels. On the other hand, based on the preset training framework, a discriminative network (including "multi-scale" unbiased adversarial units / modules) is also constructed during training, enabling the target detection model to focus on domain-independent features, thereby enhancing the learning of generalization ability.

[0059] The apparatus for target detection in optical remote sensing images provided by the present invention will be described below. The apparatus for target detection in optical remote sensing images described below and the method for target detection in optical remote sensing images described above can be referred to in correspondence with each other.

[0060] Figure 5 This is a schematic diagram of the structure of an apparatus for target detection in optical remote sensing images provided in an exemplary embodiment of the present invention. Figure 5 As shown, an apparatus for target detection in optical remote sensing images includes: an image acquisition module 510 configured to acquire a target image; wherein the target image is a target domain optical remote sensing image for which target detection is required; and a target detection execution module 520 configured to input the target image into a preset target detection model and output a target detection result for the target image via the preset target detection model; wherein the target detection result includes information characterizing the position and category of the detected target in the target image; wherein the preset target detection model is trained based on a preset training framework and at least one set of training data, each set of training data in the at least one set of training data includes a first type of training data and a second type of training data, the first type of training data including a source domain sample image and a corresponding sample ground truth label, and the second type of training data including a target domain sample image.

[0061] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a method for target detection on an optical remote sensing image. This method includes: acquiring a target image; wherein the target image is a target domain optical remote sensing image for which target detection is required; inputting the target image into a preset target detection model, and outputting a target detection result for the target image through the preset target detection model; wherein the target detection result includes information characterizing the position and category of the detected target in the target image; wherein the preset target detection model is trained based on a preset training framework and at least one set of training data, each set of training data including a first type of training data and a second type of training data, the first type of training data including a source domain sample image and a corresponding sample ground truth label, and the second type of training data including a target domain sample image.

[0062] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0063] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the methods provided above for target detection of optical remote sensing images. The method includes: acquiring a target image; wherein the target image is a target domain optical remote sensing image for which target detection is required; inputting the target image into a preset target detection model, and outputting a target detection result for the target image through the preset target detection model; wherein the target detection result includes information characterizing the position and category of the detected target in the target image; wherein the preset target detection model is trained based on a preset training framework and at least one set of training data, each set of training data in the at least one set of training data includes a first type of training data and a second type of training data, the first type of training data including a source domain sample image and a corresponding sample ground truth label, and the second type of training data including a target domain sample image.

[0064] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for target detection of optical remote sensing images provided by the methods described above. The method includes: acquiring a target image; wherein the target image is a target domain optical remote sensing image for which target detection is required; inputting the target image into a preset target detection model, and outputting a target detection result for the target image through the preset target detection model; wherein the target detection result includes information characterizing the position and category of the detected target in the target image; wherein the preset target detection model is trained based on a preset training framework and at least one set of training data, each set of training data including a first type of training data and a second type of training data, the first type of training data including a source domain sample image and a corresponding sample ground truth label, and the second type of training data including a target domain sample image.

[0065] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0066] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for target detection in optical remote sensing images, characterized in that, include: Acquire the target image; wherein the target image is an optical remote sensing image of the target domain for which target detection is required; The target image is input into a preset target detection model, and the preset target detection model outputs a target detection result for the target image; wherein, the target detection result includes information characterizing the position and category of the detected target in the target image; The preset target detection model is trained based on a preset training framework and at least one set of training data. Each set of training data includes a first type of training data and a second type of training data. The first type of training data includes a source domain sample image and a corresponding sample ground truth label. The second type of training data includes a target domain sample image. The preset training framework includes a cascaded basic detection network, a fusion network, and a discriminant network. The basic detection network includes a student model to be trained and a teacher model to be trained, which are identical in structure and connected in parallel. The input terminal of the student model to be trained is used to receive the source domain sample image, and the input terminal of the teacher model to be trained is used to receive the target domain sample image. The fusion network includes a confidence mask unit, a feature fusion unit, and a post-processing enhancement unit. The confidence mask unit generates a confidence mask based on the confidence information of the detection region output by the teacher model to be trained. The feature fusion unit generates a preliminary fusion map based on the confidence mask, the source domain prediction result output by the student model to be trained, and the target domain prediction result output by the teacher model to be trained. The post-processing enhancement unit enhances the preliminary fusion map to obtain a fused image. The discriminant network includes a cascaded student model to be trained, a gradient reversal layer, and an unbiased adversarial unit, with the output of the unbiased adversarial unit serving as the first output of the preset training framework. The unbiased adversarial unit includes a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, a fifth convolutional block, a sixth convolutional block, a seventh convolutional block, and a domain classifier; The first convolutional block, the second convolutional block, the third convolutional block, and the fourth convolutional block are connected in parallel; The output of the first convolutional block and the output of the second convolutional block are spliced ​​together via a channel and then cascaded with the fifth convolutional block; The output of the fifth convolutional block is connected to the output of the third convolutional block via a channel and then cascaded with the sixth convolutional block. The output of the sixth convolutional block is connected to the output of the fourth convolutional block via a channel and then cascaded with the seventh convolutional block. The seventh convolutional block is cascaded with the domain classifier; The first convolutional block includes a cascaded 64-channel convolutional layer, a 128-channel downsampling layer, and a 32-channel convolutional layer; The second convolutional block includes cascaded 128-channel and 32-channel convolutional layers; The third convolutional block includes a cascaded 256-channel convolutional layer and a 64-channel convolutional layer; The fourth convolutional block includes a cascaded 512-channel convolutional layer, a 128-channel convolutional layer, and a 128-channel convolutional layer. The fifth convolutional block includes a cascaded 32-channel convolutional layer and a 128-channel downsampling layer; The sixth convolutional block includes a cascaded 64-channel convolutional layer and a 96-channel downsampling layer; The seventh convolutional block includes a cascaded 32-channel convolutional layer and a 1-channel convolutional layer; The output of the student model to be trained in the basic detection network serves as the second output of the preset training framework. The training steps for training the preset target detection model based on the preset training framework include: Obtain the at least one set of training data; The source domain sample image and the target domain sample image in each set of training data are input into the preset training framework. The student model to be trained in the basic detection network outputs the corresponding source domain prediction result, the teacher model to be trained in the basic detection network outputs the corresponding target domain prediction result, and the discriminant network outputs the corresponding fusion prediction result. The fusion prediction result includes the first prediction result of the fused image in the source domain and the second prediction result in the target domain. Using a preset comprehensive loss function, the function value of the comprehensive loss function is obtained based on the sample ground truth labels in each group of training data and the corresponding source domain prediction results, target domain prediction results and fusion prediction results; Based on the function value of the comprehensive loss function, the preset training framework is trained until the preset training completion condition is met, and the preset target detection model is obtained from the student model to be trained.

2. The method according to claim 1, characterized in that, The student model to be trained is a neural network model based on YOLOv5; The YOLOv5-based neural network model includes a cascaded feature extraction backbone unit, a neck unit, and a head unit. The neck unit includes a feature pyramid subunit and a path aggregation subunit.

3. The method according to claim 1, characterized in that, The structure of the preset target detection model is the same as the structure of the student model to be trained.

4. The method according to claim 1, characterized in that, The step of using a preset comprehensive loss function to obtain the function value of the comprehensive loss function based on the ground truth labels of samples in each group of training data and the corresponding source domain prediction results, target domain prediction results, and fusion prediction results includes: Based on the ground truth labels of the samples in each set of training data and the corresponding source domain prediction results, target domain prediction results, fused images, and fused prediction results, the function values ​​of the source domain data supervision loss function, the region selection consistency loss function, and the unbiased adversarial discrimination loss function are calculated respectively. Using preset weights, the function values ​​of the source domain data supervision loss function, the region selection consistency loss function, and the unbiased adversarial discrimination loss function are weighted and fused to obtain the function value of the comprehensive loss function.

5. The method according to claim 4, characterized in that, Based on the ground truth labels of the samples in each set of training data and the corresponding source domain prediction results, the function value of the source domain data supervision loss function is calculated, including: The function value of the source domain data supervision loss function is calculated using the following formula (1); （1） in, This represents the source domain prediction result in each set of training data; Indicates the true value label of each sample in each training data set; This represents the loss function used to perform supervised loss calculations using the source domain prediction results and the sample ground truth labels; Based on the ground truth labels of samples, target domain prediction results, fused images, and fused prediction results in each set of training data, the function value of the region selection consistency loss function is calculated, including: The function value of the region selection consistency loss function is calculated using the following formulas (2) to (3); （2） （3） in, This represents the bounding box loss function calculated for the target domain portion of the fused image; This represents the bounding box loss function calculated for the source region portion of the fused image; This represents the prediction result for the target domain; This represents the true value label of each sample in the training data set, corresponding to the label of the source domain portion in the fusion domain; This represents the prediction result of the fused image in the target domain; This represents the prediction result of the fused image in the source domain; This represents the student model to be trained; This represents the fused image; Based on the ground truth labels of the samples in each set of training data and the corresponding fused image, the function value of the unbiased adversarial discriminative loss function is calculated, including: The function value of the unbiased adversarial discriminant loss function is calculated using the following formulas (4) to (5); （4） （5） in, This indicates the number of rows and columns of the image blocks divided in the fused image; Indicates the first i In the fused image located at The domain label of the image patch at the location, where, when When the image patch at the location corresponds to the source domain The value is 1, when When the image patch at the location corresponds to the target domain The value is 0; This represents the coordinates of the position of an image block, determined based on the number of rows and columns of the image blocks divided in the fused image; No. i In the fused image located at The predicted class probability of the image patch at the location; This represents the feature extraction backbone unit in the discriminant network; express Feature extractor in; Indicates the first i A fused image; The step of using preset weights to weight and fuse the function values ​​of the source domain data supervision loss function, the region selection consistency loss function, and the unbiased adversarial discrimination loss function to obtain the function value of the comprehensive loss function includes: The value of the comprehensive loss function is calculated using the following formula (6); （6） in, The weighting coefficients of the region selection consistency loss function are represented. The weight coefficients represent the unbiased adversarial discriminant loss function.

6. The method according to claim 1, characterized in that, Based on the function value of the comprehensive loss function, the preset training framework is trained until the preset training completion conditions are met, including: The value of the comprehensive loss function is then passed back to the student model to be trained; The student model to be trained adjusts the first network parameters of each network layer according to the function value of the comprehensive loss function; The teacher model to be trained uses the first network parameters to adjust the second network parameters of each network layer based on an exponential moving average strategy. The process iteratively executes the steps from obtaining the function value of the comprehensive loss function to adjusting the second network parameters of each network layer, until the preset training completion conditions are met.

7. An apparatus for target detection in optical remote sensing images, characterized in that, include: The image acquisition module is configured to acquire a target image; wherein the target image is a target domain optical remote sensing image that needs to be detected. The target detection execution module is configured to: input the target image into a preset target detection model, and output a target detection result for the target image through the preset target detection model; wherein, the target detection result includes information characterizing the position and category of the detected target in the target image; The preset target detection model is trained based on a preset training framework and at least one set of training data. Each set of training data includes a first type of training data and a second type of training data. The first type of training data includes a source domain sample image and a corresponding sample ground truth label. The second type of training data includes a target domain sample image. The preset training framework includes a cascaded basic detection network, a fusion network, and a discriminant network. The basic detection network includes a student model to be trained and a teacher model to be trained, which are identical in structure and connected in parallel. The input terminal of the student model to be trained is used to receive the source domain sample image, and the input terminal of the teacher model to be trained is used to receive the target domain sample image. The fusion network includes a confidence mask unit, a feature fusion unit, and a post-processing enhancement unit. The confidence mask unit generates a confidence mask based on the confidence information of the detection region output by the teacher model to be trained. The feature fusion unit generates a preliminary fusion map based on the confidence mask, the source domain prediction result output by the student model to be trained, and the target domain prediction result output by the teacher model to be trained. The post-processing enhancement unit enhances the preliminary fusion map to obtain a fused image. The discriminant network includes a cascaded student model to be trained, a gradient reversal layer, and an unbiased adversarial unit, with the output of the unbiased adversarial unit serving as the first output of the preset training framework. The unbiased adversarial unit includes a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, a fifth convolutional block, a sixth convolutional block, a seventh convolutional block, and a domain classifier; The first convolutional block, the second convolutional block, the third convolutional block, and the fourth convolutional block are connected in parallel; The output of the first convolutional block and the output of the second convolutional block are spliced ​​together via a channel and then cascaded with the fifth convolutional block; The output of the fifth convolutional block is connected to the output of the third convolutional block via a channel and then cascaded with the sixth convolutional block. The output of the sixth convolutional block is connected to the output of the fourth convolutional block via a channel and then cascaded with the seventh convolutional block. The seventh convolutional block is cascaded with the domain classifier; The first convolutional block includes a cascaded 64-channel convolutional layer, a 128-channel downsampling layer, and a 32-channel convolutional layer; The second convolutional block includes cascaded 128-channel and 32-channel convolutional layers; The third convolutional block includes a cascaded 256-channel convolutional layer and a 64-channel convolutional layer; The fourth convolutional block includes a cascaded 512-channel convolutional layer, a 128-channel convolutional layer, and a 128-channel convolutional layer. The fifth convolutional block includes a cascaded 32-channel convolutional layer and a 128-channel downsampling layer; The sixth convolutional block includes a cascaded 64-channel convolutional layer and a 96-channel downsampling layer; The seventh convolutional block includes a cascaded 32-channel convolutional layer and a 1-channel convolutional layer; The output of the student model to be trained in the basic detection network serves as the second output of the preset training framework. The training steps for training the preset target detection model based on the preset training framework include: Obtain the at least one set of training data; The source domain sample image and the target domain sample image in each set of training data are input into the preset training framework. The student model to be trained in the basic detection network outputs the corresponding source domain prediction result, the teacher model to be trained in the basic detection network outputs the corresponding target domain prediction result, and the discriminant network outputs the corresponding fusion prediction result. The fusion prediction result includes the first prediction result of the fused image in the source domain and the second prediction result in the target domain. Using a preset comprehensive loss function, the function value of the comprehensive loss function is obtained based on the sample ground truth labels in each group of training data and the corresponding source domain prediction results, target domain prediction results and fusion prediction results; Based on the function value of the comprehensive loss function, the preset training framework is trained until the preset training completion condition is met, and the preset target detection model is obtained from the student model to be trained.

Images