Remote sensing image target detection method and device, electronic equipment and storage medium
By using a generative adversarial network to generate multi-type meteorological image samples, the problem of low detection accuracy of existing object detection models under complex meteorological conditions is solved, and a more efficient remote sensing image object detection effect is achieved.
Patent Information
- Application Number
- CN202510121311.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-26
- Publication Date
- 2025-05-13
AI Technical Summary
The existing object detection model has low detection accuracy and poor detection effect under complex meteorological conditions, mainly due to the lack of image samples for training in the corresponding environment.
Multi-type weather image samples are generated using a Cycle-Style GAN based on cyclic style migration. By converting the original sample image to different target weather domains, sample data carrying detection target labels are generated, which is used to train the target detection model.
By generating high-quality and diverse meteorological image samples, the trained object detection model can detect targets in remote sensing images more accurately and efficiently under complex meteorological conditions, improving recognition accuracy and robustness.
Smart Images
Figure CN119992365A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the technical field of remote sensing image target detection, and more specifically, relates to a remote sensing image target detection method, device, electronic equipment and storage medium. Background Art
[0002] Complex scene optical remote sensing image target detection is to use visible light remote sensing images to automatically extract the location and category information of one or more typical detection targets (such as ships, aircraft, tanks, vehicles, etc.) from complex scenes such as ports and airports. It has a wide range of applications in both civil and military fields. In the civilian field, remote sensing target detection is widely used in environmental pollution monitoring, disaster monitoring and other fields. In the military field, remote sensing image target automatic detection technology has become an important means of reconnaissance and early warning. The use of optical remote sensing images to accurately detect and monitor detection targets in military facilities (such as airports, ports, etc.) can play a key role in military applications such as national defense security.
[0003] However, due to complex and severe weather conditions, the target detection and recognition process of remote sensing images is easily affected by the occlusion of clouds, fog, rain and snow under complex meteorological conditions, which has become a key problem restricting the improvement of target detection effect. For remote sensing target detection under the interference of complex meteorological conditions, the training process of existing target detection models is usually based on clear image samples, which are not interfered by clouds, fog, rain, snow or other environmental stray light, and lack of image samples of corresponding environments, resulting in low detection accuracy and poor detection effect of target detection models.
[0004] Therefore, how to better realize the detection of remote sensing image targets under complex meteorological conditions has become a technical problem that needs to be solved urgently in the industry. Summary of the invention
[0005] In view of the defects of the prior art, the purpose of this application is to better realize the detection of remote sensing image targets under complex meteorological conditions, aiming to solve the problems of low detection accuracy and poor detection effect of existing target detection models under complex meteorological conditions.
[0006] To achieve the above objectives, in a first aspect, the present application provides a remote sensing image target detection method, comprising: Acquire the remote sensing image to be measured; Inputting the remote sensing image to be measured into a target detection model to obtain a target detection result of the remote sensing image to be measured output by the target detection model; The target detection model is trained based on multi-type meteorological image samples carrying detection target labels; the multi-type meteorological image samples are sample data obtained by converting the original sample images from the original meteorological domain to different target meteorological domains using a generative adversarial network based on cyclic style transfer.
[0007] Optionally, the generative adversarial network based on cyclic style transfer includes a first generator network, a second generator network, a first discriminator network and a second discriminator network, and is obtained by performing adversarial training using the first discriminator network, the second discriminator network, the first generator network and the second generator network; the multi-type meteorological image samples are generated using the trained first generator network; The first generator network is used to convert an input original sample image in the original meteorological domain into a generated image in the target meteorological domain; the original sample image contains the detection target; The second generator network is used to convert the generated image in the target meteorological domain into a restored image in the original meteorological domain; The first discriminator network is used to distinguish the original sample image from the restored image; The second discriminator network is used to distinguish the real image of the target meteorological domain from the generated image.
[0008] Optionally, the first generator network and the second generator network both include an encoding network and a decoding network; the encoding network includes multiple convolutional layers and multiple residual modules; The multi-layer convolutional layer is used to perform feature encoding on the first input image; The multiple residual modules are used to extract meteorological style features from the feature graph output by the multi-layer convolutional layer; The decoding network includes a deconvolution layer and an output convolution layer; The deconvolution layer is used to restore the size of the feature map output by the encoding network to the same size as the first input image; The output convolution layer is used to extract and fuse features of the feature map output by the deconvolution layer.
[0009] Optionally, the first discriminator network and the second discriminator network both include multiple convolutional layers; Except for the last convolution layer in the multi-layer convolution layer, all other convolution layers are connected to the target activation function to extract image features of the second input image; The last convolutional layer is used to determine whether the second input image is real image data based on the image features.
[0010] Optionally, before inputting the remote sensing image to be measured into a target detection model to obtain a target detection result of the remote sensing image to be measured output by the target detection model, the method further includes: Taking each meteorological image sample in the multiple types of meteorological image samples and its corresponding detection target label as a group of training samples, and obtaining multiple groups of the training samples; The target detection model is trained using multiple groups of the training samples.
[0011] Optionally, the training of the target detection model using the multiple groups of training samples includes: For any training sample in the plurality of groups of training samples, input the any training sample into the target detection model, and output prediction information corresponding to the any training sample; Calculate the loss value based on the prediction information corresponding to any one of the training samples and the detection target label corresponding to the training sample by using a preset loss function; Based on the loss value, adjusting the model parameters of the target detection model until the loss value is less than a preset threshold or the number of training times reaches a preset number of times; The model parameters obtained when the loss value is less than the preset threshold or the number of training times reaches the preset number of times are used as the model parameters of the trained target detection model.
[0012] In a second aspect, the present application provides a remote sensing image target detection device, comprising: An acquisition module, used for acquiring the remote sensing image to be measured; A detection module, used for inputting the remote sensing image to be measured into a target detection model, and obtaining a target detection result of the remote sensing image to be measured output by the target detection model; The target detection model is obtained by training based on multiple types of meteorological image samples carrying detection target labels; the multiple types of meteorological image samples are generated using a generative adversarial network based on cyclic style transfer.
[0013] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.
[0014] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.
[0015] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.
[0016] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.
[0017] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art: The present application provides a remote sensing image target detection method, device, electronic device and storage medium, which use a generative adversarial network of cyclic style transfer to finely control the style of meteorological images, accurately capture image style features, and fuse style information with image spatial features, thereby generating a large number of high-quality and diverse meteorological image samples, and use the generated multi-type meteorological image data sets to train and optimize the target detection model, so that the trained target detection model can accurately and efficiently output the target detection results of the remote sensing image to be tested, which can effectively improve the recognition accuracy and robustness of the target detection model under complex meteorological conditions, and enhance the effect of remote sensing image target detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 It is a flowchart of a remote sensing image target detection method provided in an embodiment of the present application; Figure 2 This is one of the structural diagrams of a generative adversarial network based on cyclic style transfer provided in an embodiment of the present application; Figure 3 This is the second structural diagram of the generative adversarial network based on cyclic style transfer provided in the embodiment of the present application; Figure 4 This is the third structural diagram of the generative adversarial network based on cyclic style transfer provided in the embodiment of the present application; Figure 5 It is a schematic diagram of the structure of a generator network in a generative adversarial network based on cyclic style transfer provided in an embodiment of the present application; Figure 6 It is a schematic diagram of the structure of the discriminator network in the generative adversarial network based on cyclic style transfer provided in an embodiment of the present application; Figure 7 is a schematic diagram of the structure of a remote sensing image target detection device provided in an embodiment of the present application; Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0019] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0020] The terms "first" and "second" in the specification and claims of this application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first generator network and a second generator network are used to distinguish generator networks with different functions rather than to describe a specific order of generator networks.
[0021] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0022] In the description of the embodiments of the present application, unless otherwise specified, “multiple” means two or more than two. For example, multiple residual modules refer to two or more residual modules.
[0023] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.
[0024] Figure 1 is a flow chart of a remote sensing image target detection method provided in an embodiment of the present application, such as Figure 1 As shown, including: Step S1, obtaining a remote sensing image to be measured; Step S2, inputting the remote sensing image to be tested into the target detection model, and obtaining the target detection result of the remote sensing image to be tested output by the target detection model; The target detection model is trained based on multi-type meteorological image samples carrying detection target labels; the multi-type meteorological image samples are sample data obtained by converting the original sample images from the original meteorological domain to different target meteorological domains using a generative adversarial network based on cyclic style transfer.
[0025] Specifically, the remote sensing image to be detected described in the embodiment of the present application refers to a remote sensing image used for target detection, which contains at least one detection target. For example, the remote sensing image to be detected is a remote sensing image of a port area to be detected, which may include a detection target ship.
[0026] The multi-type meteorological image samples described in the embodiment of the present application refer to image sample data generated by inputting remote sensing images of different meteorological styles containing detection targets into a generative adversarial network based on cyclic style transfer for processing, which may specifically include remote sensing image sample data under multiple types of complex meteorological conditions, including but not limited to image data under various weather conditions such as clear, cloudy, rainy, snowy, and dense fog.
[0027] Among them, the Cycle-Style Generative Adversarial Networks (Cycle-Style GAN) based on cyclic style transfer is a deep learning model that combines the Cycle-GAN model and the Style-GAN model technology.
[0028] The traditional GAN model only focuses on the authenticity of the generated image without considering the constraints. If the traditional GAN model is used in the meteorological generation task, although the image generated by the generator is very realistic, the number of objects and the spatial structure in the image are very different from the original image, and the correlation with the original image is low. Therefore, in the embodiment of the present application, a meteorological generation method based on the Cycle-Style GAN model is proposed, and the sample data generated by the Cycle-Style GAN model is used to form multi-type meteorological image samples.
[0029] The original meteorological domain described in the embodiment of the present application refers to the source domain corresponding to the input original sample image, which may include any one of the weather meteorological domains such as sunny, cloudy, rainy, snowy, and dense fog.
[0030] The target meteorological domain described in the embodiment of the present application refers to the target domain obtained by converting the original sample image through image source domain migration, which may also include any one of the meteorological domains such as sunny, cloudy, rainy, snowy, dense fog, etc., and the original meteorological domain is different from the target meteorological domain.
[0031] Figure 2 is one of the structural diagrams of the generative adversarial network based on cyclic style transfer provided in the embodiment of the present application, such as Figure 2 As shown, in the embodiment of the present application, the Cycle-Style GAN model uses two generators G and F, and two discriminators D X and D Y The generator G is used to accept the original image X and transfer it from the original meteorological domain to the target meteorological domain, thereby generating the target image Y, while the purpose of the generator F is to use the target image Y as input to reversely generate the original image X. X and D Y Used to determine whether an image comes from a real image or a generated image.
[0032] Since the original image x The input generator G obtains ,after Then input generator F to get By calculating x and The difference can be measured from the original image x and generate images The smaller the gap between the two, the higher the content similarity between the original image and the generated image, especially when F(G( x )) = x When x And generate images There is only a change in style, without changing the types of objects and spatial structure.
[0033] The detection target label described in the embodiment of the present application refers to the detection target label corresponding to each meteorological image sample, which can be specifically implemented by manual labeling.
[0034] The target detection model described in the embodiment of the present application is trained based on multiple types of meteorological image samples carrying detection target labels, and is used to detect detection targets in remote sensing images under different meteorological conditions.
[0035] It should be noted that in the embodiments of the present application, the target detection model can be constructed based on a deep neural network. The deep neural network can specifically adopt a target detection algorithm model, such as a YOLOv5 model, a YOLOv7 model, etc., or other deep neural networks for remote sensing target detection, which are not specifically limited in the embodiments of the present application.
[0036] Among them, the model training samples are composed of multiple groups of meteorological image samples carrying detection target labels.
[0037] The detection target label is predetermined according to different types of meteorological image samples and corresponds to each meteorological image sample one by one. In other words, each meteorological image sample in the training sample is predetermined to carry the corresponding detection target label.
[0038] In an embodiment of the present application, in step S1, a target area where a detection target is located is monitored by remote sensing monitoring technology, and a remote sensing image of the target area to be detected can be obtained.
[0039] Furthermore, in an embodiment of the present application, in step S2, a sample data set constructed by detection target labels corresponding to a large number of high-quality and diversified meteorological image samples generated in advance using the Cycle-Style GAN model is used to train the target detection model. During this period, data enhancement techniques such as random cropping, rotation, color transformation, etc., as well as model optimization strategies such as transfer learning and multi-scale training, can be introduced to improve the recognition accuracy and robustness of the model under complex meteorological conditions, and finally obtain a trained target detection model. Then, the remote sensing image to be measured obtained in step S1 is input into the target detection model, and after the target detection model recognizes the detection target in the remote sensing image to be measured, the target detection result of the remote sensing image to be measured can be effectively output.
[0040] The remote sensing image target detection method of the embodiment of the present application uses a generative adversarial network with cyclic style transfer to finely control the style of meteorological images, accurately capture image style features, and fuse style information with image spatial features, thereby generating a large number of high-quality and diverse meteorological image samples, and using the generated multi-type meteorological image data sets to train and optimize the target detection model, so that the trained target detection model can accurately and efficiently output the target detection results of the remote sensing image to be tested, which can effectively improve the recognition accuracy and robustness of the target detection model under complex meteorological conditions, and enhance the effect of remote sensing image target detection.
[0041] Based on the content of the above embodiment, as an optional embodiment, the Cycle-Style GAN model includes a first generator network, a second generator network, a first discriminator network, and a second discriminator network, which is obtained by performing adversarial training using the first discriminator network, the second discriminator network, the first generator network, and the second generator network; multiple types of meteorological image samples are generated using the trained first generator network; The first generator network is used to convert an input original sample image in the original meteorological domain into a generated image in the target meteorological domain; the original sample image contains a detection target; The second generator network is used to convert the generated image in the target meteorological domain into a restored image in the original meteorological domain; The first discriminator network is used to distinguish the original sample image from the restored image; The second discriminator network is used to distinguish between real images and generated images in the target meteorological domain.
[0042] Specifically, in an embodiment of the present application, the Cycle-Style GAN model can be composed of a first generator network, a second generator network, a first discriminator network and a second discriminator network, which can be obtained by using the first discriminator network and the second discriminator network to respectively perform true and false discrimination on the image data generated by the first generator network and the second generator network, and continuously performing adversarial training.
[0043] like Figure 3 As shown, the first generator network can be expressed as Generator A2B, the second generator network can be expressed as Generator B2A, the first discriminator network can be expressed as Discriminator A, and the second discriminator network can be expressed as Discriminator B. In the scenario where the remote sensing image is migrated from the original meteorological domain sunny weather to the target meteorological domain cloudy weather style, the first generator network Generator A2B is used to convert the input original sample image of the sunny weather meteorological domain ( Input_A) is converted to the generated image in the cloudy weather meteorological domain ( Generated_B ); The second generator network Generator B2A is used to generate images of cloudy weather meteorological domain ( Generated_B ) is converted to the restored image in the clear weather meteorological domain ( Cyclic_A ).
[0044] Among them, the first discriminator network Discriminator A is used to distinguish the original sample images in the sunny weather domain through the Decision [0, 1] operation ( Input_A ) and restore the image ( Cyclic_A ); The second discriminator network Discriminator B is used to distinguish the real image and the generated image in the cloudy weather domain through the Decision [0, 1] operation ( Generated_B ).
[0045] like Figure 4 As shown in the figure, in the scenario where the remote sensing image is migrated from the cloudy weather in the original meteorological domain to the sunny weather style in the target meteorological domain, the second generator network Generator B2A is used to convert the input original sample image of the cloudy weather meteorological domain ( Input_B ) is converted to the generated image in the clear weather meteorological domain ( Generated_A ); The first generator network GeneratorA2B is used to generate images in the meteorological domain of clear weather ( Generated_A ) is converted into a restored image in the cloudy weather meteorological domain ( Cyclic_B ).
[0046] Among them, the first discriminator network Discriminator A is used to distinguish the real image of the clear weather meteorological domain from the generated image through the Decision [0, 1] operation ( Generated_A ); The second discriminator network Discriminator B is used to distinguish the original sample images of cloudy weather domain through Decision [0, 1] operation ( Input_B ) and restore the image ( Cyclic_B ).
[0047] It can be understood that the Cycle-Style GAN model in the embodiment of the present application is not limited to being applied only to scenarios where remote sensing images are migrated from sunny weather styles to cloudy weather styles, but can also be applied to scenarios where other meteorological domain image data source domains are migrated, for example, remote sensing images are migrated from sunny weather styles to snowy weather styles, or remote sensing images are migrated from sunny weather styles to foggy weather styles, etc.
[0048] Furthermore, in an embodiment of the present application, by performing adversarial training using the first discriminator network Discriminator A, the second discriminator network Discriminator B, the first generator network Generator A2B and the second generator network Generator B2A, a trained Cycle-Style GAN model can be obtained. Then, the trained first generator network Generator A2B can be used to generate multiple types of meteorological image samples to constitute a training sample data set, so as to optimize and train the subsequent target detection model.
[0049] The method of the embodiment of the present application constructs a Cycle-Style GAN model by utilizing two generator networks and two discriminator networks, and optimizes the model performance by adopting an adversarial strategy for training, utilizing continuous competition and learning between the generator network and the discriminator network. This enables the model to generate increasingly realistic samples while improving the classification accuracy of the discriminator, thereby providing comprehensive and reliable remote sensing image sample data for the training of subsequent target detection models.
[0050] Based on the content of the above embodiment, as an optional embodiment, the first generator network and the second generator network both include an encoding network and a decoding network; The encoding network includes multiple convolutional layers and multiple residual modules; Multiple convolutional layers are used to encode features of the first input image; Multiple residual modules are used to extract meteorological style features from the feature maps output by the multi-layer convolutional layers; The decoding network includes a deconvolution layer and an output convolution layer; The deconvolution layer is used to restore the size of the feature map output by the encoding network to the same size as the first input image; The output convolution layer is used to extract and fuse the feature maps output by the deconvolution layer.
[0051] Specifically, in an embodiment of the present application, the original satellite remote sensing image is input into the generator network, and the generator network can output the satellite remote sensing image of the target meteorological domain. Among them, the first generator network Generator A2B and the second generator network Generator B2A can be specifically composed of an encoding network and a decoding network. After the original satellite remote sensing image is input into the generator network, first, the input image data is down-sampled and encoded through an encoding network including multiple convolutional layers and multiple residual modules, and then up-sampled and decoded through a decoding network including deconvolution layers and convolution layers, and finally a satellite remote sensing image of the target meteorological domain is generated.
[0052] It should be noted that, for the first generator network Generator A2B, the first input image is the original sample image of the original meteorological domain; for the second generator network Generator B2A, the first input image is the generated image of the target meteorological domain.
[0053] In a specific embodiment of the present application, Figure 5 As shown, in the downsampling encoding process, the first input image (size is k ) First, it passes through a convolutional layer with a step size of S=1 and a kernel size of 7×7 for feature encoding, and then passes through two convolutional layers with a step size of S=2 and a kernel size of 3×3 to reduce the size of the feature map; then, the feature map output by the multi-layer convolutional layer passes through multiple residual blocks to learn the meteorological style features of the image. Finally, in the upsampling decoding process, the network uses a deconvolutional layer with a step size of S=2 and a kernel size of 3×3 to restore the image size to the same size as the first input image, reduce the number of channels of the image, and extract and fuse the feature map output by the deconvolution layer by a convolutional layer with a step size of S=1 and a kernel size of 7×7, and finally outputs the target satellite remote sensing image that has been migrated in the meteorological domain.
[0054] The method of the embodiment of the present application, by using the codec network to construct a generator network, and introducing convolutional layers and residual modules to construct an encoding network, can extract effective features of the input image, and help train deeper convolutional neural networks to improve the expressive power of the model; by introducing a deconvolution network and a convolutional layer to construct a decoding network, features at different levels can be fused to generate richer image details and improve the quality of the generated image.
[0055] Based on the content of the above embodiment, as an optional embodiment, the first discriminator network and the second discriminator network both include multiple convolutional layers; Except for the last convolution layer in the multi-convolution layer, all other convolution layers are connected to the target activation function to extract the image features of the second input image; The last convolutional layer is used to determine whether the second input image is real image data based on image features.
[0056] Specifically, in an embodiment of the present application, the first discriminator network Discriminator A and the second discriminator network Discriminator B may both be composed of multiple convolutional layers.
[0057] It should be noted that, for the first discriminator network Discriminator A, the second input image is the original sample image in the original meteorological domain; for the first discriminator network Discriminator B, the second input image is the generated image in the target meteorological domain.
[0058] In a specific embodiment of the present application, the discriminator network structure is as follows: Figure 6 As shown, it can specifically include five convolutional layers, among which the step size of the first four convolutional layers is S=2, the convolution kernel size is 4×4, and each convolutional layer is connected to the target activation function, such as the Leaky RELU activation function; the step size of the last convolutional layer is S=2, the convolution kernel size is 8×8, and the feature map finally passes through the convolutional layer to output a judgment result of 0 or 1, where 0 represents fake and 1 represents true.
[0059] It should be noted that although convolutional layers can effectively extract local features of input data, their operations are linear, and the Leaky ReLU target activation function can introduce nonlinear characteristics to the network. This nonlinear characteristic helps to improve the accuracy of the discriminator in distinguishing real data from fake data.
[0060] In this embodiment, the discriminator network structure can be divided into two parts: feature extraction and image judgment. Among them, the first four convolutional layers are used to extract the image features of the second input image, and the last convolutional layer is used to map the extracted image features to a scalar value in the range of [0, 1] through the Decision [0,1] operation, and then through the discriminator network, it can be accurately judged whether the input second input image is a real image.
[0061] The method of the embodiment of the present application introduces nonlinear characteristics to the discriminator network by introducing the LeakyReLU activation function by considering that the operation of extracting features by the convolution layer is linear, which enables the discriminator to learn and express complex function mapping relationships, thereby improving the classification accuracy and generalization ability of the discriminator network.
[0062] Based on the content of the above embodiment, as an optional embodiment, before inputting the remote sensing image to be measured into the target detection model to obtain the target detection result of the remote sensing image to be measured output by the target detection model, the method further includes: Each meteorological image sample in the multiple types of meteorological image samples and its corresponding detection target label are used as a group of training samples, and multiple groups of training samples are obtained; The target detection model is trained using multiple sets of training samples.
[0063] Specifically, in an embodiment of the present application, before inputting the remote sensing image to be measured into the target detection model and obtaining the target detection result of the remote sensing image to be measured output by the target detection model, the target detection model needs to be trained to obtain a trained target detection model.
[0064] In an embodiment of the present application, a large amount of satellite remote sensing image data under different types of meteorological conditions is generated by utilizing the Cycle-Style GAN model to form a comprehensive and diverse training sample data set, which may include remote sensing image data under various meteorological conditions such as clear, cloudy, rainy, snowy, and dense fog, thereby forming a data set including multi-type meteorological image samples and enhancing the complexity and diversity of the training sample data set.
[0065] In the embodiment of the present application, the target detection model is trained using the above sample data set, and the specific training process is as follows: Each meteorological image sample in the multi-type meteorological image samples and its corresponding detection target label are taken as a group of training samples, that is, each meteorological image sample with a detection target label is taken as a group of training samples, thereby obtaining multiple groups of training samples.
[0066] In the embodiment of the present application, each meteorological image sample and the detection target label it carries are in one-to-one correspondence.
[0067] Then, after obtaining multiple sets of training samples, the multiple sets of training samples are sequentially input into the target detection model, and the target detection model is trained using the multiple sets of training samples, namely: The meteorological image samples in each group of training samples and the detection target labels they carry are simultaneously input into the target detection model. According to each output result in the target detection model, the model parameters in the target detection model are adjusted by calculating the loss function value. When the preset training termination conditions are met, the entire training process of the target detection model is finally completed to obtain a trained target detection model.
[0068] After training is completed, the performance of the target detection model can be evaluated through cross-validation and actual application scenario testing. Based on the evaluation results, the model parameters and optimization strategies can be further adjusted to achieve the best recognition effect of the target detection model.
[0069] Among them, the final detection average precision (AP) scores of the models trained with the original dataset and the meteorological enhanced dataset are shown in Table 1.
[0070] Table 1
[0071] Therefore, it can be seen that under the same experimental environment, the accuracy of the target detection model trained using a dataset of multiple types of meteorological image samples in the embodiment of the present application is higher, indicating that the image generated by this method is more realistic. In addition, the increase in the AP index and the final value of the model trained using a dataset of multiple types of meteorological image samples are better than those of the original dataset, which shows that it is feasible to use the Cycle-Style GAN model for remote sensing image meteorological generation tasks, and that high-quality target satellite remote sensing images under different meteorological conditions can be generated to achieve target detection tasks under extreme weather conditions.
[0072] The method of the embodiment of the present application uses a data set including multiple types of meteorological image samples to perform model training, and uses each meteorological image sample and its corresponding detection target label as a group of training samples. The target detection model is trained using multiple groups of training samples, which is beneficial to improving the model accuracy of the trained target detection model.
[0073] Based on the content of the above embodiment, as an optional embodiment, the target detection model is trained using multiple groups of training samples, including: For any training sample in the multiple groups of training samples, any training sample is input into the target detection model, and prediction information corresponding to any training sample is output; Using a preset loss function, the loss value is calculated based on the prediction information corresponding to any training sample and the detection target label corresponding to the training sample; Based on the loss value, adjust the model parameters of the target detection model until the loss value is less than a preset threshold or the number of training times reaches a preset number; The model parameters obtained when the loss value is less than a preset threshold or the number of training times reaches a preset number are used as the model parameters of the trained target detection model.
[0074] Specifically, the preset loss function described in the embodiment of the present application refers to a loss function pre-set in the target detection model for model evaluation.
[0075] The preset threshold described in the embodiment of the present application refers to a threshold pre-set by the model, which is used to obtain the minimum loss value to complete model training.
[0076] The preset number of times described in the embodiment of the present application refers to the maximum number of pre-set model iteration training times.
[0077] In an embodiment of the present application, after obtaining multiple groups of training samples, for any group of training samples, the meteorological image samples in each group of training samples and their corresponding detection target labels are simultaneously input into the target detection model, and the prediction probability corresponding to the training sample is output.
[0078] On this basis, the preset loss function is used to calculate the loss value according to the prediction probability corresponding to the training sample and the detection target label corresponding to the training sample.
[0079] Furthermore, after the loss value is calculated, the training process ends. The model parameters of the target detection model are adjusted based on the loss value to update the weight parameters of each layer of the model in the target detection model, and then the next training is carried out, and the model training is repeated iteratively.
[0080] During the training process, if the training results for a certain group of training samples meet the preset training termination conditions, such as the corresponding calculated loss value is less than the preset threshold, or when the current number of iterations reaches the preset number, the model loss value can be controlled within the convergence range, then the model training ends. At this point, the obtained model parameters can be used as the model parameters of the trained target detection model, and the target detection model training is completed, thereby obtaining a trained target detection model.
[0081] In addition, in the embodiments of the present application, the performance of the trained target detection model can also be evaluated and iterated: the performance of the trained model is regularly evaluated, and the evaluation indicators may include indicators such as root mean square error (RMSE), recall rate (Recall), click rate (CTR), etc. According to the evaluation results, the model is further iterated and optimized to improve the performance of the target detection model.
[0082] The method of the embodiment of the present application controls the loss value of the target detection model within a convergence range by repeatedly iteratively training the target detection model using multiple groups of training samples, which is beneficial to improving the accuracy of the output results of the target detection model and improving the accuracy of remote sensing target detection under complex meteorological conditions.
[0083] The remote sensing image target detection device provided in the present application is described below. The remote sensing image target detection device described below and the remote sensing image target detection method described above can be referenced to each other.
[0084] Figure 7 is a schematic diagram of the structure of a remote sensing image target detection device provided in an embodiment of the present application, such as Figure 7 As shown, it includes: an acquisition module 10 and a detection module 20 connected in sequence.
[0085] Wherein, the acquisition module 10 is used to acquire the remote sensing image to be measured; The detection module 20 is used to input the remote sensing image to be tested into the target detection model to obtain the target detection result of the remote sensing image to be tested output by the target detection model; The target detection model is trained based on multi-type meteorological image samples carrying detection target labels; multi-type meteorological image samples are generated using a generative adversarial network based on cyclic style transfer.
[0086] It can be understood that the detailed functional implementation of each of the above-mentioned units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.
[0087] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.
[0088] The remote sensing image target detection device of the embodiment of the present application uses a generative adversarial network of cyclic style transfer to finely control the style of meteorological images, accurately capture image style features, and fuse style information with image spatial features, thereby generating a large number of high-quality and diverse meteorological image samples, and uses the generated multi-type meteorological image data sets to train and optimize the target detection model, so that the trained target detection model can accurately and efficiently output the target detection results of the remote sensing image to be tested, which can effectively improve the recognition accuracy and robustness of the target detection model under complex meteorological conditions, and improve the effect of remote sensing image target detection.
[0089] Based on the method in the above embodiment, the embodiment of the present application provides an electronic device, such as Figure 8 As shown, the electronic device may include: a processor (Processor) 810, a communication interface (Communications Interface) 820, a memory (Memory) 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logic instructions in the memory 830 to execute the method in the above embodiment.
[0090] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.
[0091] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.
[0092] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.
[0093] It is understandable that the processor in the embodiment of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.
[0094] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.
[0095] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.
[0096] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.
[0097] It should be understood that expressions such as "including" and "may include" that may be used in the present application indicate the existence of the disclosed functions, operations, or constituent elements, and do not limit one or more additional functions, operations, and constituent elements. In the present application, terms such as "including" and / or "having" may be interpreted as indicating specific characteristics, numbers, operations, constituent elements, components, or combinations thereof, but may not be interpreted as excluding the existence or possibility of adding one or more other characteristics, numbers, operations, constituent elements, components, or combinations thereof.
[0098] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.
Claims
1. A remote sensing image target detection method, characterized in that: include: Acquire the remote sensing image to be measured; Inputting the remote sensing image to be measured into a target detection model to obtain a target detection result of the remote sensing image to be measured output by the target detection model; The target detection model is trained based on multi-type meteorological image samples carrying detection target labels; the multi-type meteorological image samples are sample data obtained by converting the original sample images from the original meteorological domain to different target meteorological domains using a generative adversarial network based on cyclic style transfer.
2. The remote sensing image target detection method according to claim 1, characterized in that: The generative adversarial network based on cyclic style transfer includes a first generator network, a second generator network, a first discriminator network and a second discriminator network, and is obtained by performing adversarial training using the first discriminator network, the second discriminator network, the first generator network and the second generator network; the multi-type meteorological image samples are generated using the trained first generator network; The first generator network is used to convert an input original sample image in the original meteorological domain into a generated image in the target meteorological domain; the original sample image contains the detection target; The second generator network is used to convert the generated image in the target meteorological domain into a restored image in the original meteorological domain; The first discriminator network is used to distinguish the original sample image from the restored image; The second discriminator network is used to distinguish the real image of the target meteorological domain from the generated image.
3. The remote sensing image target detection method according to claim 2, characterized in that: The first generator network and the second generator network each include an encoding network and a decoding network; the encoding network includes multiple convolutional layers and multiple residual modules; The multi-layer convolutional layer is used to perform feature encoding on the first input image; The multiple residual modules are used to extract meteorological style features from the feature graph output by the multi-layer convolutional layer; The decoding network includes a deconvolution layer and an output convolution layer; The deconvolution layer is used to restore the size of the feature map output by the encoding network to the same size as the first input image; The output convolution layer is used to extract and fuse features of the feature map output by the deconvolution layer.
4. The remote sensing image target detection method according to claim 2, characterized in that: The first discriminator network and the second discriminator network both include multiple convolutional layers; Except for the last convolution layer in the multi-layer convolution layer, all other convolution layers are connected to the target activation function to extract image features of the second input image; The last convolutional layer is used to determine whether the second input image is real image data based on the image features.
5. The remote sensing image target detection method according to any one of claims 1 to 4, characterized in that: Before inputting the remote sensing image to be measured into the target detection model to obtain the target detection result of the remote sensing image to be measured output by the target detection model, the method further includes: Taking each meteorological image sample in the multiple types of meteorological image samples and its corresponding detection target label as a group of training samples, and obtaining multiple groups of the training samples; The target detection model is trained using multiple groups of the training samples.
6. The remote sensing image target detection method according to claim 5, characterized in that: The method of training the target detection model using multiple groups of training samples includes: For any training sample in the plurality of groups of training samples, input the any training sample into the target detection model, and output prediction information corresponding to the any training sample; Calculate the loss value based on the prediction information corresponding to any one of the training samples and the detection target label corresponding to the training sample by using a preset loss function; Based on the loss value, adjusting the model parameters of the target detection model until the loss value is less than a preset threshold or the number of training times reaches a preset number of times; The model parameters obtained when the loss value is less than the preset threshold or the number of training times reaches the preset number of times are used as the model parameters of the trained target detection model.
7. A remote sensing image target detection device, characterized in that: include: An acquisition module, used for acquiring the remote sensing image to be measured; A detection module, used for inputting the remote sensing image to be measured into a target detection model, and obtaining a target detection result of the remote sensing image to be measured output by the target detection model; The target detection model is obtained by training based on multiple types of meteorological image samples carrying detection target labels; the multiple types of meteorological image samples are generated using a generative adversarial network based on cyclic style transfer.
8. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 6.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 6.
10. A computer program product, characterized in that When the computer program product runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 6.