Optical remote sensing image target detection method, device and medium

Abstract

The present invention discloses a method, device and medium for detecting targets in optical remote sensing images, relates to the technical field of optical remote sensing target detection, and is used to realize target detection. In view of the problem that the detection rate of target detection by the Yolov5 model needs to be further improved, a method for detecting targets in optical remote sensing images is provided. By introducing an attention module, position information is retained when the network model extracts image features, and attention to spatial information is increased while capturing long-term dependencies. This allows the network model to pay more attention to areas of interest and suppress other unimportant parts, thereby enhancing attention to spatial information and further improving the performance of the network model by the attention model. The target detection network model improved based on this method has better robustness, thereby improving the detection rate of targets in optical remote sensing images, and has better performance in the recognition of small targets, in particular.

Description

Technical Field

[0001] The present invention relates to the field of optical remote sensing target detection, and in particular to an optical remote sensing image target detection method, device and medium. Background Art

[0002] Object detection is an important task in optical remote sensing image analysis. It aims to automatically identify and locate objects such as houses, airplanes, cars, playgrounds, and ships in optical remote sensing images. It has important applications in intelligent transportation, public safety, national defense, and other fields.

[0003] With the rapid rise of deep learning in the field of optical remote sensing target detection, deep learning-based detection methods, leveraging the powerful feature extraction capabilities of neural networks, are gradually replacing traditional target detection algorithms and becoming the mainstream. Common methods include Yolov5 (You Only Look Once version 5), a computer vision-based target detection algorithm. However, due to the complex backgrounds and multi-scale targets of optical remote sensing images, the increasing complexity of practical applications places higher performance demands on target detection algorithms. This is particularly true for the detection rate of small targets (i.e., targets smaller than typical targets), placing even higher demands on target detection algorithms.

[0004] Therefore, those skilled in the art are in urgent need of an optical remote sensing image target detection method to further improve the detection rate of optical remote sensing image targets, especially small targets. Summary of the Invention

[0005] The purpose of the present invention is to provide a method, device and medium for detecting targets in optical remote sensing images, so as to further improve the detection rate of targets in optical remote sensing images, especially small targets.

[0006] To solve the above technical problems, the present invention provides a method for detecting targets in optical remote sensing images, comprising:

[0007] Acquire optical remote sensing image datasets;

[0008] Based on the Yolov5 detection network model, an attention module is introduced to obtain an object detection network model; wherein the attention module is used to retain position information when the Yolov5 detection network model extracts image features, and to increase attention to spatial information while capturing long-term dependencies;

[0009] Training the target detection network model using the optical remote sensing image dataset, and persistently storing the trained target detection network model;

[0010] Deploy the trained target detection network model and perform target detection on actual optical remote sensing images.

[0011] In one possible embodiment, the attention module is used to:

[0012] Decomposing channel attention into an encoding process of aggregating features in two different directions, a first direction and a second direction; wherein the encoding process along the first direction retains the position information of the image features to obtain the first features; and the encoding process along the second direction captures the long-term dependency of the image features to obtain the second features;

[0013] focusing the spatial information of the image feature to obtain a third feature;

[0014] The first feature, the second feature, and the third feature are fused to obtain and output a target feature.

[0015] In a possible embodiment, the attention module includes: a residual connection module, a first direction aggregation module, a second direction aggregation module, a spatial focusing module and a reweighting module;

[0016] The output end of the residual connection module is connected to the input ends of the spatial focusing module, the first direction aggregation module, and the second direction aggregation module, so as to obtain the image features from the previous stage of the attention module;

[0017] The first direction aggregation module is used to retain the position information of the image feature along the first direction by using an average pooling technique to obtain the first feature;

[0018] The second direction aggregation module is used to capture the long-term dependency of the image features along the second direction by using an average pooling technique to obtain the second features;

[0019] The spatial focusing module is used to focus the spatial information through a convolution layer to obtain the third feature;

[0020] The input end of the re-weighting module is connected to the output ends of the first direction aggregation module, the second direction aggregation module and the spatial focusing module, and is used to fuse the first feature, the second feature and the third feature to obtain the target feature.

[0021] In a possible embodiment, the cross-stage local structure of the Neck part in the target detection network model further includes: a residual branch;

[0022] The input end of the residual branch is connected to the input end of the cross-stage local structure, and the output end of the residual branch is connected to the input end of the splicing module in the cross-stage local structure.

[0023] In a possible embodiment, the positioning loss in the loss function of the target detection network model is:

[0024]

[0025] Among them, L loc is the positioning loss; b, b gt are the prediction box and the label box respectively; ρ is the Euclidean distance between the center point of the prediction box and the center point of the label box; c is the minimum diagonal distance of the box that contains both the prediction box and the label box; w, w gt are the widths of the prediction box and label box respectively; h, h gt are the heights of the prediction box and the label box respectively; e is a natural constant.

[0026] In a possible embodiment, acquiring the optical remote sensing image dataset includes:

[0027] Collect multiple historical datasets of optical remote sensing images through cloud platforms;

[0028] Analyzing each of the optical remote sensing image historical data sets, and if there is an optical remote sensing image that is not marked with a location tag, marking it with the location tag;

[0029] Integrating the optical remote sensing image historical data sets into one data set to obtain the optical remote sensing image data set;

[0030] Training the target detection network model using the optical remote sensing image dataset and persistently storing the trained target detection network model includes:

[0031] The optical remote sensing image dataset is loaded into the cloud platform, and the target detection network model is trained in the cloud platform using the optical remote sensing image dataset. The trained small target prediction network model is persistently stored in the cloud platform.

[0032] In a possible embodiment, after integrating the optical remote sensing image historical data sets into one data set to obtain the optical remote sensing image data set, the method further includes:

[0033] Dividing all data in the optical remote sensing image dataset into a training set and a test set according to a preset ratio;

[0034] Training the target detection network model using the optical remote sensing image dataset and persistently storing the trained target detection network model includes:

[0035] The target detection network model is trained using the training set, and the detection rate of the target detection network model is tested using the test set;

[0036] If the detection rate meets the preset requirement, the target detection network model is persistently stored; otherwise, the process returns to the step of training the target detection network model using the training set.

[0037] To solve the above technical problems, the present invention further provides an optical remote sensing image target detection device, comprising:

[0038] A data acquisition module, used to acquire optical remote sensing image data sets;

[0039] A model building module is configured to introduce an attention module based on a Yolov5 detection network model to obtain an object detection network model; wherein the attention module is configured to retain position information when the Yolov5 detection network model extracts image features and to increase attention to spatial information while capturing long-term dependencies;

[0040] A model training module, configured to train the target detection network model using the optical remote sensing image dataset and persistently store the trained target detection network model;

[0041] The target detection module is used to deploy the trained target detection network model and perform target detection on actual optical remote sensing images.

[0042] To solve the above technical problems, the present invention further provides an optical remote sensing image target detection device, comprising:

[0043] memory for storing computer programs;

[0044] The processor is configured to implement the steps of the optical remote sensing image target detection method as described above when executing the computer program.

[0045] To solve the above technical problems, the present invention further provides a non-volatile storage medium having a computer program stored thereon, which implements the steps of the above-mentioned optical remote sensing image target detection method when executed by a processor.

[0046] The present invention provides a method for detecting targets in optical remote sensing images. The method improves the original Yolov5 detection network model by introducing an attention module to retain position information when the network model extracts image features, and increases attention to spatial information while capturing long-term dependencies. This allows the network model to pay more attention to areas of interest and suppress other unimportant parts, thereby enhancing attention to spatial information and further improving the performance of the attention model on the network model. Based on the improved target detection network model of this method, a more robust network model can be obtained on the basis of the original Yolov5 detection network model, thereby improving the detection rate of targets in optical remote sensing images, especially for small targets, with better performance.

[0047] The optical remote sensing image target detection device and non-volatile storage medium provided by the present invention correspond to the above method and have the same effects as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0049] Figure 1 A schematic diagram of a Yolov5 detection network model;

[0050] Figure 2 A flowchart of a method for detecting targets in optical remote sensing images provided by an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of an attention model provided by an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of an R-CSP structure provided in an embodiment of the present invention;

[0053] Figure 5 Schematic diagram of the CSP1-x structure in the Yolov5 detection network model;

[0054] Figure 6 Schematic diagram of the CSP2-x structure in the Yolov5 detection network model;

[0055] Figure 7 Schematic diagram of the residual structure in the Yolov5 detection network model;

[0056] Figure 8 Schematic diagram of the CBS structure in the Yolov5 detection network model;

[0057] Figure 9 A schematic diagram of a target detection network model provided by an embodiment of the present invention;

[0058] Figure 10 An application architecture diagram of an optical remote sensing image target detection method provided by an embodiment of the present invention;

[0059] Figure 11 A structural diagram of an optical remote sensing image target detection device provided by an embodiment of the present invention;

[0060] Figure 12 This is a structural diagram of another optical remote sensing image target detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0062] The core of the present invention is to provide a method, device and medium for detecting targets in optical remote sensing images.

[0063] In order to enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0064] In related technologies, methods for detecting different types of targets in optical remote sensing images can be roughly divided into four categories: template matching-based methods, knowledge-based methods, target image analysis-based methods, and deep learning-based methods. Deep learning-based detection methods, leveraging the powerful feature extraction capabilities of neural networks, have gradually replaced traditional target detection algorithms and become the mainstream target detection method for optical remote sensing. Yolov5 (You Only Look Once version 5), a computer vision target detection algorithm, is a relatively mainstream target detection algorithm in practical applications.

[0065] The target detection network model based on the Yolov5 algorithm (hereinafter referred to as the Yolov5 detection network model) is as follows Figure 1 As shown in Figure 2, the network model mainly consists of three parts:

[0066] 1. Backbone part: This part is a pre-trained convolutional neural network used to extract features from the input image.

[0067] 2. Neck part: This part is used to collect feature maps at different stages and fuse the extracted features.

[0068] 3. Detect part: This part is used to perform the final calculation output and predict the location and category of the target.

[0069] Based on the above and Figure 1 The Yolov5 detection network model shown in the figure can be trained with sample data to obtain an object detection model for optical remote sensing image object detection. However, the performance and detection rate of this Yolov5 detection network model when identifying smaller objects need to be improved.

[0070] Based on this, the present invention provides an optical remote sensing image target detection method, such as Figure 2 Shown, including:

[0071] S1: Obtain optical remote sensing image dataset.

[0072] S2: Based on the Yolov5 detection network model, the attention module is introduced to obtain the target detection network model.

[0073] Among them, the attention module is used to retain position information when the Yolov5 detection network model extracts image features, and to increase attention to spatial information while capturing long-term dependencies.

[0074] S3: Trains the target detection network model using the optical remote sensing image dataset and persistently stores the trained target detection network model.

[0075] S4: Deploy the trained target detection network model and perform target detection on actual optical remote sensing images.

[0076] Step S1 involves obtaining a dataset for training the target detection model. Generally speaking, standard optical remote sensing image datasets used for training target detection models typically include optical remote sensing images and location markers applied to the images. In practical applications, the more optical remote sensing images a dataset contains, the better the performance of the trained target detection model. However, this also increases the training time. Therefore, you can freely select an appropriate optical remote sensing image dataset for training the target detection model based on your actual needs.

[0077] Secondly, step S2 is also the core step of this method. This method proposes a target detection network model based on an improved original Yolov5 detection network model. Compared with the original Yolov5 detection network model, the improved target detection network model of this method newly introduces an attention module (S-CA). Among them, the S-CA module is used to retain position information when the network model extracts image features, and increase the attention to spatial information while capturing long-term dependencies, thereby further improving the performance of the network model, especially for the improvement of small target detection rate can be significantly improved.

[0078] Furthermore, this embodiment also provides a possible implementation scheme for the above-mentioned attention module, which is specifically used to:

[0079] The channel attention is decomposed into an encoding process of aggregating features in two different directions: the first direction and the second direction. The encoding process along the first direction retains the position information of the image features to obtain the first feature; the encoding process along the second direction captures the long-term dependency of the image features to obtain the second feature.

[0080] The spatial information of the image features is focused to obtain the third feature.

[0081] The first feature, the second feature, and the third feature are fused to obtain and output the target feature.

[0082] That is, in this embodiment, the attention module decomposes the channel attention into an encoding process of aggregating features in two different directions, the first direction and the second direction, so as to retain the position information along one direction (the first direction) and capture the long-term dependency along the other direction (the second direction). Furthermore, by fusing the features in these two directions (i.e., the first feature and the second feature), the network model can pay more attention to the area of ​​interest and suppress other unimportant areas. Afterwards, the image features extracted by the model are further fused with the first and second features through the third feature obtained after focusing on the spatial information, which can further enhance the attention to the spatial information, thereby improving the performance of the network model.

[0083] Furthermore, for the specific implementation of the above attention module, this embodiment provides a possible implementation scheme, such as Figure 3 As shown, the attention model includes: a residual connection module (Residual), a first direction aggregation module, a second direction aggregation module, a spatial focusing module and a re-weighting module (Re-weight).

[0084] Among them, the output end of the residual connection module is connected to the input end of the spatial focusing module, the first direction aggregation module and the second direction aggregation module, so as to obtain image features from the previous stage of the attention module.

[0085] The first direction aggregation module is used to retain the position information of the image feature along the first direction through the average pooling technology to obtain the first feature.

[0086] The second direction aggregation module is used to capture the long-term dependency of image features along the second direction through the average pooling technique to obtain the second feature.

[0087] like Figure 3 As shown, the first direction aggregation module and the second direction aggregation module specifically include: an average pooling module (AvgPool; wherein, Figure 3 The first direction is called X direction and the second direction is called Y direction, so the average pooling module in the first direction aggregation module is Figure 3 The X Avg Pool in the second direction aggregation module is the average pooling module. Figure 3 Y AvgPool in), connection and two-bit convolution layer module (Concat: splicing function; Conv2d: two-bit convolution layer), batch normalization and nonlinear transformation module (BatchNorm: batch normalization; Non-linear: nonlinear transformation), two-bit convolution layer module (Conv2d) and activation function module (Sigmoid).

[0088] The spatial focusing module is used to focus the spatial information through the convolution layer to obtain the third feature.

[0089] like Figure 3 As shown in the figure, the spatial focusing module specifically includes: two 7*7 convolutional layer modules (7*7Conv) and an activation function module (Sigmoid).

[0090] The input end of the re-weighting module is connected to the output ends of the first direction aggregation module, the second direction aggregation module and the spatial focusing module, and is used to fuse the first feature, the second feature and the third feature to obtain the target feature.

[0091] Based on the attention module provided in this embodiment, the features extracted by the network model can be decomposed into two different directions, X and Y, and the encoding process of the aggregated features can be performed along these two directions. Specifically, the position information is retained in the X direction, and the long-term dependency is captured in the Y direction. After the features in these two directions are fused, the entire network model can pay more attention to the area of ​​interest and suppress other unimportant areas. Furthermore, the self-attention module also uses the convolution layer to focus the extracted features on spatial information, and fuses the focused features with the features extracted in the above-mentioned X and Y directions. On the basis of the above-mentioned X and Y feature fusion, the focus on spatial information can be further enhanced, thereby further improving the detection rate of the network model.

[0092] Step S3 is the process of training the target detection network model improved and constructed by step S2 of the present method using the optical remote sensing image data set obtained by step S1. Since the target detection network model in the present method is a network model improved by introducing the attention module on the basis of the original Yolov5 detection network model, there is no obvious difference from the original Yolov5 detection network model in terms of training itself. The target detection network model can be trained according to the method of training the Yolov5 detection network model, and this embodiment will not be repeated. In addition, the trained target detection network model is also persistently stored in step S3, so that when the subsequent step S4 actually performs target detection of the optical remote sensing image, the trained small target detection network can be directly called to perform target detection without retraining, thereby improving the efficiency of target detection.

[0093] In summary, the present invention provides a method for target detection in optical remote sensing images. By introducing an attention module, the network model retains positional information and captures long-term dependencies when extracting features, while also increasing its focus on spatial information. This further enhances the attention module's impact on network performance. Target detection in optical remote sensing images based on this improved network model can improve the target detection rate, particularly in small target detection scenarios.

[0094] On the other hand, the above embodiment mainly adds an attention module to the original Yolov5 detection network model to improve the performance of the network model. This embodiment also provides another model optimization solution, such as Figure 4 As shown:

[0095] The cross-stage local structure of the Neck part in the target detection network model also includes a residual (ResUnit) branch.

[0096] Among them, the input end of the residual branch is connected to the input end of the cross-stage local structure, and the output end of the residual branch is connected to the input end of the concatenation module (Concat) in the cross-stage local structure.

[0097] First, let's explain the Cross Stage Partial (CSP) structure in the original Yolov5 detection network model: There are two cross-stage partial structure designs in the original Yolov5 detection network model, namely CSP1_x and CSP2_x. Here, x represents the number of components or repetitions in the module.

[0098] Specifically, CSP1_x is mainly used in the backbone network. The specific structure of CSP1_x is as follows Figure 5As shown in Figure 1, CSP1_x divides the input into two branches, one branch passes through the convolution layer and multiple residual (ResUnit) structures, and the other branch directly performs convolution. The results of the two branches are then spliced ​​to enhance feature extraction capabilities and reduce computational complexity.

[0099] CSP2-x is mainly used in the Neck part. The specific structure of CSP1_x is as follows: Figure 6 As shown in Figure 2. Unlike CSP1-x, CSP2-x replaces the residual structure with more convolutional layers to accommodate the needs of the Neck network. This design also aims to improve the network's feature extraction capabilities and reduce computational complexity by splitting and merging feature maps.

[0100] Among them, the residual structure is as follows Figure 7 As shown in Figure 1, it includes multiple CBSs. CBS is a basic convolution module widely used in Yolo series models for feature extraction and conversion. Figure 8 As shown in the figure, C stands for Conv, which represents the convolution layer; B stands for BatchNorm, abbreviated as BN, which represents the batch normalization layer; S stands for SiLU (an activation function), which represents the SiLU activation function layer.

[0101] As can be seen from the above, this embodiment mainly improves the cross-stage local structure CSP2-x in the Neck part of the original Yolov5 detection network model. The improved cross-stage local structure can be distinguished as the R-CSP module. R-CSP adds a residual (ResUnit) structure as a branch based on the original CSP2-x structure, allowing the network to extract more fine-grained features. The features extracted by the main branch are reorganized and integrated with high-level semantic features and low-level detail features through three branches, which can further improve the detection accuracy of the target.

[0102] Based on the above embodiments, the present invention proposes two improvements to the network structure of the original Yolov5 detection network model: 1. Introducing the attention mechanism, that is, adding Figure 3 2. Design a multi-scale feature extraction structure, that is, modify the original Figure 6 The CSP2-x structure shown is as follows Figure 4 The R-CSP structure shown.

[0103] It should be noted that the above two improvements can be implemented separately or together, and this embodiment does not limit this. Any improvement implemented alone can achieve the technical effects described in the above embodiment. Similarly, implementing the two improvements together can simultaneously bring about the technical effects that can be achieved by the two improvements. For example, in a possible example, the above two improvements are implemented together, and the improved Yolov5 detection network model is as follows: Figure 9 shown.

[0104] On the other hand, in addition to the improvements made to the network structure of the Yolov5 detection network model in the above embodiment, this embodiment also provides another improvement solution for the Yolov5 detection network model:

[0105] The positioning loss in the loss function of the improved target detection network model in this embodiment is shown as follows:

[0106]

[0107]

[0108] Where, L loc is the positioning loss; b, b gt are the prediction box and the label box respectively; ρ is the Euclidean distance between the center point of the prediction box and the center point of the label box; c is the minimum diagonal distance of the box that contains both the prediction box and the label box; w, w gt are the widths of the prediction box and label box respectively; h, h gt are the heights of the prediction box and the label box respectively; e is a natural constant.

[0109] Similarly, before explaining the improvements of this embodiment, the loss function of the original Yolov5 detection network model is first explained. The loss function of the original Yolov5 detection network model mainly consists of three parts: classification loss, positioning loss, and confidence loss, as shown in the following formula:

[0110] Loss=λ1L cls +λ2L obj +λ3L loc (4);

[0111] Where Loss is the loss function; L cls 、L obj 、L loc are classification loss, confidence loss and positioning loss respectively; λ1, λ2 and λ3 are the balance coefficients of classification loss, confidence loss and positioning loss respectively.

[0112] Among them, the positioning loss is generally implemented using Loss_CIoU (Complete Intersection over Union, CIoU, a loss function). Loss_CIoU includes overlap loss, center distance loss, and width and height loss. The specific calculation formula is as follows:

[0113]

[0114] Where Loss_CIoU represents the positioning loss of the original Yolov5 detection network model; α is the weight coefficient. Equation (6) is consistent with Equation (2) above, representing the ratio of the area of ​​the intersection of the prediction box and the label box to the area of ​​the union area, which is the overlap loss mentioned above.

[0115] As shown in Equation (7), the width-to-height loss component of the Loss_CIoU loss function measures aspect ratio consistency and involves inverse trigonometric functions. Therefore, the computational overhead is high, resulting in slower overall training time. Furthermore, different values ​​for the weight coefficient α can significantly impact the performance of the loss function and require careful adjustment.

[0116] In order to solve the above problems, this embodiment improves the original Loss_CIoU loss function. From formulas (1), (3), (5), and (7), it can be seen that the positioning loss function in this embodiment is (1-e -ww ) 2 +(1-e -wh ) 2 Replacing αv in the original Loss_CIoU loss function to represent the target shape loss can eliminate the influence of the weight coefficient α and remove the calculation of the inverse trigonometric function in the loss function, which can speed up the training of the model.

[0117] It should also be noted that this embodiment is another improvement scheme in addition to the above two improvement schemes for the original Yolov5 detection network model. It improves the positioning loss function of the Yolov5 detection network model to eliminate the influence of the weight coefficient α and eliminates the calculation part of the inverse trigonometric function in the loss function, thereby speeding up the training speed of the model. That is, as of this embodiment, the present invention has proposed three improvement schemes for the original Yolov5 detection network model, namely: 1. Introducing the attention mechanism; 2. Designing a multi-scale feature extraction structure; 3. Optimizing the loss function. It should be noted that the above three improvements have no dependency on each other and can be implemented separately, in combination with any two, or all three together. Implementing a certain improvement can bring about the technical effect corresponding to the improvement. The improvement scheme can be freely selected according to actual needs, and this embodiment does not limit this.

[0118] On the other hand, during the training process of a deep learning model, the dataset used to train the model, which is outside the model itself, will also affect the performance of the trained network model. Based on this, this embodiment provides a corresponding implementation plan, and the above-mentioned step S1 specifically includes:

[0119] S11: Collect multiple historical datasets of optical remote sensing images through the cloud platform.

[0120] S12: Analyze each optical remote sensing image historical data set, and if there is an optical remote sensing image that is not marked with a location label, mark it with a location label.

[0121] S13: Integrate the optical remote sensing image historical data sets into one data set to obtain an optical remote sensing image data set.

[0122] Accordingly, the above-mentioned step S13: training the target detection network model using the optical remote sensing image dataset and persistently storing the trained target detection network model further includes:

[0123] The optical remote sensing image dataset is loaded into the cloud platform, and the target detection network model is trained using the optical remote sensing image dataset in the cloud platform. The trained small target prediction network model is persistently stored in the cloud platform.

[0124] In this embodiment, by taking advantage of the ultra-large scale of the cloud platform, a large number of historical optical remote sensing image datasets can be collected. These historical optical remote sensing image datasets can be historical data collected through actual applications, or public datasets collected or purchased from the Internet or open source platforms, thereby obtaining a sample dataset with a data volume far exceeding that of ordinary datasets, namely the above-mentioned optical remote sensing image dataset. Figure 10 As shown, this embodiment obtains multiple public optical remote sensing image data sets through the cloud platform to obtain a large optical remote sensing image data set with a larger data volume, thereby increasing the training sample data of the target detection model to improve the model performance.

[0125] As can be seen from the above examples, when training an object detection model, optical remote sensing images in a standard dataset should be position-tagged, which is a supervised learning training method. However, given that this example collects a large number of datasets on a cloud platform, some of these datasets may be non-standard, meaning that optical remote sensing images are not position-tagged. These datasets should be position-tagged after analysis so that they can participate in subsequent model training.

[0126] After that, this embodiment also takes advantage of the cloud platform's ultra-high computing power to train the target detection network model constructed above using an optical remote sensing image dataset. As can be seen from the above, since the optical remote sensing image dataset obtained in this embodiment is obtained by integrating multiple datasets used in conventional model training, when training the target detection network model using the optical remote sensing image dataset, the requirements for hardware computing power are also higher. Therefore, in this embodiment, the cloud platform's ultra-high computing power is used to overcome this difficulty and complete the training of the target detection network model. It is not difficult to know that when a deep learning model is trained based on sample data with a larger amount of data, the performance of the model obtained is theoretically better. That is, this embodiment provides a solution that is conducive to improving model performance from the perspective of the dataset used to train the model.

[0127] Furthermore, based on the large optical remote sensing image dataset obtained by the cloud platform in the above embodiment, this embodiment also provides a possible implementation scheme. After the above step S13, the method further includes:

[0128] S14: Divide all data in the optical remote sensing image dataset into a training set and a test set according to a preset ratio.

[0129] Accordingly, step S3: training the target detection network model using the optical remote sensing image dataset and persistently storing the trained target detection network model further includes:

[0130] S31: Train the target detection network model through the training set, and test the detection rate of the target detection network model through the test set.

[0131] S32: If the detection rate meets the preset requirements, the target detection network model is persistently stored.

[0132] Otherwise, return to step S31.

[0133] It should be noted that this embodiment does not limit the specific value of the preset ratio, and any ratio value can be selected according to actual needs. In a possible embodiment, the preset ratio can be 7:3, that is, the training set:test set=7:3.

[0134] That is, Figure 10 As shown, this embodiment further divides the large optical remote sensing image dataset obtained in the above embodiment. All data is divided into a training set and a test set according to a preset ratio. The training set is used to train the model, while the test set is used to test whether the trained model performance meets expectations. The solution based on this embodiment can better ensure the performance of the trained object detection network model.

[0135] While the above embodiments describe a method for detecting objects in optical remote sensing images in detail, the present invention also provides a corresponding embodiment of an apparatus for detecting objects in optical remote sensing images. It should be noted that the present invention describes the apparatus from two perspectives: one based on functional modules and the other based on hardware.

[0136] Based on the perspective of functional modules, such as Figure 11 As shown, this embodiment provides an optical remote sensing image target detection device, including:

[0137] A data acquisition module 11 is used to acquire an optical remote sensing image dataset;

[0138] A model building module 12 is used to introduce an attention module based on the Yolov5 detection network model to obtain an object detection network model; wherein the attention module is used to retain position information when the Yolov5 detection network model extracts image features and to increase attention to spatial information while capturing long-term dependencies;

[0139] The model training module 13 is used to train the target detection network model using the optical remote sensing image dataset and to persistently store the trained target detection network model;

[0140] The target detection module 14 is used to deploy the trained target detection network model and perform target detection on the actual optical remote sensing image.

[0141] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, and they will not be repeated here.

[0142] Figure 12 A structural diagram of an optical remote sensing image target detection device provided by another embodiment of the present invention is shown in FIG. Figure 12 As shown, an optical remote sensing image target detection device includes: a memory 20 for storing a computer program;

[0143] The processor 21 is configured to implement the steps of the optical remote sensing image target detection method of the above embodiment when executing a computer program.

[0144] The optical remote sensing image target detection device provided in this embodiment may include but is not limited to a mobile terminal, a personal computer, a workstation, etc.

[0145] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0146] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of an optical remote sensing image target detection method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include but is not limited to an optical remote sensing image target detection method, etc.

[0147] In some embodiments, an optical remote sensing image target detection device may further include a display screen 22 , an input / output interface 23 , a communication interface 24 , a power supply 25 , and a communication bus 26 .

[0148] Those skilled in the art will understand that Figure 12 The structure shown in the figure does not constitute a limitation on an optical remote sensing image target detection device, and may include more or fewer components than shown in the figure.

[0149] An embodiment of the present invention provides an optical remote sensing image target detection device, which includes a memory and a processor. When the processor executes a program stored in the memory, it can implement the following method: an optical remote sensing image target detection method.

[0150] In addition to the embodiments of the optical remote sensing image target detection method and apparatus provided in the above embodiments, the present invention also provides corresponding embodiments of a computer program product. This computer program product includes a computer program / instructions that, when executed by a processor, implements the steps of the optical remote sensing image target detection method described in any of the above embodiments.

[0151] Since the embodiments of the computer program product part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the computer program product part, and will not be repeated here.

[0152] Finally, the present invention also provides an embodiment corresponding to a non-volatile storage medium. The non-volatile storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiment.

[0153] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium and executes all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0154] The above is a detailed introduction to an optical remote sensing image target detection method, device and medium provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, the present invention can also be improved and modified in several ways, and these improvements and modifications also fall within the scope of protection of the present invention.

[0155] It should also be noted that, in this specification, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

Claims

1. A method for detecting targets in optical remote sensing images, characterized in that: include: Acquire optical remote sensing image datasets; Based on the Yolov5 detection network model, an attention module is introduced to obtain a target detection network model; wherein the attention module is used to: decompose channel attention into an encoding process of aggregating features in two different directions, a first direction and a second direction; the encoding process along the first direction retains the position information of the image features to obtain a first feature; the encoding process along the second direction captures the long-term dependency of the image features to obtain a second feature; the spatial information of the image features is focused to obtain a third feature; and the first feature, the second feature, and the third feature are fused to obtain and output a target feature; Training the target detection network model using the optical remote sensing image dataset, and persistently storing the trained target detection network model; Deploying the trained target detection network model and performing target detection on actual optical remote sensing images; The cross-stage local structure of the Neck part in the target detection network model further includes: a residual branch; an input end of the residual branch is connected to an input end of the cross-stage local structure, and an output end of the residual branch is connected to an input end of a splicing module in the cross-stage local structure; The positioning loss in the loss function of the target detection network model is: ； ； ； in, For positioning loss; They are prediction box and label box respectively; is the Euclidean distance between the center point of the prediction box and the center point of the label box; is the minimum box diagonal distance that contains both the prediction box and the label box; are the widths of the prediction box and label box respectively; are the heights of the prediction box and the label box respectively; e is a natural constant.

2. The optical remote sensing image target detection method according to claim 1, characterized in that: The attention module includes: a residual connection module, a first direction aggregation module, a second direction aggregation module, a spatial focusing module and a reweighting module; The output end of the residual connection module is connected to the input ends of the spatial focusing module, the first direction aggregation module, and the second direction aggregation module, so as to obtain the image features from the previous stage of the attention module; The first direction aggregation module is used to retain the position information of the image feature along the first direction by using an average pooling technique to obtain the first feature; The second direction aggregation module is used to capture the long-term dependency of the image features along the second direction by using an average pooling technique to obtain the second features; The spatial focusing module is used to focus the spatial information through a convolution layer to obtain the third feature; The input end of the re-weighting module is connected to the output ends of the first direction aggregation module, the second direction aggregation module and the spatial focusing module, and is used to fuse the first feature, the second feature and the third feature to obtain the target feature.

3. The optical remote sensing image target detection method according to claim 1 or 2, characterized in that: Acquiring an optical remote sensing image dataset comprises: Collect multiple historical datasets of optical remote sensing images through cloud platforms; Analyzing each of the optical remote sensing image historical data sets, and if there is an optical remote sensing image that is not marked with a location tag, marking it with the location tag; Integrating the optical remote sensing image historical data sets into one data set to obtain the optical remote sensing image data set; Training the target detection network model using the optical remote sensing image dataset and persistently storing the trained target detection network model includes: The optical remote sensing image dataset is loaded into the cloud platform, and the target detection network model is trained in the cloud platform using the optical remote sensing image dataset, and the trained target prediction network model is persistently stored in the cloud platform.

4. The optical remote sensing image target detection method according to claim 3, characterized in that: After integrating the optical remote sensing image historical data sets into one data set to obtain the optical remote sensing image data set, the method further includes: Dividing all data in the optical remote sensing image dataset into a training set and a test set according to a preset ratio; Training the target detection network model using the optical remote sensing image dataset and persistently storing the trained target detection network model includes: The target detection network model is trained using the training set, and the detection rate of the target detection network model is tested using the test set; If the detection rate meets the preset requirement, the target detection network model is persistently stored; otherwise, the process returns to the step of training the target detection network model using the training set.

5. An optical remote sensing image target detection device, characterized in that: include: A data acquisition module, used to acquire optical remote sensing image data sets; A model building module is used to introduce an attention module based on the Yolov5 detection network model to obtain a target detection network model; wherein the attention module is used to: decompose channel attention into an encoding process of aggregating features in two different directions, a first direction and a second direction; the encoding process along the first direction retains the position information of the image features to obtain a first feature; the encoding process along the second direction captures the long-term dependency of the image features to obtain a second feature; focuses on the spatial information of the image features to obtain a third feature; and fuses the first feature, the second feature, and the third feature to obtain and output a target feature; A model training module, configured to train the target detection network model using the optical remote sensing image dataset and persistently store the trained target detection network model; A target detection module is used to deploy the trained target detection network model and perform target detection on actual optical remote sensing images; The cross-stage local structure of the Neck part in the target detection network model further includes: a residual branch; an input end of the residual branch is connected to an input end of the cross-stage local structure, and an output end of the residual branch is connected to an input end of a splicing module in the cross-stage local structure; The positioning loss in the loss function of the target detection network model is: ； ； ； in, For positioning loss; They are prediction box and label box respectively; is the Euclidean distance between the center point of the prediction box and the center point of the label box; is the minimum box diagonal distance that contains both the prediction box and the label box; are the widths of the prediction box and label box respectively; are the heights of the prediction box and the label box respectively; e is a natural constant.

6. An optical remote sensing image target detection device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the optical remote sensing image target detection method according to any one of claims 1 to 4 when executing the computer program.

7. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, which, when executed by a processor, implements the steps of the optical remote sensing image target detection method according to any one of claims 1 to 4.

Images