Remote sensing image-based oil tank detection method and oil tank detection model training method

Abstract

The invention provides a remote sensing image-based oil tank detection method and an oil tank detection model training method, which can be applied to the technical field of computer vision. The oil tank detection method based on the remote sensing image comprises the following steps: performing feature extraction on the remote sensing image containing a to-be-detected oil tank according to different scales to obtain a multi-scale feature map; performing feature enhancement processing on the multi-scale feature map to obtain position features of the to-be-detected oil tank; and determining position information of the to-be-detected oil tank based on the position features.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, specifically to the field of object detection, and more specifically to an oil tank detection method based on remote sensing images and an oil tank detection model training method. Background Art

[0002] An oil tank is a storage facility for liquid petroleum products. The petroleum products stored in it are an important part of urban energy. Accurate and timely acquisition of information such as the quantity and location of oil tanks is of great significance to the safety and development of the city.

[0003] In the process of realizing the concept of the present disclosure, there are at least the following problems in the related art: the oil tanks in the remote sensing images occupy relatively few pixels and can provide limited feature information, resulting in low accuracy of the oil tank detection results, and the possibility of missed target detections, false detections, etc. Summary of the invention

[0004] In view of the above problems, the present disclosure provides an oil tank detection method based on remote sensing images and an oil tank detection model training method.

[0005] According to a first aspect of the present disclosure, a remote sensing image-based oil tank detection method is provided, the method comprising: performing feature extraction on a remote sensing image containing an oil tank to be detected at different scales to obtain a multi-scale feature map; performing feature enhancement processing on the multi-scale feature map to obtain position features of the oil tank to be detected; and determining position information of the oil tank to be detected based on the position features.

[0006] According to an embodiment of the present disclosure, the above-mentioned feature extraction of the remote sensing image containing the oil tank to be inspected at different scales to obtain a multi-scale feature map includes: static convolution of the basic features of the oil tank in the above-mentioned remote sensing image to obtain an initial feature map; dynamic convolution of the abstract features of the oil tank in the above-mentioned initial feature map to obtain a multi-scale feature map.

[0007] According to an embodiment of the present disclosure, the above-mentioned dynamic convolution of the oil tank abstract features in the above-mentioned initial feature map to obtain a multi-scale feature map includes: using multiple first convolution layers to dynamically convolve the oil tank abstract features in the above-mentioned initial feature map to obtain an initial multi-scale feature map; using a second convolution layer to downsample the above-mentioned initial multi-scale feature map to obtain the above-mentioned multi-scale feature map; wherein, the attention coefficient of the convolution kernel space of each of the multiple dynamic convolution kernels in the above-mentioned first convolution layer, the attention coefficient of the convolution kernel input channel, the attention coefficient of the convolution kernel output channel and the attention coefficient of the number of convolution kernels are obtained through training.

[0008] According to an embodiment of the present disclosure, the above-mentioned feature enhancement processing is performed on the above-mentioned multi-scale feature map to obtain the position features of the above-mentioned oil tanks to be detected, including: inputting the above-mentioned multi-scale feature map into the encoder to obtain the initial position features; initializing the anchor frame information of each of the above-mentioned oil tanks to be detected based on the above-mentioned initial position features to obtain prior position information; inputting the above-mentioned position prior information and the above-mentioned multi-scale feature map into the decoder, and outputting the above-mentioned position features.

[0009] According to an embodiment of the present disclosure, the multi-scale feature map is input into an encoder to obtain an initial position feature, including: performing a dimensional transformation on the multi-scale feature map to obtain a one-dimensional multi-scale feature; and embedding the spatial position information of the remote sensing image into the one-dimensional multi-scale feature to obtain the initial position feature.

[0010] A second aspect of the present disclosure provides a remote sensing image-based oil tank detection model training method, comprising: inputting a sample remote sensing image into an initial oil tank detection model so that the initial oil tank detection model performs feature extraction on the sample remote sensing image at different scales to obtain a multi-scale feature map; performing feature enhancement processing on the multi-scale feature map to obtain position features of the oil tank in the sample remote sensing image; and determining position information of the oil tank based on the position features; determining a loss value of the initial oil tank detection model based on a sample label of the sample remote sensing image and the position information; and repeatedly performing the above operations until the loss value meets a preset condition to obtain an oil tank detection model.

[0011] According to an embodiment of the present disclosure, the loss value of the initial oil tank detection model is determined based on the sample label of the sample remote sensing image and the position information, including: performing bounding box modeling on the predicted position information based on a two-dimensional Gaussian distribution to obtain multiple predicted bounding boxes; for each of the predicted bounding boxes, calculating the Gaussian similarity between the predicted bounding box and the standard bounding box, wherein the standard bounding box is a sample label corresponding to the sample remote sensing image input into the initial oil tank detection model; and determining the loss value of the initial oil tank detection model according to the Gaussian similarity.

[0012] According to an embodiment of the present disclosure, the method further includes: when there are multiple predicted bounding boxes, matching the multiple predicted bounding boxes with multiple standard bounding boxes to calculate the Gaussian similarity, wherein the multiple standard bounding boxes are the sample labels.

[0013] According to an embodiment of the present disclosure, the above-mentioned oil tank detection model includes a first convolution layer and a second convolution layer; the above-mentioned first convolution layer includes multiple dynamic convolution kernels; the above-mentioned method also includes: using the above-mentioned sample remote sensing image to train the convolution kernel space, convolution kernel input channel, convolution kernel output channel and number of convolution kernels of each of the multiple dynamic convolution kernels, so as to use the multiple above-mentioned first convolution layers to dynamically convolve the abstract features of the oil tank in the initial feature map to obtain the initial multi-scale feature map; using the above-mentioned second convolution layer to downsample the above-mentioned initial multi-scale feature map to obtain the above-mentioned multi-scale feature map; wherein the above-mentioned initial feature map is obtained by static convolution of the basic features of the oil tank in the above-mentioned sample remote sensing image.

[0014] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0015] According to an embodiment of the present disclosure, before feature enhancement is performed on a remote sensing image, feature extraction is first performed on the remote sensing image at different sizes to capture the features of the oil tank to be detected in the remote sensing image at different resolutions, and the features of multiple size levels of the oil tank to be detected are retained in a multi-size feature map. By performing feature enhancement processing on the multi-scale feature map, the position features of the oil tank to be detected are obtained, so that the features of the oil tank to be detected can be highlighted while suppressing irrelevant background information, thereby improving the recognition accuracy of the oil tank to be detected, and thus determining the position information of the oil tank to be detected based on the position features, thereby reducing missed detections, false detections, etc. of the oil tank to be detected, and improving the accuracy of the position information of the oil tank to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above contents and other purposes, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0017] Figure 1 The application scenario diagram of the oil tank detection method based on remote sensing images according to an embodiment of the present disclosure is schematically shown;

[0018] Figure 2 The flowchart of the oil tank detection method based on remote sensing images according to the embodiment of the present disclosure is schematically shown;

[0019] Figure 3 A schematic diagram of a network structure for obtaining a multi-scale feature map according to an embodiment of the present disclosure is shown;

[0020] Figure 4 The network structure diagram of the convolution layer according to the embodiment of the present disclosure is schematically shown;

[0021] Figure 5 A flowchart of a method for training an oil tank detection model based on remote sensing images according to an embodiment of the present disclosure is schematically shown;

[0022] Figure 6 The structure diagram of the initial oil tank detection model according to the embodiment of the present disclosure is schematically shown;

[0023] Figure 7 A schematic diagram of training an initial oil tank detection model according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 8 A diagram schematically shows a comparison of the accuracy of the oil tank detection model according to an embodiment of the present disclosure and other detection models;

[0025] Fig. 9 A block diagram of an electronic device suitable for implementing an oil tank detection method based on remote sensing images and an oil tank detection model training method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0026] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0027] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0028] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0029] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0030] In the technical solution of the present disclosure, the information involved (including but not limited to image information, device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0032] In remote sensing images, the size of oil tank targets is usually small relative to the size of the entire image. Taking common high-resolution remote sensing images as an example, an image may cover an area of ​​several square kilometers, and an oil tank may be just a small object in it. Neural networks mainly extract features by analyzing pixels in images. When the proportion of oil tank pixels is small, the network can obtain relatively limited feature information about the oil tank itself. For example, for an oil tank detection model based on a convolutional neural network, the convolution layer extracts features on the image by sliding windows. Due to the small number of oil tank pixels, after several convolution operations, the number of effective neuron activations used to represent the oil tank features may be very small, making it difficult to fully characterize the complete features of the oil tank.

[0033] In deep learning, the model usually needs to learn the characteristic patterns that are unique to the target and can be distinguished from other objects. Since the oil tank has few pixels, these pixels can be easily masked by other background information or features of similar objects in the feature space of the entire image. For example, in a remote sensing image containing buildings, roads, and oil tanks, some simple shape features of the oil tank (such as a circular outline) may be confused with features such as the circular structure of the building or the circular intersection of the road at the low-level features, making it difficult for the model to accurately identify the oil tank.

[0034] In the related art, simple neural networks have certain limitations. For example, shallow neural networks or networks with small receptive fields cannot effectively cover the entire oil tank target and its surrounding contextual information. The receptive field refers to the area of ​​the input image that a neuron in the neural network can perceive. For the detection of oil tanks, it is necessary not only to pay attention to the characteristics of the oil tanks themselves, but also to consider the environmental information around them, such as the protective levees around the oil tanks, the pipes connecting the oil tanks, etc. If the receptive field of the network is too small, only the local features of the oil tanks can be obtained, while ignoring these important contextual information. For example, a neural network with only 3x3 convolution kernels and a small number of layers has a limited receptive field range and may not be able to capture the oil tank and its surrounding related facilities at the same time, and thus cannot accurately determine whether the target is an oil tank.

[0035] In summary, in the process of detecting remote sensing images, due to the small number of target pixels of oil tanks and the limited feature information provided, when the feature extraction of the neural network is insufficient, the oil tank target may not be recognized. Especially in complex background environments, such as when there are a large number of buildings or vegetation with similar textures around the oil tank, the network may ignore the existence of the oil tank. In addition, if the neural network does not take into account some special features of the oil tank, it is easy to miss the detection, such as the shadow shape at a specific angle, the connection method with the pipeline, etc. For example, in a remote sensing image taken under low light conditions, the shadow part of the oil tank may be an important recognition feature, but a simple neural network may not learn this feature, thereby missing the oil tank.

[0036] On the other hand, limited feature information may cause the model to mistake other objects for oil tanks. For example, some buildings or water storage facilities with a round appearance are similar in shape to oil tanks. When the model cannot obtain enough distinguishing features, it will make a wrong judgment. Alternatively, the lack of effective use of contextual information can also lead to false detections. For example, without considering the spatial relationship between the oil tank and the surrounding facilities, the model may mistake a single circular object as an oil tank, when in fact it may be another object unrelated to the oil tank. The combined effect of these missed detections and false detections ultimately reduces the accuracy of the oil tank detection results.

[0037] Based on the above problems, an embodiment of the present disclosure provides an oil tank detection method based on remote sensing images, which performs feature extraction on a remote sensing image containing an oil tank to be detected at different scales to obtain a multi-scale feature map; performs feature enhancement processing on the multi-scale feature map to obtain position features of the oil tank to be detected; and determines the position information of the oil tank to be detected based on the position features.

[0038] Figure 1 The application scenario diagram of the oil tank detection method based on remote sensing images according to an embodiment of the present disclosure is schematically shown.

[0039] like Figure 1 As shown, the application scenario 100 according to this embodiment may include: Figure 1 As shown, the application scenario 100 according to this embodiment may include an image acquisition device 101, a data receiving device 102, a network 103 and a server 104. The network 103 is used to provide a medium for a communication link between the data receiving device 102 and the server 104. The network 103 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0040] The image acquisition device 101 is used to acquire remote sensing images, and the image acquisition device 101 may include: aerial remote sensing, satellite remote sensing, etc. The data receiving device 102 is used to receive the remote sensing images acquired by the image acquisition device 101, and the data receiving device 102 may include a ground base station, etc.

[0041] The server 104 may be a server that provides various services, for example, processing and analyzing remote sensing images received through the network 103. The server 104 may be a cloud server or a terminal server so as to retrieve the captured remote sensing images in real time, which is not specifically limited in the embodiment of the present invention. It should be noted that the oil tank detection method based on remote sensing images and the oil tank detection model training method based on remote sensing images provided in the embodiment of the present disclosure can generally be executed by the server 104. The oil tank detection method based on remote sensing images and the oil tank detection model training method based on remote sensing images provided in the embodiment of the present disclosure can also be executed by a server different from the server 104 or a server cluster. Accordingly, the electronic device provided in the embodiment of the present disclosure can generally be arranged in the server 104, or arranged in a server different from the server 104 or a server cluster.

[0042] It should be understood that Figure 1 The number of image acquisition devices, data receiving devices, networks and servers in the embodiment is only for illustration. Any number of image acquisition devices, data receiving devices, networks and servers may be provided according to the implementation requirements.

[0043] The following will be based on Figure 1 The scene described by Figure 2~Figure 3 The oil tank detection method based on remote sensing images of the disclosed embodiment is described in detail.

[0044] Figure 2 The flowchart of the oil tank detection method based on remote sensing images according to an embodiment of the present disclosure is schematically shown.

[0045] like Figure 2 As shown, the oil tank detection method based on remote sensing images of this embodiment includes operations S210 to S230.

[0046] In operation S210, feature extraction is performed on the remote sensing image containing the oil tank to be inspected according to different scales to obtain a multi-scale feature map.

[0047] In operation S220, feature enhancement processing is performed on the multi-scale feature map to obtain position features of the oil tank to be detected.

[0048] In operation S230, location information of the oil tank to be inspected is determined based on the location feature.

[0049] According to an embodiment of the present disclosure, a remote sensing image is an image taken by a remote sensing satellite or other equipment of an oil tank to be inspected. The scale is the size of the spatial resolution of the remote sensing image. The larger the scale, the fewer details of the object presented on the remote sensing image, and the wider the coverage; the smaller the scale, the richer the details presented on the remote sensing image, but the smaller the coverage. For example, for a remote sensing image containing an oil tank, if it is observed at a large scale, you may only be able to see the approximate location of the oil tank and its relationship with the surrounding large area; if it is observed at a small scale, you can see some details on the oil tank, such as the oil inlet and outlet, the texture of the tank surface, etc.

[0050] According to the embodiments of the present disclosure, for the same remote sensing image, different scales can be obtained by changing the resolution of the image. Specifically, a large-scale image can be obtained by downsampling the remote sensing image to reduce the resolution of the image; a small-scale image can be obtained by enlarging a local area of ​​the remote sensing image or performing a multi-scale ...

[0051] According to an embodiment of the present disclosure, feature extraction of remote sensing images containing oil tanks to be inspected at different scales can be achieved by using different layers in a convolutional neural network. In each layer of the convolutional neural network, a convolution operation can be performed on the remote sensing image in the form of a sliding window to control the receptive field of view of the remote sensing image, thereby obtaining a multi-scale feature map. Feature extraction of remote sensing images containing oil tanks to be inspected at different scales is performed to obtain macroscopic features such as the overall shape and position of the oil tanks to be inspected, as well as detailed features such as the surface texture and oil inlet and outlet of the oil tanks to be inspected, thereby providing rich features for oil tank detection.

[0052] According to an embodiment of the present disclosure, a remote sensing image may include one or more oil tanks, and one or more oil tanks in the remote sensing image are oil tanks to be detected. The position features of the oil tanks to be detected are feature matrices obtained after feature enhancement processing is performed on the multi-scale feature map. After the multi-scale feature map is obtained, the multi-scale feature map can be input into a coding and decoding network for feature enhancement processing, wherein the coding and decoding network can be a Transformer network. By performing feature enhancement processing on the multi-scale feature map, a richer and more accurate feature representation is obtained, thereby improving the target detection performance.

[0053] According to an embodiment of the present disclosure, the location information may be the geographical location of the oil tank to be detected, and the location information of the oil tank to be detected represented by the location feature may be identified based on the feature data corresponding to each pixel point in the location feature. Specifically, the location feature may be divided into multiple subsequences, each subsequence corresponds to an attention head, and each attention head is used to calculate the attention weight and attention distribution of the corresponding subsequence to obtain the final attention distribution, and then the location information of the oil tank is determined based on the final attention distribution and the information that can represent the spatial position of the pixel point.

[0054] According to an embodiment of the present disclosure, before feature enhancement is performed on a remote sensing image, feature extraction is first performed on the remote sensing image at different sizes to capture the features of the oil tank to be detected in the remote sensing image at different resolutions, and the features of multiple size levels of the oil tank to be detected are retained in a multi-size feature map. By performing feature enhancement processing on the multi-scale feature map, the position features of the oil tank to be detected are obtained, so that the features of the oil tank to be detected can be highlighted while suppressing irrelevant background information, thereby improving the recognition accuracy of the oil tank to be detected, and thus determining the position information of the oil tank to be detected based on the position features, thereby reducing missed detections, false detections, etc. of the oil tank to be detected, and improving the accuracy of the position information of the oil tank to be detected.

[0055] According to the embodiments of the present disclosure, in order to obtain more oil tank information, a prediction head for identifying the category information of the oil tank to be detected can be trained to determine the type information of the oil tank to be detected in the remote sensing image according to the position feature, wherein the type information may include vertical oil storage tanks, horizontal oil storage tanks, etc. Since the position feature enhances the multi-scale feature map, and the multi-scale feature map is obtained by extracting features from the remote sensing image containing the oil tank to be detected at different scales, the multi-scale feature map retains rich oil tank features and contextual features of the oil tank. Therefore, the type information of the oil tank to be detected is determined using the position feature, thereby improving the accuracy of the type information.

[0056] According to an embodiment of the present disclosure, feature extraction is performed on a remote sensing image containing an oil tank to be detected at different scales to obtain a multi-scale feature map, including: static convolution of basic features of the oil tank in the remote sensing image to obtain an initial feature map; dynamic convolution of abstract features of the oil tank in the initial feature map to obtain a multi-scale feature map.

[0057] According to an embodiment of the present disclosure, the basic features of the oil tank characterize the features of the oil tank to be detected, and the basic features of the oil tank may include edge features, texture features, shape features, etc. The abstract features of the oil tank characterize the features of the relative relationship between the oil tank to be detected and the background, and the abstract features of the oil tank may include features that characterize the contextual information of the oil tank to be detected. The basic features of the oil tank focus on the details and local information of the oil tank to be detected in the remote sensing image, and the abstract features of the oil tank focus on the overall content and semantic understanding of the image.

[0058] According to the embodiments of the present disclosure, for the relatively stable basic features of the oil tank, the basic features of the oil tank in the remote sensing image are extracted by static convolution to obtain an initial feature map, so as to improve the stability of the operation of extracting the basic features of the oil tank through the fixedness of static convolution. For the abstract features of the oil tank that are more dependent on contextual information, the abstract features of the oil tank in the initial feature map are extracted by dynamic convolution to obtain a multi-scale feature map, so as to more accurately identify and distinguish the oil tank from other objects and improve the accuracy of oil tank detection.

[0059] According to the embodiments of the present disclosure, static convolution is used to extract the basic features of the oil tank with relatively stable features; dynamic convolution is used to extract the abstract features of the oil tank whose expression is more dependent on contextual information. The mixed use of dynamic convolution and static convolution improves the ability to capture different features while maintaining computational efficiency, thereby improving the accuracy of the detection results.

[0060] According to an embodiment of the present disclosure, dynamic convolution is performed on the abstract features of the oil tank in the initial feature map to obtain a multi-scale feature map, including: using multiple first convolution layers to dynamically convolve the abstract features of the oil tank in the initial feature map to obtain an initial multi-scale feature map; using a second convolution layer to downsample the initial multi-scale feature map to obtain a multi-scale feature map; wherein the attention coefficients of the convolution kernel space, the attention coefficients of the convolution kernel input channels, the attention coefficients of the convolution kernel output channels, and the attention coefficients of the number of convolution kernels of each of the multiple dynamic convolution kernels in the multiple first convolution layers are obtained through training.

[0061] According to an embodiment of the present disclosure, the first convolution layer is used to extract abstract features of the oil tank in the initial feature map. The attention coefficient of the convolution kernel space of the dynamic convolution kernel in each first convolution layer, the attention coefficient of the convolution kernel input channel, the attention coefficient of the convolution kernel output channel and the attention coefficient of the number of convolution kernels are all obtained through training, so that the dynamic convolution kernels in different first convolution layers can dynamically adjust parameters according to different initial feature maps, thereby enhancing the adaptability to complex semantics and spatial changes in the dynamic convolution process and improving the accuracy of position information.

[0062] According to an embodiment of the present disclosure, the step size of the dynamic convolution kernel in the second convolution layer is greater than the step size of the dynamic convolution kernel in the first convolution layer. The second convolution layer is used to downsample the initial multi-scale feature map to reduce the resolution of the initial multi-scale feature map while maintaining a high semantic expression capability, thereby improving the computational efficiency of detecting the oil tank to be detected. By giving dynamic properties to the parameters of the four dimensions of the convolution kernel, it is possible to dynamically adjust the size and shape of the dynamic convolution kernel to better adapt to changes in input features.

[0063] According to an embodiment of the present disclosure, the attention coefficient of the convolution kernel space dimension, the attention coefficient of the convolution kernel input channel dimension, the attention coefficient of the convolution kernel output channel dimension and the attention coefficient of the number of convolution kernels are introduced into the multiple first convolution layers, so that the dynamic convolution is expressed as formula (1):

[0064] (1);

[0065] in, represents the attention coefficient of the convolution kernel space dimension, Represents the attention coefficient of the convolution kernel input channel dimension, Represents the attention coefficient of the convolution kernel output channel dimension, The attention coefficient representing the number of convolution kernels, represents the weight parameter of the convolution kernel, represents the initial feature map, where i=1...n; is the matrix multiplication symbol. These four attention coefficients are complementary.

[0066] According to the embodiments of the present disclosure, the dynamic convolution operation of the first convolutional layer makes the convolution operation have differences in different dimensions on the features of the input remote sensing image, providing the model with richer scale and context information, improving the accuracy of the location information, and will not increase the model with too many parameters, thereby improving the detection efficiency.

[0067] Figure 3 A network structure diagram for obtaining a multi-scale feature map according to an embodiment of the present disclosure is schematically shown.

[0068] like Figure 3 As shown, the network structure for obtaining the multi-scale feature map includes a preprocessing stage S1, a static convolution stage S2, a dynamic convolution stage S3 and a downsampling stage S4, wherein the preprocessing stage S1 may include an initial convolution layer 301 and a maximum pooling layer 302; the static convolution stage S2 includes a static convolution layer 303; the dynamic convolution stage S3 includes a first convolution layer a304, a first convolution layer b305, and a first convolution layer c306, and the downsampling stage S4 includes a second convolution layer 307.

[0069] In the first convolution layer a304, the first convolution layer b305, the first convolution layer c306, and the second convolution layer 307, the number of channels of the output features of each layer can be uniformly configured to be 256. In the first convolution layer a304, the first convolution layer b305, and the first convolution layer c306, the dynamic convolution kernel step size of each layer can be configured to be 1. The step size of the convolution kernel in the second convolution layer 307 can be configured to be 2, so as to achieve effective downsampling by further halving the spatial resolution of the feature map while maintaining a high semantic expression capability.

[0070] A comparative experiment was conducted on two schemes: setting only the first convolution layer a304 as dynamic convolution, and setting the first convolution layer a304, the first convolution layer b305, and the first convolution layer c306 for dynamic convolution. The first convolution layer a304, the first convolution layer b305, and the first convolution layer c306 were all set to perform dynamic convolution, which resulted in a higher improvement in the accuracy of sample remote sensing images of dense small target types.

[0071] Figure 4 The network structure diagram of the convolutional layer according to an embodiment of the present disclosure is schematically shown.

[0072] like Figure 4 As shown, the first structure includes a convolution kernel 401 with a convolution kernel dimension of m1*m1 and a number of output channels or convolution kernels of p1, and a convolution kernel 402 with a convolution kernel dimension of m2*m2 and a number of output channels or convolution kernels of p1. The static convolution kernel 303 can be composed of multiple first structures.

[0073] The second structure includes a dynamic convolution kernel 403 with a convolution kernel dimension of n1*n1 and a number of output channels or convolution kernels of p2; and a dynamic convolution kernel 404 with a convolution kernel dimension of m2*m2 and a number of output channels or convolution kernels of p2; the first convolution layer a304, the first convolution layer b305, and the first convolution layer c306 can all be composed of multiple dynamic convolution structures of the second structure. For the second structure of the same dynamic convolution layer or different dynamic convolution layers, the values ​​of p2, n1, and n2 can be different.

[0074] According to an embodiment of the present disclosure, feature enhancement processing is performed on a multi-scale feature map to obtain position features of the oil tank to be detected, including: inputting the multi-scale feature map into an encoder to obtain initial position features; initializing anchor frame information of each oil tank to be detected based on the initial position features to obtain prior position information; inputting the position prior information and the multi-scale feature map into a decoder to output position features.

[0075] According to an embodiment of the present disclosure, a multi-scale feature map is input into an encoder, and the encoder extracts features through a series of convolutional layers and pooling layers, and gradually reduces the spatial dimension while increasing the depth of the feature to obtain an initial position feature. The initial position feature contains preliminary spatial information of the oil tank to be detected. The anchor frame information is used to characterize the preset bounding box of the position of the oil tank to be detected. The anchor frame information may include the center point coordinates of the preset bounding box, and the width and height of the bounding box. The anchor frame information of the oil tank to be detected is initialized using the initial position feature to adjust the size and position of the anchor frame to obtain prior position information.

[0076] According to an embodiment of the present disclosure, the position prior information and the multi-scale feature map are input into the decoder. The decoder can gradually restore the spatial resolution of the multi-scale feature map through upsampling and convolution operations, while refining the position information and outputting more precise oil tank position features, so as to more accurately predict the position information of the oil tank to be detected.

[0077] According to an embodiment of the present disclosure, the encoder extracts initial position features from a multi-scale feature map to obtain initial position features. The multi-scale feature map contains visual information at different levels, such as the edge, texture, shape, contextual information, etc. of the oil tank to be detected. Based on the initial position features, the anchor frame information is initialized, which helps the model to locate the approximate position of the oil tank to be detected in the image. The initialized position information and the feature information in the multi-scale feature map are input into the decoder. The decoder refines the position information through an iterative update process and outputs more accurate position features. In complex scenes or when the target is small, the accuracy of identifying the oil tank to be detected can be significantly improved.

[0078] According to an embodiment of the present disclosure, a multi-scale feature map is input into an encoder to obtain an initial position feature, including: performing a dimensional transformation on the multi-scale feature map to obtain a one-dimensional multi-scale feature; and embedding the spatial position information of the remote sensing image into the one-dimensional multi-scale feature to obtain the initial position feature.

[0079] According to the embodiments of the present disclosure, in order to ensure accurate identification of position features in a multi-scale feature map and reduce the degree of feature weakening, after obtaining the multi-scale feature map, the multi-scale feature map may also be transformed in dimension. Specifically, the multi-scale feature map may be flattened to obtain one-dimensional multi-scale features. Specifically, the image flattening process mainly involves two steps: edge detection and flattening. First, the edge of the multi-scale feature map is extracted by an edge detection algorithm to determine the lines and areas in the image that need to be flattened. The edge detection algorithm may include the Sobel operator and the Canny edge detection algorithm. The extracted edges are flattened using a flattening filter algorithm (such as a Gaussian filter or a mean filter) to make the image edge of the oil tank to be detected softer to eliminate the jagged effect.

[0080] According to an embodiment of the present disclosure, in order to better understand the position of the oil tank to be detected in the one-dimensional multi-scale feature, the one-dimensional multi-scale feature is processed by using a position embedding method. Specifically, the spatial position information is encoded into the one-dimensional multi-scale feature to represent the position information of one or more pixels by adding an additional dimension to the multi-scale feature map, wherein the spatial position information may be information characterizing the geographical location of the pixel, for example, the spatial position information may be the longitude and latitude of one or more pixels. The specific mapping relationship between the spatial position information and the one-dimensional multi-scale feature can be configured based on the specific detection requirements of the oil tank.

[0081] The second aspect of the present disclosure provides a method for training an oil tank detection model based on remote sensing images, including: inputting a sample remote sensing image into an initial oil tank detection model so that the initial oil tank detection model extracts features from the sample remote sensing image at different scales to obtain a multi-scale feature map; performing feature enhancement processing on the multi-scale feature map to obtain position features of the oil tank in the sample remote sensing image; and determining the position information of the oil tank based on the position features. Based on the sample label and position information of the sample remote sensing image, determine the loss value of the initial oil tank detection model. Repeat the operation until the loss value meets the preset conditions to obtain the oil tank detection model.

[0082] Figure 5 The flowchart of the oil tank detection model training method based on remote sensing images according to an embodiment of the present disclosure is schematically shown.

[0083] like Figure 5 As shown, the oil tank detection method based on remote sensing images of this embodiment includes operations S510 to S540.

[0084] In operation S510, the sample remote sensing image is input into the initial oil tank detection model so that the initial oil tank detection model extracts features from the sample remote sensing image at different scales to obtain a multi-scale feature map; performs feature enhancement processing on the multi-scale feature map to obtain position features of the oil tank in the sample remote sensing image; and determines the position information of the oil tank based on the position features.

[0085] In operation S520, a loss value of the initial oil tank detection model is determined based on the sample label and position information of the sample remote sensing image.

[0086] In operation S530, it is determined whether the loss value satisfies a preset condition.

[0087] If the loss value satisfies the preset condition, operation S540 is performed; otherwise, operation S510 is performed.

[0088] In operation S540, the initial oil tank detection model is used as the oil tank detection model.

[0089] According to an embodiment of the present disclosure, the initial oil tank detection model represents an oil tank detection model that has not been trained. The sample remote sensing image is a sample image used to train the initial oil tank detection model, and the sample remote sensing image can be selected from three public data sets. The sample remote sensing image is input into the initial oil tank detection model, and the initial oil tank detection model extracts features of the sample remote sensing image according to different scales to obtain a multi-scale feature map; the multi-scale feature map is subjected to feature enhancement processing to obtain the position features of the oil tank in the sample remote sensing image; and based on the position features, the position information of the oil tank is determined.

[0090] According to an embodiment of the present disclosure, the sample label of the sample remote sensing image may be information characterizing the oil tank to be detected relative to the sample remote sensing image. For example, the sample label may include the coordinates of the center point of the bounding box where the oil tank to be detected is located, and the width and height of the bounding box. The loss value of the initial oil tank detection model is calculated based on the degree of similarity between the sample label and the position information of the sample remote sensing image.

[0091] According to an embodiment of the present disclosure, the preset condition may be that the loss value is less than or equal to the preset value. The initial oil tank detection model is trained by selecting different sample remote sensing images to reduce the loss value until the loss value is less than or equal to the preset value, and the initial oil tank detection model whose loss value meets the preset condition is determined as the oil tank detection model.

[0092] According to an embodiment of the present disclosure, before feature enhancement is performed on a sample remote sensing image, feature extraction is first performed on the remote sensing image at different sizes to obtain macro features such as the shape and position of the oil tank to be detected in the sample remote sensing image from a small-size feature map, as well as detail features such as surface texture and oil inlet and outlet from a large-size feature map. Features of different sizes are retained in a multi-size feature map, providing richer feature information for subsequent feature enhancement and determination of position information operations. Feature enhancement is performed on the multi-scale feature map to obtain rich oil tank information and contextual information of the oil tank in the multi-scale feature map, thereby improving the accuracy of the oil tank detection model.

[0093] Figure 6 The structure diagram of the initial oil tank detection model according to an embodiment of the present disclosure is schematically shown.

[0094] like Figure 6 As shown, the initial oil tank detection model includes a backbone network 601, an encoding and decoding network 602, a first prediction head 603 and a second prediction head 604. The sample remote sensing image is input into the initial oil tank detection model. Since the backbone network 601 extracts features of the sample remote sensing image according to different scales, a multi-scale feature map is obtained; the encoding and decoding network 602 performs feature enhancement processing on the multi-scale feature map to obtain the position features of the oil tank in the sample remote sensing image; the first prediction head 603 determines the position information of the oil tank based on the position features; the second prediction head 604 determines the type information of the oil tank based on the position features. When training the category information of the oil tank to be detected, there is no need to initialize the predicted category information output by the initial oil tank detection model. The loss calculation can be performed by configuring it as a learnable parameter to optimize the model.

[0095] According to the embodiments of the present disclosure, during training, in order to improve the accuracy of the oil tank detection model and expand its application to other remote sensing images with uniform resolution, the idea of ​​transfer learning can be adopted. First, a mixed data set with a large sample size is used for training. After the model converges, the model weights are taken out as the initial oil tank detection model weights, and then training is performed again using a self-made data set.

[0096] Figure 7 A schematic diagram of training an initial oil tank detection model according to an embodiment of the present disclosure is schematically shown.

[0097] like Figure 7As shown, the original oil tank detection model 701 is trained with the first sample of the mixed data set, and when the model converges, the model weight is taken out, and the model weight is injected into the initial oil tank detection model 702, and the initial oil tank detection model 702 is trained with the second sample of the self-made training sample set to train the model weight, and the initial oil tank detection model 702 whose loss value meets the preset conditions is used as the oil tank detection model to predict the location information. According to an embodiment of the present disclosure, the self-made training sample set may include a small number of sample remote sensing images without oil tanks and a large number of sample remote sensing images containing oil tanks. For example, in a training sample set containing 5344 sample remote sensing images, 800 sample remote sensing images without oil tanks can be added as negative samples of the training sample set.

[0098] According to the embodiments of the present disclosure, in order to improve the training efficiency, the sample remote sensing images can be adjusted to a uniform size of 800×800 and integrated into a data set. Among all the sample remote sensing images, 70% of the images can be used for training, 20% of the sample remote sensing images can be used for verification, and the remaining 10% of the sample remote sensing images can be used for testing. In the process, annotation tools can be used to screen the sample labels of the sample remote sensing images and correct some errors.

[0099] According to an embodiment of the present disclosure, an oil tank detection model includes a first convolution layer and a second convolution layer; the first convolution layer includes multiple dynamic convolution kernels; the method also includes: using sample remote sensing images to train the attention coefficients of the convolution kernel space of each of the multiple dynamic convolution kernels, the attention coefficients of the convolution kernel input channels, the attention coefficients of the convolution kernel output channels, and the attention coefficients of the number of convolution kernels, so as to use multiple first convolution layers to convolve the abstract features of the oil tanks in the initial feature map to obtain an initial multi-scale feature map; using the second convolution layer to downsample the initial multi-scale feature map to obtain a multi-scale feature map; wherein the initial feature map is obtained by static convolution of the basic features of the oil tanks in the sample remote sensing image.

[0100] According to the embodiments of the present disclosure, the internal parameters and the adding position of the dynamic convolution kernel can be changed. The parameter N in the dynamic convolution kernel represents the number of dynamic convolution kernels used, which is used to control the balance between the size and accuracy of the oil tank detection model. The dynamic convolution kernel is optimized for the amount of calculation. It is selected to set only one first convolution layer as a dynamic convolution, and to set multiple first convolution layers for dynamic convolution. The two schemes are compared in experiments. Multiple first convolution layers are set to perform dynamic convolution, which results in a higher improvement in the accuracy of sample remote sensing images of dense small target types. Setting the first convolution layer for dynamic convolution can better enhance the information of the details of the target object, so that the extracted multi-scale features can better distinguish the difference between the oil tank to be detected and the background, thereby improving the accuracy of the oil tank detection model.

[0101] According to the embodiments of the present disclosure, the attention coefficient of the spatial dimension of the convolution kernel, the attention coefficient of the input channel dimension of the convolution kernel, the attention coefficient of the output channel dimension of the convolution kernel and the attention coefficient of the number of convolution kernels are introduced into the first convolution layer. The four types of attention make the convolution operation have differences in different dimensions on the features of the input remote sensing image, provide the model with richer scale and context information, can bring higher improvement to the detection accuracy of the model, and will not increase the model with too many parameters, thereby improving the detection efficiency of the oil tank detection model.

[0102] According to an embodiment of the present disclosure, the receptive field of the convolution operation is adaptively adjusted through dynamic convolution, so that the network can better capture the features of small targets at the low-level feature layer, thereby enhancing the detection performance of the oil tank detection model.

[0103] According to an embodiment of the present disclosure, based on the sample label and position information of the sample remote sensing image, the loss value of the initial oil tank detection model is determined, including: performing bounding box modeling on the predicted position information based on a two-dimensional Gaussian distribution to obtain multiple predicted bounding boxes; for each predicted bounding box, calculating the Gaussian similarity between the predicted bounding box and the standard bounding box, wherein the standard bounding box is the sample label corresponding to the sample remote sensing image input into the initial oil tank detection model; and determining the loss value of the initial oil tank detection model according to the Gaussian similarity.

[0104] According to an embodiment of the present disclosure, after the initial oil tank detection model outputs the position information, the position information can be modeled into a bounding box based on a two-dimensional Gaussian distribution, and the Gaussian similarity between the predicted bounding box and the standard bounding box can be calculated, wherein the position feature is modeled into a bounding box based on a two-dimensional Gaussian distribution. The bounding box can be expressed using formula (2) and formula (3):

[0105] (2);

[0106] and (3);

[0107] in, Represents the coordinates of the center point of the predicted bounding box, Represents the horizontal coordinate of the center point of the predicted bounding box, Represents the ordinate of the center point of the predicted bounding box; Represents the size of the predicted bounding box, w is the width of the predicted bounding box; h is the height of the predicted bounding box.

[0108] According to an embodiment of the present disclosure, for each predicted bounding box, the Gaussian similarity between the predicted bounding box and the standard bounding box is calculated, and the calculation process method can be expressed by formula (4) as follows:

[0109] (4);

[0110] in, A metric that represents the Gaussian similarity between the predicted bounding box and the standard bounding box. represents the predicted bounding box, represents the standard bounding box, Represents the horizontal coordinate of the center point of the standard bounding box, Represents the ordinate of the center point of the standard bounding box, represents the width of the standard bounding box, represents the height of the standard bounding box, Represents the horizontal coordinate of the center point of the predicted bounding box, Represents the ordinate of the center point of the predicted bounding box, represents the width of the predicted bounding box, Represents the height of the predicted bounding box.

[0111] According to an embodiment of the present disclosure, the Gaussian similarity between the predicted bounding box and the standard bounding box is calculated based on the normalized Wasserstein distance (NWD), so as to determine the loss value of the initial oil tank detection model based on the Gaussian similarity, wherein the loss value of the initial oil tank detection model can be calculated by formula (5):

[0112] (5);

[0113] in, Represents the loss value of the initial oil tank detection model; represents the Gaussian similarity between the predicted bounding box and the standard bounding box, which can be calculated by formula (6):

[0114] (6);

[0115] Where c is a constant.

[0116] According to the embodiments of the present disclosure, the scale insensitivity and boundary fuzziness of NWD are utilized to make the oil tank detection model more robust to the position offset of small targets, retain more correctly predicted bounding boxes, reduce the missed detection probability of the oil tank detection model, and improve the recall rate of the oil tank detection model.

[0117] According to the embodiments of the present disclosure, by performing bounding box modeling on position features based on two-dimensional Gaussian distribution, the spatial distribution characteristics of the model to be detected can be effectively captured, and the Gaussian similarity of the bounding boxes calculated using the normalized distance allows the model to evaluate the similarity between bounding boxes in a continuous and scale-invariant manner, thereby improving the prediction accuracy of the initial oil tank detection model for position information. Especially when processing small targets or remote sensing images in complex backgrounds, the loss value of the initial oil tank detection model determined by Gaussian similarity can more accurately optimize the model parameters of the initial oil tank detection model during the training process, thereby accurately outputting the position information and type information of the oil tank to be detected in practical applications.

[0118] According to an embodiment of the present disclosure, the method further includes: when there are multiple predicted bounding boxes, matching the multiple predicted bounding boxes with multiple standard bounding boxes to calculate Gaussian similarity, wherein the multiple standard bounding boxes are sample labels.

[0119] According to the embodiments of the present disclosure, since there may be multiple predicted bounding boxes, a pre-step can be set when calculating the loss value to match the predicted bounding box with the standard bounding box, and the Hungarian matching algorithm can be used to match the predicted bounding box with the standard bounding box. The corresponding relationship between the multiple predicted bounding boxes and the multiple standard bounding boxes is determined by using the allocation method that minimizes the "sum of losses".

[0120] According to an embodiment of the present disclosure, in order to more intuitively demonstrate the ability of the oil tank detection model and other detection models to detect dense, small-target objects in remote sensing images, four representative samples are selected from the test set for prediction.

[0121] Figure 8 A comparison chart of the accuracy of the oil tank detection model according to an embodiment of the present disclosure and other detection models is schematically shown.

[0122] like Figure 8 As shown, images A1, B1, C1, and D1 are the original images of remote sensing images. Remote sensing image A1 includes medium-sized oil tanks to be detected that are uniform in size and neatly arranged, but one target is incomplete; the oil tanks to be detected in remote sensing image B1 have large size differences and are close in position; remote sensing image C1 contains oil tanks to be detected of different sizes and appearances, and there are objects of similar shapes interfering near the oil tanks to be detected; remote sensing image D1 contains multiple small oil tanks to be detected of different sizes densely gathered in one place.

[0123] Images A2, B2, C2, and D2 are the recognition results of the oil tank to be detected by the region-based convolutional neural network. Images A3, B3, C3, and D3 are the recognition results of the oil tank to be detected by the deep learning-based target detection (You Only Look Once version 3, YOLOv3) model. Images A4, B4, C4, and D4 are the recognition results of the oil tank to be detected by the self-supervised visual learning model (Distillation with NoLabels, DINO) model. Images A5, B5, C5, and D5 are the recognition results of the oil tank detection model for the oil tank to be detected.

[0124] like Figure 8 As shown in the figure, the prediction box boundary of A2 is not accurate. In A3, the circular trace on the side is mistakenly identified as the oil tank target. Both the DINO model and the oil tank detection model can accurately extract all the correct targets. However, a careful comparison of A3 and A4 will reveal that the accuracy of the prediction box boundary of the oil tank detection model has improved compared to DINO.

[0125] The multi-scale problem in remote sensing image target detection has always been a difficult problem. In B2, only large-sized targets were detected, and small targets near large-sized oil tanks were not identified. B3, B4, and B5 all detected oil tanks of different scales to be detected, but the prediction results of YOLOv3 and DINO on small targets showed false detection and missed detection. In the original image C1, there are many objects with similar colors and shapes, and the target is similar to the background color, which increases the difficulty of detection. In C2, C3, and C4, two distinctive white oil tank targets are detected, but the 6 gray targets below cannot be detected. Only C5 of the oil tank detection model detects all the targets.

[0126] The oil tanks to be detected often appear as small objects in remote sensing images, and there will be dense distribution of small targets as shown in D1. Such targets often cannot be accurately extracted. In the prediction results of each model, D2 recognizes that there are targets in the area, but cannot accurately give a predicted bounding box. D3 and D4 recognize some targets, but there are still many densely distributed targets that cannot give an accurate predicted bounding box. The oil tank detection model with optimized bounding box evaluation indicators accurately predicts all dense small targets.

[0127] Fig. 9 A block diagram of an electronic device suitable for implementing an oil tank detection method based on remote sensing images and an oil tank detection model training method according to an embodiment of the present disclosure is schematically shown.

[0128] like Fig. 9As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 to a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0129] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0130] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0131] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0132] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0133] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the oil tank detection method based on remote sensing images and the oil tank detection model training method provided by the embodiment of the present disclosure.

[0134] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 901. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0135] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0136] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0137] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0138] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0139] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0140] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for detecting oil tanks based on remote sensing images, characterized in that: The method comprises: Extract features from the remote sensing image containing the oil tank to be inspected at different scales to obtain a multi-scale feature map; Performing feature enhancement processing on the multi-scale feature map to obtain the position feature of the oil tank to be detected; and The position information of the oil tank to be detected is determined based on the position feature.

2. The method according to claim 1, characterized in that The feature extraction of the remote sensing image containing the oil tank to be detected is performed at different scales to obtain a multi-scale feature map, including: Performing static convolution on basic features of the oil tank in the remote sensing image to obtain an initial feature map; Dynamic convolution is performed on the oil tank abstract features in the initial feature map to obtain a multi-scale feature map.

3. The method according to claim 2, characterized in that The step of dynamically convolving the oil tank abstract features in the initial feature map to obtain a multi-scale feature map includes: Using multiple first convolutional layers to dynamically convolve the oil tank abstract features in the initial feature map to obtain an initial multi-scale feature map; Downsampling the initial multi-scale feature map using a second convolutional layer to obtain the multi-scale feature map; Among them, the attention coefficients of the convolution kernel space, the attention coefficients of the convolution kernel input channels, the attention coefficients of the convolution kernel output channels and the attention coefficients of the number of convolution kernels of each of the multiple dynamic convolution kernels in the first convolution layer are obtained through training.

4. The method according to claim 1, characterized in that The performing feature enhancement processing on the multi-scale feature map to obtain the position feature of the oil tank to be detected includes: Inputting the multi-scale feature map into an encoder to obtain an initial position feature; Initializing the anchor frame information of each of the oil tanks to be detected based on the initial position features to obtain prior position information; The position prior information and the multi-scale feature map are input into a decoder, and the position feature is output.

5. The method according to claim 4, characterized in that The step of inputting the multi-scale feature map into an encoder to obtain an initial position feature comprises: Performing dimension transformation on the multi-scale feature map to obtain a one-dimensional multi-scale feature; The spatial position information of the remote sensing image is embedded into the one-dimensional multi-scale feature to obtain the initial position feature.

6. A method for training an oil tank detection model based on remote sensing images, characterized in that: The method comprises: Inputting the sample remote sensing image into the initial oil tank detection model so that the initial oil tank detection model performs feature extraction on the sample remote sensing image at different scales to obtain a multi-scale feature map; performing feature enhancement processing on the multi-scale feature map to obtain position features of the oil tank in the sample remote sensing image; and determining the position information of the oil tank based on the position features; Determining a loss value of the initial oil tank detection model based on the sample label of the sample remote sensing image and the position information; The above operation is repeated until the loss value meets the preset conditions to obtain the oil tank detection model.

7. The method according to claim 6, characterized in that The step of determining the loss value of the initial oil tank detection model based on the sample label of the sample remote sensing image and the position information includes: Performing bounding box modeling on the predicted position information based on a two-dimensional Gaussian distribution to obtain a plurality of predicted bounding boxes; For each of the predicted bounding boxes, calculating the Gaussian similarity between the predicted bounding box and a standard bounding box, wherein the standard bounding box is a sample label corresponding to the sample remote sensing image input into the initial oil tank detection model; A loss value of the initial oil tank detection model is determined according to the Gaussian similarity.

8. The method according to claim 7, characterized in that The method further comprises: In the case that there are multiple predicted bounding boxes, the multiple predicted bounding boxes are matched with multiple standard bounding boxes to calculate the Gaussian similarity, wherein the multiple standard bounding boxes are the sample labels.

9. The method according to claim 6, characterized in that The oil tank detection model includes a first convolution layer and a second convolution layer; the first convolution layer includes a plurality of dynamic convolution kernels; The method further comprises: The sample remote sensing image is used to train the convolution kernel space, convolution kernel input channel, convolution kernel output channel and number of convolution kernels of each of the multiple dynamic convolution kernels, so as to use the multiple first convolution layers to dynamically convolve the abstract features of the oil tank in the initial feature map to obtain an initial multi-scale feature map; the initial multi-scale feature map is downsampled using the second convolution layer to obtain the multi-scale feature map; wherein the initial feature map is obtained by static convolution of the basic features of the oil tank in the sample remote sensing image.

10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 9.

Images