SAR image bridge detection method and device based on segnet and storage medium

Abstract

The embodiment of the present specification provides a SAR image bridge detection method and device based on SegNet and a storage medium, the method comprises: collecting a SAR image of a target to be detected, the SAR image comprising a river area and a bridge area; taking the SAR image as input, performing image segmentation on the river area through a preset first neural network model to obtain a river area image and river area feature data; taking the river area feature data and the image as input, performing image segmentation on the bridge area through a preset second neural network model to obtain a bridge area image. The technical scheme provided in the present application is used to solve the problems of inter-class imbalance and loss of position information when directly using deep learning semantic segmentation for bridge detection.

Description

Technical Field

[0001] This document relates to the field of image processing technology, and in particular to a method, apparatus and storage medium for bridge detection in SAR images based on SegNet. Background Technology

[0002] In recent years, deep learning has achieved many results in fields such as machine vision and object detection, bringing new opportunities for the development of bridge detection in SAR images.

[0003] Existing technologies typically use deep learning semantic segmentation directly for bridge detection.

[0004] However, directly applying deep learning semantic segmentation to bridge detection does not yield good performance. The main reason is that in bridge extraction tasks, bridge elements occupy a very small portion of the entire remote sensing image, leading to severe inter-class imbalance compared to other background pixels. Furthermore, while the convolutional layers of the network have some localization capabilities, the pooling operations in deep learning semantic segmentation networks often lose positional information from the image. Summary of the Invention

[0005] In view of the above analysis, this application aims to propose a method, apparatus and storage medium for bridge detection in SAR images based on SegNet, so as to solve at least one of the above-mentioned technical problems.

[0006] Firstly, one or more embodiments of this specification provide a SAR image bridge detection method based on SegNet, including:

[0007] Acquire SAR images of the target to be detected, including river areas and bridge areas;

[0008] Using the SAR image as input, the river region is segmented using a preset first neural network model to obtain river region images and river region feature data;

[0009] Using the river region feature data and the image as input, the bridge region is segmented using a preset second neural network model to obtain a bridge region image.

[0010] Furthermore, the training methods for the first neural network model and the second neural network model include:

[0011] Acquire training images and label river areas and bridge areas in the training images respectively;

[0012] The first neural network model is used to extract images corresponding to the river region from the images to be trained.

[0013] According to the extracted image, a feature image corresponding to the river region is determined;

[0014] According to the feature image, the second neural network model is used to extract an image corresponding to the bridge region from the to-be-trained image;

[0015] The first loss function of the first neural network model and the second loss function of the second neural network model are calculated respectively;

[0016] According to the first loss function and the second loss function, a total loss function is determined;

[0017] The total loss function is used as a new loss function of the first loss function and the second loss function.

[0018] Further, the first neural network model comprises a decoder and an encoder;

[0019] The image segmentation of the river region by the preset first neural network model to obtain river region feature data comprises:

[0020] The first image obtained by the first convolutional layer of the encoder, the second image obtained by the soft layer of the decoder, and the third image obtained by any intermediate layer of the decoder are collected respectively;

[0021] The first image, the second image, and the third image are image spliced to obtain a fourth image;

[0022] Point-by-point multiplication operation is performed on the fourth image and the SAR image to obtain river region feature data.

[0023] Further, the encoder of the first neural network model comprises four encoding modules;

[0024] Each of the encoding modules comprises two convolutional layers with a convolution kernel of 3x3 and a maximum pooling layer of 2x2.

[0025] Further, the decoder of the first neural network model comprises four decoding modules and a soft layer;

[0026] Each of the decoding modules comprises two deconvolutional layers with a convolution kernel of 3x3 and a de-pooling layer of 2x2;

[0027] During decoding, the to-be-decoded data first passes through the four decoding modules and then passes through the soft layer.

[0028] Further, the encoder of the second neural network model comprises an independent convolutional layer with a convolution kernel of 3x3, a pyramid pooling module, a first encoding module, and a second encoding module.

[0029] The first encoding module and the second encoding module each comprise a convolution layer with two convolution kernels of 3x3 and a convolution layer with a step of 2.

[0030] During encoding, the data to be encoded sequentially passes through the independent convolution layer, the pyramid pooling module, the first encoding module and the second encoding module.

[0031] Further, the decoder of the second neural network model comprises three decoding modules and a soft layer.

[0032] Each of the decoding modules comprises two deconvolution layers with a convolution kernel of 3x3 and a de-pooling layer.

[0033] During decoding, the data to be decoded first passes through the three decoding modules and then passes through the soft layer.

[0034] In a second aspect, an embodiment of the present application provides a SAR image bridge detection device based on SegNet, comprising a collection module and a data processing module.

[0035] The collection module is configured to collect a SAR image of a target to be detected, and the SAR image comprises a river region and a bridge region.

[0036] The data processing module is configured to perform image segmentation on the river region by using a preset first neural network model and taking the SAR image as input, to obtain a river region image and river region feature data; and perform image segmentation on the bridge region by using a preset second neural network model and taking the river region feature data and the image as input, to obtain a bridge region image.

[0037] Further, the data processing module is further configured to collect a first image obtained by the first convolution layer of the encoder, a second image obtained by the soft layer of the decoder and a third image obtained by any intermediate layer of the decoder, respectively; perform image stitching on the first image, the second image and the third image to obtain a fourth image; and perform point-by-point multiplication operation on the fourth image and the SAR image to obtain the river region feature data.

[0038] In a third aspect, an embodiment of the present application provides a storage medium, comprising computer executable instructions for storing, which, when executed, implement the following processes:

[0039] Collecting a SAR image of a target to be detected, wherein the SAR image comprises a river region and a bridge region.

[0040] The river region is segmented by a preset first neural network model to obtain river region images and river region feature data.

[0041] The bridge region is segmented by a preset second neural network model to obtain bridge region images.

[0042] Compared with the prior art, the application can achieve at least the following technical effects:

[0043] 1. The feature data of the easily recognizable river image is extracted first, and then the image of the bridge is determined based on the feature data of the river image, which improves the problem of easy overfitting in the process of extracting the bridge feature, thereby improving the calculation accuracy of the neural network.

[0044] 2. The feature data of the river image can ensure the calculation accuracy of the model, so the number of training samples can be appropriately reduced, and the training samples of the SAR image are not easy to obtain, that is, the application can solve the problem of insufficient training samples in the field of SAR image processing.

[0045] 3. Based on the feature data of the river image, the relationship between the river image and the bridge image in the image processing process is expressed by a mathematical model, so as to establish a new loss function according to the correlation between the river image and the bridge image. Then, the new function is brought into the first neural network and the second neural network, thereby further improving the calculation accuracy of the data. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the one or more embodiments of the present specification or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0047] Figure 1 A flowchart of a SAR image bridge detection method based on SegNet provided by one or more embodiments of the present specification;

[0048] Figure 2 A schematic diagram of the first neural network model and the second neural network model provided by one or more embodiments of the present specification;

[0049] Figure 3 A schematic diagram of the pyramid pooling module provided by one or more embodiments of the present specification. DETAILED DESCRIPTION

[0050] In order to make the person skilled in the art better understand the technical solutions in one or more embodiments of the present specification, the technical solutions in one or more embodiments of the present specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of the present specification. Obviously, the described embodiments are only a part of the embodiments of the present specification, not all. Based on one or more embodiments of the present specification, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the present document.

[0051] Synthetic Aperture Radar (SAR) is a kind of microwave active imaging radar, which has strong penetration and long range, and has the ability of all-weather observation and imaging at all times, and has been widely used in national economy and national defense and military fields. In SAR target detection, since the water bridge is a key hub of transportation and has strong strategic value, how to efficiently, quickly and accurately extract and detect bridge information is particularly important. Therefore, SAR image bridge target detection has important applications in civil and military fields, and is also a research hotspot in recent years.

[0052] Deep learning semantic segmentation technology can not only reliably predict the existence and rough position of objects in the image, but also accurately point out their exact contours, so there are more research and application in image target detection, road scene recognition, semantic understanding and other directions. Therefore, in recent years, deep learning semantic segmentation technology has made many achievements in machine vision, target detection and other fields, bringing new opportunities for the development of SAR image bridge detection.

[0053] However, there are the following problems in directly applying deep learning semantic segmentation technology to SAR image bridge detection:

[0054] 1. The shooting range corresponding to the SAR image is large, so the occupied pixels of the bridge in the SAR image are relatively small, so it is easy to overfit when performing semantic segmentation and image feature extraction.

[0055] 2. SAR images are not publicly disclosed in most cases, making it difficult to obtain training samples for SAR images.

[0056] The embodiment of the present specification provides a SAR image bridge detection method based on SegNet, as shown in FIG. 1, which includes the following steps:

[0057] Step 1, collect the SAR image of the target to be detected.

[0058] In the embodiment of the present application, the SAR image includes a river area and a bridge area.

[0059] Step 2, taking the SAR image as input, performing image segmentation on the river region by a preset first neural network model to obtain a river region image and river region feature data.

[0060] In the first neural network model in the embodiment of the present application, the first neural network model comprises a decoder and an encoder.

[0061] The encoder of the first neural network model comprises four encoding modules.

[0062] Each encoding module comprises two convolution layers with a convolution kernel of 3x3 and one maximum pooling layer with a size of 2x2.

[0063] The decoder of the first neural network model comprises four decoding modules and a soft layer.

[0064] Each decoding module comprises two deconvolution layers with a convolution kernel of 3x3 and one de-pooling layer with a size of 2x2; during decoding, the data to be decoded first passes through the four decoding modules and then passes through the soft layer.

[0065] It should be noted that, since the SAR image corresponds to a large shooting range and the river occupies a small number of pixels, compared with the prior art, the present application reduces the number of modules of the encoder and the decoder and the convolution layers in each module, thereby preventing feature loss. Each decoding module corresponds to a preset image decoding size, and each encoding module corresponds to a preset image encoding size.

[0066] Based on the above structure, the specific process of step 2 is as follows: collecting a first image obtained by the first convolution layer of the encoder, a second image obtained by the soft layer of the decoder, and a third image obtained by any intermediate layer of the decoder, respectively;

[0067] performing image stitching on the first image, the second image, and the third image to obtain a fourth image;

[0068] performing point-by-point multiplication operation on the fourth image and the SAR image to obtain river region feature data.

[0069] Specifically, the present application provides a river region detection subnetwork (first neural network model), as shown in Figure 2As shown, the network is an improved SegNet semantic segmentation network, which reduces the original SegNet network encoder from 5 modules to 4, and the last two modules are reduced to 2 convolution layers and 1 max pooling layer. Each module has 2 convolution layers with a kernel size of 3x3 and 1 max pooling layer with a size of 2x2. The decoder is also processed symmetrically. The encoder extracts deep and abstract features through convolution layers, max pooling layers, etc. The decoder performs deconvolution and upsampling operations, and finally outputs the final semantic segmentation result image through the softmax classifier. The deep abstract features extracted by the encoder are converted into a dense label image with the same size as the input image. The obtained river area detection network encoder has 4 modules, each module has 2 convolution layers with a kernel size of 3x3 and a step size of 1 (each convolution layer is followed by a batch normalization (Batch Normalization, BN) layer and a ReLU activation layer), and 1 max pooling layer with a size of 2x2; After the input image enters the encoder, the size gradually becomes 256x256, 128x128, 64x64, and 32x32. The decoder has 4 modules, the first three modules have 1 de-pooling layer to gradually restore the size of the feature map, and 2 deconvolution (Deconvolution, DConv) layers (each deconvolution layer is followed by a batch normalization layer and a ReLU activation layer), and the second deconvolution layer of the last module does not contain a batch normalization layer and a ReLU activation layer.

[0070] Based on the above results, the process of obtaining river area feature data is as follows:

[0071] Collect ① the result of the input original SAR image through the first convolution layer of the river area detection sub-network encoder (first image), ② the result output by the softmax layer of the decoder of the river area detection sub-network (second image), and ③ the result after adjusting the size of the output feature map of an optional middle layer of the decoder of the river area detection sub-network. Among them, ③ usually selects the 256x256 feature map output by the first convolution layer of the third module of the decoder, and performs bilinear interpolation upsampling on the feature map to restore its resolution to 512x512 (third image). Then, the three feature maps of ①-③ are spliced, and the channel number is adjusted to 1 by 1x1 convolution. This helps to combine deep and shallow semantic features related to the river area, and obtain an attention feature map (fourth image). Multiply the original SAR image point by point to obtain more reliable river area feature data Fig P .

[0072] Step 3, input the river area feature data and the image into the preset second neural network model to perform image segmentation on the bridge area, and obtain a bridge area image.

[0073] In the embodiment of the present application, the second neural network model comprises a decoder and an encoder. The encoder of the second neural network model comprises an independent convolutional layer with a convolution kernel of 3x3, a pyramid pooling module, a first encoding module and a second encoding module;

[0074] The first encoding module and the second encoding module each comprise two convolutional layers with a convolution kernel of 3x3 and one convolutional layer with a step of 2; during encoding, the data to be encoded sequentially passes through the independent convolutional layer, the pyramid pooling module, the first encoding module and the second encoding module.

[0075] The decoder of the second neural network model comprises three decoding modules and a soft layer.

[0076] Each decoding module comprises two deconvolutional layers with a convolution kernel of 3x3 and one de-pooling layer.

[0077] During decoding, the data to be decoded first passes through the three decoding modules and then passes through the soft layer.

[0078] It should be noted that, since the SAR image corresponds to a large range of shooting and the bridge occupies a small number of pixels, compared with the prior art, the present application reduces the number of modules of the encoder and the decoder and the convolutional layers in each module, thereby preventing feature loss and matching with the first neural network model. Each decoding module corresponds to a preset image decoding size, and each encoding module corresponds to a preset image encoding size.

[0079] Specifically, the present application provides a bridge detection subnetwork (second neural network model), as shown in FIG. 1, which is an improved SegNet semantic segmentation network with a pyramid pooling module Figure 2

[0080] The improvement of the bridge detection subnetwork to the original SegNet network is that the encoder is reduced from 5 modules to 3, and the maximum pooling layer is replaced by a convolutional layer with a step of 2. Among them, the first module is composed of one convolutional layer with a convolution kernel of 3x3 (followed by one batch normalization layer and one ReLU activation layer after convolution) and a pyramid pooling module, and the last two modules are reduced to two convolutional layers with a convolution kernel of 3x3 (each convolutional layer is followed by one batch normalization layer and one ReLU activation layer after convolution) and one convolutional layer with a step of 2. The encoder gradually reduces the Fig P ​The dimensions become 256×256, 128×128, and 64×64. The decoder also processes the data symmetrically according to the simplified method of the encoder, but without the pyramid pooling module. Its first two modules each have one unpooling layer to progressively restore the feature map size, and two deconvolution layers (each deconvolution layer adds a batch normalization layer and a ReLU activation layer after deconvolution). The second deconvolution layer of the last module does not contain a batch normalization layer or a ReLU activation layer. The decoder progressively restores the feature map size output by the encoder to 128×128, 256×256, and 512×512, and finally outputs the final bridge semantic segmentation result image (Fig) after passing through a softmax classifier. B .

[0081] Among them, the pyramid pooling module is as follows Figure 3 As shown, pooling windows of sizes 2×2, 3×3, 5×5, and 7×7 are used to transform feature maps into feature maps of different scales through pooling at four different scales, preserving the original features extracted by the convolutional layers to the maximum extent and reducing the generation of blur blocks. The resulting images are then subjected to a 1×1 convolution, reducing the number of channels to 1 / N of the original, where N represents the level of refinement (N=4 here). The resulting feature maps are then upsampled using bilinear interpolation to restore the feature maps of different sizes to their original size. These five feature maps are then concatenated together with the result of a 1×1 convolution of the original feature maps, and a new feature map with the same number of channels is obtained through a 1×1 convolution.

[0082] In this embodiment of the application, the training method for the first neural network model and the second neural network model includes:

[0083] Collect training images and label river areas and bridge areas in the training images respectively;

[0084] The first neural network model is used to extract images corresponding to river regions from the images to be trained.

[0085] Based on the extracted images, determine the feature images corresponding to the river region;

[0086] Based on the feature images, the images corresponding to the bridge region are extracted from the images to be trained using a second neural network model;

[0087] Calculate the first loss function of the first neural network model and the second loss function of the second neural network model respectively;

[0088] Determine the total loss function based on the first loss function and the second loss function;

[0089] The total loss function is used as a new loss function for the first and second loss functions.

[0090] Specifically, the first loss function is:

[0091]

[0092]

[0093] where R + and R - represent the number of all positive and negative pixels in the input image X, i.e., the total number of river and non-river pixels. P j represents the class of each pixel point of the prediction map, P j ∈{0,1}, W R represents the weight of the current river region detection network, β1 represents the class loss weight of negative pixels, and 1-β1 represents the class loss weight of positive pixels. Pr(·)∈[0,1] represents the probability of a pixel being negative or positive in the prediction map.

[0094] The second loss function is:

[0095]

[0096]

[0097] where B + and B - represent the number of all positive and negative pixels in the input image X, i.e., the total number of bridges and non-bridge pixels. W B represents the weight of the current river region detection network. β2 represents the class loss weight of negative pixels, and 1-β2 represents the class loss weight of positive pixels. Pr(·)∈[0,1] represents the probability of a pixel being negative or positive in the prediction map.

[0098] Then, the total loss function is calculated, and the specific steps are:

[0099] First, the weight decay loss is calculated. Weight decay can regularize model parameters, used to select the minimum weight vector of the learning task to suppress the irrelevant components of the weight vector, suppress static noise, and improve the generalization ability of the model. The weight decay loss is defined as:

[0100]

[0101] where W represents all parameters of the R-B SegNet model, λ is a hyperparameter that controls the degree of weight penalty, and is set to 2e-4.

[0102] Then the total loss of the network is calculated, and the total loss function is defined as:

[0103] L total =αL R +γL R +ηLw

[0104] Finally, the model parameters are updated by backpropagation. Wherein a, g and h are the weights of the three types of loss, taking [0.4, 0.4, 0.2].

[0105] It should be noted that in order to ensure the accuracy of image processing, the image resolution of the training sample and the test sample is required to be greater than 64x64. Preferably, the image resolution is 512x512. When the image resolution is greater than 512x512, the image needs to be segmented, and the specific segmentation process is as follows:

[0106] The high-resolution SAR image to be detected is input into the R-B SegNet, and the upward integer value of the length and width divided by 512 is calculated, and the specific calculation formula is

[0107] N wide =Wide(Img) / N window -1

[0108] N height =Height(Img) / N height -1

[0109] Wherein Img represents the high-resolution SAR image to be detected, N window is the fixed side length of the picture input into the neural network each time, taking 512. The Wide(·) and Height(·) functions represent the width and height of the original image, N wide and N height represent the number of times the sliding window needs to move to the right and down, respectively. In this example, N wide =1, N height =2.

[0110] The original image is cut from left to right and from top to bottom with a sliding window of size 512x512 pixels, i.e. divided into image blocks of pixel value 512x512. In order to prevent the water body and bridge in the edge area from being missed, there is an overlapping area of 60 pixels between the image blocks, i.e. the window is moved to the right by 452 pixels each time, and the right end of the last time needs to coincide with the right end of the original image. After the end of each row, it returns to the leftmost end, moves down by 452 pixels, and then moves to the right again. Repeat 2 times, and the lower end of the last time coincides with the lower end of the original image. Finally, the image is divided into image blocks with a size of no more than 512x512.

[0111] After each movement intercepts a 512*512 image block, it is immediately sent to the trained R-B SegNet for detection to obtain the semantic segmentation result containing the water body and the bridge. The center position of each region (the average value of the maximum and minimum values of the up, down, left and right boundary coordinates of the irregular region) is taken as the bridge position point. The image blocks obtained after detection are sequentially spliced according to the position during segmentation, and the bridge overlapping problem that may occur at the splicing position is detected again, that is, the relative distance of the bridge position points in all overlapping regions is calculated, and if it is less than 15 pixels, it is considered as the same bridge. Thus, the SAR image bridge target detection is realized. Finally, splicing is performed.

[0112] The embodiment of the present application provides a SAR image bridge detection device based on SegNet, which comprises a collection module and a data processing module.

[0113] The collection module is used to collect a SAR image of a target to be detected, and the SAR image comprises a river area and a bridge area.

[0114] The data processing module is used to take the SAR image as input, perform image segmentation on the river area through a preset first neural network model to obtain a river area image and river area feature data, and perform image segmentation on the bridge area through a preset second neural network model to obtain a bridge area image.

[0115] In the embodiment of the present application, the data processing module is further used to collect a first image obtained by the first convolutional layer of the encoder, a second image obtained by the soft layer of the decoder and a third image obtained by any intermediate layer of the decoder respectively; perform image splicing on the first image, the second image and the third image to obtain a fourth image; and perform point-by-point multiplication operation on the fourth image and the SAR image to obtain the river area feature data.

[0116] The embodiment of the present application provides a storage medium, comprising:

[0117] The computer executable instructions are used to store computer executable instructions, and the computer executable instructions are used to realize the following processes when executed:

[0118] The SAR image of the target to be detected is collected, and the SAR image comprises a river area and a bridge area.

[0119] The SAR image is taken as input, the river area is segmented through a preset first neural network model to obtain a river area image and river area feature data.

[0120] The bridge region is segmented by a preset second neural network model with the river region feature data and the image as inputs, to obtain a bridge region image.

[0121] The above describes certain embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different than the order in which they are recited, and still achieve desirable results. Additionally, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or necessary.

[0122] In the 1930s, it was quite obvious to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structure of diodes, transistors, switches, etc.) or in software (improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by a hardware entity module. For example, a Programmable Logic Device (PLD) (e.g., a Field Programmable Gate Array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating an integrated circuit chip, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, called a hardware description language (HDL), of which there are many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., the most commonly used being VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It should be clear to those skilled in the art that, by simply logically programming a method flow in one of the above hardware description languages and programming it into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0123] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, e.g. software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.

[0124] The systems, apparatuses, modules or units illustrated by the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0125] For the sake of description, the above apparatuses are described in various units with functions respectively. Of course, the functions of the units can be implemented in one or more software and / or hardware in implementing the embodiments of the present specification.

[0126] Those skilled in the art will understand that one or more embodiments of the present specification can be provided as a method, a system or a computer program product. Therefore, one or more embodiments of the present specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0127] The specification is presented with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include fewer than all of the steps, operations, or functions shown in the figures. Additionally, the steps, operations, or functions need not be implemented in the order shown in the figures. Further, the steps, operations, or functions can be implemented using additional or different steps, operations, or functions. Figure 1 These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks. The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include fewer than all of the steps, operations, or functions shown in the figures. Additionally, the steps, operations, or functions need not be implemented in the order shown in the figures. Further, the steps, operations, or functions can be implemented using additional or different steps, operations, or functions.

[0128] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include fewer than all of the steps, operations, or functions shown in the figures. Additionally, the steps, operations, or functions need not be implemented in the order shown in the figures. Further, the steps, operations, or functions can be implemented using additional or different steps, operations, or functions. Figure 1 These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks. The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include fewer than all of the steps, operations, or functions shown in the figures. Additionally, the steps, operations, or functions need not be implemented in the order shown in the figures. Further, the steps, operations, or functions can be implemented using additional or different steps, operations, or functions.

[0129] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include fewer than all of the steps, operations, or functions shown in the figures. Additionally, the steps, operations, or functions need not be implemented in the order shown in the figures. Further, the steps, operations, or functions can be implemented using additional or different steps, operations, or functions. Figure 1 These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flow diagrams and / or block diagrams block or blocks. The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include fewer than all of the steps, operations, or functions shown in the figures. Additionally, the steps, operations, or functions need not be implemented in the order shown in the figures. Further, the steps, operations, or functions can be implemented using additional or different steps, operations, or functions.

[0130] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0131] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer-readable media.

[0132] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0133] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that processes, methods, articles or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed, or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device that includes the element.

[0134] One or more embodiments of the present specification can be described in the general context of computer-executable instructions being executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types. One or more embodiments of the present specification can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0135] Each embodiment in the present specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0136] The above merely provides the example of the present document and is not intended to limit the present document. For those skilled in the art, the present document can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present document shall be included in the scope of claims of the present document.

Claims

1. A bridge detection method for SAR images based on SegNet, characterized in that, include: Acquire SAR images of the target to be detected, including river areas and bridge areas; Using the SAR image as input, the river region is segmented using a preset first neural network model to obtain river region images and river region feature data; Using the river region feature data and the image as input, the bridge region is segmented using a preset second neural network model to obtain a bridge region image; The encoder of the first neural network model includes: 4 encoding modules; Each of the aforementioned encoding modules includes: two convolutional layers with 3×3 kernels and a 2×2 max pooling layer; The decoder of the first neural network model includes: 4 decoding modules and a soft layer; Each of the decoding modules includes: two deconvolutional layers with 3×3 kernels and one 2×2 unpooling layer; During decoding, the data to be decoded first passes through the four decoding modules and then through the soft layer; The encoder of the second neural network model includes: an independent convolutional layer with a 3×3 kernel, a pyramid pooling module, a first encoding module, and a second encoding module; Both the first encoding module and the second encoding module include: two convolutional layers with 3×3 kernels and one convolutional layer with a stride of 2; During encoding, the data to be encoded passes sequentially through the independent convolutional layer, the pyramid pooling module, the first encoding module, and the second encoding module; The decoder of the second neural network model includes: 3 decoding modules and a soft layer; Each of the decoding modules includes: two deconvolutional layers with 3×3 kernels and one unpooling layer; During decoding, the data to be decoded first passes through the three decoding modules and then through the soft layer.

2. The method according to claim 1, characterized in that, The training methods for the first neural network model and the second neural network model include: Acquire training images and label river areas and bridge areas in the training images respectively; The first neural network model is used to extract images corresponding to the river region from the images to be trained. Based on the extracted images, determine the feature images corresponding to the river region; Based on the feature image, the image corresponding to the bridge region is extracted from the image to be trained using the second neural network model; Calculate the first loss function of the first neural network model and the second loss function of the second neural network model respectively; Determine the total loss function based on the first loss function and the second loss function; The total loss function is used as a new loss function for both the first loss function and the second loss function.

3. The method according to claim 1, characterized in that, The first neural network model includes: a decoder and an encoder; The step of segmenting the river region using a preset first neural network model to obtain river region feature data includes: The first image obtained from the first convolutional layer of the encoder, the second image obtained from the soft layer of the decoder, and the third image obtained from any intermediate layer of the decoder are respectively acquired. The first image, the second image, and the third image are stitched together to obtain the fourth image; Point-by-point multiplication is performed on the fourth image and the SAR image to obtain the river region feature data.

4. A bridge detection device based on SegNet in SAR images, characterized in that, include: Acquisition module and data processing module; The acquisition module is used to acquire SAR images of the target to be detected, including river areas and bridge areas; The data processing module is used to segment the river region using the SAR image as input and a preset first neural network model to obtain the river region image and river region feature data; and to segment the bridge region using the river region feature data and the image as input and a preset second neural network model to obtain the bridge region image. The encoder of the first neural network model includes: 4 encoding modules; Each of the aforementioned encoding modules includes: two convolutional layers with 3×3 kernels and a 2×2 max pooling layer; The decoder of the first neural network model includes: 4 decoding modules and a soft layer; Each of the decoding modules includes: two deconvolutional layers with 3×3 kernels and one 2×2 unpooling layer; During decoding, the data to be decoded first passes through the four decoding modules and then through the soft layer; The encoder of the second neural network model includes: an independent convolutional layer with a 3×3 kernel, a pyramid pooling module, a first encoding module, and a second encoding module; Both the first encoding module and the second encoding module include: two convolutional layers with 3×3 kernels and one convolutional layer with a stride of 2; During encoding, the data to be encoded passes sequentially through the independent convolutional layer, the pyramid pooling module, the first encoding module, and the second encoding module; The decoder of the second neural network model includes: 3 decoding modules and a soft layer; Each of the decoding modules includes: two deconvolutional layers with 3×3 kernels and one unpooling layer; During decoding, the data to be decoded first passes through the three decoding modules and then through the soft layer.

5. The apparatus according to claim 4, characterized in that, The data processing module is also used to acquire the first image obtained from the first convolutional layer of the encoder, the second image obtained from the soft layer of the decoder, and the third image obtained from any intermediate layer of the decoder; to perform image stitching on the first image, the second image, and the third image to obtain a fourth image; and to perform point-by-point multiplication on the fourth image and the SAR image to obtain river area feature data.

6. A storage medium, characterized in that, include: Used to store computer-executable instructions, which, when executed, implement the following process: Acquire SAR images of the target to be detected, including river areas and bridge areas; Using the SAR image as input, the river region is segmented using a preset first neural network model to obtain river region images and river region feature data; Using the river region feature data and the image as input, the bridge region is segmented using a preset second neural network model to obtain a bridge region image; The encoder of the first neural network model includes: 4 encoding modules; Each of the aforementioned encoding modules includes: two convolutional layers with 3×3 kernels and a 2×2 max pooling layer; The decoder of the first neural network model includes: 4 decoding modules and a soft layer; Each of the decoding modules includes: two deconvolutional layers with 3×3 kernels and one 2×2 unpooling layer; During decoding, the data to be decoded first passes through the four decoding modules and then through the soft layer; The encoder of the second neural network model includes: an independent convolutional layer with a 3×3 kernel, a pyramid pooling module, a first encoding module, and a second encoding module; Both the first encoding module and the second encoding module include: two convolutional layers with 3×3 kernels and one convolutional layer with a stride of 2; During encoding, the data to be encoded passes sequentially through the independent convolutional layer, the pyramid pooling module, the first encoding module, and the second encoding module; The decoder of the second neural network model includes: 3 decoding modules and a soft layer; Each of the decoding modules includes: two deconvolutional layers with 3×3 kernels and one unpooling layer; During decoding, the data to be decoded first passes through the three decoding modules and then through the soft layer.

Images