A waterlogging identification method based on deep learning, and a device and storage medium thereof

Abstract

The application discloses a waterlogging identification method and device based on deep learning and a storage medium. First, a data set is acquired, and waterlogging images of the data set are subjected to data enhancement. Second, a waterlogging identification model is established, which comprises a coarse identification module, a fine identification module and a fusion module. The coarse identification module comprises a feature extraction network, a classification network and a position prediction network. The fine identification module is specifically Mask R-CNN. Then, the waterlogging images subjected to data enhancement are taken as samples and input into the waterlogging identification model for training. Finally, a waterlogging image to be identified is subjected to data enhancement and input into the trained waterlogging identification model, so that a waterlogging area is identified. The application has high efficiency and high accuracy.

Description

Technical Field

[0001] This invention relates to computer vision technology, and more particularly to a method, apparatus and storage medium for flood identification based on deep learning. Background Technology

[0002] Floods are a common natural disaster facing human society, posing a significant threat to human life and property. How to quickly and accurately identify flood images and achieve intelligent early warning is a pressing problem. Traditional flood identification methods use conventional target detection algorithms, which suffer from false alarms and missed alarms, and there is still considerable room for improvement in timely early warning and rapid response to floods.

[0003] Deep learning technology has been widely applied in the field of object detection and has achieved remarkable results. Among them, Mask R-CNN, as an excellent object detection algorithm, has achieved good results in many tasks. However, as an algorithm that uses traditional anchor-based detection methods, Mask R-CNN requires setting a large number of prior boxes when performing object detection, and is easily affected by the different sizes, proportions, and positions of objects, resulting in low recognition efficiency and accuracy. Summary of the Invention

[0004] Purpose of the invention: This invention addresses the problems existing in the prior art by providing a waterlogging identification method, device, and storage medium based on deep learning that has higher identification efficiency and accuracy.

[0005] Technical solution: The deep learning-based flood identification method of the present invention includes:

[0006] (1) Obtain a dataset, which includes several flood images and corresponding labels identifying flood areas;

[0007] (2) Data augmentation of the flood images in the dataset;

[0008] (3) Establish a flood identification model, including a coarse identification module, a fine identification module and a fusion module. The coarse identification module includes a feature extraction network, a classification network and a location prediction network. The feature extraction network extracts multi-scale feature maps from flood images. The location prediction network predicts candidate boxes of flood areas based on the multi-scale feature maps. The classifier learns the category of each candidate box based on the multi-scale feature maps. The fine identification module performs fine-grained identification based on the output of the coarse identification module to identify fine flood areas. The fusion module merges the candidate boxes belonging to the flood area category in the output of the location prediction network with the output of the fine identification module, and selects the final flood area identification result from the fusion result.

[0009] (4) Use the data-enhanced flood images as samples and input them into the flood recognition model for training;

[0010] (5) Data augmentation is performed on the flood image to be identified and input into the trained flood identification model to identify the flood area.

[0011] Furthermore, the data enhancement includes one or more combinations of image scaling, displacement variation, contrast variation, and color variation.

[0012] Furthermore, the feature extraction network is a pre-trained convolutional neural network. The classification network is specifically a convolutional neural network. The location prediction network is specifically a convolutional neural network.

[0013] Furthermore, the fusion module specifically includes:

[0014] The fusion unit is used to sort the output of the location prediction network and the output of the fine identification module according to their confidence levels and merge them into a list.

[0015] The non-maximum suppression unit is used to perform non-maximum suppression on the fused list to remove waterlogged areas that are detected repeatedly.

[0016] The confidence threshold filtering unit is used to filter the list of results processed by the non-maximum suppression unit with confidence threshold, remove waterlogged areas with confidence thresholds below the preset confidence threshold, and obtain the final list of waterlogged area identification results.

[0017] Furthermore, when training the flood identification model, the classification network uses the cross-entropy loss function. When training the flood identification model, the location prediction network uses the smoothed L1 loss function.

[0018] The flood identification device of the present invention includes a processor and a computer program stored in a memory and executable on the processor, wherein the processor implements the above-described method when executing the program.

[0019] The storage medium containing computer-executable instructions described in this invention is used to perform the above-described method when executed by a computer processor.

[0020] Beneficial effects: Compared with the prior art, the significant advantages of this invention are: this invention first performs coarse-grained detection on the flood image, then uses Mask R-CNN for fine-grained detection, and then fuses the results of coarse and fine detection to screen out the flood areas that meet the conditions. It has high recognition efficiency, high accuracy and high robustness. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the waterlogging identification method based on deep learning provided by the present invention;

[0022] Figure 2 This is a schematic diagram of the structure of the flood identification model of the present invention;

[0023] Figure 3 This is a schematic diagram of the structure of the waterlogging identification device based on deep learning provided by the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Example 1

[0026] This embodiment presents a deep learning-based flood identification method, such as... Figure 1 As shown, the specific steps include the following:

[0027] (1) Obtain a dataset, which includes several flood images and corresponding labels that identify flood areas.

[0028] (2) Data augmentation of the flood images in the dataset.

[0029] Data augmentation includes one or more combinations of image scaling, displacement adjustment, contrast adjustment, and color adjustment. Other data augmentation methods may also be included.

[0030] (3) Establish a flood identification model, including a coarse identification module, a fine identification module and a fusion module.

[0031] Among them, such as Figure 2 As shown, the coarse recognition module includes a feature extraction network, a classification network, and a location prediction network, while the fine recognition module is specifically Mask R-CNN.

[0032] The feature extraction network is specifically a pre-trained convolutional neural network, such as ResNet or VGG. These networks have been trained on large image datasets and have good feature extraction capabilities. The pre-processed flood image is input into the selected convolutional neural network, and features are extracted through multiple convolutional and pooling layers to obtain multi-scale feature maps. These feature maps contain image feature information at different resolutions, which helps improve the accuracy of object detection.

[0033] The classification network is specifically a convolutional neural network, consisting of several convolutional layers. It predicts the probability of candidate box categories based on multi-scale feature maps, and its output channel count is the number of categories multiplied by the number of candidate boxes at each anchor point. The classification network can classify candidate boxes and obtain candidate boxes for different object categories.

[0034] The location prediction network is a convolutional neural network, which includes several convolutional layers. It can predict candidate boxes for flooded areas based on multi-scale feature maps. Its output channel number is 4 times the number of candidate boxes at each anchor point (the four coordinate values ​​of the bounding box).

[0035] Mask R-CNN uses the flooded area candidate boxes output by the location prediction network as regions of interest for fine-grained object detection, including bounding boxes and masks. Flooded area candidate boxes are selected as input for detection.

[0036] The fusion module includes a fusion unit, a non-maximum suppression unit, and a confidence threshold filtering unit. The fusion unit is used to sort the candidate boxes belonging to the flooded area classification in the output of the location prediction network and the output of the fine recognition module according to confidence and merge them into a list. The non-maximum suppression unit is used to perform non-maximum suppression operation on the fused list to remove the flooded areas that are detected repeatedly. The confidence threshold filtering unit is used to perform confidence threshold filtering on the result list after processing by the non-maximum suppression (NMS) unit to remove flooded areas with a confidence threshold lower than the preset confidence threshold, so as to obtain the final list of flooded area recognition results.

[0037] (4) Use the data-enhanced flood images as samples and input them into the flood recognition model for training.

[0038] When training the flood identification model, the classification network uses the cross-entropy loss function, and the location prediction network uses the smoothed L1 loss function.

[0039] (5) Data augmentation is performed on the flood image to be identified and input into the trained flood identification model to identify the flood area.

[0040] Example 2

[0041] Figure 3 This is a schematic diagram of a waterlogging identification device based on deep learning provided by an embodiment of the present invention. The embodiment of the present invention provides services for the implementation of the method of the above embodiment one of the present invention. Figure 3 A block diagram of an exemplary device 12 suitable for implementing embodiments of the present invention is shown. Figure 3 The device 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0042] like Figure 3As shown, device 12 is represented as a general-purpose computing device. Components of device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting different system components, including system memory 28 and processing unit 16.

[0043] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0044] Device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by device 12, including volatile and non-volatile media, removable and non-removable media.

[0045] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 3 Not shown; usually referred to as a "hard drive"). Although Figure 3 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0046] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0047] Device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with device 12, and / or with any device that enables device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. Figure 3 As shown, network adapter 20 communicates with other modules of device 12 via bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0048] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the method provided in Embodiment 1 of the present invention.

[0049] Example 3

[0050] This invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the method of Embodiment 1.

[0051] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0052] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0053] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0054] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0055] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the above-described method operations, but can also perform related operations in the methods provided in any embodiment of the present invention.

Claims

1. A waterlogging identification method based on deep learning, characterized in that... The method includes: (1) Obtain a dataset, which includes several flood images and corresponding labels identifying flooded areas; (2) Perform data augmentation on the flood images in the dataset; (3) Establish a flood identification model, including a coarse identification module, a fine identification module, and a fusion module. The coarse identification module includes a feature extraction network, a classification network, and a location prediction network. The feature extraction network extracts multi-scale feature maps from the flood image. The location prediction network predicts candidate boxes for flooded areas based on the multi-scale feature maps. The classification network learns the category of each candidate box based on the multi-scale feature maps. The fine identification module is specifically Mask R-CNN, which identifies fine flooded areas based on the output of the coarse identification module. The fusion module merges the candidate boxes belonging to the flooded area category in the output of the location prediction network with the output of the fine identification module, and selects the final flooded area identification result from the fusion result. The fusion module specifically includes: The fusion unit is used to sort the output of the location prediction network and the output of the fine identification module according to their confidence levels and merge them into a list. The non-maximum suppression unit is used to perform non-maximum suppression on the fused list to remove waterlogged areas that are detected repeatedly. The confidence threshold filtering unit is used to filter the list of results processed by the non-maximum suppression unit with confidence threshold, remove waterlogged areas below the preset confidence threshold, and obtain the final list of waterlogged area identification results. (4) Use the data-enhanced flood images as samples and input them into the flood recognition model for training; (5) Data augmentation is performed on the flood image to be identified and input into the trained flood identification model to identify the flood area.

2. The deep learning-based flood identification method according to claim 1, characterized in that: The data enhancement includes one or more combinations of image scaling, displacement changes, contrast changes, and color changes.

3. The deep learning-based flood identification method according to claim 1, characterized in that: The feature extraction network is a pre-trained convolutional neural network.

4. The waterlogging identification method based on deep learning according to claim 1, characterized in that: The classification network is specifically a convolutional neural network.

5. The deep learning-based flood identification method according to claim 1, characterized in that: The location prediction network is specifically a convolutional neural network.

6. The deep learning-based flood identification method according to claim 1, characterized in that: When training the flood identification model, the classification network uses the cross-entropy loss function.

7. The deep learning-based flood identification method according to claim 1, characterized in that: When training the flood identification model, the location prediction network uses a smoothed L1 loss function.

8. A waterlogging identification device based on deep learning, comprising a processor and a computer program stored in a memory and executable on the processor, characterized in that: When the processor executes the program, it implements the method described in any one of claims 1-7.

9. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the method as described in any one of claims 1-7.

Images