Post-disaster house danger level evaluation method and device based on deep learning

Abstract

Embodiments of the present application provide a post-disaster house danger level evaluation method and device based on deep learning. The method comprises acquiring a house image, generating an image dataset; preprocessing the image dataset, and dividing the preprocessed data into training set data and test set data; constructing a MyNet network training model; obtaining the training of the MyNet network training model based on the training set data, obtaining the trained MyNet network training model, and obtaining the optimal weight parameter; inputting the test set data into the trained MyNet network training model, outputting a danger level probability, and taking the highest level of the danger level probability as the house danger level. In this way, the efficiency of post-disaster rural house danger level identification can be improved, and a large amount of human and financial resources can be saved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of house danger level identification, and more particularly, to a post-disaster house danger level evaluation method and device based on deep learning. BACKGROUND

[0002] Flood long-term soaking causes damage to the building foundation and foundation. Therefore, the method and technology for post-disaster damaged house identification and classification are research hotspots. The difference between adjacent grade damaged houses is small, and the characteristics are similar. The traditional method is to identify the damaged house grade through on-site investigation by professional technicians. This detection method not only takes a long time and is low in efficiency, but also has many uncontrollable factors.

[0003] With the rapid development of deep machine learning, it has become a trend to use image processing technology and artificial intelligence to identify the danger level of post-disaster rural houses. However, there are many forms of damage to post-disaster damaged houses, and the number of houses with some forms of damage is small, which will lead to poor network training results and ultimately unsatisfactory detection effect. SUMMARY

[0004] According to an embodiment of the present application, a post-disaster house danger level evaluation scheme based on deep learning is provided. This scheme can improve the efficiency of post-disaster rural house danger level identification and save a lot of manpower and financial resources.

[0005] In a first aspect of the present application, a post-disaster house danger level evaluation method based on deep learning is provided. The method comprises:

[0006] Obtaining a house image to generate an image data set;

[0007] Preprocessing the image data set, and dividing the preprocessed data into training set data and test set data;

[0008] Constructing a MyNet network training model;

[0009] Training the MyNet network training model based on the training set data to obtain a trained MyNet network training model and obtain optimal weight parameters;

[0010] Inputting the test set data into the trained MyNet network training model, outputting a danger level probability, and taking the level with the highest danger level probability as the house danger level.

[0011] Further, the MyNet network training model comprises a training network model, a transfer learning model and a detection classification model; an image feature extraction model is used as the training network model, which comprises a feature extraction module shallow layer and a feature extraction module deep layer;

[0012] The input data sequentially passes through the shallow feature extraction module, the deep feature extraction module, the transfer learning model and the detection and classification model, and outputs a probability value of the category to which the image belongs.

[0013] Further, the shallow feature extraction module is sequentially composed of a first convolutional layer, a first max-pooling layer, a second convolutional layer, a second max-pooling layer and two Fused-MBConv modules; the Fused-MBConv module includes a third convolutional layer and a Dropout layer.

[0014] Further, the deep feature extraction module is composed of three MBConv modules; the input data sequentially passes through the three MBConv modules and outputs feature information; the MBConv module is sequentially composed of a fourth convolutional layer, an SE module, a fifth convolutional layer and a Dropout layer.

[0015] Further, the SE module is sequentially composed of a global average pooling layer, a first full connection layer and a second full connection layer; the input data sequentially passes through the average pooling layer, the first full connection layer and the second full connection layer and is output.

[0016] Further, the transfer learning model is sequentially composed of a sixth convolutional layer, an average pooling layer and three full connection layers, and the input data sequentially passes through the sixth convolutional layer, the average pooling layer and the three full connection layers and is output.

[0017] Further, the detection and classification model includes a Softmax layer, and the data output by the transfer learning model is input into the Softmax layer and outputs a probability value of the category to which the image belongs.

[0018] In the second aspect of the present application, a post-disaster house danger level evaluation device based on deep learning is provided. The device comprises:

[0019] An acquisition module is configured to acquire a house image and generate an image data set;

[0020] A preprocessing module is configured to preprocess the image data set and divide the preprocessed data into training set data and test set data;

[0021] A model construction module is configured to construct a MyNet network training model;

[0022] A training module is configured to train the MyNet network training model based on the training set data, obtain a trained MyNet network training model, and obtain optimal weight parameters;

[0023] An evaluation module is configured to input the test set data into the trained MyNet network training model, and output a hazard level probability, wherein a hazard level with the highest hazard level probability is taken as the hazard level of the house.

[0024] In a third aspect of the present application, an electronic device is provided. The electronic device comprises at least one processor; and a memory connected to the at least one processor in communication; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the first aspect of the present application.

[0025] In a fourth aspect of the present application, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method of the first aspect of the present application.

[0026] It should be understood that the content described in the summary section is not intended to limit the key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0027] The above and other features, advantages, and aspects of embodiments of the present application will become more apparent by describing in detail the following embodiments with reference to the attached drawings. In the drawings, the same or similar reference numerals refer to the same or similar elements, and:

[0028] Figure 1 A flowchart of a deep learning-based post-disaster house hazard level evaluation method according to an embodiment of the present application is shown;

[0029] Figure 2 A training set and validation set accuracy rate curve diagram according to an embodiment of the present application is shown;

[0030] Figure 3 A training set and validation set loss rate curve diagram according to an embodiment of the present application is shown;

[0031] Figure 4 A block diagram of a deep learning-based post-disaster house hazard level evaluation device according to an embodiment of the present application is shown;

[0032] Figure 5 A block diagram of an exemplary electronic device capable of implementing an embodiment of the present application is shown;

[0033] Wherein, 500 is an electronic device, 501 is a computing unit, 502 is a ROM, 503 is a RAM, 504 is a bus, 505 is an I / O interface, 506 is an input unit, 507 is an output unit, 508 is a storage unit, and 509 is a communication unit. DETAILED DESCRIPTION

[0034] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0035] In addition, the term “and / or” in this paper only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character “ / ” in this paper generally represents that the front and rear associated objects are in an “or” relationship.

[0036] Figure 1 A flowchart of a post-disaster house danger level evaluation method based on deep learning according to an embodiment of the present application is shown.

[0037] The method comprises:

[0038] S101, acquiring a house image to generate an image data set.

[0039] In this embodiment, the house image is a rural house damage image, and each image has a label of danger level. For example, the image data set includes picture data of rural house damage after the “7.20” heavy rainstorm flood disaster in Zhengzhou, Henan, and the pictures have been labeled with danger levels by a post-disaster house appraisal technical expert team. The damaged house image is collected in a bright and sufficient environment, and is divided into indoor and outdoor when shooting. The image has a front view (one-point perspective view) and a side view (two-point perspective view) of the entire house, and the damaged part of the house is included in the image.

[0040] In the above embodiment, the danger level is divided into four levels: a, b, c and d. According to the “Rural Housing Danger Appraisal Standard” JGJ / T363-2014, the overall characteristics of the four levels are as follows:

[0041] a level: no damage, basically intact;

[0042] b level: slight damage, slight danger, but does not affect continued use;

[0043] c level: moderate damage, moderate danger, should be repaired and reinforced in time and can continue to use;

[0044] d level: serious damage, serious danger, should be stopped using immediately and rebuilt.

[0045] S102, pre-process the image data set, and divide the pre-processed data into training set data and test set data.

[0046] In this embodiment, by pre-processing the image data set, the information unrelated to the surrounding of the picture in the image data set can be cropped and filtered.

[0047] In this embodiment, since the original image sizes in the initially collected post-disaster damaged house image data set are different, the pre-processing performs cropping, stretching and scaling of the original image by Python programming, and unifies the original image size to 224x224. The cropped image is subjected to data enhancement methods such as horizontal flip, image brightness and contrast adjustment to solve the model overfitting phenomenon.

[0048] In this embodiment, the pre-processed data is divided into training set data and test set data, and the pre-processed data can be divided into training set, validation set and test set in the ratio of 8:1:1.

[0049] In some embodiments, the ratio of the training set, the validation set and the test set can also be divided into 7:2:1 and 6:2:2, but after testing and comparison, the test result shows that when the ratio is 8:1:1, the accuracy is higher.

[0050] S103, construct a MyNet network training model.

[0051] In this embodiment, the MyNet network training model includes a training network model (image feature extraction model), a transfer learning model and a detection classification model (classifier); the structure of the MyNet network training model is shown in Table 1:

[0052]

[0053] Table 1

[0054] Among them, the training network model (image feature extraction model) is Stage0~Stage8, wherein Stage0~Stage5 is the shallow layer of the feature extraction module, and Stage6~Stage8 is the deep layer of the feature extraction module.

[0055] In the shallow layer of the feature extraction module, Stage0 is a single-layer convolution layer, i.e., Convolution1, where the size of the convolution kernel is 5x5 and the number of convolution kernels is 8; Stage1 is a single-layer max-pooling layer, i.e., Maxpooling1, where the size of the max-pooling layer is 2x2; Stage2 is a single-layer convolution layer, i.e., Convolution2, where the size of the convolution kernel is 3x3 and the number of convolution kernels is 16; and stage3 is a single-layer max-pooling layer, i.e., Maxpooling2, where the size of the max-pooling layer is 2x2. Stage4 and Stage5 are composed of Fused-MBConv modules, which include convolution layers and Dropout layers; after the input data is subjected to convolution operation, the data is output through the Dropout layers.

[0056] In the deep layer of the feature extraction module, Stage6-Stage8 are all MBConv modules; the MBConv modules not only include convolution layers and Dropout layers, but also introduce an attention mechanism SE module, i.e., the MBConv module is composed of a convolution layer, an SE module, a convolution layer and a Dropout layer in sequence. The SE module includes a global average pooling layer and a fully connected layer 1 (FC1) and a fully connected layer 2 (FC2). After the input data is subjected to convolution operation and dimensionality is increased, the data is processed using the SE module, then subjected to a convolution operation to realize dimensionality reduction, and finally the extracted feature information is output through the Dropout layer.

[0057] In this embodiment, the transfer learning model sequentially includes a 1x1 convolution layer in Stage9, an average pooling layer and a fully connected layer 1 (FC1); and a fully connected layer 2 (FC2) and a fully connected layer 3 (FC3) in Stage9. The feature information obtained from the image feature extraction module is sequentially output to the detection classification model through the fully connected layer 1 (FC1), the fully connected layer 2 (FC2) and the fully connected layer 3 (FC3).

[0058] By obtaining the ImageNet competition network weight parameters provided by tensorflow officially, the ImageNet competition official weight parameters are called as the feature extractor for transfer learning, the convolution layers and the pooling layers before the fully connected layers of the MyNet model are frozen, and two fully connected layers, i.e., the fully connected layer 2 (FC2) and the fully connected layer 3 (FC3) of Stage10 in Table 1, are added between the fully connected layer 1 (FC1) and the softmax output layer.

[0059] In this embodiment, the detection classification model includes a Softmax layer, which has only 4 nodes. The data output by the transfer learning model is input to the Softmax layer, and the probability value of the class to which the image belongs is output.

[0060] In general, the number of classifications of the original convolutional neural network softmax layer is 1000, and the network finally increases the network classification performance through two fully connected layers. The first fully connected layer requires 1011776 parameters for training, and the second fully connected layer requires 337924 parameters. The original convolutional neural network fully connected layer has too many parameters and cannot improve the accuracy of the model. In the above embodiment, the original two-layer fully connected structure with too many parameters is improved to a three-layer fully connected structure with [256, 128, 4] fewer parameters, and the fully connected layer parameters of the improved network are 524544, 32896 and 516 respectively. The fully connected layer of the improved network is 557956, which is 59% less than the fully connected layer parameter of the original network 1349700.

[0061] S104, training the MyNet network training model based on the training set data to obtain a trained MyNet network training model and obtain optimal weight parameters.

[0062] In this embodiment, the MyNet model independently built is used for training, and the best weight parameter is saved after a number of iterations. One iteration is a complete training cycle of the entire house data set, which is used to evaluate the loss function in gradient descent and update the weight to achieve the optimal training result.

[0063] As Figure 2 The training set and validation set accuracy rate curve of the present application is shown. The horizontal coordinate is the number of iterations, and the vertical coordinate is the value of the accuracy rate. As can be seen, the training accuracy increases rapidly in the early 1 to 30, increases very slowly from 30 to 40, and remains stable during 40 to 50 with the increase of the number of iterations.

[0064] As Figure 3 The training set and validation set loss rate curve of the present application is shown. The horizontal coordinate is the number of iterations, and the vertical coordinate is the value of the loss rate. As can be seen, the training loss rate decreases rapidly in the early 1 to 30, decreases slowly from 30 to 40, and remains stable during 40 to 50 with the increase of the number of iterations.

[0065] S105, inputting the test set data into the trained MyNet network training model, outputting a risk level probability, and taking the level with the highest risk level probability as the house risk level.

[0066] In this embodiment, the class with the maximum probability is the classification probability of the final picture.

[0067] For example, the output result is class: a prob:0.994, class: b prob:0.00567, class: c prob:1.11e-05, class: d prob:1.16e-07, that is, the probability of the house being a class a house is 0.994, that is, 99.4%; the probability of the house being a class b, c, and d house is very small. The house is a class a house.

[0068] For example, the output result indicates that the probability of the risk level being a is 0.0703, the probability of being b is 0.918, and the probability is the largest, while the probabilities of c and d are very small. Therefore, the final determination of the risk level of the house is a class b house.

[0069] According to the embodiments of the present application, the accuracy of the classification of the damage level of the rural house after the flood disaster can be improved, the online intelligent classification can be quickly performed, and a large amount of time, material and financial resources can be saved.

[0070] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0071] The above is the introduction of the method embodiment, and the scheme described in the present application will be further described through the device embodiment.

[0072] As shown in Figure 4 The device 400 includes:

[0073] The acquisition module 410 is configured to acquire a house image and generate an image data set.

[0074] The preprocessing module 420 is configured to preprocess the image data set and divide the preprocessed data into training set data and test set data.

[0075] The model construction module 430 is configured to construct a MyNet network training model.

[0076] The training module 440 is configured to acquire the training set data, train the MyNet network training model based on the training set data, obtain a trained MyNet network training model, and obtain an optimal weight parameter.

[0077] The evaluation module 450 is configured to input the test set data into the trained MyNet network training model, and output a risk level probability, and take the risk level probability with the highest value as the risk level of the house.

[0078] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the described modules can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0079] In the technical solutions of the present application, the acquisition, storage and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0080] According to the embodiments of the present application, the present application further provides an electronic device and a readable storage medium.

[0081] Figure 5 A schematic block diagram of an electronic device 500 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0082] The device 500 includes a computing unit 501 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0083] Various components in the device 500 are connected to the I / O interface 505, including an input unit 506 such as a keyboard, a mouse, etc., an output unit 507 such as various types of displays, speakers, etc., the storage unit 508 such as a magnetic disk, an optical disk, etc., and a communication unit 509 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0084] The computing unit 501 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs various methods and processes described above, such as the methods S101-S105. For example, in some embodiments, the methods S101-S105 can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded onto the RAM 503 and executed by the computing unit 501, one or more steps of the methods S101-S105 described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the methods S101-S105 by any other suitable means, such as by means of firmware.

[0085] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0086] Program code for carrying out methods of the present application can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0087] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store program for use by or in connection with an instruction execution system, apparatus, or device. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable medium can include, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage medium would include one or more lines of electrical wire, portable computer diskette, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0088] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0089] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0090] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server can arise by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0091] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which are not limited herein.

[0092] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for assessing the post-disaster building hazard level based on deep learning, characterized in that, include: Acquire images of houses and generate an image dataset; The image dataset is preprocessed, and the preprocessed data is divided into training set data and test set data; Build the MyNet network training model; The MyNet network training model is trained based on the training set data to obtain the trained MyNet network training model and the optimal weight parameters are obtained. The test set data is input into the trained MyNet network training model, and the risk level probability is output. The risk level with the highest risk level probability is taken as the building risk level. The MyNet network training model includes a training network model, a transfer learning model, and a detection and classification model; the image feature extraction model is used as the training network model, including a shallow feature extraction module and a deep feature extraction module. The input data passes through the shallow feature extraction module, the deep feature extraction module, the transfer learning model, and the detection and classification model in sequence, and then outputs the probability value of the category to which the image belongs. The shallow layer of the feature extraction module consists of a first convolutional layer, a first max pooling layer, a second convolutional layer, a second max pooling layer, and two Fused-MBConv modules; the Fused-MBConv module includes a third convolutional layer and a Dropout layer. The feature extraction module is composed of three MBConv modules; the input data passes through the three MBConv modules in sequence and then outputs feature information. The MBConv module consists of a fourth convolutional layer, an SE module, a fifth convolutional layer, and a Dropout layer, in sequence. The SE module consists of a global average pooling layer, a first fully connected layer, and a second fully connected layer in sequence; the input data is output after passing through the average pooling layer, the first fully connected layer, and the second fully connected layer in sequence.

2. The method according to claim 1, characterized in that, The transfer learning model consists of a sixth convolutional layer, an average pooling layer, and three fully connected layers. The input data is output after passing through the sixth convolutional layer, the average pooling layer, and the three fully connected layers.

3. The method according to claim 1, characterized in that, The detection and classification model includes a Softmax layer. The data output by the transfer learning model is input into the Softmax layer, and the output is the probability value of the category to which the image belongs.

4. A deep learning-based device for assessing the hazard level of buildings after disasters, characterized in that, include: The acquisition module is used to acquire images of houses and generate image datasets; The preprocessing module is used to preprocess the image dataset and divide the preprocessed data into training set data and test set data. The model building module is used to build the MyNet network training model; The training module is used to acquire the training set data to train the MyNet network training model, obtain the trained MyNet network training model, and obtain the optimal weight parameters. The evaluation module is used to input the test set data into the trained MyNet network training model and output the hazard level probability, with the highest hazard level probability being taken as the building hazard level. The deep learning-based post-disaster building hazard assessment device is used to perform the method described in any one of claims 1-3.

5. An electronic device comprising at least one processor; and A memory communicatively connected to the at least one processor; characterized in that, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-3.

6. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-3.

Images