River dam erosion detection method and device and computer equipment
Through the drone collecting river dam images and using segmentation and GAN network models for erosion detection, the accuracy of dam erosion detection in the existing technology is solved, and efficient erosion positioning and early warning are achieved.
Patent Information
- Application Number
- CN202411293772.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2025-01-03
AI Technical Summary
The existing river channel embankment erosion detection methods are difficult to detect erosion in a timely manner, especially due to the location inaccuracy caused by wind speed and GPS errors, which leads to some erosions being unrecognized.
The drone was used to collect river embankment images, extract the river embankment model through segmentation network model, and perform binarization and simulate erosion processing, and generate mask images as data sets of GAN network model for erosion detection and positioning.
It improves the accuracy of dam erosion detection, reduces misjudgment, can automatically set detection thresholds, adapt to different degrees of erosion prediction and early warning levels, and uses the generative adversarial network to enhance the direction of detection.
Smart Images
Figure CN119251709A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of dam detection, and in particular to a method, device and computer equipment for detecting erosion of river dams. Background Art
[0002] The safety of river embankments is related to the safety of life and property of the people in the entire region. During the usual daily inspection process, in addition to paying attention to most of the defects, the place where the embankment borders the river may be easily eroded and form gaps due to daily scouring. If the gaps are not handled in time, it is easy to cause the risk of large-scale collapse. Therefore, erosion detection of river embankments is particularly critical.
[0003] At present, in the existing drone dam health inspection process, people tend to only pay attention to the large-scale collapse defects of the dam, which are often large and easy to find; but for the erosion of river dams, few people care about this kind of harm. At present, the traditional inspection of river dams is often mainly manual, and the detection of this kind of erosion is difficult to detect in time at this stage. The emergence of drone patrol technology and imaging technology provides an effective solution to this problem.
[0004] For this type of erosion detection, one of the common methods is to compare the before and after pictures, and the result can be obtained by comparison. Analogously to the erosion detection at the edge of the river embankment, the shape of the actual embankment detected can be compared with the ideal embankment to obtain the eroded area. However, for the drone inspection platform, due to reasons such as wind speed and GPS errors, it is difficult to obtain the ideal embankment shape from previous real pictures, which makes some river erosion unable to be identified. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide a method, device and computer equipment for detecting river dam erosion, aiming to improve the accuracy of dam erosion detection.
[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for detecting river embankment erosion, comprising:
[0007] The river embankment images are collected by drones and divided into a test set and a training set;
[0008] The river channel dam image is input into the segmentation network model for model training, and the river channel dam model is extracted from the river channel dam image;
[0009] Binarize the river dam image segmented by the segmentation network model to extract the mask image;
[0010] The mask images are processed for simulated erosion of river dikes to increase the number of mask images of erosion samples;
[0011] The processed mask image is used as the data set of the GAN network model, and constraints are added to train the GAN network model so that the data generated by the GAN network model has directionality;
[0012] The input river embankment image is segmented by the trained segmentation network model, and the mask image is extracted by binarization. The mask image is input into the GAN network model for erosion detection and locating the erosion position.
[0013] Furthermore, the collecting of river channel dam images by using a drone and dividing the river channel dam images into a test set and a training set includes:
[0014] Control the drone to mount an optical camera and fly along the river embankment to collect images of the river embankment;
[0015] Use the Labelme labeling tool to label the river channel on the river embankment image;
[0016] The labeled river embankment images are divided into a test set and a training set using the 2:8 criterion, where the test set is 2 and the training set is 8.
[0017] Furthermore, the river channel dam image is input into the segmentation network model for model training, and the river channel dam model is extracted from the river channel dam image, including:
[0018] The river embankment image is input into the segmentation network model. The segmentation network model reversely propagates the error according to the loss of the sample's label data and the predicted data, and then changes the network parameters in the direction of smaller loss until the training stops.
[0019] Furthermore, the step of binarizing the river dam image segmented by the segmentation network model to extract a mask image includes:
[0020] The segmented river dam is subjected to image binarization to extract the mask image, the pixel values of the river dam part are set to white values, and the other parts are set to black values, thereby completing the extraction of the river dam mask image.
[0021] Furthermore, the river dam image is input into the segmentation network model for model training, and the river dam model in the river dam image is extracted. The segmentation network model is a segmentation network model of the YOLO series, and its loss function L is as follows:
[0022] L=λ class L class +λ local L loal +λconf L conf +λ mask L mask
[0023] Among them, L class , L local , L conf , L mask They are classification loss, positioning loss, confidence loss and segmentation loss respectively;
[0024] The segmentation network model takes the minimization of the loss function as the criterion, and uses the gradient descent method to gradually reduce the segmentation network model loss until the training is completed; for L mask , λ is a hyperparameter, A represents the total number of anchor boxes, N represents the total number of recognition masks in an anchor box, M represents the mask of an object, and m i Indicates the probability that a pixel in the mask is assigned to the object. is the label value of the pixel, and finally the binary cross entropy loss is calculated, and the mask losses of all objects are added together as the segmentation loss.
[0025] Furthermore, the processed mask image is used as a data set of the GAN network model, and constraints are added to train the GAN network model, including:
[0026] The style transfer model Pix2Pix is used as a conditional constraint for the GAN network model to ensure that the generator generates pictures that match the conditions. The loss function of the style transfer model Pix2Pix is as follows:
[0027] L Pix2Pix (D, G) = E x,y [log D(x,y)]+E x，z [log(1-D(x,G(x)))]+λL1
[0028] L1=E x，y，z [||yG(x,z)|| 1 ]
[0029] Among them, y represents the original style image, z is random noise, and Pix2Pix directly inputs the original style picture, so that the generator G must generate a picture that matches the original input. In addition to judging whether the generated data conforms to the target style, the discriminator D must also judge whether it matches the original input, thereby making the generation of GAN controllable.
[0030] Furthermore, the GAN network model includes a generator network and a discriminator network. When performing the GAN network model, the discriminator network is trained first, then the generator network is trained, and then the discriminator network is trained, and this is repeated until a Nash equilibrium is reached. The loss function L of the generator model and the discriminator model is G , L D They are as follows:
[0031] L G =H(1,D(G(z)))
[0032] L D =H(1,D(G(x)))+H(0,D(G(z)))
[0033] Among them, x is the real data, z represents random input data, H represents the cross entropy function, G represents the generation network, D represents the discriminant network, 1 represents absolutely real data, 0 represents absolutely false data, L G It represents the distance between the data generated by the discriminator and the absolute real data, that is, the generator loss, L D It represents the sum of the distance that the discriminator judges the true data as true and the distance that the false data is false, that is, the discriminator loss.
[0034] The present invention also provides a river dam erosion detection device, comprising:
[0035] An image acquisition module is used to collect river channel dam images through a UAV, and divide the river channel dam images into a test set and a training set;
[0036] A segmentation network model training module is used to input the river channel dam image into the segmentation network model for model training, and extract the river channel dam model from the river channel dam image;
[0037] The mask image extraction module is used to extract the mask image by binarizing the river dam image segmented by the segmentation network model;
[0038] An image erosion processing module is used to perform simulated erosion processing of river dams on mask images to increase the number of mask images of erosion samples;
[0039] The GAN network model training module uses the processed mask image as the data set of the GAN network model and adds constraints to train the GAN network model so that the data generated by the GAN network model has directionality;
[0040] The erosion position positioning module is used to input the river embankment image into the trained segmentation network model for segmentation, binarize and extract the mask image, and input the mask image into the GAN network model for erosion detection and positioning of the erosion position.
[0041] The present invention also provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the river dam erosion detection method as described in any one of the above items is implemented.
[0042] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, can implement the river embankment erosion detection method as described in any one of the above items.
[0043] The beneficial effects of the present invention are as follows: by inputting a river channel dam image into a segmentation network model for model training, a river channel dam model in the river channel dam image is extracted; the river channel dam image segmented by the segmentation network model is binarized to extract a mask image; the mask image is subjected to river channel dam simulation erosion processing to increase the number of mask images of erosion samples; the processed mask image is used as a data set of a GAN network model, constraints are added to train the GAN network model, so that the data generated by the GAN network model has directionality; the river channel dam image is input into the trained segmentation network model for segmentation, the mask image is binarized to extract the mask image, and the mask image is input into the GAN network model for erosion detection and locating the erosion position. The erosion detection training of this scheme only requires a small number of sample photos for training. This technical scheme can greatly avoid the influence of some other factors on the dam and reduce misjudgment by using segmented and masked images; this technical scheme can automatically set the erosion detection threshold according to the situation to cope with different degrees of erosion prediction detection and warning levels; use generative adversarial networks for erosion detection, and constrain the network based on the fact that erosion will only occur on the side bordering the river, thereby improving the accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The specific structure of the present invention is described in detail below in conjunction with the accompanying drawings.
[0045] Figure 1 This is a flow chart of a method for detecting river dam erosion according to an embodiment of the present invention;
[0046] Figure 2 A network architecture diagram of a YOLO segmentation network model according to an embodiment of the present invention;
[0047] Figure 3 Instance segmentation and semantic segmentation diagram of an embodiment of the present invention;
[0048] Figure 4 A schematic diagram of mask extraction of a river dam according to an embodiment of the present invention;
[0049] Figure 5 This is a structural diagram of a GAN network model according to an embodiment of the present invention;
[0050] Figure 6A schematic diagram of simulated erosion of a dam mask according to an embodiment of the present invention;
[0051] Figure 7 This is a flowchart of the algorithm for detecting erosion according to an embodiment of the present invention;
[0052] Figure 8 A flowchart of experimental verification of erosion detection according to an embodiment of the present invention;
[0053] Fig. 9 The river embankment erosion detection effect of the embodiment of the present invention Figure 1 ;
[0054] Fig.10 The river embankment erosion detection effect of the embodiment of the present invention Figure 2 ;
[0055] Fig.11 A schematic block diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0057] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0058] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0059] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0060] like Figure 1 As shown, the first embodiment of the present invention is: a method for detecting river dam erosion, comprising the steps of:
[0061] S10, collecting river channel dam images through drones, and dividing the river channel dam images into a test set and a training set;
[0062] S20, inputting the river channel dam image into the segmentation network model for model training, and extracting the river channel dam model from the river channel dam image;
[0063] S30, binarizing the river channel dam image segmented by the segmentation network model to extract a mask image;
[0064] S40, performing a simulated erosion process of a river dam on the mask image to increase the number of mask images of erosion samples;
[0065] S50, using the processed mask image as a data set of the GAN network model, adding constraints to train the GAN network model, so that the data generated by the GAN network model has directionality;
[0066] S60, input the river dam image into the trained segmentation network model for segmentation, binarize and extract the mask image, input the mask image into the GAN network model for erosion detection and locating the erosion position.
[0067] In this technical solution, erosion detection training only requires a small number of sample photos for training. This technical solution can greatly avoid the influence of some other factors on the embankment, such as weeds, roads, etc., by using segmented and masked images, thereby reducing misjudgment; this technical solution can automatically set the erosion detection threshold according to the situation to cope with different degrees of erosion prediction detection and warning levels; use generative adversarial networks for erosion detection, and constrain the network based on the fact that erosion will only occur on the side bordering the river, thereby improving the accuracy of detection.
[0068] Among them, step S10, collecting river channel dam images by using a drone, and dividing the river channel dam images into a test set and a training set includes:
[0069] S11, control the drone to mount an optical camera and fly along the river embankment to collect images of the river embankment;
[0070] S12, label the river channel using the Labelme labeling tool on the river channel dam image;
[0071] S13, dividing the marked river embankment images into a test set and a training set using the 2:8 criterion, where the test set is 2 and the training set is 8.
[0072] Among them, step S20, inputting the river channel dam image into the segmentation network model for model training, and extracting the river channel dam model from the river channel dam image includes:
[0073] The river embankment image is input into the segmentation network model. The segmentation network model reversely propagates the error according to the loss of the sample's label data and the predicted data, and then changes the network parameters in the direction of smaller loss until the training stops.
[0074] The data set is further divided to train the segmentation network model and extract the river dam model including:
[0075] 1) The segmentation network model specifically refers to instance segmentation, not semantic segmentation. This is because there may be multiple river dam areas in an image. If semantic segmentation is used, it does not specifically distinguish which dam area is which;
[0076] 2) According to the data set, the segmentation network model will reversely propagate the error based on the loss of the sample's label data and predicted data, and then change the network parameters in the direction of smaller loss until the training stops.
[0077] Furthermore, the instance segmentation and semantic visibility of river embankments Figure 3 As shown: Instance segmentation treats each target as an independent segmentation object, such as Figure 3 The colors of the dams in b are different, and semantic segmentation means that the same type of targets are the same segmentation object, such as Figure 3 As shown in a.
[0078] The segmented river embankment is further subjected to image binarization to extract the mask image, including: the river embankment segmented by YOLO instance will be attached with a layer of mark, see Figure 4 a. According to the marked color, a specific threshold is set to detect the river channel and dam parts in binary form to obtain a mask image. Figure 4 b.
[0079] Wherein, step S30, binarizing the river dam image segmented by the segmentation network model to extract a mask image comprises:
[0080] The segmented river dam is subjected to image binarization to extract the mask image, the pixel values of the river dam part are set to white values, and the other parts are set to black values, thereby completing the extraction of the river dam mask image.
[0081] The mask image is further processed to improve the quality of the mask data set. For river dam erosion detection, the number of samples is very small, which is very unfavorable for the training of the GAN network. It is necessary to manually process the mask image to simulate the eroded dam, which can greatly increase the number of erosion samples. Figure 5is the structure diagram of the GAN network model; in addition, for the river dam, the erosion must occur on the side bordering the river, so when simulating the erosion sample, only the river dam on the side bordering the river is processed, as shown in the following figure. Figure 6 As shown, in this way, the subsequent GAN network generator will only be interested in the mask on the side bordering the river, further improving the directionality and accuracy of the data generated by the GAN network model generator.
[0082] The processed images are further used as the data set of the GAN network, and constraints are added to train the GAN network model to further ensure that the data generated by the GAN network is directional. In order to further ensure that the pictures generated by the generator are consistent with the style of the original mask images, the style transfer model is used to further constrain the generator model to ensure that the generated images are consistent with the style of the input images. The GAN network model training includes the training of the generator G and the discriminator D. The two are trained in the form of mutual game. The final effect is that the data generated by the generator is realistic and the discriminator has excellent discrimination ability.
[0083] Further input the river embankment picture, the segmentation network model first performs segmentation, binarization is performed to extract the mask image, and the mask image is input into the GAN network model for erosion detection and positioning, including: after the segmentation network model and the GAN network model are trained, the river embankment picture collected by the drone can be sent to the segmentation network model for detection, and then binarization is performed to obtain the river embankment mask image, and the mask image is sent to the GAN network model for erosion detection and positioning.
[0084] In step S20, the river dam image is input into the segmentation network model for model training, and the river dam model in the river dam image is extracted. The segmentation network model is a segmentation network model of the YOLO series, and its loss function L is as follows:
[0085] L=λ class L class +λ local L local +λ conf L conf +λ mask L mask
[0086]
[0087] Among them, L class , L local , L conf , L mask They are classification loss, positioning loss, confidence loss and segmentation loss respectively;
[0088] The segmentation network model takes the minimization of the loss function as the criterion, and uses the gradient descent method to gradually reduce the segmentation network model loss until the training is completed; for L mask , λ is a hyperparameter, A represents the total number of anchor boxes, N represents the total number of recognition masks in an anchor box, M represents the mask of an object, and m i Indicates the probability that a pixel in the mask is assigned to the object. is the label value of the pixel, and finally the binary cross entropy loss is calculated, and the mask losses of all objects are added together as the segmentation loss.
[0089] The mask image is further processed to improve the quality of the mask data set: Since erosion rarely occurs in the actual river dam data set samples, it is very unfavorable for the subsequent GAN network training. Therefore, the mask image is simulated for erosion. The mask image of the river dam that is actually eroded by river water is partially transformed into the non-river dam part. The main operation of the binary image is to corrode the specified area and binarize the pixel values of the non-river dam part. In this way, the model of river erosion is completed. This operation greatly increases the number of erosion samples. In addition, erosion must occur on the side of the river dam that borders the river water. Therefore, in the process of simulating river erosion, only the side eroded by the river water is simulated for erosion. This is in line with the actual erosion phenomenon, and it can also allow the subsequent GAN to only focus on the image on the side bordering the river water, ensuring the rationality and accuracy of generating an ideal river dam.
[0090] Further game training of the generator and discriminator of the GAN network model. The GAN network model has two very critical parts. One is the generator model, which is responsible for generating data based on samples. The goal is to make the generated data indistinguishable from the real thing, that is, the image data generated in this method must deceive the discriminator; the discriminator is responsible for distinguishing the generator from the real data, and the goal is to distinguish the real pictures from the fake pictures as much as possible; the relationship between the two is like the relationship between a spear and a shield. In the process of mutual game, they enhance their own capabilities.
[0091] Among them, step S50, using the processed mask image as a data set of the GAN network model, adding constraints to train the GAN network model includes:
[0092] The style transfer model Pix2Pix is used as a conditional constraint for the GAN network model to ensure that the generator generates pictures that match the conditions. The loss function of the style transfer model Pix2Pix is as follows:
[0093] L Pix2Pix (D, G) = E x，y [log D(x,y)]+E x，z[log(1-D(x,G(x)))]+λL1
[0094] L1=E x，y，z [||yG(x,z)|| 1 ]
[0095] Among them, y represents the original style image, z is random noise, and Pix2Pix directly inputs the original style picture, so that the generator G must generate a picture that matches the original input. In addition to judging whether the generated data conforms to the target style, the discriminator D must also judge whether it matches the original input, thereby making the generation of GAN controllable.
[0096] The GAN network model includes a generator network and a discriminator network. When performing the GAN network model, the discriminator network is trained first, then the generator network is trained, and then the discriminator network is trained, and this is repeated until the Nash equilibrium or the set condition is reached; the loss function L of the generator model and the discriminator model is G , L D They are as follows:
[0097] L G =H(1,D(G(z)))
[0098] L D =H(1,D(G(x)))+H(0,D(G(z)))
[0099] Among them, x is the real data, z represents random input data, H represents the cross entropy function, G represents the generation network, D represents the discriminant network, 1 represents absolutely real data, 0 represents absolutely false data, L G It represents the distance between the data generated by the discriminator and the absolute real data, that is, the generator loss, L D It represents the sum of the distance that the discriminator judges the true data as true and the distance that the false data is false, that is, the discriminator loss.
[0100] Further input the river dam picture, the segmentation network model first performs segmentation, binarization extracts the mask image, and inputs the mask image into the GAN network for erosion detection and positioning, including:
[0101] YOLO segmentation network model detection, the first step of the input model is to enter the YOLO segmentation network model detection, segment the dam instance, and obtain the river dam segmentation image; the network architecture of the YOLO segmentation network model is as follows Figure 2 shown.
[0102] Binarization is performed on the segmented river channel dam instance segmentation object to extract the mask image;
[0103] The mask image is sent to the GAN network generator to generate an ideal river dam mask image;
[0104] Perform pixel difference calculation on the input mask image and the image generated by the generator to determine how much the different pixel parts account for the river embankment part. If it is greater than the set threshold, it is judged as erosion, otherwise it is not eroded;
[0105] For the eroded river embankment, the parts with different pixels, i.e. the parts where erosion occurs, are further obtained, and the contours are described to locate the eroded parts, and the contours are returned to the original image.
[0106] Further testing of river bank erosion combined with Figure 7 Note: After training the YOLO segmentation network model and the GAN network model, the entire algorithm training phase is completed. The drone flies along the river dam. The image collected by the camera first enters the YOLO network segmentation network model to extract the river dam part. Then the segmented image is binarized to obtain the mask image of the entire image. The river and dam parts will be white, and the other parts will be black. The mask image at this time is called M1. The M1 image is sent to the GAN network model. The generator of the GAN network model will generate a mask image based on the knowledge learned during training. The mask image M2 is based on the mask image M1. M2 is the mask image of an ideal river embankment. The ideal image M2 is generated by subtracting the original mask image M1. Since M2 is an ideal and intact embankment mask image, there are two situations at this time. If the embankment is eroded, then M2 minus M1 will definitely result in a large difference Dx. If no erosion occurs, the images of M1 and M2 are similar, and the calculated difference Dx is very small. The ratio of the difference Dx to the original M1 value is used as an indicator to determine whether erosion has occurred. If it is greater than a certain threshold, it is determined to be eroded, otherwise it is not eroded.
[0107] A simulation experiment of dam and river erosion was conducted. The whole algorithm flow chart for simulating and detecting dam and river erosion is shown in the attached figure. Figure 8As shown in the figure: In order to demonstrate the actual effect of the algorithm, the effect of river dam erosion is simulated. In the dam mask M1 actually identified and extracted, there is no mask, so preprocessing is needed to match the input requirements of the generator. The river area can be extracted by gradient calculation, so the recognition box is first calculated with a gradient, and then the white area is searched for contours after dilation, erosion, and black and white inversion to obtain the contour with the largest enclosed area, and its interior is assigned to white, and the rest is assigned to black, so that the river area is white. Mask C is obtained, and then M1 is dilated and subtracted from the original M1 to obtain the outer perimeter of the dam mask G, and then the overlapped part of G and C is assigned to 100, and the rest is assigned to 0 to obtain the river water mask M2. Finally, M1 and M2 are added to obtain the dam mask M3 with mask as the generator input. M3 is input into the ideal dam shape generator to obtain the ideal dam mask M4. Then M4 is subtracted from M1. The burrs are first removed by an erosion operation. Finally, the repaired area of the bank is obtained by contour search (corresponding to the defective area D in reality). x , i.e. the dam and river channel erosion area), when the area of the repaired area is larger than T% of the mask area (T is an adjustable parameter used to control the warning level, here it is 5), it is retained and finally marked on the recognition result. Fig. 9 , Fig.10 Shown is the result of dam erosion detection.
[0108] Another embodiment of the present invention is: a river dam erosion detection device, comprising:
[0109] An image acquisition module is used to collect river channel dam images through a UAV, and divide the river channel dam images into a test set and a training set;
[0110] A segmentation network model training module is used to input the river channel dam image into the segmentation network model for model training, and extract the river channel dam model from the river channel dam image;
[0111] The mask image extraction module is used to extract the mask image by binarizing the river dam image segmented by the segmentation network model;
[0112] An image erosion processing module is used to perform simulated erosion processing of river dams on mask images to increase the number of mask images of erosion samples;
[0113] The GAN network model training module uses the processed mask image as the data set of the GAN network model and adds constraints to train the GAN network model so that the data generated by the GAN network model has directionality;
[0114] The erosion position positioning module is used to input the river embankment image into the trained segmentation network model for segmentation, binarize and extract the mask image, and input the mask image into the GAN network model for erosion detection and positioning of the erosion position.
[0115] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned river dam erosion detection device can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.
[0116] The above-mentioned river dam erosion detection device can be implemented in the form of a computer program. The computer program can be used in Fig.11 Runs on the computer device shown.
[0117] See also Fig.11 , Fig.11 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a terminal or a server, wherein the terminal may be an electronic device with communication functions such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device. The server may be an independent server or a server cluster composed of multiple servers.
[0118] See also Fig.11 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .
[0119] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a river dam erosion detection method.
[0120] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .
[0121] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a river dam erosion detection method.
[0122] The network interface 505 is used to communicate with other devices over the network. Fig.11The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0123] The processor 502 is used to run the computer program 5032 stored in the memory to implement the river dam erosion detection method as described above.
[0124] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0125] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.
[0126] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the above-mentioned river channel dam erosion detection method.
[0127] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, or any other computer-readable storage medium that can store program codes.
[0128] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0129] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
[0130] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0131] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.
[0132] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention, and these modifications or substitutions should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A method for detecting river embankment erosion, characterized in that: include: The river embankment images are collected by drones and divided into a test set and a training set; The river channel dam image is input into the segmentation network model for model training, and the river channel dam model is extracted from the river channel dam image; Binarize the river dam image segmented by the segmentation network model to extract the mask image; The mask images are processed for simulated erosion of river dikes to increase the number of mask images of erosion samples; The processed mask image is used as the data set of the GAN network model, and constraints are added to train the GAN network model so that the data generated by the GAN network model has directionality; The input river embankment image is segmented by the trained segmentation network model, and the mask image is extracted by binarization. The mask image is input into the GAN network model for erosion detection and locating the erosion position.
2. The method for detecting river dam erosion according to claim 1, characterized in that: The collecting of river channel dam images by using a drone and dividing the river channel dam images into a test set and a training set includes: Control the drone to mount an optical camera and fly along the river embankment to collect images of the river embankment; Use the Labelme labeling tool to label the river channel on the river embankment image; The labeled river embankment images are divided into a test set and a training set using the 2:8 criterion, where the test set is 2 and the training set is 8.
3. The river dam erosion detection method according to claim 1, characterized in that: The river dam image is input into the segmentation network model for model training, and the river dam model is extracted from the river dam image, including: The river embankment image is input into the segmentation network model. The segmentation network model reversely propagates the error according to the loss of the sample's label data and the predicted data, and then changes the network parameters in the direction of smaller loss until the training stops.
4. The method for detecting river dam erosion according to claim 1, characterized in that: The step of binarizing the river channel dam image segmented by the segmentation network model to extract a mask image comprises: The segmented river dam is subjected to image binarization to extract the mask image, the pixel values of the river dam part are set to white values, and the other parts are set to black values, thereby completing the extraction of the river dam mask image.
5. The method for detecting river dam erosion according to claim 1, characterized in that: The river dam image is input into the segmentation network model for model training, and the river dam model in the river dam image is extracted. The segmentation network model is a segmentation network model of the Y0L0 series, and its loss function L is as follows: L=λ class L class +λ local L local +λ conf L conf +λ mask L mask Among them, L class , L local , L conf , L mask They are classification loss, positioning loss, confidence loss and segmentation loss respectively; The segmentation network model takes the minimization of the loss function as the criterion, and uses the gradient descent method to gradually reduce the segmentation network model loss until the training is completed; for L mask , λ is a hyperparameter, A represents the total number of anchor boxes, N represents the total number of recognition masks in an anchor box, M represents the mask of an object, and m i Indicates the probability that a pixel in the mask is assigned to the object. is the label value of the pixel, and finally the binary cross entropy loss is calculated, and the mask losses of all objects are added together as the segmentation loss.
6. The method for detecting river dam erosion according to claim 1, characterized in that: The method of using the processed mask image as a data set of the GAN network model and adding constraints to train the GAN network model includes: The style transfer model Pix2Pix is used as a conditional constraint for the GAN network model to ensure that the generator generates pictures that match the conditions. The loss function of the style transfer model Pix2Pix is as follows: L Pix2Pix (D,G)=E x，y log D(x,y)]+E x，z [log(1-D(x,G(x)))]+λL1 L1=E x，y，z [||yG(x,z)||1] Among them, y represents the original style image, z is random noise, and Pix2Pix directly inputs the original style picture, so that the generator G must generate a picture that matches the original input. In addition to judging whether the generated data conforms to the target style, the discriminator D must also judge whether it matches the original input, thereby making the generation of GAN controllable.
7. The river dam erosion detection method according to claim 6, characterized in that: The GAN network model includes a generator network and a discriminator network. When performing the GAN network model, the recognition network is trained first, then the generator network is trained, and then the recognition network is trained, and this is repeated until a Nash equilibrium is reached. The loss function L of the generator model and the discriminator model is G , L D They are as follows: L G =H(1,D(G(z))) L D =H(1,D(G(x)))+H(0,D(G(z))) Among them, x is the real data, z represents random input data, H represents the cross entropy function, G represents the generation network, D represents the discriminant network, 1 represents absolutely real data, 0 represents absolutely false data, L G It represents the distance between the data generated by the discriminator and the absolute real data, that is, the generator loss, L D It represents the sum of the distance that the discriminator judges the true data as true and the distance that the false data is false, that is, the discriminator loss.
8. A river dam erosion detection device, characterized in that: include: An image acquisition module is used to collect river channel dam images through a UAV, and divide the river channel dam images into a test set and a training set; A segmentation network model training module is used to input the river channel dam image into the segmentation network model for model training, and extract the river channel dam model from the river channel dam image; The mask image extraction module is used to extract the mask image by binarizing the river dam image segmented by the segmentation network model; An image erosion processing module is used to perform simulated erosion processing of river dams on mask images to increase the number of mask images of erosion samples; The GAN network model training module uses the processed mask image as the data set of the GAN network model and adds constraints to train the GAN network model so that the data generated by the GAN network model has directionality; The erosion position positioning module is used to input the river embankment image into the trained segmentation network model for segmentation, binarize and extract the mask image, and input the mask image into the GAN network model for erosion detection and positioning of the erosion position.
9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the river dam erosion detection method according to any one of claims 1 to 7 when executing the computer program.
10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the river embankment erosion detection method according to any one of claims 1 to 7 can be implemented.