Dam defect identification and analysis method and device adaptive to multi-scale characteristics

Abstract

The invention provides a dam defect identification and analysis method and device adaptive to multi-scale characteristics, and the method comprises the steps: obtaining a dam appearance image of a current evaluation period, and carrying out the classification of the image based on a binarization technology, and obtaining a to-be-identified defect region image; the to-be-identified defects comprise cracks and / or water seepage; extracting shape features of the to-be-recognized defect from the to-be-recognized defect area image; inputting the shape features of the to-be-identified defects into a defect identification model based on multi-scale features to obtain a first defect identification result of the dam in the current evaluation period; on the basis of the first defect identification result of the dam in the current evaluation period and the second defect identification result of the dam in the previous evaluation period, dam apparent overall safety score information is obtained; and evaluating the overall performance health state of the dam based on the dam apparent overall safety score information. The problems of human resource waste, low efficiency, large subjective influence on results and the like existing in manual inspection can be solved.

Description

Technical Field

[0001] The present application relates to the field of dam safety management, as well as artificial intelligence technologies such as deep learning, and in particular to a dam defect identification and analysis method and device that are adaptable to multi-scale characteristics. Background Art

[0002] With the rapid development of artificial intelligence (AI), intelligent inspection methods such as robots and drones have become an alternative to superficial manual dam inspections. The widespread use of technologies like drones and robots has also led to the emergence of massive amounts of image data and the continuous improvement of defect recognition algorithms, providing strong data support for dam safety assessments. Summary of the Invention

[0003] The embodiments of the present application provide a dam defect identification and analysis method and device that are adaptable to multi-scale characteristics.

[0004] According to a first aspect of an embodiment of the present application, a dam defect identification and analysis method adapted to multi-scale features is provided, comprising the following steps:

[0005] Obtaining a surface image of the dam in a current assessment period, and classifying the surface image of the dam based on a binarization technique to obtain an image of a defect area to be identified; the defect to be identified includes cracks and / or water seepage;

[0006] Extracting shape features of the defect to be identified from the image of the defect area to be identified;

[0007] Inputting the shape features of the defect to be identified into a defect recognition model based on multi-scale features to obtain a first defect recognition result of the dam in the current assessment period;

[0008] Obtaining apparent overall safety score information of the dam based on a first defect identification result of the dam in the current assessment cycle and a second defect identification result of the dam in the previous assessment cycle;

[0009] Based on the apparent overall safety score information of the dam, the overall health status of the dam is evaluated.

[0010] According to a second aspect of an embodiment of the present application, a dam defect identification and analysis device adapted to multi-scale features is provided, comprising:

[0011] A classification module is used to obtain an apparent image of the dam in the current assessment period and classify the apparent image of the dam based on a binarization technique to obtain an image of a defective area to be identified; the defect to be identified includes cracks and / or water seepage;

[0012] An extraction module, configured to extract shape features of the defect to be identified from the image of the defect area to be identified;

[0013] an identification module, configured to input the shape features of the defect to be identified into a defect identification model based on multi-scale features to obtain a first defect identification result of the dam in the current assessment period;

[0014] A score acquisition module, configured to acquire information on the apparent overall safety score of the dam based on the first defect identification result of the dam in the current assessment cycle and the second defect identification result of the dam in the previous assessment cycle;

[0015] An evaluation module is used to evaluate the overall health status of the dam based on the apparent overall safety score information of the dam.

[0016] According to a third aspect of the embodiments of the present application, there is provided an electronic device, including:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0020] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, which stores instructions. When the instructions are executed on an electronic device, the electronic device executes the method described in the first aspect above.

[0021] According to a fifth aspect of the embodiments of the present application, a program product is provided, which implements the steps of the method described in the first aspect when the instructions in the program product are executed by a processor.

[0022] According to the technical solution of this application, defect identification is performed based on the shape characteristics of the defects to be identified, which can optimize the model input structure, improve algorithm efficiency and simulation accuracy, achieve real-time information perception, and quickly discover dam defects and safety hazards. Based on the first defect identification result of the dam in the current assessment cycle and the second defect identification result of the dam in the previous assessment cycle, the dam's apparent overall safety score information is obtained, and based on the dam's apparent overall safety score information, the overall health status of the dam is evaluated to achieve scientific management and control of dam safety. This can solve the problems of human resource waste, low efficiency, subjective influence on results, and lack of scientificity in manual inspections, and also provide efficient support for the intelligent development of inspections and digital management.

[0023] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0025] Figure 1 A schematic flow chart of a dam defect identification and analysis method adapted to multi-scale characteristics provided in an embodiment of the present application;

[0026] Figure 2 A schematic diagram of the model structure of a defect recognition model based on multi-scale features provided in an embodiment of the present application;

[0027] Figure 3 A block diagram of a dam defect identification and analysis device adapted to multi-scale features provided in an embodiment of the present application;

[0028] Figure 4 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0030] It should be noted that, in the description of this application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in this article is merely a way to describe the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0031] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application. The singular forms "a", "an" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0032] It should be understood that although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to a determination."

[0033] The following describes a dam defect identification and analysis method and apparatus adapted to multi-scale features according to an embodiment of the present application with reference to the accompanying drawings.

[0034] It should be noted that the dam defect identification and analysis method adapted for multi-scale features in the embodiments of the present application may be performed by a dam defect identification and analysis device adapted for multi-scale features. This device may be implemented using software and / or hardware and may be configured in an electronic device. Exemplary electronic devices may include, but are not limited to, terminals and servers.

[0035] Figure 1 This is a flow chart of the dam defect identification and analysis method adapted to multi-scale characteristics provided in the embodiment of the present application. Figure 1 As shown, the dam defect identification and analysis method adapted to multi-scale features may include but is not limited to the following steps.

[0036] In step 101, the surface image of the dam in the current evaluation period is obtained, and the surface image of the dam is classified based on the binarization technology to obtain an image of the defect area to be identified; the defects to be identified include cracks and / or water seepage.

[0037] In the embodiments of the present application, the apparent image of the dam can be obtained by capturing images of the dam using an image acquisition device. For example, the image acquisition device can be a camera, a mobile phone, a robot with image acquisition capabilities, or a drone. For example, the apparent image data of the dam can be captured using a drone.

[0038] In an embodiment of the present application, a dam surface image can be periodically collected, and dam defects within the current evaluation period can be identified and analyzed based on the dam surface image. For example, the evaluation period can be one week, that is, when the evaluation period is reached, the dam surface image can be collected by a drone. After obtaining the dam surface image, the dam surface image can be binarized to obtain a corresponding binary image. Taking the dam cracks and seepage areas as black as an example, it can be determined based on the grayscale value of each pixel in the binary image whether the dam has a defect area to be identified. If there are pixels with a grayscale value of 0 in the binary image, it can be considered that the dam has a defect area to be identified, and the binary image or the dam surface image can be used as the image of the defect area to be identified; if there are no pixels with a grayscale value of 0 in the binary image, it can be considered that the dam does not have a defect area to be identified.

[0039] In step 102, shape features of the defect to be identified are extracted from the image of the defect area to be identified.

[0040] In the embodiment of the present application, taking the image of the defect area to be identified as a binary image as an example, the shape features of the defect to be identified can be extracted from the binary image based on the grayscale value of each pixel in the binary image. In the embodiment of the present application, taking the image of the defect area to be identified as a color dam surface image as an example, a pre-trained neural network model can be used to extract the shape features of the defect to be identified from the image of the defect area to be identified, but it is not limited to this. Other technical means can also be used to extract the shape features of the defect to be identified, for example, the shape features of the defect to be identified can be extracted based on the pixel value of each pixel in the image, etc.

[0041] In step 103, the shape features of the defects to be identified are input into a defect identification model based on multi-scale features to obtain a first defect identification result of the dam in the current assessment cycle.

[0042] In an embodiment of the present application, the defect recognition model based on multi-scale features can be improved based on the traditional defect recognition model. In some embodiments, the network structure of the defect recognition model based on multi-scale features can be a YOLOv5 network structure, including: an input end, a backbone network, a fusion network and a prediction network, wherein the input end can be used to perform Mosaic data enhancement on the shape features of the input defect to be identified; the backbone network can include a Focus structure, a CSP structure and an SPPF structure, the defect recognition model can extract layer features through a cross-stage local network CSP structure, and complete multi-scale feature fusion based on an optimized spatial pyramid pooling module SPPF, the Focus structure at the front end of the network can be used to slice the input data; the fusion network can be combined with the ASFF module in the neck network Neck to fuse image features of different scales; the prediction network can be used to output three sets of feature maps of different scales and corresponding prediction boxes.

[0043] In other words, the adaptive spatial feature fusion (ASFF) module can be superimposed on the YOLOv5 network to improve the accuracy of the model in image defect recognition. The network structure of the defect recognition model based on multi-scale features can be composed of four parts, such as Figure 2 As shown in the figure, they are the input, backbone, fusion network (Neck-ASFF), and prediction network. Conv represents the parameters related to the convolution module, which can include: the number of input channels, the number of output channels, the convolution kernel size, and the stride. C3 and SPPF can include the number of input channels and the number of output channels of the module. At the input, the YOLOv5 model mainly performs mosaic data enhancement on the image. In the backbone network, it mainly includes Focus, CSP, and SPPF structures. The model extracts layer features through the cross-stage local network CSP and then completes multi-scale feature fusion based on the optimized spatial pyramid pooling (SPPF) module. The Focus structure, located at the front end of the network, mainly slices the input data, effectively improving data extraction quality. The fusion network mainly combines the ASFF module in the neck network, which can greatly enhance the fusion and transmission of features at different scales. In the prediction network, the integrated output is obtained to obtain three sets of feature maps of different scales and corresponding prediction boxes.

[0044] Adaptive spatial feature fusion (ASFF) can fuse features of images at different scales to improve recognition performance. The principle is that during the feature fusion process, the model automatically assigns weights to feature maps of different scales through learning. In some embodiments, the ASFF module is used to automatically assign weights to feature maps of different scales through learning during the feature fusion process. The fusion process is as follows: the second feature map X is added to the three feature maps output by the neck network Neck using the upsampling method.2 , the third feature map X 3 The scale and number of channels are converted to the first feature map X 1 Consistent, get the feature map X 1→1 , feature map X 2→1 , and feature map X 3→1 ; Along the channel direction, the feature map X 1→1 With feature map X 2→1 and feature map X 3→1 Calculate the fusion and use a 1×1 convolution kernel to adjust the number of channels to 3. Finally, calculate the weight of each layer during fusion based on the Softmax function. The calculation formula is as follows:

[0045]

[0046] in, is the output feature map Y l The feature vector of the location (i, j); is the feature vector at position (i, j) converted from level n to level l feature map; are the weights of the fusion of feature maps of three different scales, and are calculated in the same way; Represents the Softmax function fusion layer The corresponding control parameters when the parameters meet the conditions

[0047] For example, Figure 2 As shown, X 1 、X 2 、X 3 They represent the feature maps output by the YOLOv5 path aggregation network, Y 1 It is the new feature map after fusion. Before fusion, the feature map X is up-sampled. 2 、X 3 The scale and number of channels are converted to the same as X 1 Consistent, then follow the channel direction to move X 1→1 With the new feature map X 2→1 、X 3→1 Calculate the fusion and use a 1×1 convolution kernel to adjust the number of channels to 3. Finally, calculate the weight of each layer during fusion based on the Softmax function. The specific calculation formulas are shown in the above formulas (1) and (2).

[0048] It should be noted that the multi-scale feature-based defect recognition model can be pre-trained. In some embodiments, the training method of the multi-scale feature-based defect recognition model may include the following steps 1) to 3):

[0049] Step 1) Based on the dam underlying database index and calling the defect images in the dam defect image database, a model input data set is established to provide data input for the defect recognition model. The input data set contains image data of dam cracks and seepage.

[0050] It should be noted that water seepage and cracks are the most common surface defects in dams and are a key focus of daily attention for hydraulic structures. Therefore, when constructing a dam underlying database, we collected past and current image data from various dam locations and other basic facility data to form the dam image database. We used 3D panoramic and GIS technology to capture image information of the dam's exterior and interior, correlating it with the dam's location and environmental information to achieve precise positioning and 3D visualization of the image data. Furthermore, because past inspection results often consisted of unstructured data (non-standardized text descriptions, images, videos, etc.) or image size and resolution were inconsistent, dam image data was preprocessed to normalize image length and width to optimize the model input structure and improve algorithm efficiency. Then, using binarization techniques, we classified the images in the dam image database, marking those containing cracks and water seepage. The shape features of these cracks and water seepage were statistically analyzed to form the dam defect image feature database. The dam underlying database includes four interfaces: an image index retrieval query interface, a database call interface, a 3D display interface, and a comprehensive analysis and evaluation system. The image index query interface retrieves images of the dam section based on the input location, image location, and shooting time; the database call interface is responsible for data table operations such as connection, query, modification, and storage between system users and the database; the three-dimensional display interface retrieves the three-dimensional data of the data model to realize three-dimensional display; the comprehensive analysis and evaluation system analyzes the safety status of the dam based on the current inspection results.

[0051] Step 2) Use two-thirds of the input data set as the training set for the defect recognition model, and the remaining one-third as the validation set. The training set data is used to train the multi-scale feature-based defect recognition model and calibrate the model's hyperparameters. The network structure of the multi-scale feature-based defect recognition model can be found in the description of the previous embodiment and will not be repeated here.

[0052] Step 3) Input the validation set data into the trained multi-scale feature-based defect recognition model, perform defect recognition on the validation set data, and compare it with the corresponding manually labeled defect images to complete the recognition effect evaluation. The evaluation method can adopt an indicator evaluation method, and the evaluation indicators can include precision (Precision, P), recall (Recall, R) and mean average precision (mAP). The calculation formulas for each evaluation indicator are as follows:

[0053] Accuracy:

[0054]

[0055] Recall:

[0056]

[0057] Mean Average Precision:

[0058]

[0059] In formulas (3) and (4), TP, FP, and FN represent the number of defects correctly detected by the model, the number of defects incorrectly identified by the model, and the number of defects not identified by the model, respectively. In formula (5), N represents the number of categories, and AP represents the mean of the precision P at different recall rates R, and its value is the integral of the PR curve.

[0060] In summary, the image data of various locations of the dam in the past and present and other basic data of facilities can be collected as a dam image database, and the images in the dam image database can be used as training data to complete the training of the defect recognition model based on multi-scale features. By using the pre-trained defect recognition model based on multi-scale features to identify the shape features of the defects to be identified, the first defect recognition result of the dam in the current assessment cycle can be obtained. As an example, the first defect recognition result can include whether there are new cracks (for example, it can be specifically divided into none, a small amount, a large amount, severe, etc.), and it can also include whether there are new seepage points (for example, it can be specifically divided into none, a small amount, a large amount, severe, etc.), and it can also include but not be limited to three groups of feature maps of different scales and corresponding prediction boxes, etc. Optionally, the first defect recognition result can also include the crack area and / or seepage area, etc.

[0061] In step 104, based on the first defect identification result of the dam in the current assessment cycle and the second defect identification result of the dam in the previous assessment cycle, the apparent overall safety score information of the dam is obtained.

[0062] In some embodiments, the crack area change rate and the seepage area change rate can be determined based on the first defect identification result of the dam in the current evaluation cycle and the second defect identification result of the dam in the previous evaluation cycle; the safety score information of each evaluation indicator can be determined based on the first defect identification result, the crack area change rate, the seepage area change rate, and the working status information of other facilities of the dam; and the weighted summation processing is performed based on the safety score information of each evaluation indicator and the weight of each evaluation indicator to obtain the apparent overall safety score information of the dam.

[0063] In some embodiments, the above-mentioned evaluation indicators may include crack condition information, water seepage condition information, and other facility condition information. In one possible implementation, the optional implementation method of determining the safety score information of each evaluation indicator based on the first defect identification result, the crack area change rate, the water seepage area change rate, and the working condition information of other facilities of the dam may include: determining the safety score information of the crack condition information based on the first defect identification result and the crack area change rate and its corresponding weight; determining the safety score information of the water seepage condition information based on the first defect identification result and the water seepage area change rate and its corresponding weight; assigning values ​​to each sub-indicator in the other facility condition information based on the working condition information of other facilities of the dam and the sub-indicator assignment rule information to obtain the score information of each sub-indicator, and determining the safety score information of the other facility condition information based on the score information of each sub-indicator and its corresponding weight.

[0064] In some embodiments, an optional implementation method for determining the safety score information of the crack condition information based on the first defect identification result, the crack area change rate, and their corresponding weights includes: assigning a value to the first defect identification result based on the first defect identification result and the crack area change rate in combination with the sub-indicator assignment rule information to obtain first scoring information for the first defect identification result, and determining the safety score information of the crack condition information based on the first scoring information of the first defect identification result, the crack area change rate, and their respective weights, wherein the first defect identification result includes whether there are new cracks. As an example, the calculation formula for the safety score information of the crack condition information is as follows: Safety score information of the crack condition information = first scoring information × α1 + (1-crack area change rate) × β1, where α1 and β1 are weights, respectively.

[0065] In some embodiments, the optional implementation method for determining the safety score information of the water seepage condition information based on the first defect identification result and the water seepage area change rate and their corresponding weights includes: assigning a value to the first defect identification result based on the first defect identification result and the water seepage area change rate combined with the sub-indicator assignment rule information to obtain second score information of the first defect identification result, and determining the safety score information of the water seepage condition information based on the second score information of the first defect identification result, the water seepage area change rate, and their respective weights, wherein the first defect identification result includes whether there are new water seepage points. As an example, the calculation formula for the safety score information of the water seepage condition information is expressed as follows: Safety score information of the water seepage condition information = second score information × α2 + (1-crack area change rate) × β2, where α2 and β2 are weights respectively.

[0066] For example, a dam apparent safety rating evaluation system can be developed to analyze the safety status of the dam based on apparent defects. This can include three steps: establishing an overall inspection safety rating and formulating scoring standards; building an indicator system and determining each sub-item indicator; and calculating the weighted scores for each inspection content based on the "indicator evaluation method" and "scoring method" to derive the dam apparent defect safety score, thus forming a scientific and intelligent dam apparent defect inspection safety evaluation system.

[0067] (1) Overall inspection safety level classification

[0068] The apparent overall inspection safety level of the dam can be divided into four levels: A, B, C, and D. The evaluation levels are normal, basically normal, abnormal, and seriously abnormal. The lower the score level, the lower the score. The specific score ranges and evaluation levels of each level are shown in Table 1.

[0069] Table 1 Overall safety rating standards

[0070]

[0071] (2) Grading index scoring

[0072] Determine the content of the sub-indicators and divide each evaluation indicator into four levels: a, b, c, and d. Use the expert scoring method to determine the weight of each sub-indicator. The corresponding grading standards, scoring intervals, and evaluation principles (i.e., sub-indicator assignment rules) are shown in Table 2. The calculation principle of each indicator safety score can be as follows:

[0073] Crack area change rate L = (crack area in comparison period - crack area in evaluation period) / crack area in comparison period, where the crack area in comparison period is the crack area in the previous evaluation period (e.g., it can be determined based on the shape characteristics of the cracks), and the crack area in the evaluation period can be understood as the crack area in the current evaluation period;

[0074] Water seepage area change rate C = (water seepage area in comparison period - water seepage area in evaluation period) / water seepage area in comparison period, where the water seepage area in comparison period is the water seepage area in the previous evaluation period (e.g., it can be determined based on the shape characteristics of the seepage), and the water seepage area in the evaluation period can be understood as the water seepage area in the current evaluation period;

[0075] Crack status = presence or absence of new cracks × α1 + (1-crack area change rate L) × β1;

[0076] Water seepage condition = whether there are new water seepage points × α2 + (1-water seepage area change rate C) × β2;

[0077] Other facilities status = working status of water-stopping facilities × α3 working status of drainage facilities × β3;

[0078] Table 2 Evaluation grading table of sub-item indicators

[0079]

[0080] For example, the value assigned to the "Presence of New Cracks" sub-item indicator can be determined from Table 2 based on the specific content of "Presence of New Cracks" and the crack area change rate. For example, assuming that the specific content of "Presence of New Cracks" is "A Small Amount" and the crack area change rate L is 0.4, the value assigned to the "Presence of New Cracks" sub-item indicator can be based on "70 < n ≤ 90" in Table 2. For example, a value can be randomly selected from 70 < n ≤ 90 as the value of the "Presence of New Cracks" sub-item indicator, such as n = 80. Similarly, the value assigned to the "Presence of New Seepage Points" sub-item indicator can be implemented. For the sub-indicators of "Working condition of water-stopping facilities" and "Working condition of drainage facilities" in the evaluation indicators of "other facilities conditions", a value can be randomly selected from the corresponding value range of n based on the specific content of the "Working condition of water-stopping facilities" to assign a value to the sub-indicator of "Working condition of water-stopping facilities". Similarly, a value can be randomly selected from the corresponding value range of n based on the specific content of the "Working condition of drainage facilities" to assign a value to the sub-indicator of "Working condition of drainage facilities", thereby realizing the assignment of values ​​to each sub-indicator. Based on the assignment of each sub-indicator and the above-mentioned safety score calculation formula for the evaluation indicators, the safety score information of the above-mentioned crack conditions, water seepage conditions, and other facilities conditions evaluation indicators can be obtained.

[0081] (3) Dam apparent defect safety score

[0082] Based on the determined evaluation indicators (such as crack conditions, water seepage, and other facilities), each indicator is assigned a corresponding weight to calculate the dam's apparent overall safety score. For example, the weights for each evaluation indicator can be based on the weights shown in Table 3. The formula for calculating the dam's apparent overall safety score N can be as follows: Dam's apparent overall safety score N = Crack conditions × 0.5 + Water seepage × 0.3 + Other facilities × 0.2.

[0083] Table 3 Weight classification of each indicator in the comprehensive score of dam apparent safety

[0084]

[0085] In step 105, the overall health status of the dam is evaluated based on the dam apparent overall safety score information.

[0086] In some embodiments, when the assessment result of the overall health status of the dam is abnormal or seriously abnormal, corresponding emergency control measures information is generated based on the assessment result, and the emergency control measures information is pushed to the corresponding maintenance personnel. The emergency control measures information is used to indicate that corresponding measures should be taken for the defective parts of the dam.

[0087] In an embodiment of the present application, when the apparent overall safety score information of the dam is obtained, the apparent overall safety score information of the dam can be compared with the evaluation threshold in the preset health status evaluation grade standard to determine within which evaluation threshold value range the apparent overall safety score information of the dam falls. The health status evaluation grade corresponding to the evaluation threshold is then used as the evaluation result of the overall health status of the dam.

[0088] For example, using Table 1 above, if the dam's apparent overall safety score is less than or equal to 60 points, the safety level is D, indicating a severe abnormal condition. If the dam's apparent overall safety score is greater than 60 and less than or equal to 70 points, the safety level is C, indicating an abnormal condition. If the dam's apparent overall safety score is greater than 70 and less than or equal to 90 points, the safety level is B, indicating a basically normal condition. If the dam's apparent overall safety score is greater than 90 and less than or equal to 100 points, the safety level is A, indicating a normal condition. When the safety condition is normal or basically normal, it means the dam can operate normally. Defects must be tracked and their conditions recorded promptly. When the safety condition is abnormal, the defective parts must be repaired promptly to prevent further changes that may affect the normal operation of the dam. When the safety condition is seriously abnormal, it must be recorded and reported immediately, and appropriate safety measures must be taken to avoid causing damage or accidents.

[0089] In the above-mentioned embodiments, the present application uses the shape features of the defects to be identified for defect recognition, which can optimize the model input structure, improve algorithm efficiency and simulation accuracy. Furthermore, the present application adopts a multi-scale feature defect recognition model and integrates the ASFF module into the neck network (Neck) of the traditional YOLOv5 model, significantly enhancing the fusion and transmission effect of features at different scales, improving the model's accuracy in image defect recognition, and meeting the demand for dam defect recognition accuracy in different scenarios. The present application constructs a safety assessment system based on dam surface images. The safety assessment system seamlessly integrates with the current "unmanned inspection" technology of dams, enabling a one-time rapid safety assessment for daily dam inspections, addressing the problems of manual inspections and assessments, such as waste of human resources, low efficiency, subjective influence on results, and lack of scientificity. The present application has image index query, database call, defect recognition, safety assessment, and 3D display functions. After obtaining daily dam inspection images, it automatically identifies dam surface defects and evaluates their safety level, and provides corresponding treatment recommendations, providing an efficient and scientific solution for dam safety management.

[0090] Figure 3 This is a block diagram of a dam defect identification and analysis device adapted to multi-scale features provided in an embodiment of the present application. Figure 3 As shown, the dam defect identification and analysis device adapted to multi-scale features may include: a classification module 301 , an extraction module 302 , an identification module 303 , a score acquisition module 304 and an evaluation module 305 .

[0091] Among them, the classification module 301 is used to obtain the dam surface image of the current evaluation period, and classify the dam surface image based on the binarization technology to obtain the image of the defect area to be identified; the defects to be identified include cracks and / or water seepage.

[0092] The extraction module 302 is used to extract the shape features of the defect to be identified from the image of the defect area to be identified.

[0093] The identification module 303 is used to input the shape features of the defects to be identified into the defect identification model based on multi-scale features to obtain the first defect identification result of the dam in the current assessment cycle.

[0094] The score acquisition module 304 is used to obtain the apparent overall safety score information of the dam based on the first defect identification result of the dam in the current assessment cycle and the second defect identification result of the dam in the previous assessment cycle.

[0095] The evaluation module 305 is used to evaluate the overall health status of the dam based on the dam's apparent overall safety score information.

[0096] In some embodiments, the scoring acquisition module 304 is used to: determine the crack area change rate and the seepage area change rate based on the first defect identification result of the dam in the current evaluation cycle and the second defect identification result of the dam in the previous evaluation cycle; determine the safety score information of each evaluation indicator based on the first defect identification result, the crack area change rate, the seepage area change rate, and the working status information of other facilities of the dam; perform weighted summation processing based on the safety score information of each evaluation indicator and the weight of each evaluation indicator to obtain the apparent overall safety score information of the dam.

[0097] In some embodiments, each evaluation indicator includes information on crack conditions, water seepage conditions, and other facility conditions. In this embodiment of the present application, the score acquisition module 304 is configured to: determine safety score information for the crack conditions based on the first defect identification result, the crack area change rate, and its corresponding weight; determine safety score information for the water seepage conditions based on the first defect identification result, the water seepage area change rate, and its corresponding weight; assign values ​​to each sub-indicator in the other facility condition information based on the working condition information of the dam's other facilities and information on sub-indicator assignment rules to obtain score information for each sub-indicator; and determine safety score information for the other facility condition information based on the score information for each sub-indicator and its corresponding weight.

[0098] In some embodiments, the scoring acquisition module 304 is used to: assign a value to the first defect identification result based on the first defect identification result and the crack area change rate combined with the sub-indicator assignment rule information to obtain first scoring information of the first defect identification result, and determine the safety scoring information of the crack status information based on the first scoring information of the first defect identification result, the crack area change rate and their respective weights, wherein the first defect identification result includes whether there are new cracks.

[0099] In some embodiments, the scoring acquisition module 304 is used to: assign a value to the first defect identification result based on the first defect identification result and the seepage area change rate combined with the sub-indicator assignment rule information, to obtain second scoring information of the first defect identification result, and determine the safety scoring information of the seepage condition information based on the second scoring information of the first defect identification result, the seepage area change rate and their respective weights, wherein the first defect identification result includes whether there are new seepage points.

[0100] In some embodiments, the calculation formula for the safety score information of the crack condition information is expressed as follows: Safety score information of the crack condition information = first score information × α1 + (1-crack area change rate) × β1, where α1 and β1 are weights respectively; the calculation formula for the safety score information of the water seepage condition information is expressed as follows: Safety score information of the water seepage condition information = second score information × α2 + (1-crack area change rate) × β2, where α2 and β2 are weights respectively.

[0101] In some embodiments, the evaluation module 305 is also used to: when the evaluation result of the overall health status of the dam is abnormal or seriously abnormal, generate corresponding emergency control measures information based on the evaluation result, and push the emergency control measures information to corresponding maintenance personnel. The emergency control measures information is used to indicate that corresponding measures should be taken for the defective parts of the dam.

[0102] In some embodiments, the network structure of the defect recognition model based on multi-scale features is a YOLOv5 network structure, including: an input end, a backbone network, a fusion network and a prediction network, wherein the input end is used to perform Mosaic data enhancement on the shape features of the input defects to be identified; the backbone network includes a Focus structure, a CSP structure and an SPPF structure. The defect recognition model extracts layer features through a cross-stage local network CSP structure, and completes multi-scale feature fusion based on an optimized spatial pyramid pooling module SPPF. The Focus structure located at the front end of the network is used to slice the input data; the fusion network combines the ASFF module in the neck network Neck to fuse image features of different scales; the prediction network is used to output three sets of feature maps of different scales and corresponding prediction boxes.

[0103] In some embodiments, the ASFF module is used to automatically assign weights to feature maps of different scales through learning during the feature fusion process. The fusion process is as follows: the second feature map X is added to the three feature maps output by the neck network Neck by upsampling. 2 , the third feature map X 3 The scale and number of channels are converted to the first feature map X 1 Consistent, get the feature map X 1→1 , feature map X 2→1 , and feature map X 3→1 ; Along the channel direction, the feature map X 1→1 With feature map X 2→1 and feature map X 3→1 Calculate the fusion and use a 1×1 convolution kernel to adjust the number of channels to 3. Finally, calculate the weight of each layer during fusion based on the Softmax function. The calculation formula is as follows:

[0104]

[0105] in, is the output feature map Y l The feature vector of the location (i, j); is the feature vector at position (i, j) converted from level n to level l feature map; are the weights of the fusion of feature maps of three different scales, and are calculated in the same way; Represents the Softmax function fusion layer The corresponding control parameters when the parameters meet the conditions

[0106] It should be noted that the above explanation of the embodiment of the dam defect identification and analysis method adapted to multi-scale characteristics is also applicable to the dam defect identification and analysis device adapted to multi-scale characteristics of this embodiment, and will not be repeated here.

[0107] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0108] like Figure 4 , is a block diagram of an electronic device according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0109] like Figure 4 As shown, the electronic device includes: one or more processors 401, a memory 402, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 401 is taken as an example.

[0110] Memory 402 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor, causing the at least one processor to execute the multi-scale feature-adapted dam defect identification and analysis method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to execute the multi-scale feature-adapted dam defect identification and analysis method provided in this application.

[0111] The memory 402 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the dam defect identification and analysis method adapted to multi-scale features in the embodiment of the present application (for example, the attached Figure 3The processor 401 executes the non-transient software programs, instructions, and modules stored in the memory 402 to execute various functional applications and data processing of the server, thereby implementing the dam defect identification and analysis method adapted to multi-scale features in the above-mentioned method embodiment.

[0112] The memory 402 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 402 may optionally include a memory remotely located relative to the processor 401, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0113] The electronic device may further include: an input device 403 and an output device 404. The processor 401, the memory 402, the input device 403 and the output device 404 may be connected via a bus or other means. Figure 4 The bus connection is taken as an example.

[0114] The input device 403 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device, such as input devices such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, and a joystick. The output device 404 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0115] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0116] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0117] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0118] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0119] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0120] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0121] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A dam defect identification and analysis method adapted to multi-scale characteristics, characterized in that: The following steps are involved: Obtaining a surface image of the dam in a current assessment period, and classifying the surface image of the dam based on a binarization technique to obtain an image of a defect area to be identified; the defect to be identified includes cracks and / or water seepage; Extracting shape features of the defect to be identified from the image of the defect area to be identified; Inputting the shape features of the defect to be identified into a defect recognition model based on multi-scale features to obtain a first defect recognition result of the dam in the current assessment period; Obtaining apparent overall safety score information of the dam based on a first defect identification result of the dam in the current assessment cycle and a second defect identification result of the dam in the previous assessment cycle; Based on the apparent overall safety score information of the dam, the overall health status of the dam is evaluated.

2. The method according to claim 1, wherein The obtaining of the apparent overall safety score information of the dam based on the first defect identification result of the dam in the current assessment cycle and the second defect identification result of the dam in the previous assessment cycle includes: Determining a crack area change rate and a seepage area change rate based on a first defect identification result of the dam in the current evaluation cycle and a second defect identification result of the dam in the previous evaluation cycle; Determining safety score information for each evaluation indicator based on the first defect identification result, the crack area change rate, the seepage area change rate, and working condition information of other facilities of the dam; The apparent overall safety score information of the dam is obtained by performing weighted summation based on the safety score information of each evaluation indicator and the weight of each evaluation indicator.

3. The method according to claim 2, wherein The evaluation indicators include crack condition information, water seepage condition information, and other facility condition information; the safety score information of each evaluation indicator is determined based on the first defect identification result, the crack area change rate, the water seepage area change rate, and the working condition information of other facilities of the dam, including: Determining safety score information of the crack condition information based on the first defect identification result, the crack area change rate and its corresponding weight; Determining safety score information of the water seepage condition information based on the first defect identification result, the water seepage area change rate and its corresponding weight; Based on the working status information of other facilities of the dam and the sub-indicator assignment rule information, each sub-indicator in the other facility status information is assigned a value to obtain the scoring information of each sub-indicator, and based on the scoring information of each sub-indicator and its corresponding weight, the safety scoring information of the other facility status information is determined.

4. The method according to claim 3, wherein The determining of the safety score information of the crack condition information based on the first defect identification result, the crack area change rate and its corresponding weight includes: assigning a value to the first defect identification result based on the first defect identification result and the crack area change rate in combination with sub-item indicator assignment rule information to obtain first scoring information for the first defect identification result; and determining safety scoring information for the crack condition information based on the first scoring information for the first defect identification result, the crack area change rate, and their respective weights, wherein the first defect identification result includes whether new cracks are present; The determining of the safety score information of the water seepage condition information based on the first defect identification result, the water seepage area change rate and its corresponding weight includes: Based on the first defect identification result and the seepage area change rate combined with the sub-indicator assignment rule information, the first defect identification result is assigned to obtain the second scoring information of the first defect identification result, and based on the second scoring information of the first defect identification result, the seepage area change rate and their respective weights, the safety scoring information of the seepage condition information is determined, wherein the first defect identification result includes whether there are new seepage points.

5. The method according to claim 4, wherein The calculation formula of the safety score information of the crack status information is as follows: the safety score information of the crack status information = the first score information × α1 + (1-crack area change rate) × β1, where α1 and β1 are weights respectively; The calculation formula of the safety score information of the water seepage condition information is as follows: the safety score information of the water seepage condition information = the second score information × α2 + (1-crack area change rate) × β2, where α2 and β2 are weights respectively.

6. The method according to any one of claims 1 to 5, wherein The method further comprises: When the assessment result of the overall health status of the dam is abnormal or seriously abnormal, corresponding emergency control measures information is generated based on the assessment result, and the emergency control measures information is pushed to corresponding maintenance personnel. The emergency control measures information is used to instruct corresponding measures to be taken for the defective parts of the dam.

7. The method according to claim 1, wherein The network structure of the multi-scale feature-based defect recognition model is a YOLOv5 network structure, including: an input end, a backbone network, a fusion network, and a prediction network, wherein: The input end is used to perform Mosaic data enhancement on the input shape features of the defect to be identified; The backbone network includes Focus, CSP, and SPPF structures. The defect recognition model extracts layer features through the cross-stage local network CSP structure and completes multi-scale feature fusion based on the optimized spatial pyramid pooling module SPPF. The Focus structure at the front of the network is used to slice the input data. The fusion network combines the ASFF module in the neck network Neck to fuse image features of different scales; The prediction network is used to output three sets of feature maps of different scales and corresponding prediction boxes.

8. The method according to claim 7, wherein The ASFF module is used to automatically assign weights to feature maps of different scales through learning during the feature fusion process. The fusion process is as follows: The second feature map X among the three feature maps output by the neck network Neck is up-sampled by the up-sampling method. 2 , the third feature map X 3 The scale and number of channels are converted to the first feature map X 1 Consistent, get the feature map X 1→1 , feature map X 2→1 , and feature map X 3→1 ; The feature map X is transformed along the channel direction 1→1 With feature map X 2→1 and feature map X 3→1 Calculate the fusion and use a 1×1 convolution kernel to adjust the number of channels to 3. Finally, calculate the weight of each layer during fusion based on the Softmax function. The calculation formula is as follows: in, is the output feature map Y l The feature vector of the location (i, j); is the feature vector at position (i, j) converted from level n to level l feature map; are the weights of the fusion of feature maps of three different scales, and are calculated in the same way; Represents the Softmax function fusion layer The corresponding control parameters when the parameters meet the conditions 9. A dam defect identification and analysis device adapted to multi-scale characteristics, characterized in that: include: A classification module is used to obtain the dam surface image of the current assessment period and classify the dam surface image based on a binarization technique to obtain an image of the defect area to be identified; The defects to be identified include cracks and / or water seepage; An extraction module, configured to extract shape features of the defect to be identified from the image of the defect area to be identified; an identification module, configured to input the shape features of the defect to be identified into a defect identification model based on multi-scale features to obtain a first defect identification result of the dam in the current assessment period; A score acquisition module, configured to acquire information on the apparent overall safety score of the dam based on the first defect identification result of the dam in the current assessment cycle and the second defect identification result of the dam in the previous assessment cycle; An evaluation module is used to evaluate the overall health status of the dam based on the apparent overall safety score information of the dam.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images