Bridge image multi-disease segmentation algorithm based on perceptual analysis and quantitative calculation
Through the improved DCNv2 model and jump connection channel-space attention module, combined with numerical integration and geometric modeling, the problem of identification and segmentation of multiple diseases in bridge images is solved, and the precise segmentation and quantitative calculation of bridge diseases is realized, which improves the The accuracy and robustness of the inspection provide a scientific basis for bridge maintenance.
Patent Information
- Application Number
- CN202411540275.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-10-31
AI Technical Summary
The prior art is difficult to accurately identify and segment a variety of complex diseases in bridge images, especially because the model structure is simple or the resolution is insufficient, and the appearance of different types of diseases is similar, resulting in misclassification, lack of spatial context understanding and multi-scale characteristics. Extraction capability.
Using a variety of disease segmentation algorithms for bridge images based on perception analysis, the jump connection channel-space attention module and PSPNet pyramid pooling module are introduced by building an improved DCNv2 model, combining module stacking rules and the depth and width scaling rules of the normalized network model. , enhance the model's context feature capture capability, and calculate disease attribute values using numerical integration and geometric modeling.
Improve the precise segmentation and quantitative calculation ability of bridge diseases, enhance the model's ability to identify and locate multiple diseases, realize the comprehensive perception and analysis of multiple diseases in bridge images, and improve the accuracy and robustness of detection. , provide a scientific basis for bridge maintenance.
Smart Images

Figure CN119027676A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of bridge defect detection, and specifically to a bridge image multiple defect segmentation algorithm and quantitative calculation based on perception analysis. Background Art
[0002] In the field of bridge maintenance and management, efficient segmentation and accurate quantification of various defects in bridge images are crucial. However, current technologies face many challenges in handling this task. Existing technical solutions mainly rely on processing relatively simple public datasets, such as Crack500. These datasets often contain only a single type of defect and cannot fully reflect the complexity and diversity of actual bridge defects. In actual applications, bridges often face the coexistence of multiple defects, such as tiny cracks, local corrosion, material aging, etc. These defects may appear as extremely small areas with blurred edges in the image, which places extremely high demands on the resolution and detail recognition capabilities of the model. The model may have difficulty accurately identifying subtle defects due to its overly simple structure or insufficient resolution. Different types of defects may be misclassified due to their similar appearance. In the absence of spatial context understanding, the model has difficulty accurately identifying defects and their surrounding structures and texture information. At the same time, defects may appear at different scales, requiring the model to have multi-scale feature extraction capabilities, but existing models often find it difficult to adapt to this requirement. To this end, we propose a segmentation algorithm and quantitative calculation of multiple defects in bridge images based on perceptual analysis. Summary of the invention
[0003] The purpose of the present invention is to provide a bridge image multiple disease segmentation algorithm and quantitative calculation based on perceptual analysis.
[0004] To achieve the above object, the present invention provides the following technical solution: a bridge image multiple disease segmentation algorithm and quantitative calculation based on perceptual analysis, comprising the following steps: Step 1: Source and construction of bridge dataset: using drones to obtain bridge bottom images and construct bridge disease dataset; Step 2: Dataset division: divide the bridge disease dataset constructed in step 1 into 7:3, which are used as the training set and test set of the model respectively; Step 3: Construct a multi-task semantic segmentation algorithm based on perceptual analysis and a quantitative calculation method for bridge disease parameters, improve DCNv2, propose an improved DCNv2, use the improved DCNv2 as the core operator, build basic modules, propose module stacking rules, and determine model parameters according to the rules, normalize the depth and width scaling rules of the network model, use restriction factors and compound coefficients to scale the depth and width, apply the pyramid pooling module (PPM) from PSPNet in the last layer of the backbone network, equip each classifier with an independent convolution head, and apply the skip connection channel-spatial attention module (skip connection channel-spatial attention) to enhance the model's ability to capture contextual features; Step 4: Calculate the bridge damage attribute values, and calculate the crack length, crack width and damage area in turn.
[0005] As a further solution of the present invention: in the step 1, the bridge disease data set includes 13 types of diseases, including cracks, crazing cracks, exposed steel bars, moisture, salt precipitation, rust, honeycombs, hollow areas, cavities, spalling, graffiti, weathering and erosion. The locations where the bridge disease data set is collected include retained formwork, supports, expansion joints, drainage, protective equipment and joint strips.
[0006] As a further solution of the present invention: in the step 2, the training set is used for training and learning the model, the test set is used to evaluate the performance and accuracy of the model, and the data is cleaned before dividing the data set.
[0007] As a further solution of the present invention: in step 3, the improved DCNv2 expression is as follows:
[0008] In the formula, Indicates the total number of aggregation groups. Groups, represents the position-independent projection weight of the group, where K represents the total number of sampling points, p0 represents the current pixel point, represents the kth position in the predefined grid sampling position, represents the group dimension, Indicates Group The sample points are adjusted by the softmax function. represents the input feature map after slicing, It is The corresponding grid sampling position in the group The offset of
[0009] As a further solution of the present invention: In the step three, the basic module is composed of two layers of normalization and a feedforward network, and the initial layer and the downsampling layer follow the design principles of traditional convolutional neural networks. Each initial layer contains two convolutional layers, two LN layers and a GELU activation layer, and the downsampling layer consists of a convolutional layer and a LN layer.
[0010] As a further solution of the present invention: In step 3, there are two rules for module stacking. The first rule is the number of channels in the last three stages. By the number of channels in the first stage Decision, that is The second rule is that the module group number corresponds to the number of channels in each stage. ,Third, the stacking mode is fixed to “AABA”, that is, the number of module stacking in stages 1, 2, and 4 is the same and no more than that in stage 3, i.e. , choose the model with a parameter volume of 30M as the basis, and its specific parameters are: the number of Steam output channels The number of groups is 1 / 16 of the number of input channels in each stage. The number of module stacks in the 1st, 2nd, and 4th stages is 4, the number of modules stacked in the third stage is 18, the model parameter is 30M, and in step 3, the depth and width scaling rules of the normalized network model are scaled by using the limiting factor α and the compound coefficient β to scale the depth D and width C, that is, D=αD' and C=βC', where , D' and C' are original values, D and C are scaled values, α=1.09, β=1.36, in step 3, the order of operation of applying the pyramid pooling module from PSPNet in the last layer of the backbone network is, before the extracted features are input to the bottom-up branch of the feature pyramid network, the model is based on the residual network ResNet, and the final feature map of each stage of ResNet is marked as , and the series of feature maps output by FPN are expressed as ,in is the feature map following the PPM module, and the downsampling rates of these feature maps are In step 3, the specific method of equipping each classifier with an independent convolution head is to fuse feature layers of different scales. , and adjust them to the same value through bilinear interpolation. The same size is used, and these layers are connected. Then a convolution layer is applied to fuse features from different levels and reduce the channel dimension. The number of output channels of all additional non-classifier convolution layers in FPN is 512, and batch normalization (BN) technology is used after the convolution layer.
[0011] As a further solution of the present invention: the poly learning rate strategy is adopted in the model training process, that is, the current iterative learning rate is equal to the initial learning rate multiplied by The initial learning rate and power exponent are set to 0.02 and 0.9 respectively, the weight decay is 0.0001 and the momentum is 0.9, and the parameters of each layer in the backbone network are initialized using the weights pre-trained on the ImageNet dataset. The skip connection channel-spatial attention module can compensate for the loss of spatial information caused by downsampling, and enhance the global dimensional interaction for the multi-scale features of each layer of the decoder, so as to achieve the recovery of fine details when generating the output mask. Specifically, the feature vector is obtained by concatenating the encoder and decoder outputs. , the channel attention submodule adopts a reduced ratio The encoder-decoder structure of the multi-layer perception (MLP) is used to amplify the channel-space dependency across dimensions, using two Convolutional layer, also with reduced ratio Focusing on the spatial information from the channel attention submodule, given the input feature map , through the intermediate state After that, the final output Defined as,
[0012] In the formula, Represent the output results of the channel attention submodule and the spatial attention submodule respectively, It is the abbreviation of Fully connected layer.
[0013] As a further solution of the present invention: In step 4, the crack length is calculated as follows: the crack is composed of a series of pixel points, where the position of each pixel point is calculated using a two-dimensional coordinate system. To describe, its gray value function is defined as If the gray value of a pixel is 255, it means that the point is located in the crack area. If it is 0, it means that the point does not belong to the crack part. In order to calculate the actual length of the crack, the length of this continuous curve is approximately calculated by numerical integration. The specific formula is as follows:
[0014] In the formula, is the coordinate value of geometric calibration, is the crack unit length, Indicates the ratio of a single pixel to the actual size. The crack width is calculated as follows: and Identify the boundary contours on both sides of the crack, construct a coordinate system for the edge of the image, and the main axis is In addition, for the curve The tangent line is perpendicular to The angle between straight lines, using the angle function The specific formula is as follows:
[0015]
[0016]
[0017] In the formula, for and The vertical line and and The angles are equal, and the angles of each point are obtained by The main calculation process is as follows: traverse all the pixels in the crack image to determine the specific coordinates of each pixel, build a model for each coordinate position, and count the number of pixels inside the crack in the vertical direction area corresponding to each pixel. , where these pixels are represented as In coordinate form, calculate the midpoint coordinate function of the vertical area , the tilt angle at the above specific pixel point is recorded as ,use Calculate the width of the crack at that point and get the width of the crack at each point one by one After completing the calculation of the crack widths of all pixels, these width values are accumulated and their arithmetic mean is calculated to obtain the average width value. .
[0018] As a further solution of the present invention: In the step 4, the disease area is calculated by geometrically modeling the disease boundary and defining two boundary curves of the disease as follows: and , use the area integration principle to integrate the area enclosed by the two curves to get the overall area size. The specific formula is as follows:
[0019] In the formula, As the area of the finite element of the disease, the width of the disease is , length using To express, Indicates the ratio of a single pixel to the actual size.
[0020] By adopting the above technical solution, compared with the prior art, the beneficial effects of the present invention are: 1. The present invention reduces the memory usage of the model and enhances the model's ability to capture and distinguish multiple diseases by designing a dynamic sparse convolution operator and introducing a multi-group mechanism in the encoder stage. In the decoder stage, the patch feature expansion layer and the skip connection channel-spatial attention module are used to reduce the model complexity and improve the precise positioning and segmentation performance of different types of disease areas. In addition, by fusing multi-scale features and unifying the output, the comprehensive perception and analysis of bridge diseases are achieved, and the overall detection accuracy and robustness are improved. 2. The present invention improves DCNv2 and uses it as the core operator to build a basic module. It combines the module stacking rules and the depth and width scaling rules of the normalized network model to enhance the model's ability to capture contextual features. In addition, the introduction of the skip connection channel-spatial attention module further enhances the model's ability to distinguish and locate bridge defects. The algorithm can achieve accurate segmentation of various defects in bridge images, providing a basis for subsequent quantitative calculations. 3. The present invention can calculate the various attribute values of bridge defects by using numerical integration, geometric modeling and area integration principle methods, providing a scientific basis for the maintenance and management of bridges. In particular, when calculating the length and width of cracks, a sophisticated pixel-level analysis method is used to ensure the accuracy of the calculation results. At the same time, when calculating the defect area, the integral is converted into the accumulation of defect pixels, thereby achieving accurate measurement of the area of the defect region.
[0021] Other advantages, objectives and features of the present invention will be set forth in part in the following description and, in part, will be apparent to those skilled in the art based on an examination of the following or may be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 A schematic diagram of the method steps in an embodiment of the present invention; Figure 2 This is a schematic diagram of a basic module in an embodiment of the present invention; Figure 3 This is a schematic diagram of the skip connection channel-spatial attention module process in an embodiment of the present invention; Figure 4 This is a schematic diagram of bridge crack calculation in an embodiment of the present invention; Figure 5 This is a flow chart of the bridge image multiple disease segmentation algorithm in an embodiment of the present invention. DETAILED DESCRIPTION
[0023] The specific embodiments of the present invention will be further described below in conjunction with the accompanying drawings. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.
[0024] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0025] Please see attached Figure 1 -Attached Figure 5 The present invention provides a bridge image multiple disease segmentation algorithm and quantitative calculation based on perceptual analysis, comprising the following steps: Step 1: Source and construction of bridge dataset: using drones to obtain bridge bottom images and construct bridge disease dataset; Step 2: Dataset division: divide the bridge disease dataset constructed in step 1 into 7:3, which are used as the training set and test set of the model respectively; Step 3: Build a multi-task semantic segmentation algorithm based on perceptual analysis and a quantitative calculation method for bridge disease parameters, improve DCNv2, propose an improved DCNv2, use the improved DCNv2 as the core operator, build basic modules, propose module stacking rules, and determine model parameters according to the rules, normalize the depth and width scaling rules of the network model, use the restriction factor and compound coefficient to scale the depth and width, and apply the pyramid pooling module (PPM) from PSPNet in the last layer of the backbone network; Step 4: Calculate the bridge damage attribute values, and calculate the crack length, crack width and damage area in turn.
[0026] Embodiment 1: In step 1, the bridge disease data set includes 13 types of diseases, including cracks, cracks, exposed steel bars, moisture, salt precipitation, rust, honeycomb, hollow areas, cavities, spalling, graffiti, weathering and erosion. The bridge disease data set collection locations include retained templates, supports, expansion joints, drainage, protective equipment and joints. In step 2, the training set is used for model training and learning, and the test set is used to evaluate the performance and accuracy of the model. Before dividing the data set, the data is cleaned. In step 3, the improved DCNv2 expression is as follows:
[0027] In the formula, Indicates the total number of aggregation groups. Groups, represents the position-independent projection weight of the group, where K represents the total number of sampling points, p0 represents the current pixel point, represents the kth position in the predefined grid sampling position, represents the group dimension, Indicates Group The sample points are adjusted by the softmax function. represents the input feature map after slicing, It is The corresponding grid sampling position in the group The offset of Specifically, in actual image data processing, the features presented by different bridge images are very different. Some areas may be relatively sparse, while some areas are relatively dense. The dynamic sparse convolution operator can keenly perceive the sparsity of the input feature map, and then dynamically adjust the sparsity mode of the convolution kernel. When facing a sparse area, the convolution kernel will automatically adjust to a sparser state, reducing unnecessary calculations and greatly reducing the memory usage of the model. At the same time, when processing dense areas, it can flexibly adjust to a more adaptive mode to ensure effective feature extraction; In order to further enhance the model's ability to capture and distinguish multiple diseases, the multi-group mechanism plays a key role. Dividing the input feature map into multiple groups is like providing the model with multiple different perspectives to observe the disease characteristics. Each group performs convolution operations independently, allowing the model to learn more deeply about disease characteristics of different scales and types. For example, for diseases such as cracks, different groups may pay attention to different details of the cracks. Some groups may focus more on the length characteristics of the cracks, while others may focus more on the width or depth characteristics of the cracks. This grouping method helps the model to fully understand the various properties of the disease and improve its sensitivity and accuracy to the disease.
[0028] In Example 2, in step 3, the basic module consists of two layers of normalization and a feedforward network, and the initial layer and the downsampling layer follow the design principles of traditional convolutional neural networks. Each initial layer contains two convolutional layers, two LN layers, and a GELU activation layer. The downsampling layer consists of a convolutional layer and a LN layer. In step 3, there are two module stacking rules. The first rule is the number of channels in the last three stages. By the number of channels in the first stage Decision, that is The second rule is that the module group number corresponds to the number of channels in each stage. ,Third, the stacking mode is fixed to “AABA”, that is, the number of module stacking in stages 1, 2, and 4 is the same and no more than that in stage 3, i.e. , choose the model with a parameter volume of 30M as the basis, and its specific parameters are: the number of Steam output channels The number of groups is 1 / 16 of the number of input channels in each stage. The number of module stacks in the 1st, 2nd, and 4th stages is 4, the number of modules stacked in the third stage is 18, the model parameter is 30M, and in step 3, the depth and width scaling rules of the normalized network model are scaled by using the limiting factor α and the compound coefficient β to scale the depth D and width C, that is, D=αD' and C=βC', where , D' and C' are the original values, D and C are the scaled values, α = 1.09, β = 1.36, In step 3, the order of operations from the PSPNet pyramid pooling module is applied to the last layer of the backbone network, before the extracted features are input to the bottom-up branch of the feature pyramid network. The model is based on the residual network ResNet, and the final feature map of each stage of ResNet is marked as , and the series of feature maps output by FPN are expressed as ,in is the feature map following the PPM module, and the downsampling rates of these feature maps are In step 3, the specific way to equip each classifier with an independent convolution head is to fuse feature layers of different scales. , and adjust them to the same value through bilinear interpolation. The same size, and these layers are connected, and then a convolution layer is applied to fuse features from different levels and reduce channel dimensions. All additional non-classifier convolution layers in FPN have 512 output channels, and batch normalization technology is used after the convolution layer. The poly learning rate strategy is used during model training, that is, the current iteration learning rate is equal to the initial learning rate multiplied by The initial learning rate and power exponent are set to 0.02 and 0.9 respectively, the weight decay is 0.0001 and the momentum is 0.9. The parameters of each layer in the backbone network are initialized with weights pre-trained on the ImageNet dataset, which can compensate for the loss of spatial information caused by downsampling and enhance the global dimensional interaction for the multi-scale features of each layer of the decoder, so as to achieve the recovery of fine details when generating the output mask. The specific method is to obtain the feature vector by concatenating the encoder and decoder outputs. , the channel attention submodule adopts a reduced ratio The encoder-decoder structure of the multi-layer perception (MLP) is used to amplify the channel-space dependency across dimensions, using two Convolutional layer, also with reduced ratio Focusing on the spatial information from the channel attention submodule, given the input feature map , through the intermediate state After that, the final output Defined as,
[0029] In the formula, Represent the output results of the channel attention submodule and the spatial attention submodule respectively, It is the abbreviation of fully connected layer; Specifically, the design of the decoder stage has brought new breakthroughs to the task of bridge image defect segmentation. The role of the patch feature expansion layer cannot be underestimated. In the process of the model decoding the defect features, the patch feature expansion layer provides the model with richer feature representation capabilities by expanding the dimension of the feature map, just like adding more colors and details to a blurred picture, so that the model can understand the characteristics of the defect more comprehensively. For example, when dealing with rust damage on the bridge surface, the patch feature expansion layer can expand the original relatively single rust feature into multi-dimensional features including color, texture, shape, etc., so that the model can more accurately identify and segment the rust area. The skip connection channel-spatial attention module is another key component of the decoder stage. By calculating the attention weights in channels and spaces, this module enables the model to focus on more important feature areas in a targeted manner. In bridge images, different diseased areas may have different importance. Some areas may have a greater impact on the structural safety of the bridge and require special attention. The skip connection channel-spatial attention module can automatically identify these important areas and give them higher attention weights. For example, when processing the hole area of the bridge, this module can focus on the edge and internal structure of the hole, thereby improving the accuracy of segmentation.
[0030] In the third embodiment, in step 4, the crack length is calculated as follows: the crack is composed of a series of pixel points, and the position of each pixel point is expressed using a two-dimensional coordinate system. To describe, its gray value function is defined as If the gray value of a pixel is 255, it means that the point is located in the crack area. If it is 0, it means that the point does not belong to the crack part. In order to calculate the actual length of the crack, the length of this continuous curve is approximately calculated by numerical integration. The specific formula is as follows:
[0031] In the formula, is the coordinate value of geometric calibration, is the crack unit length, Indicates the ratio of a single pixel to the actual size. The crack width is calculated as follows: and Identify the boundary contours on both sides of the crack, construct a coordinate system for the edge of the image, and the main axis is In addition, for the curve The tangent line is perpendicular to The angle between straight lines, using the angle function The specific formula is as follows:
[0032]
[0033]
[0034] In the formula, for and The vertical line and and The angles are equal, and the angles of each point are obtained by The main calculation process is as follows: traverse all the pixels in the crack image to determine the specific coordinates of each pixel, build a model for each coordinate position, and count the number of pixels inside the crack in the vertical direction area corresponding to each pixel. , where these pixels are represented as In coordinate form, calculate the midpoint coordinate function of the vertical area , the tilt angle at the above specific pixel point is recorded as ,use Calculate the width of the crack at that point and get the width of the crack at each point one by one After completing the calculation of the crack widths of all pixels, these width values are accumulated and their arithmetic mean is calculated to obtain the average width value. In step 4, the disease area is calculated by geometrically modeling the disease boundary and defining two boundary curves of the disease as and , use the area integration principle to integrate the area enclosed by the two curves to get the overall area size. The specific formula is as follows:
[0035] In the formula, As the area of the finite element of the disease, the width of the disease is , length using To express, Indicates the ratio of a single pixel to the actual size; Specifically, when calculating the length and width of cracks, a sophisticated pixel-level analysis method is key. By calculating the continuity and directionality of crack pixels, we can accurately determine the length and width of cracks. For cracks, which are long and thin disease features, pixel-level analysis can capture every subtle change, thereby ensuring the accuracy of the calculation results. For example, when dealing with curved cracks, our method can accurately calculate the length and width of the cracks based on the directional changes of the crack pixels, providing an important indicator for evaluating the severity of the cracks. When calculating the damaged area, converting the integral into the accumulation of damaged pixels is an innovative method. This method can effectively handle irregularly shaped damaged areas and achieve accurate measurement of the damaged area. For example, for diseases such as cavities and weathering, their shapes are often irregular, and traditional calculation methods may have large errors. Our method can accurately calculate the area of these irregular damaged areas by converting the integral into the accumulation of pixels, providing accurate data support for formulating reasonable maintenance plans.
[0036] Working principle: Firstly, a bridge disease dataset containing 13 types of diseases and specific collection locations is constructed by using drones to obtain images of the bottom of the bridge. The dataset is divided into training and test sets at a ratio of 7:3 and data cleaning is performed before division. This method improves DCNv2 to obtain an improved DCNv2, which is used as the core operator to build a basic module. The basic building block consists of two layers of normalization and a feedforward network. The initial layer and the downsampling layer follow the traditional convolutional neural network design principles. The module stacking rule is followed when determining the model parameters. The depth and width are scaled using the limiting factor and the compound coefficient. The PS The PNet pyramid pooling module equips each classifier with an independent convolution head and applies the skip connection channel-spatial attention module. The dynamic sparse convolution operator and multi-group mechanism in the encoder stage reduce memory usage and enhance the ability to capture and distinguish defects. The patch feature expansion layer and attention module in the decoder stage reduce model complexity and improve the performance of precise positioning and segmentation. It integrates multi-scale features and unifies the output to achieve comprehensive perception and analysis, improve detection accuracy and robustness. At the same time, numerical integration, geometric modeling and area integration principle methods are used to calculate the various attribute values of bridge defects. At this point, the entire workflow is completed.
[0037] The above-mentioned front, back, left, right, top and bottom are all based on the figures in the specification. Figure 1 As a benchmark, according to the person's observation perspective, the side of the device facing the observer is defined as the front, the left side of the observer is defined as the left, and so on.
[0038] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the scope of protection of the present invention.
[0039] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments.
[0040] It is apparent to those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the invention, and the changes still fall within the protection scope of the invention.
Claims
1. A bridge image multiple disease segmentation algorithm and quantitative calculation based on perceptual analysis, characterized by: The following steps are involved: Step 1: Source and construction of bridge dataset: using drones to obtain bridge bottom images and construct bridge disease dataset; Step 2: Dataset division: divide the bridge disease dataset constructed in step 1 into 7:3, which are used as the training set and test set of the model respectively; Step 3: Construct a multi-task semantic segmentation algorithm based on perceptual analysis and a quantitative calculation method for bridge disease parameters, improve DCNv2, propose an improved DCNv2, use the improved DCNv2 as the core operator, build basic modules, propose module stacking rules, and determine model parameters according to the rules, normalize the depth and width scaling rules of the network model, use restriction factors and compound coefficients to scale the depth and width, apply the pyramid pooling module from PSPNet in the last layer of the backbone network, equip each classifier with an independent convolution head, and apply the skip connection channel-spatial attention module to enhance the model's ability to capture contextual features; Step 4: Calculate the attribute values of bridge damage, and calculate the crack length, crack width and damage area in turn.
2. The bridge image multiple disease segmentation algorithm and quantitative calculation based on perceptual analysis according to claim 1 is characterized by: In step 1, the bridge disease data set includes 13 types of diseases, including cracks, crazing cracks, exposed steel bars, moisture, salt precipitation, rust, honeycombs, hollow areas, cavities, spalling, graffiti, weathering and erosion. The locations where the bridge disease data set is collected include retained formwork, bearings, expansion joints, drainage, protective equipment and joint strips.
3. The bridge image multiple disease segmentation algorithm and quantitative calculation based on perceptual analysis according to claim 1 is characterized by: In the step 2, the training set is used for training and learning the model, and the test set is used to evaluate the performance and accuracy of the model. Before dividing the data set, the data is cleaned.
4. The bridge image multiple disease segmentation algorithm and quantitative calculation based on perceptual analysis according to claim 1 is characterized by: In step 3, the improved DCNv2 expression is as follows: In the formula, Indicates the total number of aggregation groups. Groups, represents the position-independent projection weight of the group, where K represents the total number of sampling points, p0 represents the current pixel point, represents the kth position in the predefined grid sampling position, represents the group dimension, Indicates Group The sample points are adjusted by the softmax function. represents the input feature map after slicing, It is The corresponding grid sampling position in the group The offset of 5. The bridge image multiple disease segmentation algorithm and quantitative calculation based on perceptual analysis according to claim 1 is characterized by: In the step 3, the basic module consists of two layers of normalization and a feedforward network, and the initial layer and the downsampling layer follow the design principles of traditional convolutional neural networks. Each initial layer contains two convolutional layers, two LN layers and a GELU activation layer, and the downsampling layer consists of a convolutional layer and a LN layer.
6. The bridge image multiple disease segmentation algorithm and quantitative calculation based on perceptual analysis according to claim 1 is characterized by: In step 3, there are two rules for module stacking. The first rule is the number of channels in the last three stages. By the number of channels in the first stage Decision, that is The second rule is that the module group number corresponds to the number of channels in each stage. ,Third, the stacking mode is fixed to "AABA", that is, the number of module stacking in stages 1, 2, and 4 is the same and no more than that in stage 3, i.e. , choose the model with a parameter volume of 30M as the basis, and its specific parameters are: the number of Steam output channels The number of groups is 1 / 16 of the number of input channels in each stage. The number of module stacks in the 1st, 2nd, and 4th stages is 4, the number of modules stacked in the third stage is 18, the model parameter is 30M, and in step 3, the depth and width scaling rules of the normalized network model are scaled by using the limiting factor α and the compound coefficient β to scale the depth D and width C, that is, D=αD' and C=βC', where , D' and C' are original values, D and C are scaled values, α=1.09, β=1.36, in step 3, the order of operation of applying the pyramid pooling module from PSPNet in the last layer of the backbone network is, before the extracted features are input to the bottom-up branch of the feature pyramid network, the model is based on the residual network ResNet, and the final feature map of each stage of ResNet is marked as , and the series of feature maps output by FPN are expressed as ,in is the feature map following the PPM module, and the downsampling rates of these feature maps are In step 3, the specific method of equipping each classifier with an independent convolution head is to fuse feature layers of different scales. , and adjust them to the same value through bilinear interpolation. The same size, and these layers are connected, and then a convolution layer is applied to fuse features from different levels and reduce the channel dimension. All additional non-classifier convolution layers in FPN have an output channel number of 512, and batch normalization technology is used after the convolution layer.
7. The bridge image multiple disease segmentation algorithm and quantitative calculation based on perceptual analysis according to claim 6 is characterized by: The poly learning rate strategy is used in the model training process, that is, the current iterative learning rate is equal to the initial learning rate multiplied by The initial learning rate and power exponent are set to 0.02 and 0.9 respectively, the weight decay is 0.0001 and the momentum is 0.9, and the parameters of each layer in the backbone network are initialized using the weights pre-trained on the ImageNet dataset. The skip connection channel-spatial attention module can compensate for the loss of spatial information caused by downsampling, and enhance the global dimensional interaction for the multi-scale features of each layer of the decoder, so as to achieve the recovery of fine details when generating the output mask. Specifically, the feature vector is obtained by concatenating the encoder and decoder outputs. , the channel attention submodule adopts a reduced ratio The encoder-decoder structure of the multi-layer perceptron is used to amplify the channel-space dependency across dimensions, using two Convolutional layer, also with reduced ratio Focusing on the spatial information from the channel attention submodule, given the input feature map , through the intermediate state After that, the final output Defined as, In the formula, Represent the output results of the channel attention submodule and the spatial attention submodule respectively, It is the abbreviation of fully connected layer.
8. The bridge image multiple disease segmentation algorithm and quantitative calculation based on perceptual analysis according to claim 1 is characterized by: In step 4, the length of the crack is calculated as follows: the crack is composed of a series of pixel points, where the position of each pixel point is calculated using a two-dimensional coordinate system. To describe, its gray value function is defined as If the gray value of a pixel is 255, it means that the point is located in the crack area. If it is 0, it means that the point does not belong to the crack part. In order to calculate the actual length of the crack, the length of this continuous curve is approximately calculated by numerical integration. The specific formula is as follows: In the formula, is the coordinate value of geometric calibration, is the crack unit length, Indicates the ratio of a single pixel to the actual size. The crack width is calculated as follows: and Identify the boundary contours on both sides of the crack, construct a coordinate system for the edge of the image, and the main axis is In addition, for the curve The tangent line is perpendicular to The angle between the lines, using the angle function The specific formula is as follows: In the formula, for and The vertical line and and The angles are equal, and the angles of each point are obtained by The main calculation process is as follows: traverse all the pixels in the crack image to determine the specific coordinates of each pixel, build a model for each coordinate position, and count the number of pixels inside the crack in the vertical direction area corresponding to each pixel. , where these pixels are represented as In coordinate form, calculate the midpoint coordinate function of the vertical area , the tilt angle at the above specific pixel point is recorded as ,use Calculate the width of the crack at that point and get the width of the crack at each point one by one After completing the calculation of the crack widths of all pixels, these width values are accumulated and their arithmetic mean is calculated to obtain the average width. .
9. The bridge image multiple disease segmentation algorithm and quantitative calculation based on perceptual analysis according to claim 1 is characterized by: In step 4, the disease area is calculated by geometrically modeling the disease boundary and defining two boundary curves of the disease as follows: and , use the area integration principle to integrate the area enclosed by the two curves to get the overall area size. The specific formula is as follows: In the formula, As the area of the finite element of the disease, the width of the disease is , length using To express, Indicates the ratio of a single pixel to the actual size.
Citation Information
Patent Citations
Bridge apparent disease rapid detection method based on machine vision
CN113326846A
Computer vision method and intelligent camera system for bridge health diagnosis
CN113971660A
Pavement disease detection method based on improved YOLOv8 model
CN117788950A
Bridge crack disease detection method and device based on crack self-segmentation model
CN117809082A
Method and system for measuring corrosion and crack diseases of steel structure bridge
CN118864404A
Cited By
Crack image recognition system for cover and excavation top-down subway station
CN120071162A
Data internal feature vector generation method, medium and system
CN120596896A
Data internal feature vector generation method, medium and system
CN120596896B
Training method of underground coal mine image processing model and image processing method
CN121095691A
