Concrete slump detection method based on image semantic segmentation

Abstract

The application discloses a concrete slump detection method based on semantic segmentation, and specifically comprises the following steps: step 1, image preprocessing and dataset construction; step 2, constructing a semantic segmentation module based on a multi-scale channel attention feature; and step 3, image post-processing and result output. By using the application, the error introduction problem caused by the dependence of the existing concrete slump detection method on manual operation and subjective judgment can be overcome, the output slump detection result is more accurate, the method system of computer vision and artificial intelligence technology is enriched, and a selection for improving the concrete slump detection precision is provided.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and artificial intelligence technology, specifically relating to a concrete slump detection method based on image semantic segmentation. Background Technology

[0002] In recent years, concrete slump testing has become increasingly important in engineering construction and quality control. Concrete slump is an indicator for evaluating the fluidity and plasticity of concrete, and it refers to the degree to which concrete collapses in a slump cone test.

[0003] Traditional methods for testing the slump of concrete include the slump cone test, pressure plate method, and flow meter. These methods usually rely on manual operation and subjective judgment, which are subject to subjectivity, instability, and limitations of human resources. Summary of the Invention

[0004] The purpose of this invention is to provide a concrete slump detection method based on image semantic segmentation, which solves the problem that existing detection methods rely on manual operation and subjective judgment.

[0005] The technical solution adopted in this invention is: a concrete slump detection method based on image semantic segmentation, which is implemented according to the following steps:

[0006] Step 1: Image preprocessing and dataset construction;

[0007] Step 2: Construct a semantic segmentation module based on multi-scale channel attention features;

[0008] Step 3: Image post-processing and result output.

[0009] The invention is further characterized by:

[0010] Step 1 is implemented in the following steps:

[0011] Step 1.1: Perform image cropping and image enhancement on images of various slump buckets and concrete sample images with different slumps;

[0012] Step 1.2: Label the sample images processed in Step 1.1 in labelme. Label the slump bucket and the collapsed concrete in the sample images as slump bucket type and concrete type, respectively.

[0013] Step 1.3: Save the generated JSON file, convert it into segmented visualization images in batches, fill it with color, and generate the corresponding labeled images as GroundTruth. Together with the original images, these constitute the training set, validation set, and test set.

[0014] Step 2 is implemented in the following steps:

[0015] Step 2.1: Construct the encoder submodule, which consists of convolutional layers and downsampling layers, including original feature extraction units and multi-scale feature extraction units;

[0016] Step 2.2: Construct the decoder submodule, which consists of a channel attention layer and an upsampling layer, to gradually restore the resolution of the feature map;

[0017] Step 2.3: Set training hyperparameters and train and test the network;

[0018] Step 2.1, which involves constructing the encoder submodule, is implemented as follows:

[0019] Step 2.1.1: Construct an initial feature extraction unit, which includes one 3×3 convolution with a stride of 2 and two 3×3 convolutions with a stride of 1, to extract the initial features;

[0020] Step 2.1.2: Construct a multi-scale feature extraction module, where all convolutions are standard convolutions and dilated convolutions;

[0021] Step 2.1.2, which involves constructing the multi-scale feature extraction module, is implemented as follows:

[0022] Step 2.1.2.1: The input features are first processed through a 3×3 standard convolution to extract features and the number of channels is adjusted to half.

[0023] Step 2.1.2.2: Extract multi-scale features from three branches with different receptive fields;

[0024] Step 2.1.2.3: Add and fuse the multi-scale features of the three branches, and use a 1×1 convolution to recover the number of channels;

[0025] Step 2.1.2.4: Channel shuffling is used to exchange information between channels, and residual connections before the feature extraction branch are introduced before shuffling.

[0026] Step 2.1.2.2 is implemented as follows:

[0027] Step 2.1.2.2.1: The first branch is a 3×3 standard convolution with a receptive field of 3×3, used to extract local information;

[0028] In step 2.1.2.2.2, the second branch is a 3×3 dilated convolution with a dilation rate of d, and its receptive field is (2×d+1)×(2×d+1).

[0029] In step 2.1.2.2.3, the third branch is a 3×3 dilated convolution with a dilation rate of d+1, and its receptive field is (2×d+3)×(2×d+3).

[0030] Step 2.2, which involves constructing the decoder submodule, is implemented as follows:

[0031] Step 2.2.1: The feature map with 1 / 32 of the size of 512 channels first passes through the channel attention layer to calculate new features;

[0032] Step 2.2.2: The 1 / 32 feature map with 512 channels obtained in Step 2.2.1 is upsampled and then fused with the 1 / 16 feature map with 256 channels obtained through the channel attention layer.

[0033] Step 2.2.3: The fused features are upsampled again and then added to and fused with the feature map of size 1 / 8 of the 128 channels obtained through the channel attention layer;

[0034] Step 2.2.4: Finally, after upsampling again, it is fused with the feature map of size 1 / 4 of the 64 channels obtained through the channel attention layer;

[0035] Step 2.2.5: The feature map upsampled to half the size of 16 channels is fed into the output convolutional layer. The stride of the output convolutional layer is 2, and the output channels are two-dimensional transposed convolutions of the number of classes.

[0036] Step 2.2.1 is implemented as follows:

[0037] Step 2.2.1.1: Perform spatial global max pooling and average pooling on the input feature F of size H×W×C to obtain two channel descriptions F of size 1×1×C. avg F max ;

[0038] Step 2.2.1.2: The output of step 2.2.1.1 is fed into a multilayer perceptron network, which consists of two fully connected layers and one ReLU activation layer;

[0039] Step 2.2.1.3: Add the two features output from Step 2.2.1.2 and pass them through a Sigmoid activation function to obtain the weight coefficient Mc(F). Multiply this weight coefficient by the feature F of size H×W×C to obtain the scaled new feature. The calculation of Mc(F) is shown in the following formula:

[0040]

[0041] Step 3, image post-processing and result output, is implemented as follows:

[0042] Step 3.1: Input the before-and-after images of the collapse test into the semantic segmentation method module based on multi-scale feature extraction constructed in Step 2, and output the predicted segmentation images P1 and P2.

[0043] Step 3.2: Post-process the segmented images P1 and P2 obtained in Step 3.1, perform region connectivity and noise reduction, and ensure that P1 achieves the segmentation of the slump bucket and P2 achieves the segmentation of the concrete.

[0044] Step 3.3: Search the post-processed images P1 and P2 output in Step 3.2. Find the highest point Y1 and the lowest point Y2 of the y-axis coordinate value among all pixels in P1 that are classified as slump buckets. Find the lowest point Y3 of the y-axis coordinate value among all pixels in P2 that are classified as concretes. Calculate the absolute difference between Y1 and Y2 in the y-axis coordinate value. Calculate the ratio K of the image size to the actual size based on the actual height of the slump bucket. Then calculate the absolute difference between Y2 and Y3 in the y-axis coordinate value. Calculate the actual height of the slump bucket based on K.

[0045] Step 3.4: Calculate the concrete classification results based on slump grading. The slump is 10-40mm, which is low plasticity concrete; 50-90mm, which is plasticity concrete; 100-150mm, which is fluid concrete; and slump greater than 160mm, which is high fluidity concrete.

[0046] Step 3.5: Output P1, P2, slump and concrete classification results.

[0047] The beneficial effects of this invention are:

[0048] In the concrete slump detection method based on image semantic segmentation of this invention, data preprocessing and image postprocessing are not involved in the model training process. Semantic segmentation based on multi-scale channel attention features uses a multi-scale feature extraction module with three branches to extract multi-scale information, providing richer contextual information. Before each downsampling, the current feature map is concatenated with the three-channel input image downsampled to the same resolution and the feature map after the previous downsampling. Furthermore, except for the first upsampling, before each upsampling, the current feature map is added to a multi-scale feature map with the same number of channels and resolution after channel attention, further improving prediction performance. This method overcomes the errors introduced by existing concrete slump detection methods that rely on manual operation and subjective judgment, resulting in more accurate slump detection results. It enriches the methodological system of computer vision and artificial intelligence technologies, providing an option for improving the accuracy of concrete slump detection. Attached Figure Description

[0049] Figure 1This is a schematic diagram of the network structure based on multi-scale channel attention features in step 2 of the concrete slump detection method based on image semantic segmentation of the present invention.

[0050] Figure 2 This is a schematic diagram of the multi-scale feature extraction module in step 2.1.2 of the concrete slump detection method based on image semantic segmentation of the present invention;

[0051] Figure 3 This is a schematic diagram of the decoder module in step 2.2 of the concrete slump detection method based on image semantic segmentation of the present invention;

[0052] Figure 4 This is a schematic diagram of the channel attention layer in step 2.2.1 of the concrete slump detection method based on image semantic segmentation of the present invention. Detailed Implementation

[0053] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0054] With the development of machine learning and deep learning, automatic detection of concrete slump has been extensively studied. These methods can identify and measure the slump of concrete based on features such as color, texture, and shape. Compared with traditional methods, deep learning-based methods can learn feature representations from a large amount of data and automatically extract features useful for slump prediction. The models can adapt to different concrete properties and construction conditions and have high prediction accuracy and stability.

[0055] This invention provides a concrete slump detection method based on image semantic segmentation. It utilizes convolutional neural networks as a foundation and focuses on semantic segmentation technology. Key technologies include: image preprocessing, constructing a semantic segmentation module based on multi-scale feature extraction, and image post-processing and result output. Through the implementation of these three key technologies, the concrete slump detection task is accomplished, thus overcoming the problems of existing concrete slump detection methods relying on manual operation and subjective judgment. This is of great significance for achieving high-precision slump detection to determine whether concrete is easy to construct and has uniform and dense properties, thereby ensuring the normal progress of construction.

[0056] Example 1

[0057] The concrete slump detection method based on image semantic segmentation is implemented according to the following steps:

[0058] Step 1: Image preprocessing and dataset construction;

[0059] Step 2: Construct a semantic segmentation module based on multi-scale channel attention features;

[0060] Step 3: Image post-processing and result output.

[0061] Example 2

[0062] The concrete slump detection method based on semantic segmentation is implemented according to the following steps:

[0063] Step 1: Image preprocessing and dataset construction:

[0064] Step 1.1: Perform image segmentation and image enhancement on images of various slump buckets and concrete samples with different slumps to ensure that the proportion of the target object in the image is close to that of the real experimental environment, while ensuring the data balance of the two types of key segmentation objects.

[0065] Step 1.2: Label the sample images processed in Step 1.1 in labelme. Label the slump bucket and the collapsed concrete in the sample images as slump bucket type and concrete type, respectively.

[0066] Step 1.3: Save the generated JSON file, convert it into segmented visualization images in batches, fill it with color, and generate the corresponding labeled images as GroundTruth. Together with the original images, these constitute the training set, validation set, and test set.

[0067] Step 2: The semantic segmentation module based on multi-scale attention channel features performs semantic segmentation on the image. The network structure diagram is shown below. Figure 1 As shown, the specific steps are as follows:

[0068] Step 2.1: Construct the encoder submodule, which mainly consists of convolutional layers and downsampling layers, including original feature extraction units and multi-scale feature extraction units;

[0069] Step 2.2 involves constructing the decoder submodule, primarily composed of a channel attention layer and an upsampling layer, used to progressively restore the resolution of the feature map, such as... Figure 3 As shown, the details are as follows:

[0070] Step 2.2.1: The feature map, which is 1 / 32 the size of 512 channels, first passes through the channel attention layer to calculate new features, such as... Figure 4 As shown;

[0071] Step 2.2.1.1: Perform spatial global max pooling and average pooling on the input feature F of size H×W×C to obtain two channel descriptions F of size 1×1×C. avg F max ;

[0072] Step 2.2.1.2: The output of step 2.2.1.1 is fed into a multilayer perceptron network, which consists of two fully connected layers and one ReLU activation layer;

[0073] Step 2.2.1.3: Add the two features output from Step 2.2.1.2 and pass them through a Sigmoid activation function to obtain the weight coefficient Mc(F). Multiply this weight coefficient by the feature F of size H×W×C to obtain the scaled new feature. The calculation of Mc(F) is shown in the following formula:

[0074]

[0075] Step 2.2.2: The 1 / 32 feature map with 512 channels obtained in Step 2.2.1 is upsampled and then fused with the 1 / 16 feature map with 256 channels obtained through the channel attention layer.

[0076] Step 2.2.3: The fused features are upsampled again and then added to and fused with the feature map of size 1 / 8 of 128 obtained by passing through the channel attention layer, as shown in step 2.2.1.

[0077] Step 2.2.4: Finally, after upsampling again, it is fused with the feature map that has passed through the channel attention layer, as shown in step 2.2.1, which has 64 channels and is 1 / 4 the size.

[0078] Step 2.2.5: The feature map upsampled to half the size of 16 channels is fed into the output convolutional layer. The stride of the output convolutional layer is 2, and the output channels are two-dimensional transposed convolutions of the number of classes.

[0079] Step 2.3: Set training hyperparameters and train and test the network;

[0080] Step 3: Perform image post-processing and output the results. The specific steps are as follows:

[0081] Step 3.1: Input the before-and-after images of the collapse test into the semantic segmentation method module based on multi-scale feature extraction constructed in Step 2, and output the predicted segmentation images P1 and P2.

[0082] Step 3.2: Post-process the segmented images P1 and P2 obtained in Step 3.1, perform region connectivity and noise reduction, and ensure that P1 achieves the segmentation of the slump bucket and P2 achieves the segmentation of the concrete.

[0083] Step 3.3: Search the post-processed images P1 and P2 output in Step 3.2. Find the highest point Y1 and the lowest point Y2 of the y-axis coordinate value among all pixels in P1 that are classified as slump buckets. Find the lowest point Y3 of the y-axis coordinate value among all pixels in P2 that are classified as concretes. Calculate the absolute difference between Y1 and Y2 in the y-axis coordinate value. Calculate the ratio K of the image size to the actual size based on the actual height of the slump bucket. Then calculate the absolute difference between Y2 and Y3 in the y-axis coordinate value. Calculate the actual height of the slump bucket based on K.

[0084] Step 3.4: Calculate the concrete classification results based on slump grading. The slump is 10-40mm, which is low plasticity concrete; 50-90mm, which is plasticity concrete; 100-150mm, which is fluid concrete; and slump greater than 160mm, which is high fluidity concrete.

[0085] Step 3.5: Output P1, P2, slump and concrete classification results;

[0086] Example 3

[0087] The specific scheme adopted in this embodiment for constructing the encoder submodule in step 2.1 is as follows:

[0088] Step 2.1.1: Construct an initial feature extraction unit, which contains a 3×3 convolution with a stride of 2 and two 3×3 convolutions with a stride of 1. The function of this unit is to extract the initial features.

[0089] Step 2.1.2, construct a multi-scale feature extraction module, such as... Figure 2 As shown, in order to achieve higher segmentation performance, the convolutions used are all standard convolutions and dilated convolutions;

[0090] Step 2.1.2.1: The input features are first processed through a 3×3 standard convolution to extract features and the number of channels is adjusted to half.

[0091] Step 2.1.2.2: After extracting features from three branches with different receptive fields, multi-scale features are extracted from the three branches. The specific steps are as follows:

[0092] Step 2.1.2.2.1: The first branch is a 3×3 standard convolution with a receptive field of 3×3, used to extract local information;

[0093] In step 2.1.2.2.2, the second branch is a 3×3 dilated convolution with a dilation rate of d, and its receptive field is (2×d+1)×(2×d+1).

[0094] In step 2.1.2.2.3, the third branch is a 3×3 dilated convolution with a dilation rate of d+1, and its receptive field is (2×d+3)×(2×d+3).

[0095] Step 2.1.2.3: Add and fuse the multi-scale features of the three branches, and use a 1×1 convolution to recover the number of channels;

[0096] Step 2.1.2.4 uses channel shuffling to facilitate information exchange between channels, and introduces residual connections before the feature extraction branch before shuffling to promote training and avoid gradient vanishing.

Claims

1. A method for concrete slump detection based on image semantic segmentation, characterized in that, The method is implemented according to the following steps: Step 1, image preprocessing and dataset construction; Step 2, constructing a semantic segmentation module based on multi-scale channel attention features; Step 3, image post-processing and result output; The step 2 is implemented according to the following steps: Step 2.1, constructing an encoder submodule composed of convolutional layers and down-sampling layers, including an original feature extraction unit and a multi-scale feature extraction unit; Step 2.2, constructing a decoder submodule composed of channel attention layers and up-sampling layers, for gradually restoring the resolution of the feature map; Step 2.3, setting training hyperparameters, training and testing the network; The step 2.1 is implemented according to the following steps: Step 2.1.1, constructing an initial feature extraction unit, including one 3x3 convolution with a stride of 2 and two 3x3 convolutions with a stride of 1, to extract initial features; Step 2.1.2, constructing a multi-scale feature extraction module, with standard convolution and dilated convolution; The step 2.1.2 is implemented according to the following steps: Step 2.1.2.1, the input features are first passed through a 3x3 standard convolution for feature extraction and adjusting the channel number to half; Step 2.1.2.2, three feature extraction branches with different receptive fields are used to extract multi-scale features from each branch; Step 2.1.2.3, the multi-scale features from the three branches are added and fused, and a 1x1 convolution is used to restore the channel number; Step 2.1.2.4, channel shuffling is used for information exchange between channels, and a residual connection is introduced before the feature extraction branch; The step 2.1.2.2 is implemented according to the following steps: Step 2.1.2.2.1, the first branch is a 3x3 standard convolution with a receptive field of 3x3, used to extract local information; Step 2.1.2.2.2, the second branch is a 3x3 dilated convolution with a dilation rate of d, whose receptive field is ; Step 2.1.2.2.3, the third branch is a 3x3 dilated convolution with a dilation rate of d+1, whose receptive field is ; The step 3 is implemented according to the following steps: Step 3.1, send the contrast images before and after the slump experiment into the semantic segmentation method module based on multi-scale feature extraction constructed in step 2, output the predicted segmentation image , ; Step 3.

2. Post-processing of the segmented image obtained in step 3.1 , performing region connectivity and noise handling, ensuring implementing the segmentation of the slump bucket, implementing the segmentation of the concrete; Step 3.

3. Post-processing the output image of step 3.2 , Find the highest point of y-axis coordinate value in all pixels of the category of slump bucket in Find the highest point of y-axis coordinate value in all pixels of the category of slump bucket in and the lowest point Find the lowest point of y-axis coordinate value in all pixels of the category of concrete in Calculate the absolute difference of y-axis coordinate value between and Calculate the absolute difference of y-axis coordinate value between According to the actual height of the slump bucket, calculate the ratio K of the image size and the actual size, and then calculate and the absolute difference of y-axis coordinate value, and calculate the actual height of the slump according to K. Step 3.4, according to the slump grading calculation, the concrete classification result is obtained, wherein the slump is within 10-40mm, which is low plasticity concrete; the slump is within 50-90mm, which is plasticity concrete; the slump is within 100-150mm, which is fluidity concrete; the slump is greater than 160mm, which is high fluidity concrete; Step 3.5, output , , slump and concrete classification results. 2.The method of claim 1, wherein, The step 1 is implemented according to the following steps: Step 1.1, image cutting and image enhancement are performed on various slump buckets and concrete sample images with different slump; Step 1.2, the sample images processed in step 1.1 are labeled in labelme, and the slump bucket and the concrete after slump in the sample images are labeled as slump bucket class and concrete class respectively; Step 1.3, save the generated json file, batch convert it into segmented visualization images, and then perform color filling to generate labeled images corresponding to the images, which are used as GroundTruth, together with the original images, to form a training set, a validation set and a test set. 3.The method of claim 1, wherein, The step 2.2 is implemented according to the following steps: Step 2.2.1, the 1 / 32 size feature map with 512 channels first passes through a channel attention layer to calculate new features; Step 2.2.2, the 1 / 32 feature map with 512 channels obtained in step 2.2.1 is added and fused with the 1 / 16 feature map with 256 channels obtained by passing through the channel attention layer; Step 2.2.3, the fused features are added and fused with the 1 / 8 size feature map with 128 channels obtained by passing through the channel attention layer after upsampling; Step 2.2.4, the fused features are added and fused with the 1 / 4 size feature map with 64 channels obtained by passing through the channel attention layer after upsampling again; Step 2.2.5, the 1 / 2 size feature map with 16 channels obtained by upsampling is input into an output convolution layer, the output convolution layer has a step of 2 and outputs a two-dimensional transposed convolution with the number of channels being the number of categories. 4.The concrete slump detection method based on image semantic segmentation according to claim 3, characterized in that, The step 2.2.1 is implemented according to the following steps: Step 2.2.1.

1. Spatial global max-pooling and average-pooling are performed on the input of size respectively to obtain two channel descriptions of size , ;​ Step 2.2.1.2, the output of step 2.2.1.1 is input into a multi-layer perception network composed of two fully connected layers and a Relu activation layer; Step 2.2.1.3 Add the two features output from Step 2.2.1.2 and pass through a Sigmoid activation function to get the weight coefficient , and multiply it with the feature F of size to get the scaled new feature, where The calculation of the weight coefficient is shown as follows: （1）。

Images