A fair-faced concrete color difference detection and identification method, system, device and storage medium

Abstract

The present application belongs to the technical field of color difference analysis, and particularly relates to a method for detecting and identifying color difference of fair-faced concrete. First, a preset amount of image data of the fair-faced concrete is collected, and the image data of the fair-faced concrete is preprocessed. Then, a color difference identification model of the fair-faced concrete is constructed based on a preset semantic segmentation algorithm, and an identification result of the color difference identification model of the fair-faced concrete is output. The identification result is subjected to color space conversion, histogram equalization, Kalman filter denoising, image opening and closing operations, and image mask operation, so as to generate a preliminary quantization result of the color difference of the fair-faced concrete. Finally, a color difference quantization model of the fair-faced concrete is constructed and trained, the preliminary quantization result of the color difference of the fair-faced concrete is taken as an input parameter of the color difference quantization model of the fair-faced concrete, and a quantization result of the color difference of the fair-faced concrete is output. The present application can solve the problems of strong subjectivity and poor generalization ability of the image recognition technology used for detecting and identifying the color difference of the fair-faced concrete.

Description

Technical Field

[0001] This invention pertains to color difference analysis technology, and particularly relates to a method, system, equipment, and storage medium for detecting and identifying color differences in fair-faced concrete. Background Technology

[0002] After fair-faced concrete and its precast components are poured or hoisted, no secondary decoration work is required, which greatly reduces the use of surface decoration materials. Therefore, it is an environmentally friendly and beautiful green building material. Compared with ordinary concrete, fair-faced concrete uses its natural surface as the decorative surface. While meeting mechanical and durability requirements, it also has a higher quality appearance. It is often used in civil fields, highway and railway fields, urban overpass fields, etc.

[0003] During construction, the surface of fair-faced concrete no longer requires chiseling, repair, or plastering, reducing construction waste at the site. Therefore, the elimination of secondary decoration processes requires that the surface of fair-faced concrete achieves a beautiful and clean effect after a single construction, meeting the architectural design expectations. Thus, the acceptance process for the appearance quality of fair-faced concrete buildings is extremely important. The main defects in the appearance quality of fair-faced concrete include color difference, cracks, and bubbles. The control of color difference on the decorative surface of fair-faced concrete affects the overall effect of the building's appearance. Therefore, color difference is a key point in the appearance control of fair-faced concrete.

[0004] Current research attempts to identify color differences in fair-faced concrete using image processing techniques and deep learning algorithms. This primarily involves subjectively adjusting the contrast of unstructured data such as images, trimming noise points, enhancing edge data points, data transposition, and grayscale processing to achieve qualitative and quantitative analysis. By reading the texture features of unstructured data, image classification techniques can detect cracks and air bubbles to some extent. Image processing techniques often target the data features of the image itself, have a relatively small computational scale, and can quickly provide detection results. However, in the inspection of the appearance quality of fair-faced concrete, image processing techniques rely on manually set grayscale thresholds, which inevitably leads to subjectivity. Therefore, while current image processing techniques are suitable for the current experimental data, they lack the ability to generalize to similar data. Summary of the Invention

[0005] The technical problem solved by this invention is to provide a method, system, device and storage medium for color difference detection and identification of fair-faced concrete, so as to solve the problems of strong subjectivity and poor generalization ability of the image recognition technology currently used for color difference detection and identification of fair-faced concrete.

[0006] The basic solution provided by this invention is a method for detecting and identifying color difference in fair-faced concrete, comprising:

[0007] S1: According to the preset fair-faced concrete image acquisition scheme, acquire a preset amount of fair-faced concrete image data and preprocess the fair-faced concrete image data.

[0008] S2: Construct a fair-faced concrete color difference recognition model based on a preset semantic segmentation algorithm. Use the preprocessed fair-faced concrete image data as the input parameters of the fair-faced concrete color difference recognition model, output the recognition result of the fair-faced concrete color difference recognition model, and perform color space conversion, histogram equalization, Kalman filter noise reduction, image opening and closing operations, and image masking operations on the recognition result to generate preliminary quantification results of fair-faced concrete color difference.

[0009] S3: Construct and train a color difference quantification model for fair-faced concrete, use the preliminary color difference quantification results of fair-faced concrete as input parameters for the color difference quantification model, and output the color difference quantification results of fair-faced concrete.

[0010] Furthermore, the color difference quantification model for fair-faced concrete is constructed based on an improved ResNet50 semantic segmentation algorithm, and S3 includes:

[0011] S3-1: Divide the image data in the preliminary quantification result of the color difference of fair-faced concrete into different gray ranges according to the preset average gray value, and generate image data of different ranges;

[0012] S3-2: Mark the image data in the preliminary quantification results of color difference of fair-faced concrete with color difference blocks using the annotation tool to generate a dataset with color difference block label information;

[0013] S3-3: Construct a color difference quantization model for fair-faced concrete based on the improved ResNet50 semantic segmentation algorithm, and train the color difference quantization model for fair-faced concrete based on training metrics. Use the dataset as the input image of the trained color difference quantization model for fair-faced concrete, and output the color difference quantization result of fair-faced concrete.

[0014] Furthermore, the color difference quantization result includes color difference block mask, bounding box, color difference block number, and color difference block percentage.

[0015] Furthermore, the improved ResNet50 semantic segmentation algorithm is specifically as follows:

[0016] The ResNet50 semantic segmentation framework consists of the Backbone main network, the FCN Head sub-network, and the Deeplab sub-network.

[0017] Adjust the stride of the 1×1 convolution kernel mapping box of the left residual structure in the Backbone main network to 1, and the stride of the 3×3 convolution kernel mapping box to 2.

[0018] Add a max pooling layer with a kernel size of 2×2 and a bounding box stride of 1 to the residual structure on the right.

[0019] Furthermore, the training metrics in S3-3 include the learning rate, loss function, and accuracy evaluation metrics. The accuracy evaluation metrics include pixel accuracy, average pixel accuracy per class, intersection-over-union ratio (IoU), and average IoU. The pixel accuracy evaluation formula is:

[0020]

[0021] Where PA represents pixel accuracy, TP represents the model prediction result and the actual label result are the same, and the result category is 1, TN represents the model prediction result category is 0 and the actual label result category is 1, FP represents the model prediction result category is 1 and the actual label result category is 0, and FN represents the model prediction result and the actual label result are the same, and the result category is 0.

[0022] The formula for evaluating the average pixel accuracy of the category is:

[0023]

[0024] The crossover-union ratio evaluation formula is as follows:

[0025]

[0026] Where IoU represents the intersection-union ratio.

[0027] Furthermore, S1 includes:

[0028] S1-1: Image acquisition of fair-faced concrete using a drone equipped with a gimbal;

[0029] S1-2: The collected fair-faced concrete image data is processed by image segmentation, data augmentation, and data selection to generate a fair-faced concrete image dataset.

[0030] Furthermore, the preset semantic segmentation algorithms in S2 include the MobileNetV3 semantic segmentation algorithm, the ResNet50 semantic segmentation algorithm, and the VGGNet16 semantic segmentation algorithm. S2 includes:

[0031] S2-1: Based on the usage scenario, a preset semantic segmentation algorithm is invoked to construct a fair-faced concrete color difference recognition model. The algorithm hyperparameters of batch size, learning rate, and iteration number are automatically optimized and trained to identify the fair-faced concrete color difference.

[0032] S2-2: Input the fair-faced concrete image dataset into the trained fair-faced concrete color difference recognition model, and perform color space conversion, histogram equalization, Kalman filter noise reduction, image opening and closing operations, and image masking operations to generate quantization results of light-colored complete color difference images, light-colored incomplete color difference images, dark-colored complete color difference images, and dark-colored incomplete color difference images.

[0033] A color difference detection and identification system for fair-faced concrete, applied to the aforementioned color difference detection and identification method for fair-faced concrete, includes:

[0034] The data acquisition module is used to acquire a preset amount of fair-faced concrete image data according to the preset fair-faced concrete image acquisition scheme.

[0035] The data preprocessing module is used to preprocess the image data of fair-faced concrete.

[0036] The color difference recognition module is used to construct a color difference recognition model for fair-faced concrete based on a preset semantic segmentation algorithm. It takes the preprocessed fair-faced concrete image data as the input parameters of the color difference recognition model, outputs the recognition result of the color difference recognition model, and performs color space conversion, histogram equalization, Kalman filter noise reduction, image opening and closing operations, and image masking operations on the recognition result to generate a preliminary quantification result of the color difference of fair-faced concrete.

[0037] The color difference quantization module is used to build and train the color difference quantization model of fair-faced concrete. It takes the preliminary color difference quantization result of fair-faced concrete as the input parameter of the color difference quantization model and outputs the color difference quantization result of fair-faced concrete.

[0038] An electronic device includes a processor and a memory, wherein the memory stores programs or instructions, and the processor executes a method for detecting and identifying color differences in fair-faced concrete as described above by calling the programs or instructions stored in the memory.

[0039] A computer-readable storage medium storing a program or instructions that causes a computer to perform a method for detecting and identifying color differences in fair-faced concrete as described above.

[0040] The principle and advantages of this invention are as follows: In this application, addressing the problem of color difference detection and recognition in fair-faced concrete, image classification algorithms, image processing techniques, and semantic segmentation algorithms are comprehensively applied to identify and quantify color differences in fair-faced concrete buildings during the appearance quality acceptance stage. First, a fair-faced concrete image acquisition scheme is designed, and the image data is preprocessed to improve the accuracy, quality, and scale of the original data. Then, a fair-faced concrete color difference recognition model is constructed using a semantic segmentation algorithm. After training the model, the image data undergoes color space conversion, histogram equalization, Kalman filtering noise reduction, image opening and closing operations, and image masking. Membrane manipulation can initially achieve color difference quantification of fair-faced concrete. However, to address the insufficient generalization ability and subjectivity of image processing technology during the quantification process, a color difference quantification model for fair-faced concrete is constructed and trained by dividing the average grayscale range and using an improved ResNet50 semantic segmentation algorithm. This improves the quantification effect of fair-faced concrete, ultimately enabling effective detection and recognition of color differences in fair-faced concrete and improving detection efficiency. At the same time, it reduces the requirements for quality inspectors' experience, effectively avoids subjective problems in the quality inspection process, has a short detection time for a single image, and high actual detection accuracy, effectively improving the generalization ability of color difference defects in fair-faced concrete. Attached Figure Description

[0041] Figure 1 This is a flowchart of an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of image distortion types according to an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram showing the generalization prediction results of three models in an embodiment of the present invention;

[0044] Figure 4 This is a preliminary quantization structure diagram of an embodiment of the present invention;

[0045] Figure 5 This is a statistical chart of the quantification results for each category in an embodiment of the present invention;

[0046] Figure 6 This is a comparison diagram of the improved residual network and the original residual network in an embodiment of the present invention;

[0047] Figure 7 This is a schematic diagram of the training loss of the ResNet50 semantic segmentation framework in an embodiment of the present invention;

[0048] Figure 8 This is a quantization effect diagram after algorithm improvement according to an embodiment of the present invention;

[0049] Figure 9 This is a functional block diagram of the system in an embodiment of the present invention;

[0050] Figure 10 This is a schematic diagram of the electronic device structure according to an embodiment of the present invention. Detailed Implementation

[0051] The following detailed description illustrates the specific implementation method:

[0052] The reference numerals in the accompanying drawings include: electronic device 400, processor 401, memory 402, input device 403, and output device 404.

[0053] The basic implementation examples are as follows: Figure 1 As shown: A method for detecting and identifying color difference in fair-faced concrete, comprising:

[0054] S1: According to the preset fair-faced concrete image acquisition scheme, acquire a preset amount of fair-faced concrete image data, and preprocess the fair-faced concrete image data; wherein, S1 includes:

[0055] S1-1: Image acquisition of fair-faced concrete using a drone equipped with a gimbal;

[0056] S1-2: The collected fair-faced concrete image data is processed by image segmentation, data augmentation, and data selection to generate a fair-faced concrete image dataset.

[0057] In this embodiment, a drone equipped with a gimbal is used to move horizontally at a fixed height or to vertically ascend and descend relative to a ground control point to meet the image acquisition needs of a large-scale exposed concrete building at different elevations. Specifically, to ensure that the pixel size of the acquired image data meets the requirements of the model, the camera on the drone equipped with the gimbal has a resolution of no less than 20 megapixels. To ensure that the drone can acquire effective data during horizontal movement and vertical ascent and descent, and to achieve equal time intervals or equal distance intervals for taking pictures, the minimum shutter speed of the camera should not exceed 1 / 1000 second. To ensure the continuity of the data acquisition process and avoid data overlap and data loss caused by repeated take-off and landing of the drone, the continuous working time of the drone should not be less than 2 hours, and the storage space of the device should not be less than 4.9GB. Therefore, the drone used in this application is the DJI Movic 2 Pro drone.

[0058] Furthermore, during image acquisition, image distortion can cause deformation of the fair-faced concrete image, affecting the accuracy of color difference quantification for fair-faced concrete; for example... Figure 2 As shown, image distortion is divided into radial distortion and tangential distortion. Therefore, this application constructs radial distortion value models and tangential distortion value models. The expression for the radial distortion value model is as follows:

[0059]

[0060] Where, xd and y d ω1, ω2, and ω3 represent the radial distortion values ​​in the x and y directions, respectively; ω1, ω2, and ω3 represent the radial distortion coefficients in the x and y directions, respectively.

[0061] The expression for the tangential distortion value model is:

[0062]

[0063] Where, x s and y s λ1, λ2, λ3, and λ4 represent the tangential distortion values ​​in the x and y directions, respectively; λ1, λ2, λ3, and λ4 represent the tangential distortion coefficients in the x and y directions, respectively.

[0064] Subsequently, by constructing a fair-faced concrete grid calibration network, the radial and tangential distortion values ​​were calculated to obtain the optimal shooting distance, which in this application is 5m.

[0065] Based on the above-designed fair-faced concrete image acquisition scheme, a preset amount of fair-faced concrete image data is acquired. In this application, the acquisition amount of fair-faced concrete image data is set according to the actual situation. In this application, the acquired image data consists of 512 original images with a size of 5472×3684px. Subsequently, the acquired original images are preprocessed.

[0066] First, given the large pixel size of the acquired original images, image segmentation is performed, dividing each original image into 306 images of size 256×256px. A color difference detection model for fair-faced concrete is then constructed from the segmented image data to filter out image data containing fair-faced concrete color differences. In this application, the fair-faced concrete color difference detection model is constructed using a binary classification algorithm.

[0067] To balance the data volume of chromatic difference images and non-chromatic difference images, and to increase the complexity of the fair-faced concrete chromatic difference dataset, data augmentation is needed to optimize the image dataset. Specifically, this application uses the method of adjusting auxiliary attribute information for data augmentation, including the ADASYN method and the Borderline SMOTE method.

[0068] After data augmentation, the dataset used by the fair-faced concrete color difference detection model includes data with color difference and data without color difference. Therefore, the dataset is selected, and the data with color difference and the data without color difference are stored in different memories. The data with color difference is named abnormal, and the data without color difference is named normal.

[0069] For training the color difference detection model for fair-faced concrete, this application specifically involves creating corresponding category labels, namely abnormal and normal labels, for the image experimental data in the database. Then, all the processed image experimental data are divided into three datasets: a training set, a validation set, and a generalization prediction set. Each dataset includes 50% of the experimental data with color difference and 50% of the experimental data without color difference, in order to train the model.

[0070] At this point, the fair-faced concrete image dataset is generated. Color difference recognition is then performed using image processing techniques. Specifically, this includes S2: constructing a fair-faced concrete color difference recognition model based on a pre-defined semantic segmentation algorithm. The pre-processed fair-faced concrete image data is used as the input parameters of the color difference recognition model, which outputs the recognition results. The recognition results are then subjected to color space transformation, histogram equalization, Kalman filtering noise reduction, image opening and closing operations, and image masking operations to generate preliminary quantification results of the fair-faced concrete color difference. The pre-defined semantic segmentation algorithms in S2 include MobileNetV3 semantic segmentation, ResNet50 semantic segmentation, and VGGNet16 semantic segmentation. S2 includes:

[0071] S2-1: Based on the usage scenario, a preset semantic segmentation algorithm is invoked to construct a fair-faced concrete color difference recognition model. The algorithm hyperparameters of batch size, learning rate, and iteration number are automatically optimized and trained to identify the fair-faced concrete color difference.

[0072] S2-2: Input the fair-faced concrete image dataset into the trained fair-faced concrete color difference recognition model, and perform color space conversion, histogram equalization, Kalman filter noise reduction, image opening and closing operations, and image masking operations to generate quantization results of light-colored complete color difference images, light-colored incomplete color difference images, dark-colored complete color difference images, and dark-colored incomplete color difference images.

[0073] In this embodiment, the semantic segmentation algorithm used to construct the fair-faced concrete color difference recognition model is selected based on the usage scenario. The semantic segmentation algorithms in this application include MobileNetV3, ResNet50, and VGGNet16. The actual usage scenario involves using a drone to collect images of fair-faced concrete indoors. Therefore, the algorithms are compared in this scenario. Specifically, based on the unstructured data algorithm category, MobileNetV3, ResNet50, and VGGNet16 models are initially adopted. Then, the models are trained using algorithm hyperparameters. In this application, the hyperparameters used are batch size, learning rate, and number of iterations. The batch size refers to the total number of unstructured sample data in a batch. To ensure the reliability of the batch size hyperparameter optimization, the learning rate is uniformly set to 0.001, and the number of iterations is uniformly set to 50. In the training results of batch size, the optimal batch size parameter value appears at the position with the lowest loss function value. Therefore, in MobileNetV3, the loss function value of a batch size of 40 is 0.053, in ResNet50 the loss function value of a batch size of 32 is 0.160, and in VGGNet16 the loss function value of a batch size of 40 is 0.694.

[0074] In training the learning rate hyperparameter, the batch size was fixed according to the optimal batch size mentioned above, and the number of iterations was set to be near the convergence of the three models, which was 100. In the training results of the learning rate, the optimal learning rate value for the MobileNetV3 model was 10. -3.0 The optimal learning rate for the ResNet50 model is 10. -4.0 The optimal learning rate for the VGGNet16 model is 10. -4.6 ;

[0075] The three models were then trained with a preset iteration count of 2000. The batch size and learning rate were set as described above. The specific performance of the three models is as follows:

[0076] 1. During the training phase, the loss function of the MobileNetV3 model begins to converge after the 50th iteration and reaches its minimum value of 0.0956 at the 223rd iteration. In the second phase, the loss function begins to converge after the 550th iteration and reaches its minimum value of 0.0115 at the 632nd iteration.

[0077] 2. During the training of the ResNet50 model, the loss function began to converge after the 100th iteration and reached its minimum value of 0.0610 at the 755th iteration.

[0078] 3. During the training of the VGGNet16 model, the loss function began to converge at the 50th iteration and reached its minimum value of 0.0562 at the 653rd iteration.

[0079] For the three trained models, it is necessary to evaluate their training data. To verify the generalization prediction ability of the three models, this application sets up a fair-faced concrete generalization prediction dataset. The fair-faced concrete generalization prediction dataset includes training and validation sets, which are mechanically divided according to a ratio; therefore, both training and validation sets are from the same source. By analyzing the scatter distribution of the Prob probability value labels using the fair-faced concrete generalization prediction dataset, the generalization prediction results of the MobileNetV3, ResNet50, and VGGNet16 models on the color difference of fair-faced concrete can be obtained, such as... Figure 3 As shown, the confidence scores for experimental images with and without color difference for the MobileNetV3 model are 0.82 to 1.00 and 0.81 to 1.00, respectively; for the ResNet50 model, the confidence scores for experimental images with and without color difference are 0.88 to 1.00 and 0.86 to 1.00, respectively; and for the VGGNet16 model, the confidence scores for experimental images with and without color difference are 0.84 to 1.00 and 0.86 to 1.00, respectively. In the generalization prediction experiment, the generalization prediction accuracy of the three fair-faced concrete color difference recognition models all reached over 99%, but the ResNet50 model had higher generalization prediction ability and confidence. Therefore, this application constructs the fair-faced concrete color difference recognition model by selecting the ResNet50 model after the experiment. In other embodiments of this application, other types of semantic segmentation algorithms can also be called according to other usage scenarios. For example, the MobileNetV3 semantic segmentation algorithm is better selected on the terminal device. Therefore, the basic algorithm of the fair-faced concrete color difference recognition model used in this application can be selected differently according to different usage scenarios.

[0080] After the fair-faced concrete color difference recognition model is built and trained, the acquired fair-faced concrete dataset is input into the model. For the output recognition results, color space conversion, histogram equalization, Kalman filtering for noise reduction, image opening and closing operations, and image masking are performed. Specifically, color space conversion involves converting the RGB color space of the acquired fair-faced concrete image data to the HSV color space using OpenCV. In the HSV color space, saturation is encoded using grayscale values, which clearly quantifies the defect characteristics of color difference in fair-faced concrete. Histogram equalization involves plotting histograms and luminance histograms for each color channel to... The algorithm calculates the average, standard deviation, and median values ​​of image pixels under different channel states, enabling automatic optimization of HSV color space parameters. Kalman filtering denoising involves linear filtering of the fair-faced concrete color difference image to remove noise points in non-color difference regions of the original image. This eliminates black noise pixels scattered in non-color difference regions and white noise pixels scattered within color difference regions, while preserving the original contours of the color difference regions and the non-color difference parts within them. Image opening and closing operations are performed as follows: The opening operation uses erosion followed by dilation to separate local images, removing brighter pixels, softening scattered noise, and amplifying the aggregate features of local targets. After the opening operation, the fair-faced concrete color difference image removes scattered color differences, strengthens and amplifies the color differences in specific local areas, while weakening and shrinking non-color difference regions. The closing operation also includes erosion and dilation, but in the reverse order of the opening operation. Dilation assimilates pixels surrounding local image features, making the original features more prominent and the image contours smoother. By erosion, relatively small holes in local image features are filled, gaps in closely spaced areas are connected, and missing contour lines are reshaped, minimizing minor imperfections in the image. Therefore, the closed-operation process reduces the independence between color difference blocks in the fair-faced concrete color difference image, enhances the integrity and connectivity of individual color difference blocks, and compresses the number of color difference blocks in the image. The contours of the fair-faced concrete color difference regions are smoother than those obtained with the opening operation, and point-like color differences are further weakened and eliminated. The image masking operation involves calculating and outputting a color difference mask and label information from the sample image using an image masking algorithm. The pixel dimensions of the fair-faced concrete color difference allow for direct calculation of the relative area and percentage of color difference blocks, which is more valuable than millimeter dimensions. Therefore, the image masking algorithm directly outputs the color difference pixel dimensions. The fair-faced concrete color difference mask label includes the color difference block number, color difference block percentage, and geometric parameter features (length and width), listed from left to right. The color difference blocks are numbered from largest to smallest based on the percentage of color difference, which helps quality inspectors quickly determine the distribution of the largest color difference blocks in a local area.The percentage of color difference blocks represents the proportion of a single color difference block in the sample image, and can be further used to obtain the total percentage of color difference blocks in the entire sample image. By reconstructing and summing the detected images, the percentage of color difference in a local area or the whole of the sample building can be obtained, providing quality inspectors with a basis for evaluating the degree of color difference in fair-faced concrete; and the output of the final image masking operation, such as... Figure 4 As shown, preliminary quantification results of color differences for different categories can be obtained.

[0081] For example Figure 5 As shown, it lists Figure 4 The specific quantitative information of color difference images of different types of fair-faced concrete shows that the boundary between the color difference area and the non-color difference area in the light-colored fair-faced concrete image is clear, and the mask can accurately cover the color difference area, but it still cannot avoid the non-color difference part in the color difference area, such as... Figure 4 As shown in Figure a, the boundary between the color difference region and the non-color difference region in the color difference image of dark fair-faced concrete is blurred. The mask merges and covers the color difference regions that are closer together, thus increasing the proportion of the color difference region, as shown in Figure a. Figure 4 As shown in d, the total color difference ratio of a light-colored incomplete color difference image should be less than 34.75%, and the total color difference ratio of a dark-colored complete color difference image should be less than 35.08%. Therefore, the color difference quantization method for fair-faced concrete based on image processing can quantify different types of color difference images, but it cannot accurately quantify dark-colored color difference images, and the proportion of color difference areas in the quantization results is too large.

[0082] Therefore, further processing is needed on the preliminary quantization results output and processed by the fair-faced concrete recognition model. Specifically, this includes S3: constructing and training a fair-faced concrete color difference quantization model, using the preliminary quantization results of the fair-faced concrete color difference as input parameters to the model, and outputting the quantization results of the fair-faced concrete color difference. The fair-faced concrete color difference quantization model is built based on an improved ResNet50 semantic segmentation algorithm, and S3 includes:

[0083] S3-1: Divide the image data in the preliminary quantification result of the color difference of fair-faced concrete into different gray ranges according to the preset average gray value, and generate image data of different ranges;

[0084] S3-2: Mark the image data in the preliminary quantification results of color difference of fair-faced concrete with color difference blocks using the annotation tool to generate a dataset with color difference block label information;

[0085] S3-3: Construct a color difference quantization model for fair-faced concrete based on the improved ResNet50 semantic segmentation algorithm, train the color difference quantization model for fair-faced concrete based on training metrics, use the dataset as the input image of the trained color difference quantization model for fair-faced concrete, and output the color difference quantization result for fair-faced concrete.

[0086] In this embodiment, in order to accurately quantify the color difference of fair-faced concrete and optimize the detection performance of the fair-faced concrete color difference recognition model when the average gray value varies widely, the preliminary quantization results of fair-faced concrete are first divided into different gray ranges according to the preset average gray value. Specifically, based on the experimental results, the experimental images with average gray values ​​greater than 189 and less than 137 are relatively scattered and not suitable for further subdivision. To ensure the balance of the training data, the image data with average gray values ​​in the interval [0, 137) are... Image data with average gray values ​​in the interval [137, 150] and average gray values ​​in the interval [202, 256] are merged into the interval [189, 202). Therefore, based on the preset average gray value, the gray range is divided every 13 average gray units, resulting in five gray ranges: [0, 150), [150, 163), [163, 176), [176, 189), and [189, 256]. Each gray range contains 600 image data in this application, for a total of 3326 images.

[0087] After the grayscale range is divided, the color difference image of fair-faced concrete is labeled using a labeling tool to mark the color difference blocks in the image. Specifically, this application uses the CVAT labeling tool to mark the color difference blocks. A single fair-faced concrete experimental image contains one or more color difference blocks. CVAT is used to number these blocks starting from the first color difference label of the first experimental image in the new labeling task, thus recording the label information of all color difference blocks. For ease of retrieval, the label information is stored as a JSON file in the same computer path as the training program. Individual label information is stored in a 2×N matrix, where N represents the number of annotation points for a single label. The labeled fair-faced concrete color difference image serves as the dataset for subsequent color difference quantization.

[0088] After the dataset was generated, a model for quantizing the color difference of fair-faced concrete was constructed based on the improved ResNet50 semantic segmentation algorithm. The ResNet50 semantic segmentation algorithm consists of a backbone main network, an FCN Head sub-network, and a Deeplab sub-network. The color difference image input to the algorithm framework is transmitted through the backbone main network. The data first goes through the FCN Head sub-network for convolution conv_1 operation, and then returns to the backbone network for further learning. The Deeplab sub-network transforms the feature information extracted by the backbone network into the color difference quantization result of fair-faced concrete through dilated convolution and depthwise separable convolution.

[0089] Specifically, the FCN Head sub-network is a fully convolutional network structure. It classifies each pixel in the color difference image and has 3 data channels. It outputs a segmentation result with the same size as the input image and uses bilinear interpolation to perform upsampling to extract feature information.

[0090] The Deeplab sub-network comprises the ASPP module and the Deeplab Head module. The ResNet50 semantic segmentation algorithm framework uses the Deeplab sub-network as its backbone, restoring the original image data size and outputting detection results through the ASPP module using the feature information extracted by the FCN Head sub-network. The feature information extracted from the FCN Head sub-network undergoes simultaneous convolution operations (1×1 kernel), dilated convolution operations (3×3 kernel), adaptive pooling (Adaptive Avg Pool 2d), and bilinear interpolation operations in the ASPP module. Feature matrices of the same size are stacked together using a concat operation, and then dimensionality reduction is achieved through a 1×1 kernel convolution operation. The ASPP module transforms input data of arbitrary size into output data of arbitrary size through adaptive pooling, effectively simplifying the computation process and reducing the computational cost of pooling layers. Similar to the FCN Head sub-network, the Deeplab Head module performs pixel-level classification on the feature data after convolution by the ASPP module, achieving semantic segmentation results for the image.

[0091] To further simplify network complexity and reduce network computation, such as Figure 6 As shown, the stride of the 1×1 convolutional kernel mapping boxes in the left residual structure of the backbone network is adjusted to 1, and the stride of the 3×3 convolutional kernel mapping boxes is adjusted to 2. A max-pooling layer with a 2×2 kernel size and a mapping box stride of 1 is added to the right residual structure. This improved residual network is referred to as the ResNet50_mp residual network. The unimproved ResNet50 residual network and the improved ResNet50_mp residual network are embedded into the ResNet50 backbone network for testing. The training loss process of the two models is as follows: Figure 7 As shown. During training, the hyperparameter settings followed the optimization results of the ResNet50 algorithm, with a batch size of 32 and a learning rate of 10. -4.0The iteration count was 2000. Training results show that ResNet50 converges faster in the early stages, stabilizing the training loss region around 200 iterations. However, the convergence value of the loss function after stabilization is high, and the function fluctuates significantly. ResNet50_mp converges more slowly in the early stages, but the convergence value of the training loss function is lower than that of ResNet50 in the later stages, with smaller amplitude and fewer noise points in the later stages. Figure 7 As shown, the ResNet50 residual network converged to the minimum loss function value of 0.1497 after 1428 iterations. After 750 iterations, the loss function of the ResNet50_mp residual network decreased relatively smoothly, but a few noisy data points remained. After 1300 iterations, the noise points gradually disappeared, and the loss function showed a stable fluctuation state, converging to the minimum value of 0.1537 at the 1016th iteration. Therefore, ResNet50_mp was chosen as the residual network for the ResNet50 backbone main network in the fair-faced concrete color difference quantification model.

[0092] The completed color difference quantification model for fair-faced concrete needs to be trained according to training metrics. The training metrics in this application include pixel accuracy, average pixel accuracy per category, intersection-over-union (IoU), and average IoU. The pixel accuracy evaluation formula is as follows:

[0093]

[0094] PA represents pixel accuracy, TP represents the model prediction result and the actual label result are the same, and the result category is 1, TN represents the model prediction result category is 0 and the actual label result category is 1, FP represents the model prediction result category is 1 and the actual label result category is 0, and FN represents the model prediction result and the actual label result are the same, and the result category is 0.

[0095] In the process of quantizing the color difference of fair-faced concrete with different average grayscale ranges, the pixel accuracy was 93.63% for the average grayscale range [0, 150), 95.52% for the average grayscale range [150, 163), 94.50% for the average grayscale range [163, 176), 94.47% for the average grayscale range [176, 189), and 93.74% for the average grayscale range [189, 256]. The model showed good pixel accuracy across different average grayscale ranges, with slightly lower pixel accuracy at the extremes of the average grayscale range.

[0096] The formula for evaluating the average pixel accuracy of the category is:

[0097]

[0098] Based on the model's training results, the average pixel accuracy for each category in the average grayscale range [0, 150) is 81.36%, the average pixel accuracy for each category in the average grayscale range [150, 163) is 81.55%, the average pixel accuracy for each category in the average grayscale range [163, 176) is 83.30%, the average pixel accuracy for each category in the average grayscale range [176, 189) is 82.45%, and the average pixel accuracy for each category in the average grayscale range [189, 256] is 81.00%.

[0099] The crossover-union ratio evaluation formula is as follows:

[0100]

[0101] Where IoU represents the crossover ratio. The quantification process of color difference in fair-faced concrete with different average gray ranges includes the color difference part and the non-color difference part. The crossover ratios for the average gray range [0, 150) are 94.33% and 79.67%, for the average gray range [150, 163) are 96.20% and 82.01%, and for the average gray range [163, 176) are 94.35% and 83.76%.

[0102] The cross-union ratios for the average grayscale ranges of [176,189] are 95.50% and 84.31%, respectively, and the cross-union ratios for the average grayscale ranges of [189,256] are 92.37% and 79.33%, respectively.

[0103] The average crossover ratio (CWR) is calculated as a weighted average of the CWRs for each category. Specifically, the average CWR for the average grayscale range [0, 150) is 86.99%, for [150, 163) it is 89.10%, for [163, 176) it is 89.00%, for [176, 189) it is 89.89%, and for [189, 256] it is 85.84%.

[0104] To better demonstrate the practical effects of this application, such as Figure 8 The image shown illustrates the complete color difference effect in dark images. Compared to image processing techniques, the semantic segmentation algorithm framework based on ResNet50 provides four types of labeling information in its detection results: bounding boxes, color difference patch masks, color difference patch numbers, and color difference patch percentages. The bounding boxes are rectangular and define the coverage area of ​​the color difference patches; their length and width correspond to the maximum length and width of the color difference patches. The color difference patch mask uses the outline of the color difference patch as its boundary and fills the mask area with random colors, accurately identifying the position and shape of the color difference patches. The color difference patch percentage records the proportion of pixels occupied by different color difference patches in the entire detected image, providing a basis for calculating the total color difference percentage.

[0105] Therefore, in this application, the dataset is refined and classified according to the average gray level using the average gray level range division method, so that the average gray level difference of any experimental image in each category is controlled within 13, thereby ensuring that experimental images with different average gray level ranges have high validation set confidence. By combining the semantic segmentation algorithm framework based on ResNet50 with image processing technology, the subjective factors of gray level threshold selection are eliminated. In actual use, after operational verification, on the one hand, it reduces the requirements for quality inspection personnel's experience and effectively avoids subjective problems in the quality inspection process. The detection time for a single image (corresponding to the actual fair-faced concrete inspection surface of 4.468 x 4.468 area) is only about 1 second, and the actual detection accuracy can reach 92.1%, effectively improving the acceptance accuracy and efficiency of fair-faced concrete color difference defects. On the other hand, it improves the model's generalization prediction ability for fair-faced concrete color differences with different average gray level ranges.

[0106] like Figure 9 As shown, in another embodiment of this example, a fair-faced concrete color difference detection and recognition system is further included, applied to the above-described fair-faced concrete color difference detection and recognition method, comprising:

[0107] The data acquisition module is used to acquire a preset amount of fair-faced concrete image data according to the preset fair-faced concrete image acquisition scheme.

[0108] The data preprocessing module is used to preprocess the image data of fair-faced concrete.

[0109] The color difference recognition module is used to construct a color difference recognition model for fair-faced concrete based on the ResNet50 semantic segmentation algorithm. It takes the preprocessed fair-faced concrete image data as the input parameters of the color difference recognition model, outputs the recognition result of the fair-faced concrete color difference recognition model, and performs color space conversion, histogram equalization, Kalman filter noise reduction, image opening and closing operations, and image masking operations on the recognition result to generate preliminary quantification results of fair-faced concrete color difference.

[0110] The color difference quantization module is used to build and train a color difference quantization model for fair-faced concrete based on the improved ResNet50 semantic segmentation algorithm. It takes the preliminary color difference quantization result of fair-faced concrete as the input parameter of the color difference quantization model and outputs the color difference quantization result of fair-faced concrete.

[0111] In another embodiment of this example, an electronic device is also included, such as... Figure 10 As shown, the electronic device 400 includes one or more processors 401 and memory 402.

[0112] The processor 401 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0113] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement the chaotic encryption method based on multi-physiological feature fusion of any embodiment of the present invention described above, and / or other desired functions. Various contents such as initial extrinsic parameters and thresholds may also be stored in the computer-readable storage medium.

[0114] In one example, the electronic device 400 may further include an input device 403 and an output device 404, these components being interconnected via a bus system and / or other forms of connection mechanisms (not shown). The input device 403 may include, for example, a keyboard, a mouse, etc. The output device 404 may output various information to the outside, including warning messages, braking force, etc. The output device 404 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0115] Of course, for the sake of simplicity, Figure 10 Only some of the components of the electronic device 400 relevant to the present invention are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 400 may include any other suitable components depending on the specific application.

[0116] In addition to the methods and devices described above, embodiments of the present invention may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of the chaotic encryption method based on the fusion of multiple physiological features provided in any embodiment of the present invention.

[0117] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0118] Furthermore, embodiments of the present invention may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the chaotic encryption method based on the fusion of multiple physiological features provided in any embodiment of the present invention.

[0119] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0120] The above are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for detecting and identifying color difference in fair-faced concrete, characterized in that: include: S1: According to the preset fair-faced concrete image acquisition scheme, acquire a preset amount of fair-faced concrete image data and preprocess the fair-faced concrete image data. S2: Construct a fair-faced concrete color difference recognition model based on a preset semantic segmentation algorithm. Use the preprocessed fair-faced concrete image data as the input parameters of the fair-faced concrete color difference recognition model, output the recognition result of the fair-faced concrete color difference recognition model, and perform color space conversion, histogram equalization, Kalman filter noise reduction, image opening and closing operations, and image masking operations on the recognition result to generate preliminary quantification results of fair-faced concrete color difference. S3: Construct and train a color difference quantification model for fair-faced concrete, use the preliminary color difference quantification results of fair-faced concrete as input parameters for the color difference quantification model, and output the color difference quantification results of fair-faced concrete. The color difference quantification model for fair-faced concrete is constructed based on an improved ResNet50 semantic segmentation algorithm, wherein S3 includes: S3-1: Divide the image data in the preliminary quantification result of the color difference of fair-faced concrete into different grayscale ranges according to the preset average grayscale value, and generate image data of different ranges; the preset average grayscale value includes , , , , ; S3-2: Mark the image data with different grayscale ranges generated in S3-1 using a labeling tool to generate a dataset with color difference block label information; S3-3: Construct a color difference quantization model for fair-faced concrete based on the improved ResNet50 semantic segmentation algorithm, and train the color difference quantization model for fair-faced concrete based on training metrics. Use the dataset as the input image of the trained color difference quantization model for fair-faced concrete, and output the color difference quantization result of fair-faced concrete. The color difference quantization result includes color difference block mask, bounding box, color difference block number, and color difference block percentage; The improved ResNet50 semantic segmentation algorithm is as follows: The ResNet50 semantic segmentation framework consists of the Backbone main network, the FCN Head sub-network, and the Deeplab sub-network. The color difference image input to the algorithm framework is transmitted through the Backbone main network. The data first goes through the FCNHead sub-network for convolution conv_1 operation, and then returns to the main network for further learning. The Deeplab sub-network transforms the feature information extracted by the main network into the color difference quantization result of fair-faced concrete through dilated convolution and depth-separable convolution. Specifically, the FCN Head sub-network is a fully convolutional network structure that classifies each pixel in the color difference image; it outputs a segmentation result with the same size as the input image and uses bilinear interpolation for upsampling to extract feature information. The Deeplab subnetwork includes the ASPP module and the Deeplab Head module. The ResNet50 semantic segmentation algorithm framework uses the Deeplab subnetwork as the backbone and restores the original size of the image data by using the ASPP module to restore the feature information extracted by the FCN Head subnetwork and output the detection results. Adjust the stride of the 1×1 convolution kernel mapping box of the left residual structure in the Backbone main network to 1, and the stride of the 3×3 convolution kernel mapping box to 2; Add a max pooling layer with a kernel size of 2×2 and a bounding box stride of 1 to the residual structure on the right.

2. The method for detecting and identifying color difference in fair-faced concrete according to claim 1, characterized in that: The training metrics in S3-3 include the learning rate, loss function, and accuracy evaluation metrics. The accuracy evaluation metrics include pixel accuracy, average pixel accuracy per class, intersection-over-union (IoU), and average IoU. The pixel accuracy evaluation formula is: Where PA represents pixel accuracy, TP represents the model prediction result and the actual label result are the same, and the result category is 1, TN represents the model prediction result category is 0 and the actual label result category is 1, FP represents the model prediction result category is 1 and the actual label result category is 0, and FN represents the model prediction result and the actual label result are the same, and the result category is 0. The formula for evaluating the average pixel accuracy of the category is: The crossover-union ratio evaluation formula is as follows: Where IoU represents the intersection-union ratio.

3. The method for detecting and identifying color difference in fair-faced concrete according to claim 2, characterized in that: S1 includes: S1-1: Image acquisition of fair-faced concrete using a drone equipped with a gimbal; S1-2: The collected fair-faced concrete image data is processed by image segmentation, data augmentation, and data selection to generate a fair-faced concrete image dataset.

4. The method for detecting and identifying color difference in fair-faced concrete according to claim 3, characterized in that: The semantic segmentation algorithms preset in S2 include MobileNetV3 semantic segmentation algorithm, ResNet50 semantic segmentation algorithm and VGGNet16 semantic segmentation algorithm, and S2 includes: S2-1: Based on the usage scenario, a preset semantic segmentation algorithm is invoked to construct a fair-faced concrete color difference recognition model. The algorithm hyperparameters of batch size, learning rate, and iteration number are automatically optimized and trained to identify the fair-faced concrete color difference. S2-2: Input the fair-faced concrete image dataset into the trained fair-faced concrete color difference recognition model, and perform color space conversion, histogram equalization, Kalman filter noise reduction, image opening and closing operations, and image masking operations to generate quantization results of light-colored complete color difference images, light-colored incomplete color difference images, dark-colored complete color difference images, and dark-colored incomplete color difference images.

5. A color difference detection and identification system for fair-faced concrete, applied to the color difference detection and identification method for fair-faced concrete as described in any one of claims 1-4, characterized in that: include: The data acquisition module is used to acquire a preset amount of fair-faced concrete image data according to the preset fair-faced concrete image acquisition scheme. The data preprocessing module is used to preprocess the image data of fair-faced concrete. The color difference recognition module is used to construct a color difference recognition model for fair-faced concrete based on a preset semantic segmentation algorithm. It takes the preprocessed fair-faced concrete image data as the input parameters of the color difference recognition model, outputs the recognition result of the color difference recognition model, and performs color space conversion, histogram equalization, Kalman filter noise reduction, image opening and closing operations, and image masking operations on the recognition result to generate a preliminary quantification result of the color difference of fair-faced concrete. The color difference quantization module is used to build and train the color difference quantization model of fair-faced concrete. It takes the preliminary color difference quantization result of fair-faced concrete as the input parameter of the color difference quantization model and outputs the color difference quantization result of fair-faced concrete.

6. An electronic device, characterized in that: The method includes a processor and a memory, wherein the memory stores programs or instructions, and the processor executes a method for detecting and identifying color difference in fair-faced concrete as described in any one of claims 1-4 by calling the programs or instructions stored in the memory.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instructions that cause a computer to execute a method for detecting and identifying color differences in fair-faced concrete as described in any one of claims 1-4.

Images