Real-time evaluation method and system for construction quality of 3D printing concrete

By collecting images in real time during 3D printing and using OneFormer network and texture anomaly judgment model to detect thickness and texture anomalies between concrete layers, the accuracy problem of 3D printed concrete quality detection is solved, efficient and stable quality evaluation is achieved, and the promotion of The wide application of 3D printing technology.

CN118918060APending Publication Date: 2024-11-08EAST CHINA JIAOTONG UNIVERSITY
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410907130.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The lack of high-precision quality inspection methods in the construction of 3D printed concrete has led to unstable product quality and limited large-scale production and application.

Method used

A real-time evaluation method for 3D printed concrete construction quality is adopted, and images are collected in real time through the camera, and the OneFormer network and texture anomaly judgment model are used to detect interlayer thickness and texture anomalies to achieve high-precision quality evaluation.

Benefits of technology

It realizes high-precision and real-time quality inspection of 3D printed concrete, reduces manual monitoring costs, improves product stability, and promotes the large-scale application of 3D printing technology in the construction field.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918060A_ABST
    Figure CN118918060A_ABST
Patent Text Reader

Abstract

The invention relates to a 3D printing concrete construction quality real-time evaluation method and system, and the method comprises the steps: constructing an evaluation data set: fixing a camera on a printing nozzle, carrying out the side shooting of a printing wall body in the printing height direction, selecting a subset image which does not contain a printing piece background, is clear and visible in printing layer, and is clear in texture details, and carrying out the calculation of the evaluation data set; taking the subset image and a non-3D-printed concrete image as an original data set; performing data enhancement processing on the original data set to obtain an enhanced data set R *; carrying out two kinds of labeling processing on the enhanced data set R *, wherein the first kind of labeling processing corresponds to a binary segmentation mask of an interlayer line; the second annotation corresponds to a texture classification of a window extracted from the enhanced data set image R *; constructing a layer thickness segmentation model; and a texture anomaly judgment model is constructed, the texture anomaly judgment model is used for obtaining classification whether the texture is abnormal or not, and the printing quality is evaluated through the interlayer thickness and the texture anomaly condition. And real-time quality detection in the 3D printing concrete construction process is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of concrete construction quality assessment, and in particular to a real-time assessment method for 3D printing concrete construction quality. Background Art

[0002] Concrete is the main material of many building structures, and its quality directly affects the safety of the structure. Low-quality concrete may lead to insufficient structural strength, thereby increasing the risk of collapse or damage. High-quality concrete has better durability and can resist weathering, chemical attack and water penetration, thus extending the service life of the building.

[0003] While high-quality concrete may cost more in the short term, it can reduce maintenance and repair costs and provide better economics in the long term. The quality of concrete affects its functionality, including load-bearing capacity, crack resistance and water resistance. This is essential to ensure that a building or structure can meet its intended purpose. The production and use of high-quality concrete can reduce negative impacts on the environment, such as reducing resource waste and reducing harmful emissions.

[0004] 3D printing technology is a new type of robotic construction process that adapts to digitalization and intelligence. Its advantages lie in its high degree of automation, fast construction speed, high material utilization, ability to realize special-shaped construction, and saving template costs. At present, the application of 3D printing technology in the construction field is mainly aimed at concrete and other cement-based materials, which is also very consistent with the development concept of the intelligent construction field. However, 3D printing technology relies heavily on manual monitoring, which requires a lot of manpower and economic costs. In addition, manual monitoring cannot reach places above the height of human vision, and the quality of 3D printed concrete cannot be tested. The unstable quality of manufactured products poses structural safety risks, which limits the development and application of 3D printing in large-scale production.

[0005] Image segmentation technology is widely used in computer vision, natural language processing and other fields. Applying it to the detection of concrete quality is an ideal intelligent detection method that can achieve quality detection during the construction process. However, although the traditional quality detection network can accurately locate the uploaded image, the accuracy of the positioning result is low, and the guiding significance for the actual construction process in the project is limited. Therefore, how to use high-precision semantic segmentation algorithms to monitor and measure the quality of 3D printed concrete is an urgent need in the field. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the above technologies and provide a stable, high-precision, intelligent 3D printed concrete construction quality real-time evaluation method and system. When 3D printed concrete is realized, the quality of concrete in the printing process can be accurately and efficiently detected, including the monitoring of interlayer thickness and concrete, providing an effective means for ensuring printing quality.

[0007] In order to solve the above technical problems, the technical solution of the present invention is:

[0008] In a first aspect, the present invention provides a method for real-time evaluation of 3D printed concrete construction quality, the method comprising the following steps:

[0009] Step 1: Build an evaluation dataset:

[0010] The camera is fixed on the printing nozzle, and the printed wall is photographed from the side along the printing height direction. At this time, the camera can capture images containing the topmost layer of printed slurry strips. Each frame of the image contains 7-9 layers of printed slurry strips. The middle window area of ​​the topmost layer in each frame of the image is considered to be the window position facing the center of the nozzle. During the printing process, the camera is used to collect images in real time. During the image collection process, it is ensured that the line connecting the camera to the extrusion point of the extrusion 3D printer is perpendicular to the printing wall. From the collected images, a subset image is selected that does not contain a printed background, has a clearly visible printed layer, and has clear texture details. The subset image includes images that meet the requirements of no printed background, a clearly visible printed layer, and clear texture details obtained under different printing parameters;

[0011] The subset images and non-3D printed concrete images are used as the original data set; the original data set is enhanced using the MSR algorithm to obtain the enhanced data set R*;

[0012] Two types of annotation processing are performed on the enhanced dataset R*: the first type corresponds to the binary segmentation mask of the interlayer line. The area to be identified in the dataset R* is annotated at the pixel level using the Photoshop tool, and a corresponding semantic label is assigned to each pixel to determine whether it is an interlayer line. This forms a dataset with annotation information. After the annotation is completed, a binary image is generated. The black pixel value is 0, representing the background, and the white pixel value is 255. The width of the white pixels in these binary images is 18-22 pixels, representing the target interlayer line.

[0013] The second annotation corresponds to the texture classification of the window extracted from the enhanced dataset image R*: the window width is set to 200 pixels, and the height of the window is based on the thickness of each printed layer, but does not include adjacent layers; the image in the enhanced dataset image R* is segmented according to the window size, which is equivalent to taking a small window from each layer of concrete in the image and judging whether the small window is abnormal; the window facing the center of the printing nozzle is regarded as an abnormal window, and the window with abnormalities in all windows in each image is found. When the abnormal area exceeds 30% of the surface area of ​​the entire image, the image is provided with an abnormal label; otherwise, it is a good label;

[0014] Obtain the corresponding binary images and texture classification anomaly results of the images in the enhanced dataset R*, and use them as the evaluation dataset;

[0015] Step 2: Constructing the layer thickness segmentation model

[0016] The layer thickness segmentation model includes a OneFormer network and an interlayer thickness calculation module, which is used to detect whether the interlayer thickness of 3D printed concrete is qualified;

[0017] The evaluation data set is used to train the OneFormer network, which outputs a binary image that predicts the position information of the concrete interlayer lines, as well as the data required by the interlayer thickness calculation module (including the coordinates of all pixels in the binary image); the interlayer lines are determined from the binary image, which can segment the independent printed layers in the image, and the interlayer thickness calculation is performed separately for each layer;

[0018] The interlayer thickness calculation module is used to calculate the distance between two adjacent interlayer lines;

[0019] Step 3: Build a texture anomaly judgment model

[0020] The gray-level co-occurrence matrix of the images in the evaluation data set is calculated, and the gray-level co-occurrence matrix is ​​used to generate a heat map. The heat map includes features such as energy, contrast, and mean. Each heat map corresponds to a label of abnormality. The heat map is used as the input of the ANN network, and the binary classification result of whether the image is abnormal is output. The heat map data set obtained by using the images in the evaluation data set is used to train the ANN network to obtain a texture abnormality judgment model.

[0021] Step 4: Actual monitoring

[0022] During the 3D printed concrete construction process, images without printed background, with clearly visible printed layers and clear texture details are acquired in real time, and corresponding thermal maps are obtained. The layer thickness segmentation model is used to obtain the interlayer thickness of the image, and the texture anomaly judgment model is used to obtain the classification of whether the texture is abnormal. The interlayer thickness and texture anomalies are used to evaluate the printing quality.

[0023] Furthermore, the OneFormer network includes five backbone modules, namely, a backbone network Backbone backbone module, a Transformer decoder backbone module, a query mapper backbone module, a text mapper backbone module, and a contrast loss backbone module;

[0024] The backbone module of the backbone network adopts the enhanced efficiency network PN-EfficientNet. The specific process of the enhanced efficiency network PN-EfficientNet is:

[0025] First, the input image is output through the first convolution layer, which is a feature map with a dimension of (112×112×32), and is padded to "same", that is, the width and height of the output are reduced by half; then it enters the proxy normalization layer, and then enters the mobile flip bottleneck convolution MBConv1 after being processed by the Swish activation function. The expansion ratio of the mobile flip bottleneck convolution MBConv1 is 1, the depth convolution kernel size is 3x3, the kernel step size is 1x1, and it includes compression and excitation operations, without connection deactivation and connection skipping;

[0026] Next, it enters six serially connected moving flip bottleneck convolution MBConv6 layers, each layer has multiple MBConv6s, and the expansion factor of MBConv6 is 6;

[0027] Finally, it enters the second convolutional layer and is filled to "same", that is, the width and height of the output remain unchanged. It is then processed by the proxy normalization layer, Swish activation function, global average pooling layer, random inactivation dropout, fully connected layer, and Softmax classification layer to output a tensor containing a feature vector.

[0028] Furthermore, the calculation process of the proxy normalization layer is:

[0029] First, for each channel c, the proxy variable is calculated

[0030] in, and is the mean and variance of the proxy variable, and N indicates normal distribution;

[0031] Then, we choose layer normalization as the batch-independent normalization method, and for each batch element b and channel c, we calculate:

[0032]

[0033] Among them, μ b and σ 2 b is the mean and standard deviation of the input X of batch element b on channel c; X b...is the unnormalized pre-activation of the b-th element in the batch; Y b... is the normalized pre-activation of the bth element in the batch after batch-independent normalization; ε is the normalized numerical stability constant;

[0034] Next, for each channel c, the proxy normalized activation is calculated:

[0035] Z...c

[0036]

[0037] Where φ is the activation function, γc and βc are the normalized scaling and offset parameters, E[] and Var[] are the expected value and variance, respectively, and ~ represents the estimate or approximation of the corresponding variable or value; Z …c is the activation value after being processed by the activation function φ; Y …c represents the activation value on a specific channel c; E[] and Var[] are the expected value and variance respectively;

[0038] Finally, proxy normalization is performed to ensure that the pre-activation Y remains close to the normalized state, thereby maintaining the expressiveness and stability of the network.

[0039] Furthermore, the contrast loss backbone module adopts the MoCo contrast learning mechanism, named MoCo loss calculation, and the specific process is as follows:

[0040] First, define a query encoder to convert the input image into a feature representation; at the same time, initialize a momentum encoder whose parameters are copies of the query encoder parameters;

[0041] Then, the input image is data augmented to generate a pair of positive samples. These augmented images will be input into the query encoder and momentum encoder as query image query and key image Key respectively.

[0042] Subsequently, the query image is encoded using the query encoder to obtain the query feature representation; at the same time, the key image is encoded using the momentum encoder to obtain the key feature representation; the feature representation obtained after the data sample is encoded by the query encoder and the momentum encoder is used as the encoding key. In MoCo, the encoding key is part of the dynamic dictionary, and a queue is maintained to store and update these encoding keys for comparative learning and optimization processes;

[0043] At the same time, a queue is maintained as a dynamic dictionary, which stores the encoding keys of several previous small batches (5-10); when the encoding key of the current small batch is added to the end of the queue (enqueue), the encoding key of the oldest small batch is removed from the front of the queue (dequeue). By continuously updating the encoding keys in the queue, it is ensured that the dynamic dictionary contains the latest sample feature representation; as the training progresses, the encoding keys in the queue will be continuously updated, so that the dynamic dictionary can cover a wider sample space, and the size of the dynamic dictionary can be controlled by removing the encoding keys of the oldest small batch;

[0044] The query image is encoded by the query encoder to obtain the query feature, and the feature obtained by encoding the corresponding positive sample image by the momentum encoder is represented as the positive key feature;

[0045] In MoCo, positive sample images refer to images paired with query images, also known as positive keys, and negative sample images refer to other images that are not paired with query images, also known as negative keys; positive sample images are encoded by momentum encoders to calculate the similarity between query features and positive key features. By maximizing the similarity between query and positive key, the model learns more discriminative feature representations; negative sample images are encoded by other encoding keys in the dynamic dictionary to calculate the similarity between query features and negative key features. By minimizing the similarity between query and negative key, the model learns better visual representations and improves the discriminability of features;

[0046] Secondly, using the contrastive loss function, the similarity between the query features and the positive key features is calculated, and the dissimilarity between the query features and all the negative key features in the queue is calculated; the contrastive loss function aims to maximize the similarity between the query and the positive key, while minimizing the similarity between the query and the negative key;

[0047] Finally, repeat the above steps until the maximum number of iterations is reached.

[0048] Furthermore, in step 1, the anomaly is determined by comparing the texture features in this window with the texture feature windows at the corresponding positions of the corresponding images of the same material in the public data set. If there is a significant difference, it is an abnormal window.

[0049] Furthermore, the specific process of the interlayer thickness calculation module is as follows:

[0050] First, the fast Euclidean distance transform (EDT) is applied to the predicted binary image containing the inter-layer line. This transform calculates a distance value D(i) for each pixel point i in the image, which represents the distance from the point to the nearest inter-layer line.

[0051] D(i) uses the formula get,

[0052] Where: D(a,b): represents the measure of the shortest distance from the pixel point (a,b), that is, the distance between the pixel point in the layer and the pixel point (a′,b′) on the layer line; B represents the set of all possible positions related to the pixel (a,b) in the foreground area of ​​the binary image;

[0053] Next, find the local maximum points of the distance transform result D(i), which roughly correspond to the center position of the printed layer; convolve the distance transform result D with vertical kernels i1=(0,-1,1) and i2=(1,-1,0) to accurately locate the local maximum, that is, the layer center point i; for the layer center point i located between two interlayer lines, the local layer thickness T(i) is defined as twice the distance between the corresponding interlayer lines, and the formula is:

[0054]

[0055] Among them, S is the set of local maximum points that can be identified by applying the vertical kernel to the distance transformation result D for convolution;

[0056] M: is the area of ​​the layer currently being analyzed in the image. This area limits the range of pixels for calculating the layer thickness, ensuring that only the thickness of the current layer is considered, while ignoring other irrelevant parts of the image, which is called the region of interest;

[0057] The condition i∈S∩M means that the layer thickness of the pixel point i will be calculated only when the pixel point i is located in both the local maximum point set S and the region of interest M. If the pixel point i is not in the intersection of S and M, then T(i) will be defined as undefined;

[0058] Through the above steps, the local layer thickness T(i) at the center point i of each layer is calculated, and the local interlayer thickness is used as the predicted interlayer thickness to be compared with the layer thickness threshold to evaluate the consistency of the printed layer and detect possible anomalies; the layer thickness threshold is 7mm-13mm.

[0059] In a second aspect, the present invention provides a real-time evaluation system for 3D printing concrete construction quality, the system comprising:

[0060] An image acquisition module is used to obtain concrete images of the printed area during the construction process of 3D printing concrete construction;

[0061] The image processing module is used to enhance the concrete image of the image acquisition module, obtain a binary image and a thermal map, and perform pixel-level interlayer line annotation and texture anomaly annotation;

[0062] The layer thickness segmentation model, including the OneFormer network and the interlayer thickness calculation module, is used to detect whether the interlayer thickness of 3D printed concrete is qualified and output the abnormal layer thickness results;

[0063] The texture anomaly judgment model is used to judge whether the texture of the thermal map is abnormal. If the texture feature is identified as abnormal, an abnormal mark is made;

[0064] In the early warning module, if at least one of the output results of the layer thickness segmentation model and the texture anomaly judgment model is abnormal, the concrete quality is considered unqualified, and the defect information is promptly fed back to the background system, which prompts relevant personnel to check the material status.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] The present invention performs real-time detection on the quality of the 3D printed concrete construction process and constructs an evaluation data set for this object. Based on this, a layer thickness segmentation model and a texture anomaly judgment model are used to detect the quality and feedback abnormal situations. The printing quality can be evaluated efficiently and accurately by using the interlayer thickness and texture anomalies.

[0067] The preferred interlayer thickness in the present invention is the OneFormer network, which uses PN-EfficientNet (enhanced efficiency network) and MoCo (Memory-augmented Contrastive Learning) momentum contrast function, which greatly improves the accuracy of segmentation and prediction, enables the system to achieve real-time and high-precision measurement, helps to reduce overfitting and improve the generalization ability of the model, and can improve training efficiency and convergence speed. It has been trained and tested on multiple data sets and tasks, showing good generalization ability.

[0068] The present invention can provide the possibility for realizing larger-scale architectural-level 3D printing, further promoting the development of this field, which has great significance and broad application prospects, and also provides strong support and guarantee for the development of concrete 3D printing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a schematic diagram of the structure of the OneFormer network in the present invention;

[0070] Figure 2 It is a schematic diagram of the structure of the enhanced efficiency network PN-EfficientNet in the present invention;

[0071] Figure 3 It is a calculation flow chart of the proxy normalization layer PN in the present invention;

[0072] Figure 4It is a schematic diagram of the process of calculating MoCo loss in the present invention;

[0073] Figure 5 It is a structural diagram of the ANN network in the present invention; DETAILED DESCRIPTION

[0074] In order to more clearly describe the technical problems, technical solutions and advantages of the present invention, the following will be described in detail with reference to the drawings and embodiments. It should be noted that these embodiments are only used to illustrate the principles and application scope of the present invention and should not be regarded as limiting the present invention.

[0075] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0076] Example 1

[0077] The present invention provides a real-time evaluation method for 3D printing concrete construction quality, the method comprising the following steps:

[0078] Step 1: Build an evaluation dataset:

[0079] The camera is fixed on the printing nozzle, and the printed wall is photographed from the side along the printing height direction. At this time, the camera can capture images containing the topmost layer of printed pulp strips. Each frame of the image contains 7-9 layers of printed pulp strips. The middle window area of ​​the topmost layer in each frame of the image is considered to be the window position facing the center of the nozzle. During the printing process, the camera is used to collect images in real time. During the image collection process, it is ensured that the line connecting the camera to the extrusion point of the extrusion 3D printer is perpendicular to the printing wall. From the collected images, a subset image is selected that does not contain a printed background, has a clearly visible printed layer, and has clear texture details. The subset image includes images that satisfy the conditions of no printed background, clear visible printed layers, and clear texture details obtained under different printing parameters (including printing speed and print head temperature);

[0080] The subset images and non-3D printed concrete images are used as the original data set, and each image in the original data set is a 712x712 pixel RGB image; the original data set is enhanced by using the MSR algorithm to obtain the enhanced data set R*;

[0081] Two types of annotation processing are performed on the enhanced dataset R*: the first type corresponds to the binary segmentation mask of the interlayer line. The area to be identified in the dataset R* is annotated at the pixel level using the Photoshop tool, and a corresponding semantic label is assigned to each pixel to determine whether it is an interlayer line. This forms a dataset with annotation information. After the annotation is completed, a binary image is generated. The black pixel value is 0, representing the background, and the white pixel value is 255. The width of the white pixels in these binary images is 18-22 pixels, representing the target interlayer line.

[0082] The second annotation corresponds to the texture classification of the window extracted from the enhanced dataset image R*: the window width is set to 200 pixels, and the height of the window is based on the thickness of each printed layer, but does not include adjacent layers; the image in the enhanced dataset image R* is segmented according to the window size, which is equivalent to taking a small window from each layer of concrete in the image and judging whether the small window is abnormal; the window facing the center of the printing nozzle is regarded as an abnormal window, and the window with abnormalities in all windows in each image is found. When the abnormal area exceeds 30% of the surface area of ​​the entire image, the image is provided with an abnormal label; otherwise, it is a good label;

[0083] The anomaly is determined by comparing the texture features in this window with the texture feature windows at the corresponding position of the same material image in the public data set. If there is a significant difference, it is an abnormal window.

[0084] Obtain the corresponding binary images and texture classification anomaly results of the images in the enhanced dataset R*, and use them as the evaluation dataset;

[0085] In the present invention, the use of a camera fixed to the printing nozzle for image acquisition refers to side shooting, which can capture an image containing the topmost layer of printed pulp strips. In this embodiment, the top middle window layer is considered to be directly opposite the center of the nozzle. Adjust the lens distance, which is 30cm from the lens to the printing surface orthogonal distance, to ensure that each frame of the image contains approximately 7-9 layers, and the expected thickness of each layer is 7mm-13mm. The camera resolution is 1280×960 pixels. During image acquisition, ensure that the line connecting the camera to the extrusion point of the extrusion 3D printer is perpendicular to the printing wall to avoid blur. From the acquired images, select a subset of images that do not contain the background of the print to create a training set and a test set. The selected subset images must also meet the requirements of high clarity of the printed layer, clear texture details, and diversified printing parameters.

[0086] High print layer clarity: The print layer edges in the image should be clearly discernible, and the image should be able to clearly show the interlayer lines separating adjacent print layers to facilitate the segmentation of interlayer lines and layers.

[0087] Clear texture details: The image should be able to show the texture details of the concrete surface, and the image should have sufficient clarity and resolution and not be blurred, which is crucial for subsequent texture analysis and classification.

[0088] Images that meet the above selection criteria are obtained under different printing parameters (including printing speed and print head temperature, etc.) and conditions (different material formulations) to increase the diversity of the dataset and improve the generalization ability of the model.

[0089] The original data set of this experiment consists of 2,000 images of 3D printed concrete during the printing process and regular concrete images. Each image is a 712x712 pixel RGB image, which helps the model to more accurately identify the quality characteristics of 3D printed concrete images.

[0090] The original data set is subjected to the following operations using the MSR algorithm: ① log transform the image, ② Gaussian blur the log image (i.e., obtain the low-frequency component), and ③ use the original image and the blurred log image to make a difference to obtain the image R. The original image R, G, and B dimensions are subjected to operations ①, ②, and ③ respectively. The difference is that in ②, the Gaussian blur selects the sigmoid parameter to be set to 15, 80, and 200 respectively. Finally, the results of operations ①, ②, and ③ in the three dimensions are weighted averaged, and then the three dimensions are combined and the data is scaled to 0-255 to obtain the enhanced data set R*.

[0091] When performing texture classification and annotation, before the window is extracted, the original image is flattened to reduce the impact of shadows and flashes, while retaining the texture information. The image is divided into good and abnormal images according to the quality of the image texture features. The original image is 712*712, and the fixed width of the window is 200 pixels (about 5mm). The height of the window will be adjusted as needed to cover the entire thickness of each layer, but not the adjacent layers, which is equivalent to taking a small window for each layer of concrete in the original image, similar to a nine-square grid. Any category other than "good" is considered "abnormal".

[0092] When constructing the data set, the material formula can be set according to the material formula in the public data set. The anomaly is that the annotator compares the texture features in this window with the corresponding texture feature window of the same material in the public data set to see if there is a significant difference. If there is a significant difference, it is considered an abnormal window. In this embodiment, the window size is consistent with the window width and length in the public data set, and the window data at the same position is used for abnormal comparison. The window in the middle of the top layer of concrete needs to be ignored and not calculated.

[0093] Step 2: Constructing the layer thickness segmentation model

[0094] The layer thickness segmentation model includes a OneFormer network and an interlayer thickness calculation module, which is used to detect whether the interlayer thickness of 3D printed concrete is qualified;

[0095] The evaluation data set is used to train the OneFormer network, which outputs a binary image that predicts the position information of the concrete interlayer lines, as well as the data required by the interlayer thickness calculation module (including the coordinates of all pixels in the binary image); the interlayer lines are determined from the binary image, which can segment the independent printed layers in the image, and the interlayer thickness calculation is performed separately for each layer;

[0096] The interlayer thickness calculation module is used to calculate the distance between two adjacent interlayer lines;

[0097] Step 3: Build a texture anomaly judgment model

[0098] The gray-level co-occurrence matrix of the images in the evaluation data set is calculated, and the gray-level co-occurrence matrix is ​​used to generate a heat map. The heat map includes features such as energy, contrast, and mean. Each heat map corresponds to a label of abnormality. The heat map is used as the input of the ANN network, and the binary classification result of whether the image is abnormal is output. The heat map data set obtained by using the images in the evaluation data set is used to train the ANN network to obtain a texture abnormality judgment model.

[0099] Step 4: Actual monitoring

[0100] During the 3D printed concrete construction process, images without printed background, with clearly visible printed layers and clear texture details are acquired in real time, and corresponding thermal maps are obtained. The layer thickness segmentation model is used to obtain the interlayer thickness of the image, and the texture anomaly judgment model is used to obtain the classification of whether the texture is abnormal. The interlayer thickness and texture anomalies are used to evaluate the printing quality.

[0101] The overall architecture of the OneFormer network in this embodiment is as follows Figure 1 As shown in the figure, it is used to detect whether the interlayer thickness of 3D printed concrete is qualified, including 5 backbone modules, namely the backbone network Backbone backbone module, the Transformer decoder backbone module, the query mapper backbone module, the text mapper backbone module, and the contrast loss backbone module; the specific process is as follows:

[0102] First, the input image will pass through a backbone network Backbone module, which will perform operations such as convolution and pooling on the input image to extract the multi-scale feature representation of the image. The features extracted by the backbone network Backbone module will undergo corresponding operations to gradually compress their size from 1 / 2 to 1 / 32 of the original size. This involves using pooling operations (such as average pooling or maximum pooling) or convolution operations to reduce the spatial dimension of the feature map. The 1 / 32 scale feature representation and the 1 / 2 scale feature representation will be fed into the first Transformer decoder, and the 1 / 2 scale feature will be fed into the second Transformer decoder.

[0103] At the same time, the text mapper module encodes the input text information (text information is the segmentation layer spacing, the coordinates of the identified pixel points) through the text encoder (Text Encoder) and converts the text into a vector representation. After text encoding, the Text Mapper maps the encoded text information to the text query (TextQueries) representation required by the model. Text Mapper also considers the contextual information of the text, and associates the text information with the image content by learning the learnable text context embedding (Learnable Text Context Embeddings) to improve the model's understanding of the relationship between image and text. Then, the processed text query is combined with the image features to form a task-conditioned query, which is used to guide the model's learning and reasoning under different tasks (panoramic segmentation, instance segmentation, semantic segmentation). This embodiment uses instance segmentation and semantic segmentation, and the task-conditioned query will be input into the second Transformer decoder to generate the query vector of the model.

[0104] The newly generated query vector is then input into the Query Mapper backbone module, which searches for additional task query vectors, stacks text information to initialize the task query vector, and outputs the required feature vector through the calculation of the contrast loss backbone module.

[0105] The input feature vector is then tokenized to convert it into a token representation acceptable to the first Transformer decoder. This typically involves splitting the feature vector into a series of tokens and mapping them to the Transformer's token embedding space. The first Transformer decoder then uses a multi-head attention mechanism, and after the attention calculation, the feature vector is processed through a position-wise feedforward network (Position-wise FFN). During the calculation of each sub-layer, layer normalization is applied to normalize the feature vector, and residual connections are used to retain the original feature information.

[0106] Finally, the feature vector processed by multiple Transformer layers will be sent to the output layer to output the mask and category for the final task-related prediction. In addition, during the training phase, the model calculates the loss function based on the generated query vector and the actual label information (instance segmentation, semantic segmentation labels).

[0107] Use MoCo (Memory-augmented Contrastive Learning) to calculate the loss. Use the back propagation algorithm to update the network parameters so that the model can better learn discriminative feature representations. Repeat the above steps and continuously iterate the training model until the model converges and learns feature representations that meet the task requirements.

[0108] Furthermore, the backbone network Backbone module adopts the enhanced efficiency network PN-EfficientNet to extract the feature representation of the input image and capture the low-level and high-level features of the image.

[0109] The PN-EfficientNet (enhanced efficiency network) structure is as follows Figure 2 The specific process is as follows:

[0110] In the first stage, the following operations are performed on the input image in order to obtain the result of the first stage: first, convolution (the convolution kernel is 32 kernels 3×3×3, the step size is 2×2, and the padding is "same", that is, the width and height of the output are reduced by half). The output of the convolution operation is a feature map with a dimension of (112×112×32). Because this layer does not contain bias terms, the total number of parameters that need to be trained and learned in this layer is 864 (32x3x3x3). Then enter the proxy normalization layer, the input of this layer is a feature map of (112×112×32), so the total number of parameters in this layer is 128 (32x4), of which 64 parameters need to be trained and learned, and then the Swish activation function is used.

[0111] Then comes the second stage, which performs mobile flip bottleneck convolution MBConv1 on the 112x112x32 feature map output by the previous stage. The expansion ratio of MBConv1 is 1, the depth convolution kernel size is 3x3, the kernel step size is 1x1, it includes compression and excitation operations, no connection deactivation and connection skipping, and outputs the result of the second stage.

[0112] Next, in the third stage, a moving flip bottleneck convolution MBConv6 layer is performed on the 112x112x32 feature map outputted in the previous stage. There are two MBConv6s in the MBConv6 layer in the third stage. The MBConv6 expansion ratio is 6, the depth convolution kernel size is 3x3, the kernel step size is 1x1, and it includes compression and excitation operations, without connection deactivation and connection skipping, and outputs the result of the third stage.

[0113] Then repeat the third stage operation 5 times, where the depth convolution kernel of MBConv6 in the fourth, sixth and seventh stages is 5x5, the number of MBConv6 is 2, 3, 3 respectively, and the depth convolution kernel of MBConv6 in the fifth and eighth stages is 3x3, the number of MBConv6 is 3 and 4 respectively, until a 7x7x320 feature map is output. The total number of parameters is 722480, and the number of parameters that need to be trained and learned is 717232.

[0114] Finally, the following operations are performed on the input 7x7x320 image in order: convolution (the convolution kernel is 1280 kernel 1×1×320, the stride is 1×1, and the padding is "same", that is, the width and height of the output remain unchanged). The output of the convolution operation is a feature map with a dimension of (7×7×1280). Because this layer does not contain a bias term, the total number of parameters that need to be trained and learned for this layer is 409600 (1280x1x1x320). Proxy Normalization layer (Proxy Normalization) This layer inputs a feature map of (7×7×1280), so the total number of parameters in this layer is 5120 (1280x4), of which 2560 parameters need to be trained and learned. Swish activation function, global average pooling layer (global average pooling), this layer performs global average pooling in the channel dimension direction, and the output is a feature map of (1x1x1280). The random dropout fully connected layer has 1000 neurons. Because this layer contains bias terms, the total number of parameters is 1281000 (1000x1280+1000), Softmax activation function, and output classification results.

[0115] The present invention introduces a new batch-independent normalization method, ProxyNormalization, which retains expressiveness while offsetting two major sources of non-normalization: affine transformation and activation function, following group normalization or layer normalization. Specifically, the non-normalization is offset by assimilating the output of the group norm or layer norm into a Gaussian "proxy" variable and applying the same affine transformation and the same activation function to this proxy variable. The statistics of the non-normalized proxy variable are then used to correct the expected distribution shift in the true activation. Proxy normalization allows the group size to be maximized (i.e., using layer normalization), and even if there is no channel non-normalization problem, proxy normalization still maintains expressiveness.

[0116] The calculation process of the proxy normalization layer is as follows: Figure 3 As shown and as follows: First, for each channel c, calculate the proxy variable: in, and is the mean and variance of the proxy variable, and Y is expressed by weight decay to approximate the normalized prior. Then, if layer normalization is selected as the batch-independent normalization method, for each batch element b and channel c, calculate: As shown in Formula 1:

[0117] Among them, μ b and σ 2 b is the mean and standard deviation of the input X of batch element b on channel c, and ε is a normalized numerical stability constant;

[0118] Next, for each channel c, the proxy normalized activation is calculated:

[0119]

[0120] where φ is the activation function, γc and βc are the scaling and bias parameters of the normalization, E[] and Var[] are the expected value and variance, respectively, and ε is a numerical stability constant for the surrogate normalization.

[0121] Finally, proxy normalization is performed to ensure that the pre-activation Y remains close to the normalized state, thereby maintaining the expressiveness and stability of the network.

[0122] The PN-EfficientNet (Enhanced Efficiency Network) used in the OneFormer model of the present invention optimizes performance and efficiency through a method called "compound scaling". This method takes into account the depth, width, resolution and training strategy of the network at the same time. Through this scaling strategy, PN-EfficientNet (Enhanced Efficiency Network) achieves performance that is superior to previous models while maintaining the model size and computational efficiency. PN-EfficientNet (Enhanced Efficiency Network) is designed with special consideration of computational efficiency, so it can provide better performance at the same or lower computational cost. The architecture is relatively simple, making the model easier to train and debug, which helps reduce the risk of model overfitting, especially when the amount of data is small. Experiments have shown that PN-EfficientNet (Enhanced Efficiency Network) has been trained and tested on multiple data sets and tasks, showing good generalization capabilities.

[0123] The Transformer decoder module is used to generate the output segmentation results. It consists of multiple Transformer decoder layers, each of which contains a multi-head self-attention mechanism, a feedforward neural network, a residual connection, and a normalization layer. The final segmentation prediction is generated through the Transformer decoder.

[0124] The Query Mapper module initializes query vectors based on the specific requirements of the task and the input data. These query vectors can be considered as important information that the model needs to pay attention to when performing a specific task. By initializing the query vectors, the model can better focus on the features related to the task. Query Mapper is also responsible for calculating the contrast loss of the query text. Contrastive loss is a loss function used to measure the differences between model learning tasks. Through contrastive loss, the model is guided to learn the differences between tasks, thereby improving the generalization ability and accuracy of the model. By introducing query text and contrastive loss, the Query Mapper module helps to reduce the model's category misjudgment during the prediction process. The model can better understand the differences between different tasks, thereby improving the ability to distinguish different categories and reducing misclassification.

[0125] The Text Mapper module encodes the input text information through the Text Encoder and converts the text into a vector representation. This process usually includes operations such as text segmentation and conversion to word embedding so that the model can understand and process the text information. After text encoding, the Text Mapper maps the encoded text information to the text query representation required by the model. These text queries are usually used to guide the model to identify and segment the target object in the image segmentation task. The Text Mapper also considers the contextual information of the text and associates the text information with the image content by learning the learnable text context embeddings to improve the model's understanding of the relationship between the image and the text. Then in the inference phase, the Text Mapper can help reduce the size and computation of the model by generating text query representations in the training phase, and the Text Mapper module is not needed in the inference phase, thereby improving the efficiency and inference speed of the model.

[0126] The Contrastive Loss module is used to calculate the contrast loss of the query text. Using the MoCo (Memory-augmented Contrastive Learning) momentum contrast function for loss calculation can help the model learn more discriminative feature representations.

[0127] The process of the MoCo function is as follows Figure 4 As shown:

[0128] First, define a query encoder to convert the input image into a feature representation. At the same time, initialize a momentum encoder whose parameters are a copy of the query encoder parameters. Then, perform data augmentation on the input image to generate a pair of positive samples (e.g., by different cropping methods). These augmented images will be input into the query encoder and momentum encoder as query and key, respectively.

[0129] Subsequently, the query image is encoded using the query encoder to obtain the query feature representation. At the same time, the key image is encoded using the momentum encoder to obtain the key feature representation. The feature representation obtained after the data sample is encoded by the query encoder and the momentum encoder is used as the encoding key. In MoCo, the encoding key is part of the dynamic dictionary, and a queue is maintained to store and update these encoding keys for comparative learning and optimization processes;

[0130] At the same time, a queue is maintained as a dynamic dictionary, which stores the encoding keys of several previous small batches. The encoding key of the current small batch is added to the end of the queue (enqueue), and the encoding key of the oldest small batch is removed from the front of the queue (dequeue).

[0131] In MoCo, positive sample images refer to images paired with the query image, also called positive keys, and negative sample images refer to other images that are not paired with the query image, also called negative keys.

[0132] Positive sample images are encoded by the momentum encoder to calculate the similarity between query features and positive key features. By maximizing the similarity between the query and the positive key, the model learns a more discriminative feature representation. Negative sample images are encoded by other encoding keys in the dynamic dictionary to calculate the similarity between query features and negative key features. By minimizing the similarity between the query and the negative key, the model learns a better visual representation and improves the discriminability of features.

[0133] Second, use a contrastive loss function, such as InfoNCE, to calculate the similarity between the query features and the positive key features, and also calculate the dissimilarity between the query features and all negative key features in the queue; the contrastive loss function aims to maximize the similarity between the query and the positive keys, while minimizing the similarity between the query and the negative keys;

[0134] Finally, repeat the above steps until the maximum number of iterations is reached.

[0135] MoCo is a self-supervised learning method that provides more negative sample comparisons by building a dynamic negative sample set (memory bank), thereby helping the model learn more discriminative feature representations. This can help the OneFormer model better capture the relationship between image features, improve the effect of feature learning, improve the generalization ability and performance of the model, have higher robustness, help reduce overfitting and improve the generalization ability of the model, and can improve training efficiency and convergence speed. By more effectively using negative sample comparisons to guide model learning, the training process of the OneFormer model can be accelerated and training efficiency can be improved.

[0136] The OneFormer model will eventually output the image prediction of the concrete interlayer line position information, as well as the data required by the interlayer thickness calculation module (including the coordinates of the pixel points). The independent printing layers in the image are segmented according to the interlayer lines obtained by segmentation.

[0137] The specific process of the interlayer thickness calculation module is as follows:

[0138] First, the fast Euclidean distance transform (EDT) is applied to the predicted binary image containing the inter-layer line. This transform calculates a distance value D(i) for each pixel point i in the image, which represents the distance from the point to the nearest inter-layer line.

[0139] D(i) uses the formula get,

[0140] Where: D(a,b): represents the measure of the shortest distance from the pixel point (a,b), that is, the distance between the pixel point in the layer and the pixel point (a′,b′) on the layer line; B represents the set of all possible positions related to the pixel (a,b) in the foreground area of ​​the binary image;

[0141] Next, find the local maximum points of the distance transform result D(i), which roughly correspond to the center position of the printed layer; convolve the distance transform result D with vertical kernels i1=(0,-1,1) and i2=(1,-1,0) to accurately locate the local maximum, that is, the layer center point i; for the layer center point i located between two interlayer lines, the local layer thickness T(i) is defined as twice the distance between the corresponding interlayer lines, and the formula is:

[0142]

[0143] Among them, S is the set of local maximum points that can be identified by applying the vertical kernel to the distance transformation result D for convolution;

[0144] M: is the area of ​​the layer currently being analyzed in the image. This area limits the range of pixels for calculating the layer thickness, ensuring that only the thickness of the current layer is considered, while ignoring other irrelevant parts of the image, which is called the region of interest;

[0145] The condition i∈S∩M means that the layer thickness of the pixel point i will be calculated only when the pixel point i is located in both the local maximum point set S and the region of interest M. If the pixel point i is not in the intersection of S and M, then T(i) will be defined as undefined;

[0146] Through the above steps, the local layer thickness T(i) at the center point i of each layer is calculated, and the local interlayer thickness is used as the predicted interlayer thickness to be compared with the layer thickness threshold to evaluate the consistency of the printed layer and detect possible anomalies.

[0147] The interlayer thickness refers to the distance between two adjacent interlayer lines. If the interlayer thickness exceeds a preset threshold, it can be considered that the concrete quality is abnormal. The expected interlayer thickness (i.e., the layer thickness threshold) is 7mm-13mm.

[0148] The local layer thickness T(i) is defined as twice the distance between the corresponding interlayer lines. This is because the distance transform gives the shortest distance to the nearest interlayer line, and the thickness of the layer is the sum of the distances between two opposite interlayer lines. Through the above steps, the local layer thickness T(i) at each point i can be calculated, which helps to evaluate the consistency of the printed layer and detect possible anomalies, such as uneven layer thickness or misalignment between layers.

[0149] If the interlayer thickness exceeds the threshold, it will be judged as abnormal concrete quality and the early warning module will be triggered.

[0150] Constructing a texture anomaly judgment model: The image to be detected is then input into the ANN (artificial neural network), and the ANN network neurons are as follows: Figure 5 As shown in the figure, the ANN network recognizes texture features based on GLCM (gray-level co-occurrence matrix). First, the gray-level co-occurrence matrix is ​​used to generate a heat map, which includes features such as energy, contrast, and mean. Each heat map corresponds to a label of abnormality. The heat map is used as the input of the ANN network, and a binary classification result of whether the image is abnormal is output. The heat map data set obtained by using the images in the evaluation data set is used to train the ANN network to obtain a texture anomaly judgment model. The neural network is helped to learn and distinguish different image categories, and the concrete material status is divided into two results: qualified and abnormal.

[0151] Then, the ANN network will output the prediction result to determine whether the concrete material is qualified. If the image is identified as an abnormal state of concrete material, the warning module will be triggered to stop printing the material; then the system will automatically slow down the printing speed and arrange staff to check the material status; if the image is not identified as an abnormal image, no alarm will be issued and no mark will be made.

[0152] Example 2

[0153] The real-time evaluation system for 3D printing concrete construction quality in this embodiment includes:

[0154] An image acquisition module is used to obtain concrete images of the printed area during the construction process of 3D printing concrete construction;

[0155] The image processing module is used to enhance the concrete image of the image acquisition module, obtain a binary image and a thermal map, and perform pixel-level interlayer line annotation and texture anomaly annotation;

[0156] The layer thickness segmentation model, including the OneFormer network and the interlayer thickness calculation module, is used to detect whether the interlayer thickness of 3D printed concrete is qualified and output the abnormal layer thickness results;

[0157] The texture anomaly judgment model is used to judge whether the texture of the thermal map is abnormal. If the texture feature is identified as abnormal, an abnormal mark is made;

[0158] In the early warning module, if at least one of the output results of the layer thickness segmentation model and the texture anomaly judgment model is abnormal, the concrete quality is considered unqualified, and the defect information is promptly fed back to the background system, which prompts relevant personnel to check the material status.

[0159] A system for realizing 3D printing concrete quality detection and early warning based on the improved One Former model, the early warning system comprises:

[0160] An image acquisition module is used to obtain a concrete image of a printing area during the 3D printing concrete construction process;

[0161] The image processing module is used to segment and annotate the concrete image of the image acquisition module at the pixel level and perform data enhancement to obtain an enhanced data set.

[0162] In the present invention, the early warning module marks the abnormal results output by the layer thickness segmentation model and marks the abnormal results output by the texture abnormality judgment model. If the output results of at least one of the two models are identified as abnormal, the concrete quality is considered unqualified, and the defect information is promptly fed back to the background system, which prompts relevant personnel to check the material status.

[0163] Example 3

[0164] The real-time assessment method of 3D printed concrete build quality includes the following steps:

[0165] Step 1: The experimental dataset consists of 2,000 3D printed concrete images and 1,000 regular concrete images. Each image in the dataset is a 712x712 pixel RGB image.

[0166] The original data set is enhanced using the MSR algorithm. ① The image is log transformed, ② the log image is Gaussian blurred (i.e., the low-frequency component is obtained), and ③ the original image and the blurred log image are used to make a difference to obtain the image R. The original image R, G, and B dimensions are respectively subjected to ①②③ operations. The difference is that in ②, the Gaussian blur selects the sigmoid parameter to be set to 15, 80, and 200 respectively. Finally, the results of ①②③ operations on the three dimensions are weighted averaged, and then the three dimensions are combined and the data is scaled to 0-255 to obtain the enhanced data set R*.

[0167] Obtain the corresponding binary images and texture classification anomaly results of the images in the enhanced dataset R*, and use them as the evaluation dataset;

[0168] The images in the evaluation data set are divided into a training set plus a validation set and a test set in a ratio of 7:3, which not only ensures the amount of training data but also improves the generalization ability of the model. The training set is used to train the model, and the test set is used to evaluate the model performance under different parameter settings, so as to select the best parameter settings and make the model have better generalization ability. Using the k-fold cross-validation method during the training process can make full use of the limited data set, more accurately evaluate the model performance, and select the best model and parameters.

[0169] Step 2, training part based on One Former model: First, randomly initialize the parameters of One Former neural network, then extract the concrete image of the printing area and the corresponding annotated image during the 3D printing concrete construction process from the training set; secondly, use the feature extraction structure to extract image features; finally, complete the prediction of each pixel category of the image based on the extracted features, and combine the annotated image model to calculate the error loss using the MoCo loss function based on the generated query vector and the real label information (instance segmentation, semantic segmentation label), and then update the network weight parameters through error back propagation. Iterate and train all images in sequence. After 30,000 iterations, the training model reaches convergence, that is, when the model training gradient is close to 0, stop training and extract the optimal network parameters for prediction.

[0170] During prediction, the network first loads the trained parameters and loads the input image from the test set; it obtains feature information through image processing, feature extraction, clustering, transformer-based prototype learning (PL), convolution-based pixel descriptor (PD) and other modules in turn, and uses the CLS module (AConvolutional Layer) to calculate the category of each pixel based on the final features, thereby achieving semantic segmentation.

[0171] The calculation method of the pixel area in the picture is as follows: the binary mask of the interlayer line is obtained through the OneFormer model, that is, the interlayer line part is marked as the foreground, and the non-interlayer line part is marked as the background. The pixel counting method is used to calculate the number of pixels in the interlayer line part, that is, the number of foreground pixels, represented by N. The number of pixels N is multiplied by the square of the pixel size to obtain the area S of the interlayer line part, in pixels squared.

[0172] Step 3: Input the concrete image of the printing area in the 3D printing concrete process to be tested into the trained OneFormer network; based on the output result of OneFormer, use the interlayer thickness calculation module to calculate and determine the local thickness of the printed layer. The interlayer thickness refers to the distance between two adjacent interlayer lines. If the interlayer thickness exceeds a preset threshold, it can be considered that the concrete quality is abnormal. The expected interlayer thickness is 9-11mm.

[0173] If the interlayer thickness exceeds the threshold, it will be judged as abnormal concrete quality and the early warning module will be triggered.

[0174] Step 4: Train the texture anomaly judgment model. Calculate GLCM through the texture images annotated in the training set to obtain its heat map to train the ANN network. It is mainly divided into two parts: training and prediction. The training part first randomly initializes the network parameters, then preprocesses the heat map obtained by GLCM, propagates the input data through the network, calculates the output, and divides it into qualified and abnormal categories.

[0175] Then, the loss function is used to evaluate the difference between the predicted output and the true label. According to the gradient of the loss function, the error is back-propagated through the network, the weight and bias updates are calculated, and the weights and biases of the network are updated using the optimization algorithm. All images are iterated and trained in sequence. After 10,000 iterations, the training model reaches convergence, that is, when the model training gradient is close to 0, the training is stopped and the optimal network parameters are extracted for prediction.

[0176] During prediction, the network first loads the trained parameters, selects texture images from the test set to calculate the GLCM, obtains the heat map for testing, and then passes through the input layer, hidden layer, and output layer to output the classification result.

[0177] Step 5: Early warning feedback

[0178] The system receives images for quality inspection;

[0179] If the concrete quality is detected as unqualified, that is, the interlayer thickness of the concrete or the texture characteristics of the concrete are abnormal, the early warning module will be triggered and fed back to the back-end system to stop printing the material. The back-end system will automatically slow down the printing speed and increase the pumping rate, and arrange staff to check the material status; if the image is not identified as an image of unqualified quality, the material will be considered qualified, and no alarm or mark will be made if it is qualified;

[0180] Until the printing is completed, all areas meet the requirements and the behavior warning is completed.

[0181] The efficiency of the One Former network is compared with other algorithms using the evaluation data set established in this embodiment. The results are shown in Table 1.

[0182] Table 1

[0183]

[0184] The hardware involved in the present invention includes:

[0185] Processor: As the core component of the present invention, the processor is responsible for controlling and managing the operation of the entire system, including data acquisition, data processing, image recognition, alarm and other functions, and needs to have sufficient computing power and parallel processing capabilities to meet real-time requirements. The processor can be in different forms such as single-chip microcomputers, microprocessors, computers, etc. to meet the needs of different application scenarios;

[0186] Memory: Memory can be used to store collected data and historical data for subsequent processing and analysis. It has the characteristics of high speed, high reliability and scalability to meet the needs of long-term stable operation of the system;

[0187] Database: The back-end system needs to use a database to store and manage collected data, historical data, analysis results and other information;

[0188] Network interface: The back-end system needs to use a network interface to exchange and communicate data with the front-end system and other external systems. It has the characteristics of high speed, high stability and high security to ensure the reliability and security of data transmission.

[0189] The main feature of the computer back-end system memory is that a computer program is stored thereon, and the computer program can be executed by a processor to implement each step of the above-mentioned method.

[0190] The computer back-end system database has the main feature that the database is configured to store and manage computer application data, including various data types and structures, which are applied to each step of the above-mentioned method.

[0191] The main feature of the computer back-end system network interface is to realize communication and data transmission between computers. The network interface can provide various communication protocols and data transmission methods to meet the communication and data transmission needs of different application scenarios and different requirements, and is applied to each step of the method described above.

[0192] The system of the present invention provides the possibility of realizing larger-scale building-level 3D printing quality monitoring, further promoting the development of this field, has great significance and broad application prospects, and also provides strong support and guarantee for the development of concrete 3D printing technology.

[0193] It should be understood that each part of the present invention can be implemented by hardware, software or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software stored in a memory and executed by a suitable instruction execution system.

[0194] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0195] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A real-time evaluation method for 3D printed concrete construction quality, characterized in that: The method comprises the following steps: Step 1: Build an evaluation dataset: The camera is fixed on the printing nozzle, and the printed wall is photographed from the side along the printing height direction. At this time, the camera can capture images containing the topmost layer of printed slurry strips. Each frame of the image contains 7-9 layers of printed slurry strips. The middle window area of ​​the topmost layer in each frame of the image is considered to be the window position facing the center of the nozzle. During the printing process, the camera is used to collect images in real time. During the image collection process, it is ensured that the line connecting the camera to the extrusion point of the extrusion 3D printer is perpendicular to the printing wall. From the collected images, a subset image is selected that does not contain a printed background, has a clearly visible printed layer, and has clear texture details. The subset image includes images that meet the requirements of no printed background, a clearly visible printed layer, and clear texture details obtained under different printing parameters; The subset images and non-3D printed concrete images are used as the original data set; the original data set is enhanced using the MSR algorithm to obtain the enhanced data set R*; Two types of annotation processing are performed on the enhanced dataset R*: the first type corresponds to the binary segmentation mask of the interlayer line. The area to be identified in the dataset R* is annotated at the pixel level using the Photoshop tool, and a corresponding semantic label is assigned to each pixel to determine whether it is an interlayer line. This forms a dataset with annotation information. After the annotation is completed, a binary image is generated. The black pixel value is 0, representing the background, and the white pixel value is 255. The width of the white pixels in these binary images is 18-22 pixels, representing the target interlayer line. The second annotation corresponds to the texture classification of the window extracted from the enhanced dataset image R*: the window width is set to 200 pixels, and the height of the window is based on the thickness of each printed layer, but does not include adjacent layers; the image in the enhanced dataset image R* is segmented according to the window size, which is equivalent to taking a small window from each layer of concrete in the image and judging whether the small window is abnormal; the window facing the center of the printing nozzle is regarded as an abnormal window, and the window with abnormalities in all windows in each image is found. When the abnormal area exceeds 30% of the surface area of ​​the entire image, the image is provided with an abnormal label; otherwise, it is a good label; Obtain the corresponding relationship between the binary image and the abnormal texture classification results corresponding to the image in the enhanced dataset R*, and use this as the evaluation dataset; Step 2: Constructing the layer thickness segmentation model The layer thickness segmentation model includes a OneFormer network and an interlayer thickness calculation module, which is used to detect whether the interlayer thickness of 3D printed concrete is qualified; The evaluation data set is used to train the OneFormer network, which outputs a binary image that predicts the position information of the concrete interlayer lines, as well as the data required by the interlayer thickness calculation module (including the coordinates of all pixels in the binary image); the interlayer lines are determined from the binary image, which can segment the independent printed layers in the image, and the interlayer thickness calculation is performed separately for each layer; The interlayer thickness calculation module is used to calculate the distance between two adjacent interlayer lines; Step 3: Build a texture anomaly judgment model The gray-level co-occurrence matrix of the images in the evaluation data set is calculated, and the gray-level co-occurrence matrix is ​​used to generate a heat map. The heat map includes features such as energy, contrast, and mean. Each heat map corresponds to a label of abnormality. The heat map is used as the input of the ANN network, and the binary classification result of whether the image is abnormal is output. The heat map data set obtained by using the images in the evaluation data set is used to train the ANN network to obtain a texture abnormality judgment model. Step 4: Actual monitoring During the 3D printed concrete construction process, images without printed background, with clearly visible printed layers and clear texture details are acquired in real time, and corresponding thermal maps are obtained. The layer thickness segmentation model is used to obtain the interlayer thickness of the image, and the texture anomaly judgment model is used to obtain the classification of whether the texture is abnormal. The interlayer thickness and texture anomalies are used to evaluate the printing quality.

2. The real-time evaluation method for 3D printing concrete construction quality according to claim 1 is characterized in that: The OneFormer network includes five backbone modules, namely, the backbone network Backbone backbone module, the Transformer decoder backbone module, the query mapper backbone module, the text mapper backbone module, and the contrast loss backbone module; The backbone module of the backbone network adopts the enhanced efficiency network PN-EfficientNet. The specific process of the enhanced efficiency network PN-EfficientNet is: First, the input image is output through the first convolution layer, which is a feature map with a dimension of (112×112×32), and is padded to "same", that is, the width and height of the output are reduced by half; then it enters the proxy normalization layer, and then enters the mobile flip bottleneck convolution MBConv1 after being processed by the Swish activation function. The expansion ratio of the mobile flip bottleneck convolution MBConv1 is 1, the depth convolution kernel size is 3x3, the kernel step size is 1x1, and it includes compression and excitation operations, without connection deactivation and connection skipping; Next, we enter six cascaded mobile flip bottleneck convolution MBConv6 layers, each with multiple MBConv6s. The dilation factor of MBConv6 is 6; Finally, it enters the second convolutional layer and is filled to "same", that is, the width and height of the output remain unchanged. It is then processed by the proxy normalization layer, Swish activation function, global average pooling layer, random inactivation dropout, fully connected layer, and Softmax classification layer to output a tensor containing a feature vector.

3. The real-time evaluation method for 3D printing concrete construction quality according to claim 2 is characterized in that: The calculation process of the proxy normalization layer is: First, for each channel c, the proxy variable is calculated in, and is the mean and variance of the proxy variable, and N indicates normal distribution; Then, we choose layer normalization as the batch-independent normalization method, and for each batch element b and channel c, we calculate: Among them, μ b and σ 2 b is the mean and standard deviation of the input X of batch element b on channel c; X b... is the unnormalized pre-activation of the b-th element in the batch; Y b... is the normalized pre-activation of the bth element in the batch after batch-independent normalization; ε is the normalized numerical stability constant; Next, for each channel c, the proxy normalized activation is calculated: Z...c= Where φ is the activation function, γc and βc are the normalized scaling and offset parameters, E[] and Var[] are the expected value and variance, respectively, and ~ represents the estimate or approximation of the corresponding variable or value; Z …c is the activation value after being processed by the activation function φ; Y …c represents the activation value on a specific channel c; E[] and Var[] are the expected value and variance respectively; Finally, proxy normalization is performed to ensure that the pre-activation Y remains close to the normalized state, thereby maintaining the expressiveness and stability of the network.

4. The real-time evaluation method for 3D printing concrete construction quality according to claim 2, characterized in that: The contrast loss backbone module adopts the MoCo contrast learning mechanism, named MoCo loss calculation, and the specific process is as follows: First, define a query encoder to convert the input image into a feature representation; at the same time, initialize a momentum encoder whose parameters are copies of the query encoder parameters; Then, the input image is data augmented to generate a pair of positive samples. These augmented images will be input into the query encoder and momentum encoder as query image query and key image Key respectively. Subsequently, the query image is encoded using the query encoder to obtain the query feature representation; at the same time, the key image is encoded using the momentum encoder to obtain the key feature representation; The features obtained by encoding the data samples through the query encoder and the momentum encoder are represented as encoding keys. In MoCo, the encoding keys are part of the dynamic dictionary, and a queue is maintained to store and update these encoding keys for comparative learning and optimization processes; At the same time, a queue is maintained as a dynamic dictionary, which stores the encoding keys of several previous small batches (5-10); when the encoding key of the current small batch is added to the end of the queue (enqueue), the encoding key of the oldest small batch is removed from the front of the queue (dequeue). By continuously updating the encoding keys in the queue, it is ensured that the dynamic dictionary contains the latest sample feature representation; as the training progresses, the encoding keys in the queue will be continuously updated, so that the dynamic dictionary can cover a wider sample space, and the size of the dynamic dictionary can be controlled by removing the encoding keys of the oldest small batch; The query image is encoded by the query encoder to obtain the query feature, and the feature obtained by encoding the corresponding positive sample image by the momentum encoder is represented as the positive key feature; In MoCo, positive sample images refer to images paired with query images, also known as positive keys, and negative sample images refer to other images that are not paired with query images, also known as negative keys; positive sample images are encoded by momentum encoders to calculate the similarity between query features and positive key features. By maximizing the similarity between query and positive key, the model learns more discriminative feature representations; negative sample images are encoded by other encoding keys in the dynamic dictionary to calculate the similarity between query features and negative key features. By minimizing the similarity between query and negative key, the model learns better visual representations and improves the discriminability of features; Secondly, using the contrastive loss function, the similarity between the query features and the positive key features is calculated, and the dissimilarity between the query features and all the negative key features in the queue is calculated; the contrastive loss function aims to maximize the similarity between the query and the positive key, while minimizing the similarity between the query and the negative key; Finally, repeat the above steps until the maximum number of iterations is reached.

5. The real-time evaluation method for 3D printing concrete construction quality according to claim 1, characterized in that: In step 1, the anomaly is determined by comparing the texture features in the current window with the texture feature windows at the corresponding positions of the corresponding images of the same material in the public data set. If there is a significant difference, it is an abnormal window.

6. The real-time assessment method for 3D printing concrete construction quality according to claim 1, characterized in that: The specific process of the interlayer thickness calculation module is as follows: First, the fast Euclidean distance transform (EDT) is applied to the predicted binary image containing the inter-layer line. This transform calculates a distance value D(i) for each pixel point i in the image, which represents the distance from the point to the nearest inter-layer line. D(i) uses the formula get, Where: D(a,b): represents the measure of the shortest distance from the pixel point (a,b), that is, the distance between the pixel point in the layer and the pixel point (a′,b′) on the layer line; B represents the set of all possible positions related to the pixel (a,b) in the foreground area of ​​the binary image; Next, find the local maximum points of the distance transform result D(i), which roughly correspond to the center position of the printed layer; convolve the distance transform result D with vertical kernels i1=(0,-1,1) and i2=(1,-1,0) to accurately locate the local maximum, that is, the layer center point i; for the layer center point i located between two interlayer lines, the local layer thickness T(i) is defined as twice the distance between the corresponding interlayer lines, and the formula is: Among them, S is the set of local maximum points that can be identified by applying the vertical kernel to the distance transformation result D for convolution; M: is the area of ​​the layer currently being analyzed in the image. This area limits the range of pixels for calculating the layer thickness, ensuring that only the thickness of the current layer is considered, while ignoring other irrelevant parts of the image, which is called the region of interest; The condition i∈S∩M means that the layer thickness of the pixel point i will be calculated only when the pixel point i is located in both the local maximum point set S and the region of interest M. If the pixel point i is not in the intersection of S and M, then T(i) will be defined as undefined; Through the above steps, the local layer thickness T(i) at the center point i of each layer is calculated, and the local interlayer thickness is used as the predicted interlayer thickness to be compared with the layer thickness threshold to evaluate the consistency of the printed layer and detect possible anomalies; the layer thickness threshold is 7mm-13mm.

7. A real-time assessment system for 3D printing concrete construction quality, characterized in that: The system comprises: An image acquisition module is used to obtain concrete images of the printed area during the construction process of 3D printing concrete construction; The image processing module is used to enhance the concrete image of the image acquisition module, obtain a binary image and a thermal map, and perform pixel-level interlayer line annotation and texture anomaly annotation; The layer thickness segmentation model, including the OneFormer network and the interlayer thickness calculation module, is used to detect whether the interlayer thickness of 3D printed concrete is qualified and output the abnormal layer thickness results; The texture anomaly judgment model is used to judge whether the texture of the thermal map is abnormal. If the texture feature is identified as abnormal, an abnormal mark is made; In the early warning module, if at least one of the output results of the layer thickness segmentation model and the texture anomaly judgment model is abnormal, the concrete quality is considered unqualified, and the defect information is promptly fed back to the background system, which prompts relevant personnel to check the material status.

Citation Information

Cited By

  • Control system for industrial-grade particle 3D printer

    CN120396354A