Quality classification method and system for 3D printing concrete based on balanced adversarial network

Abstract

The invention discloses a quality classification method and system for 3D printing concrete based on a balanced adversarial network, and the method comprises the steps: obtaining original images of 3D concrete filaments with different printing widths and different flowability, and guaranteeing the illumination consistency of a detection region containing the 3D printing concrete during collection; constructing a self-encoding image data set; marking the image labels according to width and mobility to obtain an image set with classification labels, and defining the image set as a label data set; an improved balanced adversarial network is constructed, the improved balanced adversarial network comprises an auto-encoder and an adversarial module, the adversarial module comprises a generator and a discriminator, a target number of enhanced images are generated by using the trained generator, and each piece of enhanced data is provided with a label; and combining the labeling data set and the enhanced data with the labels to form a classification data set, training a ResNet-18 network by using the classification data set, and further judging the type of the 3D concrete filament so as to obtain the quality classification of the 3D printing concrete.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent construction, and in particular to a 3D printing concrete quality classification method and system based on a balanced adversarial network. Background Art

[0002] 3D printing, also known as additive manufacturing, is an advanced manufacturing technology that creates objects by stacking materials layer by layer. This technology is favored for its high design freedom, high manufacturing efficiency, high resource utilization and ability to manufacture complex structures. In the process of 3D printing concrete, concrete is extruded into long filaments and stacked layer by layer to form components and structures. This process does not require the use of traditional formwork, which is significantly different from the traditional concrete pouring process, which is mixed and poured in bulk.

[0003] However, the unique construction process of 3D printed concrete poses challenges to quality monitoring and control for practical applications. In most extrusion-based 3D printing concrete processes, although concrete is mixed in batches, its continuous extrusion nature results in different rest times between mixing and extrusion of the printed concrete. In addition, the material fluidity of fresh concrete has significant time sensitivity, and changes in this fluidity can cause instabilities in extrudability and buildability, which in turn affect the geometry, roughness, and green strength fluctuations of the printed concrete filaments. Concrete 3D printing in a dry state may result in rough surfaces, insufficient interlayer bonding, overly thin printed filaments, and even initial defects, so achieving real-time quality classification is essential to ensure the quality of 3D printed concrete structures.

[0004] Although 3D printing concrete technology has obvious advantages in improving construction efficiency, reducing material waste, and increasing design freedom, due to its technical novelty, the time from theoretical exploration to practical application is relatively short, and the accumulated practical experience and case studies in the industry are relatively insufficient. The process of 3D printing concrete involves complex physical and chemical changes, which are affected by many factors, including printing parameters, material properties, and environmental conditions, making it difficult to collect comprehensive and representative training data, and experimental data under specific conditions are difficult to generalize to a wider range of application scenarios.

[0005] In response to these problems, the present invention proposes a system specifically for unbalanced data sets, which can artificially enhance existing data, increase the number and diversity of training samples while maintaining data annotation consistency, and can monitor the printing filaments online, and establish a real-time feedback system for 3D printing concrete quality classification. Computer vision and deep learning technology are used to classify the quality of the 3D printing process, thereby solving the quality control problem of 3D printing concrete in practical applications and promoting its further development and application in the field of construction. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the above technologies and provide a quality classification method and system for 3D printed concrete based on a balanced adversarial network.

[0007] Compared with the prior art, the system of the present invention creatively uses computer vision technology to realize the quality monitoring of the concrete 3D printing process, can accurately and efficiently detect the classification of 3D printed concrete, realize real-time monitoring, and provide an effective means for ensuring printing quality.

[0008] In order to solve the above technical problems, the technical solution of the present invention is:

[0009] In a first aspect, the present invention provides a quality classification method for 3D printed concrete based on a balanced adversarial network, the steps of the quality classification method are:

[0010] Step 1: Obtain original images of 3D concrete filaments with different printing widths and different fluidity, and ensure that the illumination of the detection area containing 3D printed concrete is consistent during acquisition;

[0011] Step 2: Image preprocessing:

[0012] The original image obtained in step 1 is cropped into a small image of 600×600 pixels containing the filaments, and the filament area is marked. The size of the original image is larger than 600×600 pixels; then the brightness is normalized using the Gamma correction method.

[0013]

[0014] in, is the pixel at the i-th row and j-th column of the m-th channel in the k-th image, k = 1, 2, ..., is the image number, m = "R", "G", "B" is the channel name; i, j = 1, 2, ..., 600, indicates the pixel position; is the normalized pixel; γ is the brightness normalization parameter;

[0015] Then use the following formula to make the chromaticity of the input images of different batches of materials evenly distributed:

[0016]

[0017] in, is the pixel after chromaticity uniformity processing; β is the constant value of the chromaticity normalization process, E ij The data is normalized to brightness The average pixel expectation of

[0018] So far, the self-encoding image dataset is obtained;

[0019] Step 3: Labeling:

[0020] Label the images by width and fluidity to obtain a collection of images with classification labels, which is defined as a label dataset;

[0021] Step 4: Build an improved balanced adversarial network

[0022] The improved balanced adversarial network includes an autoencoder and an adversarial module, the adversarial module includes a generator and a discriminator, the autoencoder includes an encoder E and a decoder Δ, the encoder of the autoencoder is used to infer the distribution of different categories in the latent space, the decoder Δ matches the topological structure of the generator G, and the encoder E matches the structure of the discriminator D;

[0023] Step 5: Train the improved balanced adversarial network

[0024] First, the autoencoder dataset is used to train the autoencoder, and then the trained autoencoder is used to initialize the adversarial module. During initialization, the input is processed by the autoencoder and connected to the class conditional latent vector generator, and then divided into two branches, connecting the generator G and the discriminator D respectively; during initialization, the input image is an image with a classification label. During initialization, the encoder parameters of the autoencoder are assigned to the discriminator, and the decoder parameters of the autoencoder are assigned to the generator;

[0025] After initialization, the autoencoder is removed, and the images in the labeled dataset are used to train the improved balanced adversarial network. The category label c is input into the class conditional latent vector generator to generate a latent vector, which is then processed by the generator G and input into the discriminator D together with the original data. When training the discriminator D, the optimized sparse classification cross entropy loss function is used to match the class label of the real image and the false label of the generated image; after the sparse classification cross entropy loss function constraint, the label selected by the discriminator matches the label c used to generate the image, where the sparse classification cross entropy loss function is:

[0026]

[0027] Where n is the number of samples; C is the number of categories; y ij is the judgment value of whether the category j of sample i is true. If it is true, the value is 1, otherwise it is 0; p ij is the probability that sample i belongs to category j predicted by the model;

[0028] When the training model reaches convergence, stop training;

[0029] The trained generator is then used to generate the target number of enhanced images with a resolution of 600*600 to form enhanced data, each of which is labeled.

[0030] Step 6: Combine the labeled dataset and the enhanced data with labels to form a classification dataset, and use the classification dataset to train the ResNet-18 network to identify the type of 3D concrete filaments to obtain the quality classification of 3D printed concrete.

[0031] Further, the encoder E includes eight sequentially connected convolutional activation blocks, two dense layers and a softmax activation function, each convolutional activation block includes a convolutional layer of 3×3 filters and a LeakyRelu activation function, the input image passes through 8 convolutional layers of 3×3 filters, the number of feature maps of the 8 convolutional layers of 3×3 filters increases from 64 to 512, and after the last convolutional activation block, a dense layer, a LeakyRelu activation function, a dense layer, a softmax activation function, and a softmax activation function are sequentially connected to obtain the probability of sample classification, and then encoded into a potential vector z;

[0032] The decoder Δ includes a convolution activation block with 64 feature maps, five residual structures connected in series, each residual structure includes two 3×3 kernel convolution layers, a LeakyRelu activation function and an element-by-element summation operation, and the number of feature maps of each 3×3 kernel convolution layer is 64. After a 3×3 kernel convolution layer is processed by the LeakyRelu activation function and another 3×3 kernel convolution layer, an element-by-element summation operation is performed with the input of the residual structure to obtain an output; the output of the last residual structure is convolutionally processed and element-by-element summation is performed with the input of the first residual structure, and then processed by two sub-pixel convolution layers and enters the convolution layer to obtain the output of the decoder.

[0033] Furthermore, the sub-pixel convolution layer includes a convolution layer, two pixel masking layers and a LeakyRelu activation function, wherein the convolution layer: k=3, n=256, s=1, wherein k is the convolution kernel size, n is the number of feature maps, and s is the step size.

[0034] Furthermore, the autoencoder is constrained by using an L2 loss function during training; the convolution kernel of the last convolution layer of the decoder is 9, the number of feature maps is 3, and the step size is 1; the convolution kernel of the last residual structure connected convolution in the decoder is 9, the number of feature maps is 64, and the step size is 1.

[0035] Furthermore, the printing filaments are set to three width standards of 35mm, 30mm and 25mm, and the width evaluation grade consists of three levels, including "wide", "moderate" and "thin". Based on the three width targets, different width evaluation grades are assigned to each image, and extreme samples with a difference of >10.0mm between the actual width and the target width are excluded from training;

[0036] The fluidity range of 3D printed concrete is between 160-220mm, forming a total of 9 different label categories, specifically:

[0037] Category label 1 means that the fluidity is 160mm-180mm dry and belongs to the target width 25mm width assessment grade thin;

[0038] Category label 2 means the fluidity is 160mm-180mm dry and falls within the target width of 30mm, with a moderate width assessment grade;

[0039] Category label 3 means the flowability is 160mm-180mm dry and belongs to the target width 35mm width assessment grade wide;

[0040] Category label 4 means that the fluidity is moderate at 181mm-200mm and falls within the target width assessment grade of 25mm thin;

[0041] Category label 5 means that the fluidity is moderate between 181 mm and 200 mm and falls within the target width of 30 mm, with a moderate width assessment grade;

[0042] Category label 6 means that the fluidity is moderate between 181mm and 200mm and falls within the target width 35mm width assessment grade wide;

[0043] Category label 7 indicates that the fluidity is 201mm-220mm fluid and belongs to the target width 25mm width assessment level thin;

[0044] Category label 8 means that the fluidity is 201mm-220mm fluid and the target width is 30mm, which is a moderate width assessment level;

[0045] Category label 9 indicates that the fluidity is 201mm-220mm fluid and belongs to the target width 35mm width assessment grade wide.

[0046] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which can implement the steps of the method when executed by a processor.

[0047] In a third aspect, the present invention provides a quality classification system for 3D printed concrete based on a balanced adversarial network, wherein the system executes the method, including:

[0048] The image collection module uses an industrial camera to collect images of the printing area. The camera continuously collects RGB channel images of the printed filament and saves them in real time to the computer for detection and adjustment;

[0049] Image processing module, used to process the original image and obtain the autoencoder dataset and label dataset;

[0050] The image augmentation module is used to enhance the label data set of the image processing module using an intelligent network to enrich the number of samples;

[0051] The image classification module connects the image amplification module and the image processing module, and uses the classification data set composed of the label data set and the enhanced data to train the ResNet-18 network, and then distinguish the type of 3D concrete filaments to obtain the quality classification of 3D printed concrete.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] In practical applications, the present invention can provide a new idea and solution for quality classification monitoring of concrete 3D printing process, and at the same time provide strong support and guarantee for the development of concrete 3D printing technology.

[0054] Based on the existing technical solutions, it can not only reduce labor costs, improve printing efficiency, and ensure the quality of printed products, but also make it possible to pursue larger-scale architectural-level 3D printing. Compared with existing technologies, it has great significance and good application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a residual structure diagram in the balanced adversarial network of the present invention.

[0056] Figure 2 It is a structural diagram of the autoencoder in the balanced adversarial network of the present invention.

[0057] Figure 3 It is a schematic diagram of adversarial initialization in the balanced adversarial network of the present invention.

[0058] Figure 4 It is a structural diagram of the convolutional activation block in the balanced adversarial network of the present invention.

[0059] Figure 5 This is a structural diagram of the sub-pixel convolutional layer in the balanced adversarial network of the present invention.

[0060] Figure 6 It is a schematic diagram of adversarial training in the balanced adversarial network of the present invention.

[0061] Figure 7 It is a schematic diagram of the ResNet-18 network structure in the present invention. DETAILED DESCRIPTION

[0062] In order to more clearly describe the technical problems, technical solutions and advantages of the present invention, the following will be described in detail with reference to the drawings and embodiments. It should be noted that these embodiments are only used to illustrate the principles and application scope of the present invention and should not be regarded as limiting the present invention.

[0063] In the initialization of the confrontation module of the present invention, the confrontation module initialization structure is as follows: Figure 3 As shown in Figure 2. The generator G and the discriminator D have explicit class knowledge. In adversarial training, G is required to generate images of different categories and D is required to label images as fake images or use the category label c of the specific problem. While initializing the adversarial module, the knowledge of the autoencoder is passed into the adversarial module by initializing G as the weights in the decoder Δ, and the first layer of the discriminator D is initialized as the weights of the encoder E.

[0064] The purpose of discriminator initialization is to include meaningful features in the discriminator D that are helpful for image classification.

[0065] In the initialization of the generator, since, when adversarial training starts, the generator G is equal to the decoder Δ. Therefore, the latent vector Z input to the generator G is equivalent to a point in the latent space of the autoencoder, that is, Z can be regarded as the output of the encoder E or the input of the decoder Δ. Therefore, the encoder E maps the real image to the latent space used by the generator G.

[0066] Use a multivariate normal distribution N in the latent space c =N(μ c ,Σ c ), the mean vector μ c and the covariance matrix Σ c For each class c, calculate μ c and Σ c To match Z c =E(X c ), considering all the real images X of class c in the training dataset c And, these probability distributions are used to initialize the class-conditional latent vector generator, which is a random process that takes a class label c as input and returns a class from N c Randomly drawn latent vector Z c As output. In adversarial training, the probability distribution N c is considered invariant, forcing the generator not to deviate from the initial class encoding in the latent space.

[0067] The adversarial training process is as follows Figure 5 As shown, data flows through the generator G and the discriminator D in batches, and their weights are fine-tuned to optimize their loss functions.

[0068] The discriminator classifies the input image as belonging to one of n problem-specific classes or fake. In each batch of images provided, 1 / (n+1) are fake, and the fake data is generated as the output of G, which is a latent vector Z extracted from the class-conditional latent vector generator. cAs input. In turn, the class-conditional latent vector generator takes as input the uniformly distributed class labels c, i.e., the fake images are evenly distributed among the problem-specific classes.

[0069] When training the discriminator D, an optimized sparse categorical cross entropy loss function is used to match the class labels of real images and the fake labels of generated images. For each batch learned by the discriminator, the generator G learns a batch of the same size. To do this, a batch of conditional latent vectors Z is randomly drawn by imposing a uniform distribution on the labels c. c These vectors are processed by the generator and the output image is fed into the discriminator. The parameters in G are optimized so that the label selected by the discriminator matches the label c used to generate the image. The sparse categorical cross entropy loss function is as follows:

[0070]

[0071] Where n is the number of samples, C is the number of categories, and y ij is the judgment value of whether the category j of sample i is true. If it is true, the value is 1, otherwise it is 0; p ij is the probability that the model predicts that sample i belongs to category j. The sum in this formula is performed over all categories of each sample, but only the true label y ij Only items that are 1 are counted into the loss. This calculation method avoids considering the entire probability distribution and only focuses on the predicted probability corresponding to the true label, thereby effectively reducing the amount of calculation and memory usage in the case of multiple categories and sparse category labels.

[0072] For multi-classification problems, there are as many exclusive categories as there are categories in the model output. It is usually used with the softmax activation function.

[0073] The acquired images are used to train the improved balanced adversarial network. During the training process, the number of network layers of the generator and the discriminator is gradually increased, so that the network can first learn large-scale structural information and then gradually shift its attention to small-scale detail learning; instead of learning information on all scales at the same time, the output resolution of each stage is doubled. After 30,000 iterations, the training model reaches convergence, that is, when the model training gradient is close to 0 (less than 0.01 can be considered close to 0), the training is stopped. The trained generator is then used to generate 12,000 enhanced images with a resolution of 600*600 to constitute the enhanced data.

[0074] Example 1

[0075] The quality classification method of 3D printed concrete in this embodiment is based on a balanced adversarial network, and the steps of the quality classification method are:

[0076] Step 1: Use an industrial camera to continuously collect RGB channel images of the filaments of extruded 3D printed concrete and save them in real time to a computer for detection and adjustment. The camera is fixed in position, the camera lens is kept at a distance of 20 mm (focal length) from the 3D printed concrete filament, and a stable annular light source is fixed on the camera to ensure consistent lighting in the detection area containing 3D printed concrete.

[0077] Step 2: Image preprocessing:

[0078] The original image was cropped to a small image of 600×600 pixels containing the filaments to reduce computational cost and irrelevant noise. The size of the original image was larger than 600×600 pixels. Then the brightness was normalized using the Gamma correction method to ensure that the photos collected under different lighting environments and material reflectance characteristics can be accurately identified.

[0079]

[0080] exist Where k = 1, 2, ..., is the image number, m = "R", "G", "B" is the channel name; i, j = 1, 2, ..., 600, represents the pixel position; k is the normalized image number, m = "R", "G", "B" are the channel names, i, j = 1, 2, ..., 600, indicating the pixel position of the normalized image, and γ is the brightness normalization parameter, which is generally selected as 2.2.

[0081] Then use the following formula to make the chromaticity of the input images of different batches of materials evenly distributed:

[0082]

[0083] in k is the number of the final processed image, m = "R", "G", "B" are the channel names, i, j = 1, 2, ..., 600, representing the pixel bits of the final processed image, β is the constant value of the chromaticity normalization process, E ij The data is normalized to brightness The average pixel expectation.

[0084] So far, the self-encoding image dataset is obtained.

[0085] Step 3: Labeling:

[0086] 3000 original images of 3D concrete filaments with different printing widths and different fluidities were collected and preprocessed as mentioned above to form an autoencoding image dataset, in which the filaments were labeled. Then, the image labels were annotated according to width and fluidity to ensure that the sample number distribution of all classes was approximately equal to the sample number distribution of the entire autoencoding image dataset. The set of images with classification labels was defined as the labeled dataset.

[0087] In the present invention, three width standards of 35mm, 30mm and 25mm are set for the printing filament. The width evaluation grade consists of three levels, including "wide", "moderate" and "thin". Based on the three width targets, different width evaluation grades are assigned to each image to judge the approximate width range of different width grades. Extreme samples whose actual width differs from the target width by >10.0mm are excluded from training. In addition to considering the width of the printing filament, the fluidity of the 3D printing material is also considered. The recommended fluidity range for 3D printed concrete is between 160-220mm. Therefore, concrete filaments with different fluidities (160mm, 190mm and 220mm) are printed as reference samples to understand the image characteristics of concrete printing filaments with different material fluidities.

[0088] The main features that can be distinguished by human vision include surface roughness, edge flatness, and degree of reflection. Based on the preliminary observation of the filament images, three material fluids were set, including "dry", "moderate", and "fluid". In the "dry" material state, the 3D printed concrete filament has a rough surface, rough edges, and no reflection, while in the "fluid" material state, the printed concrete surface is smooth and rich in moisture. The proposed suitable material has fewer reflection points on the smooth surface, which is considered suitable for concrete 3D printing.

[0089] The three material fluidities of "dry", "medium" and "fluid" intersect with the three filament widths to form 9 different label categories. See the table below for specific category divisions

[0090]

[0091] Step 4: Build an improved balanced adversarial network

[0092] The improved balanced adversarial network includes an autoencoder and an adversarial module, the adversarial module includes a generator and a discriminator, the autoencoder is composed of an encoder E, a decoder Δ, and a reconstruction loss, the encoder of the autoencoder is used to infer the distribution of different categories in the latent space, the decoder Δ matches the topological structure of the generator G, and the encoder E matches the structure of the discriminator D.

[0093] The encoder E includes eight sequentially connected convolutional activation blocks, two dense layers and a softmax activation function, each convolutional activation block includes a convolutional layer of a 3×3 filter and a LeakyRelu activation function, the input image passes through 8 convolutional layers of 3×3 filters, the number of feature maps of the 8 convolutional layers of 3×3 filters increases from 64 to 512, and the number of feature maps increases by two times each time a convolutional layer of a 3×3 filter passes through, and the convolutional layer of the 3×3 filter is used to reduce the image resolution each time the number of features doubles; after the obtained graph of 512 feature maps, two dense layers and a final softmax activation function are passed to obtain the probability of sample classification, and then it is encoded into a potential vector z.

[0094] The decoder Δ includes a convolution activation block with a feature map number of 64, five residual structures connected in series, each residual structure includes two 3×3 kernel convolution layers, a LeakyRelu activation function and an element-by-element summation operation, the number of feature maps of each 3×3 kernel convolution layer is 64, a 3×3 kernel convolution layer is processed by the LeakyRelu activation function and another 3×3 kernel convolution layer, and then the element-by-element summation operation is performed with the input of the residual structure to obtain the output; the output of the last residual structure is processed by convolution (the convolution kernel is 9, the number of feature maps is 64, and the step size is 1) and then the element-by-element summation operation is performed with the input of the first residual structure, and then the output is processed by two sub-pixel convolution layers and enters the convolution layer (the convolution kernel is 9, the number of feature maps is 3, and the step size is 1) to obtain the output of the decoder; the encoder E and the decoder Δ are trained using the data set, and the L2 loss function is used for constraint to obtain the reconstructed data, the resolution of the input image is improved by two trained sub-pixel convolution layers, and finally the information of the autoencoder is passed to the adversarial module.

[0095] The sub-pixel convolution layer includes a convolution layer, two pixel masking layers and a LeakyRelu activation function, wherein the convolution layer: k=3, n=256, s=1, where k is the convolution kernel size, n is the number of feature maps, and s is the step size.

[0096] The discriminator includes eight convolutional activation blocks connected in series, each of which includes a convolutional layer of a 3×3 filter and a LeakyRelu activation function. After the eight convolutional activation blocks, the number of feature maps increases from 64 to 512. After the last convolutional activation block, a dense layer, a LeakyRelu activation function, a dense layer, and a softmax activation function are connected in sequence. After the graph of the 512 feature maps obtained, two dense layers and a final softmax activation function are passed to obtain the probability of sample classification.

[0097] Step 5: Train the improved balanced adversarial network

[0098] First, the autoencoder is trained using the autoencoder dataset, and then the trained autoencoder is used to initialize the adversarial module, which can avoid mode collapse. The training of the autoencoder extracts features through supervised learning, using the autoencoder dataset with filament annotations, without category labels, and unconditionally processing all images of the majority and minority classes.

[0099] The L2 loss minimization function is used as a constraint throughout the autoencoder training. The form of the L2 loss function is:

[0100]

[0101] Where n is the number of samples, y i is the true pixel value of the i-th sample, y i is the pixel prediction value of the i-th sample. This loss function calculates the square of the difference between the predicted value and the true value of each sample and averages the squared difference of all samples. This completes the training of the autoencoder.

[0102] After the autoencoder is trained, it is initialized. During initialization, the input is processed by the autoencoder and connected to the class conditional latent vector generator. Then it is divided into two branches, which are connected to the generator G and the discriminator D respectively. The trained autoencoder is used to initialize the adversarial module. At this time, the input image is an image with a classification label. During initialization, the encoder parameters of the autoencoder are assigned to the discriminator, and the decoder parameters of the autoencoder are assigned to the generator.

[0103] After initialization, the autoencoder is removed, and the images in the labeled dataset are used to train the improved balanced adversarial network. The category label c is input into the class-conditional latent vector generator to generate a latent vector, which is then processed by the generator G and input into the discriminator together with the original data. When training the discriminator D, the optimized sparse classification cross entropy loss function is used to match the class labels of the real images and the false labels of the generated images; for each batch learned by the discriminator, the generator G learns a batch of the same size; for this purpose, a uniform distribution is imposed on the label c through the class-conditional latent vector generator (obtained by training with sample classification probabilities), and a batch of conditional latent vectors Z is randomly drawn. c These vectors are processed by the generator G, and the output image is fed into the discriminator; the label selected by the discriminator matches the label c used to generate the image, after being constrained by the sparse categorical cross entropy loss function, where the sparse categorical cross entropy loss function is:

[0104]

[0105] Where n is the number of samples, C is the number of categories, and y ijis the judgment value of whether the category j of sample i is true. If it is true, the value is 1, otherwise it is 0. ij is the probability predicted by the model that sample i belongs to category j.

[0106] The summation in the sparse categorical cross entropy loss function is performed over all categories for each example, but only y ij Only items that are 1 are counted as losses. This calculation method avoids considering the entire probability distribution and only focuses on the predicted probability corresponding to the true label, thereby effectively reducing the amount of calculation and memory usage while ensuring prediction accuracy in the case of multiple categories and sparse category labels.

[0107] After 30,000 iterations, the training model reached convergence, that is, when the model training gradient was close to 0 (less than 0.01 can be considered close to 0), the training was stopped.

[0108] The trained generator is then used to generate 12,000 enhanced images with a resolution of 600*600 to form enhanced data, each of which is labeled.

[0109] Step 6: Combine the labeled dataset and the enhanced data with labels to form a classification dataset R. Use the classification dataset to train the ResNet-18 network to identify the type of 3D concrete filaments and obtain the quality classification of 3D printed concrete.

[0110] The classification data set R is divided into training set and test set in a ratio of 7:3 for training and verification of ResNet-18 network. After 40,000 iterations, the training model reaches convergence, that is, when the model training gradient is close to 0 (less than 0.01 can be considered close to 0), the training is stopped and the optimal network parameters are extracted for prediction. If the training gradient is not close to 0, the weight parameters are adjusted.

[0111] Example 2

[0112] This embodiment implements a quality classification system for 3D printed concrete based on a balanced adversarial network, including:

[0113] The image collection module uses an industrial camera to collect images of the printing area. The camera continuously collects RGB channel images of the printed filament and saves them in real time to the computer for detection and adjustment. During the whole process, the camera is fixed in the same position so that the lens and the 3D printed concrete filament are kept at a distance of 20 mm (focal length). A stable ring light source is fixed on the camera to ensure consistent lighting in the detection area containing 3D printed concrete. The image size captured by the camera is 1280×1024 pixels, and the maximum sampling frequency is 9 frames / second. The RGB image content consists of 3D printed concrete in 9 different states.

[0114] The image processing module captures and saves the image to the computer, and then cuts out a small image of 600×600 pixels from the initial image. In order to ensure that the photos collected under different lighting environments and material reflection characteristics can be accurately identified, the brightness is normalized using the Gamma correction method, as shown in the formula

[0115]

[0116] exist Where k = 1, 2, ..., is the image number, m = "R", "G", "B" is the channel name; i, j = 1, 2, ..., 600, represents the pixel position; k is the normalized image number, m = "R", "G", "B" are the channel names, i, j = 1, 2, ..., 600, indicating the pixel position of the image after brightness normalization, and γ is the normalization parameter, which is generally selected as 2.2.

[0117] Then use the following formula to make the chromaticity of the input images of different batches of materials evenly distributed:

[0118]

[0119] in k is the number of the final processed image, m = "R", "G", "B" are the channel names, i, j = 1, 2, ..., 600, representing the pixel position of the image after chromaticity uniformity processing, β is the constant value of the chromaticity normalization process, E ij After normalization The average pixel expectation.

[0120] 3000 original images of 3D printed concrete filaments were collected and labeled as training sets, classified by width targets and randomly selected, and the distribution of the number of samples in the 9 categories was approximately equal to the distribution of the number of samples in the entire dataset.

[0121] The image augmentation module is used to enhance the label data set of the image processing module using an intelligent network to enrich the number of samples;

[0122] The image classification module connects the image amplification module and the image processing module, and uses the classification data set composed of the label data set and the enhanced data to train the ResNet-18 network, and then distinguish the type of 3D concrete filaments to obtain the quality classification of 3D printed concrete.

[0123] The images are classified and sent to the background system, waiting for the monitoring personnel to control.

[0124] The ResNet-18 neural network is as follows Figure 7As shown in the figure, the data is input into the network, and first passes through a convolution layer with a convolution kernel size of 7*7, a stride of 2, a padding of 3, and an output channel of 64. Then it passes through a maximum pooling layer with a convolution kernel size of 3*3, a stride of 2, and a padding of 1. Then it passes through four 3*3 convolution layers with a convolution kernel size of 3*3, a stride of 1, and a padding of 1. This layer does not change the size of the data and the number of channels.

[0125] After that, when passing through the 5th 3*3 convolution layer, it first passes through a 1*1 convolution layer and undergoes a downsampling. The output channel is doubled and the output data size is halved. After that, the 9th 3*3 convolution layer also performs 1*1 convolution and downsampling. That is, the output channel is doubled and the output data size is halved. The 13th 3*3 convolution layer is the same as above, that is, the output channel is doubled and the output data size is halved. Finally, it passes through an average pooling layer and a linear layer to output the classified image.

[0126] Example 3

[0127] The hardware devices used in the system of this embodiment include the following components:

[0128] Processor: As the core component of the present invention, the processor is responsible for controlling and managing the operation of the entire system, including data acquisition, data processing, image recognition and other functions, and needs to have sufficient computing power and parallel processing capabilities to meet real-time requirements. The processor can be in different forms such as single-chip microcomputers, microprocessors, computers, etc. to meet the needs of different application scenarios;

[0129] Memory: Memory can be used to store collected data and historical data for subsequent processing and analysis. It has the characteristics of high speed, high reliability and scalability to meet the needs of long-term stable operation of the system;

[0130] Database: Use database to store and manage collected data, historical data, analysis results and other information;

[0131] Network interface: used for data exchange and communication, with the characteristics of high speed, high stability and high security, ensuring the reliability and security of data transmission.

[0132] The processor is configured to execute computer executable instructions. When the computer executable instructions are executed by the processor, the various steps of the quality classification method of 3D printed concrete based on the balanced adversarial network are implemented.

[0133] A computer program is stored in the memory, and the computer program can be executed by the processor to implement each step of the quality classification method of 3D printed concrete based on a balanced adversarial network.

[0134] The database is configured to store and manage data of a computer application, including various data types and structures, applied to various steps of a quality classification method for 3D printed concrete based on a balanced adversarial network.

[0135] The network interface realizes communication and data transmission between computers. The network interface can provide various communication protocols and data transmission methods to meet the communication and data transmission needs of different application scenarios and different requirements.

[0136] The present invention is mainly used for 3D printed concrete quality classification. It uses an automatic discrimination system to automatically identify 3D printed concrete filament images and judge the condition of 3D printed concrete.

[0137] Example 4

[0138] The quality classification method of 3D printed concrete based on the balanced adversarial network in this embodiment includes an improved balanced adversarial network and a ResNet-18 network;

[0139] The image of 3D printed concrete filament is used as the input of the improved balanced adversarial network. The input image first passes through the encoder network, through 8 convolutional layers with increasing numbers of 3×3 filter kernels, which increase from 64 feature maps to 512 feature maps by 2 times. Cross-row convolution is used to reduce the image resolution every time the number of features doubles. After the 512 feature maps are obtained, two dense layers and a final softmax activation function are passed to obtain the probability of sample classification, which is then encoded into a latent vector z. After that, the decoder network passes through a residual structure with the same layout, where the residual structure consists of two convolutional layers with 3×3 kernels and 64 feature maps, and a LeakyRelu activation function, and then two trained sub-pixel convolutional layers are used to increase the resolution of the input image. Finally, the information of the autoencoder is passed to the adversarial module.

[0140] In the adversarial module, the generator and discriminator modules first use the encoder part of the autoencoder to infer the distribution of different categories in the latent space. The decoding stage Δ of the autoencoder matches the topological structure of the generator G.

[0141] The generator consists of five residual structures with the same layout, where the residual structure consists of two convolutional layers with 3×3 kernels and 64 feature maps, and the LeakyRelu activation function and the element-by-element summation. Two trained sub-pixel convolutional layers are used to improve the resolution of the input image, and finally a convolutional layer with 9 convolution kernels, 3 feature maps and a stride of 1 is used to output the image.

[0142] The residual structure is as follows Figure 1As shown in the figure, it consists of two convolutional layers, a LeakyRelu activation function and an element-by-element summation function, with a residual structure of: k=3, n=64, s=1, where k is the convolution kernel size, n is the number of feature maps, and s is the step size.

[0143] The sub-pixel convolution layer is as follows Figure 5 As shown, it consists of a convolution layer, two pixel masking layers and a LeakyRelu activation function, sub-pixel convolution layer: k = 3, n = 256, s = 1.

[0144] In the discriminator, the encoding stage E of the autoencoder is matched with the convolutional activation block of the discriminator D, which increases by 2 times from 64 feature maps to 512 feature maps through 8 convolutional layers with increasing numbers of 3×3 filter kernels. Cross-row convolution is used to reduce the image resolution every time the number of features doubles. After the 512 feature maps obtained, two dense layers and a final softmax activation function are passed to obtain the probability of sample classification.

[0145] The convolutional activation block is as follows Figure 4 As shown in Figure 1, it consists of a convolutional layer, a LeakyRelu activation function, k=3, s=1, and the number of feature maps increases from 64 to 512.

[0146] By initializing the discriminator D as the weights of the encoder E, and finally passing the last layer of the discriminator D through a dense layer with a softmax activation function, the final discriminator output data-enhanced image is generated.

[0147] The image is input into the ResNet-18 neural network. Figure 7 As shown in the figure, there are a total of sixteen 3*3 convolutional layers connected in sequence. When the data is input into the network, it first passes through a convolutional layer with a convolution kernel size of 7*7, a step size of 2, a padding of 3, and an output channel of 64, and then passes through a maximum pooling layer. The convolution kernel size of this layer is 3*3, the step size is 2, and the padding is 1.

[0148] Then it passes through 4 3*3 convolution layers, the convolution kernel size is 3*3, the step size is 1, and the padding is 1. The 4 3*3 convolution layers do not change the size of the data and the number of channels.

[0149] After that, when passing through the 5th 3*3 convolution layer, it first passes through a 1*1 convolution layer and undergoes a downsampling. The output channel is doubled and the output data size is halved. After that, the 9th 3*3 convolution layer also performs 1*1 convolution and downsampling. That is, the output channel is doubled and the output data size is halved. The 13th 3*3 convolution layer is the same as above, that is, the output channel is doubled and the output data size is halved. Finally, after an average pooling layer and a linear layer, the classified image is output.

[0150] The present invention first utilizes an improved balanced adversarial network to expand and enhance the autoencoding data set to generate more and higher quality images.

[0151] Balanced adversarial networks are a type of generative adversarial networks designed specifically for imbalanced datasets. Artificially enhancing existing data can increase data diversity and reduce the risk of overfitting. Common data augmentation methods include rotation, flipping, and scaling. These methods can increase the number and diversity of training samples while maintaining data annotation consistency.

[0152] When image datasets are imbalanced, the accuracy of deep learning classifiers can drop significantly. Balanced adversarial networks restore the balance of datasets by generating new minority class images. This approach leverages the generative power of adversarial networks combined with the encoding advantages of autoencoders to achieve effective enhancement of minority class images. An autoencoder consists of two parts, an encoder and a decoder, which can map input data to a low-dimensional latent space and then reconstruct the original data from the latent space. In balanced adversarial networks, autoencoders are not only used for data compression and reconstruction, but also initialize the generator in the adversary through its encoder module. This initialization strategy enables balanced adversarial networks to learn accurate class conditions in the latent space, which helps control the generation process towards the desired category.

[0153] In summary, autoencoders play a vital role in balanced adversarial training, and effectively cope with the challenges of unbalanced datasets through their unique encoding and initialization functions. This not only improves the training stability of adversarial training, but also enhances the model's ability to generate various categories.

[0154] The quality classification method of the present invention is based on the training and testing of the improved balanced adversarial network, the number of parameters is reduced, and the accuracy is improved. The performance test of different models is carried out using the autoencoder data set of the present invention (the number of pictures is 3000), and the test results are shown in the following table:

[0155]

[0156] Among them, params represents the number of parameters, Accuracy represents the accuracy, FLOPs represents floating-point operations, and evaluation indicators such as mF1 (the average F1 score combines precision and recall using the harmonic mean calculation, which is particularly suitable for processing unbalanced data sets.) Compared with existing common models, the present invention not only improves the accuracy, but also reduces the number of parameters, achieving dual optimization of accuracy and recognition speed.

[0157] The present invention combines the balanced adversarial network and the ResNet-18 network to include data of all categories in the adversarial training process, learn features from the majority class and generate images of the minority class, thereby balancing the data set. This method is particularly suitable for generating data of categories with a small number of samples, effectively improving classification accuracy and solving the problem of unbalanced data sets.

[0158] In the actual implementation stage, the industrial camera acquires the image of the 3D printed concrete extruded filaments; the image of the 3D printed concrete filaments to be detected is preprocessed and input into the trained ResNet-18 network, which outputs the width category and fluidity category of the 3D printed concrete. The operator adjusts the ratio of the machine or material according to the required 3D printed concrete category to control the width and fluidity of the printed concrete.

[0159] By enhancing and classifying the data of 3D printed concrete filament images, the present invention realizes the identification of instability, especially the problem of inaccurate identification of specific concrete categories caused by instability caused by changes in the fluidity of 3D printed concrete materials. In addition, the present invention also allows the specific category of 3D printed concrete to be adjusted according to the needs of the control personnel.

[0160] The present invention introduces autoencoders and class-conditional latent spaces, successfully solving the classification problem of unbalanced data sets and significantly improving classification accuracy and data processing efficiency. This not only provides a powerful data enhancement tool for the field of 3D printing concrete, but also opens up new solutions for other application fields involving unbalanced data.

[0161] Compared with the existing technology, this technical solution has the following advantages and application prospects: it improves the efficiency of 3D printed concrete quality identification, strengthens the quality control of concrete in the 3D printing process, and provides the possibility of pursuing faster image identification. This is of great significance for 3D printed concrete filament image identification and has broad application prospects. In the future, we can further explore the application of balanced adversarial networks in other fields, such as medical image analysis, traffic sign recognition, etc., to make full use of its superior performance in unbalanced data processing.

[0162] In the description of this specification, the described specific features, structures, or characteristics may be combined in an appropriate manner in any one or more embodiments or examples.

[0163] It should be understood that each part of the present invention can be implemented by hardware, software or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or hardware stored in a memory and executed by a suitable instruction execution system.

[0164] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0165] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A quality classification method for 3D printed concrete based on balanced adversarial network, characterized in that: The steps of the quality classification method are: Step 1: Obtain original images of 3D concrete filaments with different printing widths and different fluidity, and ensure that the illumination of the detection area containing 3D printed concrete is consistent during acquisition; Step 2: Image preprocessing: The original image obtained in step 1 is cropped into a small image of 600×600 pixels containing the filaments, and the filament area is marked. The size of the original image is larger than 600×600 pixels; then the brightness is normalized using the Gamma correction method. in, is the pixel at the i-th row and j-th column of the m-th channel in the k-th image, k = 1, 2, ..., is the image number, m = "R", "G", "B" is the channel name; i, j = 1, 2, ..., 600, indicates the pixel position; is the normalized pixel; γ is the brightness normalization parameter; Then use the following formula to make the chromaticity of the input images of different batches of materials evenly distributed: in, is the pixel after chromaticity uniformity processing; β is the constant value of the chromaticity normalization process, E ij The data is normalized to brightness The average pixel expectation of So far, the self-encoding image dataset is obtained; Step 3: Labeling: Label the images by width and fluidity to obtain a collection of images with classification labels, which is defined as a label dataset; Step 4: Build an improved balanced adversarial network The improved balanced adversarial network includes an autoencoder and an adversarial module, the adversarial module includes a generator and a discriminator, the autoencoder includes an encoder E and a decoder Δ, the encoder of the autoencoder is used to infer the distribution of different categories in the latent space, the decoder Δ matches the topological structure of the generator G, and the encoder E matches the structure of the discriminator D; Step 5: Train the improved balanced adversarial network First, the autoencoder dataset is used to train the autoencoder, and then the trained autoencoder is used to initialize the adversarial module. During initialization, the input is processed by the autoencoder and connected to the class conditional latent vector generator, and then divided into two branches, connecting the generator G and the discriminator D respectively; during initialization, the input image is an image with a classification label. During initialization, the encoder parameters of the autoencoder are assigned to the discriminator, and the decoder parameters of the autoencoder are assigned to the generator; After initialization, the autoencoder is removed, and the images in the labeled dataset are used to train the improved balanced adversarial network. The category label c is input into the class conditional latent vector generator to generate a latent vector, which is then processed by the generator G and input into the discriminator D together with the original data. When training the discriminator D, the optimized sparse classification cross entropy loss function is used to match the class label of the real image and the false label of the generated image; after the sparse classification cross entropy loss function constraint, the label selected by the discriminator matches the label c used to generate the image, where the sparse classification cross entropy loss function is: Where n is the number of samples; C is the number of categories; y ij is the judgment value of whether the category j of sample i is true. If it is true, the value is 1, otherwise it is 0; p ij is the probability that sample i belongs to category j predicted by the model; When the training model reaches convergence, stop training; The trained generator is then used to generate the target number of enhanced images with a resolution of 600*600 to form enhanced data, each of which is labeled. Step 6: Combine the labeled dataset and the enhanced data with labels to form a classification dataset, and use the classification dataset to train the ResNet-18 network to identify the type of 3D concrete filaments to obtain the quality classification of 3D printed concrete.

2. The method according to claim 1, characterized in that The encoder E includes eight sequentially connected convolutional activation blocks, two dense layers and a softmax activation function, each convolutional activation block includes a convolutional layer of 3×3 filters and a LeakyRelu activation function, the input image passes through 8 convolutional layers of 3×3 filters, the number of feature maps of the 8 convolutional layers of 3×3 filters increases from 64 to 512, and after the last convolutional activation block, a dense layer, a LeakyRelu activation function, a dense layer, a softmax activation function, and a softmax activation function are sequentially connected to obtain the probability of sample classification, and then encoded into a potential vector z; The decoder Δ includes a convolution activation block with 64 feature maps, five residual structures connected in series, each residual structure includes two 3×3 kernel convolution layers, a LeakyRelu activation function and an element-by-element summation operation, and the number of feature maps of each 3×3 kernel convolution layer is 64. After a 3×3 kernel convolution layer is processed by the LeakyRelu activation function and another 3×3 kernel convolution layer, an element-by-element summation operation is performed with the input of the residual structure to obtain an output; the output of the last residual structure is convolutionally processed and element-by-element summation is performed with the input of the first residual structure, and then processed by two sub-pixel convolution layers and enters the convolution layer to obtain the output of the decoder.

3. The method according to claim 2, characterized in that The sub-pixel convolution layer includes a convolution layer, two pixel masking layers and a LeakyRelu activation function, wherein the convolution layer: k=3, n=256, s=1, where k is the convolution kernel size, n is the number of feature maps, and s is the step size.

4. The method according to claim 2, characterized in that: The autoencoder is trained using an L2 loss function for constraint; the convolution kernel of the last convolution layer of the decoder is 9, the number of feature maps is 3, and the step size is 1; The convolution kernel of the last residual structure connection in the decoder is 9, the number of feature maps is 64, and the stride is 1.

5. The method according to claim 1, characterized in that The printing filament is set to three width standards: 35mm, 30mm and 25mm. The width evaluation level consists of three levels, including "wide", "moderate" and "thin". Based on the three width targets, a different width evaluation level is assigned to each image. Extreme samples with a difference of >10.0mm between the actual width and the target width are excluded from training. The fluidity of 3D printed concrete ranges from 160-220mm, forming a total of 9 different label categories, specifically: Category label 1 indicates that the fluidity is 160mm-180mm dry and belongs to the target width 25mm width assessment level thin; Category label 2 means that the fluidity is 160mm-180mm dry and the target width is 30mm, and the width assessment level is medium; Category label 3 means that the fluidity is 160mm-180mm dry and the target width is 35mm, and the width assessment level is wide; Category label 4 means that the fluidity is 181mm-200mm moderate and the target width is 25mm, and the width assessment level is thin; Category label 5 means that the fluidity is moderate between 181mm and 200mm and the target width is 30mm, and the width assessment level is moderate; Category label 6 means that the fluidity is moderate between 181mm and 200mm and the target width is 35mm, and the width assessment level is wide; Category label 7 indicates that the fluidity is 201mm-220mm fluid and belongs to the target width 25mm width assessment level thin; Category label 8 indicates that the fluidity is 201mm-220mm and the target width is 30mm, which is a medium width assessment grade. Category label 9 indicates that the fluidity is 201mm-220mm and the target width is 35mm, which is a wide width assessment grade.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of any method described in claims 1-5 can be implemented.

7. A quality classification system for 3D printed concrete based on a balanced adversarial network, characterized in that: The system executes the method of claim 1, comprising: The image collection module uses an industrial camera to collect images of the printing area. The camera continuously collects RGB channel images of the printed filament and saves them in real time to the computer for detection and adjustment; Image processing module, used to process the original image and obtain the autoencoder dataset and label dataset; The image augmentation module is used to enhance the label data set of the image processing module using an intelligent network to enrich the number of samples; The image classification module connects the image amplification module and the image processing module, and uses the classification data set composed of the label data set and the enhanced data to train the ResNet-18 network, and then distinguish the type of 3D concrete filaments to obtain the quality classification of 3D printed concrete.

Images