3D printed bridge prefabricated box girder construction quality monitoring system based on deep learning

Abstract

The present invention is a 3D printing bridge prefabricated box girder construction quality monitoring system based on deep learning, including: a data (image) collection and processing module, which collects and processes images of the printing area through an industrial camera; a data annotation and model training module, which is connected to the data (image) collection and processing module to construct a convergence model for target detection and measurement; an information calibration and storage module, which is connected to the data annotation and model training module, inputs the test set into the trained target detection algorithm, uses the calibration algorithm to correct the conversion ratio M in the conversion function module, updates the conversion function module with the calibrated conversion ratio M', and obtains the calibrated convergence model; a defect detection and analysis module; and an early warning and prompt processing module. Not only can it reduce labor costs, improve printing efficiency, and ensure the quality of printed products, but it can also make it possible to pursue 3D printing at a larger scale and architectural level.

Description

Technical Field

[0001] The present invention relates to the fields of intelligent construction and computer vision technology, and in particular to a system and method for monitoring the construction quality of a 3D printed bridge prefabricated box girder based on deep learning. Background Art

[0002] Box girders are usually constructed by artificial construction. During the construction process, factors such as concrete pouring, maintenance, and external loads need to be considered. The construction is difficult and the construction period is long. There are also certain safety risks. In addition, due to the special properties of the box girder structure, its strength is relatively slow to form, and it takes a long time to reach the strength required by the design, which also limits the application scope and service life of the box girder. Therefore, it is crucial to build high-quality box girders in a shorter time.

[0003] 3D printing technology is a new type of construction technology that adapts to digitalization and intelligence. Its advantages lie in high degree of automation, fast construction speed, high material utilization, ability to realize special-shaped construction, and saving template costs. At present, the application of 3D printing technology in the construction field is mainly aimed at concrete and other cement-based materials, which is also very consistent with the development concept of the intelligent construction field.

[0004] At present, 3D printing technology is heavily dependent on manual monitoring, which consumes a lot of manpower and economic costs. In addition, manual monitoring cannot reach places above the height of human vision, and it is impossible to monitor the quality of the box girder structure of the bridge during the construction process. The unstable quality of the manufactured products poses a structural safety risk, which limits the development and application of 3D printing in large-scale production. In practical applications, it is currently impossible to provide solutions for automation and intelligence in this field. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the above technologies and provide an intelligent deep learning-based 3D printed bridge prefabricated box girder construction quality monitoring system and method with automatic correction.

[0006] Compared with the existing technology, the system of the present invention creatively uses computer vision technology to realize the quality monitoring of the concrete 3D printing process, which can accurately and efficiently detect defects in the printing process, realize real-time monitoring and early warning, and provide an effective means for ensuring printing quality.

[0007] In order to solve the above technical problems, the technical solution of the present invention is:

[0008] In a first aspect, the present invention provides a method for monitoring the construction quality of a 3D printed bridge prefabricated box girder based on deep learning, wherein the steps of the monitoring method are:

[0009] Obtain defect images of the printing area during the construction of the prefabricated box girder of the 3D printed bridge;

[0010] The defect images are preprocessed and labeled with defect types, and the actual size of the actual frame of the labeled defect target in reality and the pixel size in the picture are measured; the labeled defect images are divided into a training set and a test set;

[0011] Constructing a convergence model for target detection and measurement, the convergence model includes a target detection algorithm and a conversion function module,

[0012] The target detection algorithm is used to obtain the pixel size of the prediction box in the defect image, and the conversion function module is used to obtain the predicted actual size of the defect according to the pixel size of the prediction box using the conversion ratio M between the actual size of the actual box and the pixel size;

[0013] The target detection algorithm is trained using the training set, and the output is the pixel size of the prediction box. The predicted actual size of the defect is obtained through the conversion function module;

[0014] Input the test set into the trained object detection algorithm to obtain the pixel width of the test set prediction box.

[0015] The conversion ratio M in the transfer function module is corrected using a calibration algorithm, and the specific steps of the calibration algorithm are:

[0016] 1) Use the following error function f(x) to measure the difference between the predicted box and the actual box:

[0017]

[0018] where p i is the pixel width of the i-th prediction box, is the actual width of the actual box of the i-th defect target, x is the conversion ratio calibration coefficient that needs to be optimized, and N is the total number of predicted boxes in all samples in the test set;

[0019] 2) Take the average value of the actual width of all samples in the test set - the average value of the pixel width of the predicted box divided by the average value of the pixel width of the predicted box as the initial conversion ratio calibration coefficient x 0 ;

[0020] 3) Calculate the first and second order derivatives of the error function f(x) and obtain:

[0021]

[0022]

[0023] where f'(x) and f”(x) are the first and second derivatives of f(x), respectively, and sgn(·) is the sign function;

[0024] 4) Construct the iteration formula as follows:

[0025]

[0026] Among them, x k is the k-th iteration conversion ratio calibration coefficient, x k+1 is the conversion ratio calibration coefficient for the k+1th iteration;

[0027] According to the iterative formula, the conversion ratio calibration coefficient for the next iteration is obtained and substituted into the error function f(x k+1 ) expression, we can obtain the error function f(x k+1 ), and the optimal calibration coefficient X is obtained through iteration;

[0028] 5) Obtaining a calibrated conversion ratio M' according to M'=M*X, updating the conversion function module with the calibrated conversion ratio to obtain a calibrated convergence model;

[0029] The video image to be identified is input into the calibrated convergence model to detect defect targets, and the actual size of each defect is predicted. The predicted actual size is compared with the corresponding threshold to determine whether to alarm, and the printing process is adjusted according to the alarm information; until the printing is completed and all areas meet the requirements, the behavior monitoring during the construction process of the 3D printed bridge prefabricated box girder is completed.

[0030] The target detection algorithm adopts an improved YOLOv7 neural network, the improved YOLOv7 neural network adopts an LN layer, and a DSiLU activation function is used in the feedforward neural network; a CBS is reduced in the ELAN and SPPCSPC parts respectively. 1 layer.

[0031] The target detection algorithm uses an improved YOLOv7 neural network and trains the model for 300 epochs to reach convergence; when the mean average accuracy (mAP) reaches 94%, a calibrated converged model is obtained;

[0032] The improved YOLOv7 deep neural network includes a backbone network Backbone and a Head network. The input image is input into the backbone network Backbone. In the backbone network Backbone, after passing through CBS1, CBS2, CBS1, CBS2, ELAN, MP-1, ELAN, MP-1, ELAN, MP-1, ELAN, and then outputting SPPCSPC connected to the Head network. The SPPCSPC outputs result A through a CBS3 and UPsample. The output of the second-to-last ELAN is concatenated with result A through a CBS3 at the same time. The concatenated result is concatenated with the first ELAN-W, CBS3, and UPsample to output result B. The output of the second ELAN is concatenated with result B through a CBS3, and the concatenated result is concatenated with MP-2 and REP through the second ELAN-W.

[0033] The output of MP-2 is spliced ​​with the output of result A after the first ELAN-W processing, the spliced ​​output is processed by the third ELAN-W, then spliced ​​by another MP-2 and the output of SPPCSPC, and finally processed by the fourth ELAN-W;

[0034] The outputs of the second to fourth ELAN-W mentioned above are all connected to the CBM block through REP, and the outputs of the three CBM blocks are connected to each other to output a large pixel feature map as the output image of the final network.

[0035] The structures of CBS1, CBS2 and CBS3 are the same, each including a convolutional layer, an LN layer and a DSiLU function. 1 :k=3,s=1;CBS 2 :k=3,s=2;CBS 3 : k = 1, s = 1, where k is the convolution kernel size and s is the step size;

[0036] REP consists of a convolutional layer and an LN layer connected in series to form a branch. Two such branches are connected in parallel and then connected in parallel with a branch formed by an LN layer. The outputs of the three branches are added to obtain the output of REP.

[0037] MP-1 and MP-2 have the same structure, both of which include two branches. One branch is composed of the maximum pooling layer and CBS3, and the other branch is CBS 3 , CBS 2 The outputs of the two branches are concatenated to obtain the output of MP-1 or MP-2, where the number of hidden layers of MP-1 is i / 2, and the number of hidden layers of MP-2 is i, where i represents the number of input nodes;

[0038] The structure of SPPCSPC is: data passes through CBS once 3 , CBS 1 , CBS 3 After the module, it is processed by three maximum pooling processes. The results of the three maximum pooling processes are the same as the last CBS 3 The output is spliced ​​and then passes through a CBS 3 Then with the input of SPPCSPC through a CBS 3 The processed output is directly spliced ​​and finally passed through a CBS 3 Then get the output of SPPCSPC;

[0039] The input of ELAN goes through a max pooling layer and CBS 3 Output result E, ELAN input passes through CBS in sequence at the same time 3 , CBS 1 , CBS 1 , CBS 1 Output the result F, and then compare it with the result F that only passes through CBS 3 , CBS 3 With two CBS 1 Get the results G and H, then concatenate E, F, G, and H together, and finally pass CBS 3 Get the output of ELAN;

[0040] The structure of ELAN-W is similar to that of ELAN. The difference is that there are four CBS1 in the F branch. The input of ELAN passes through CBS1 in sequence. 3 , CBS 1 , CBS 1 , CBS 1 , CBS 1 Output the result F, and then compare it with the result F that only passes through CBS 3 , CBS 3 With CBS 1 , CBS 3 With two CBS 1 , CBS 3 With three CBS 1 The results G, H, I, and J are obtained, and then E, F, G, H, I, and J are spliced ​​together, and finally the output of ELAN is obtained through CBS3;

[0041] The CBM block consists of a convolutional layer, a LN layer, and a sigmoid function.

[0042] The defect images include a broken defect image, a crack defect image, a bulge image, and a collapse image.

[0043] Set the measurement parameter threshold to 5% of the crack prediction frame area to the lens detection area. If it exceeds 5%, an alarm will be triggered and the printing process will be adjusted according to the alarm information. If it does not exceed the threshold or the image is not identified as a defect, the material will be printed as qualified. If it is qualified, no alarm will be triggered and no mark will be made.

[0044] The crack defect information area ratio or defect type is transmitted to the prompt processing module; the processing module receives the defect information transmitted by the early warning module, wherein the defect information is the defect type, and if the defect type is crack, the area ratio including the crack; then the defect information is displayed and fed back to the back-end system, and the printing of the material is stopped; the back-end system automatically slows down the printing speed and increases the pumping rate and checks the material status;

[0045] In a second aspect, the present invention provides a 3D printed bridge prefabricated box girder construction quality monitoring system based on deep learning, comprising:

[0046] The data (image) collection and processing module collects images of the printing area through an industrial camera, obtains multiple frames of images including defect information, and pre-processes the images;

[0047] A data annotation and model training module is connected to the data (image) collection and processing module, annotates the defect type of the preprocessed defect image, measures the actual size of the actual frame of the annotated defect target in reality and the pixel size in the picture, and constructs a convergence model for target detection and measurement. The convergence model includes a target detection algorithm and a conversion function module. The conversion function module is used to obtain the predicted actual size of the defect according to the pixel size of the predicted frame using the conversion ratio M of the actual size of the actual frame and the pixel size;

[0048] An information calibration and storage module is connected to the data annotation and model training module, inputs the test set into the trained target detection algorithm, obtains the pixel width of the prediction box, uses the calibration algorithm to correct the conversion ratio M in the conversion function module, updates the conversion function module with the calibrated conversion ratio M', obtains the calibrated convergence model, and stores the corresponding monitoring information;

[0049] A defect detection and analysis module is connected to the information calibration and storage module, detects defect targets in the input data through the calibrated convergence model, obtains the predicted actual size of each defect, and sets a threshold to analyze the results;

[0050] The early warning and prompt processing module is connected to the defect detection and analysis module. It is used to display the measurement and early warning results after the results exceed the early warning threshold, and promptly feedback the defect information to the background system. The background system automatically reduces the movement speed of the printer nozzle, increases the discharge speed, and prompts relevant personnel to check the material status.

[0051] In a third aspect, the present invention provides a method for monitoring the construction quality of a 3D printed bridge prefabricated box girder based on deep learning. The monitoring method is implemented using the system, and comprises the following steps:

[0052] 1) Using the data (image) collection and processing module, obtaining defect images of the printing area during the construction process of the 3D printed bridge prefabricated box girder;

[0053] 2) Annotate the defect type of the preprocessed defect image, and measure the actual size of the actual frame of the marked defect target in reality and the pixel size in the picture to form a data set. The data set is divided into a training set and a test set. The training set is used to train a convergence model for target detection and measurement. The convergence model includes a target detection algorithm and a conversion function module;

[0054] 3) Input the test set into the trained target detection algorithm to obtain the pixel width of the prediction box, and transmit the detection result to the information calibration and storage module for processing. Use the calibration algorithm to correct the conversion ratio M in the conversion function module, update the conversion function module with the calibrated conversion ratio M', obtain the calibrated convergence model, and store the corresponding monitoring information;

[0055] 4) Input the video image to be identified into the calibrated convergence model to detect defect targets, obtain the predicted actual size of each defect, and set a threshold to analyze the results;

[0056] 5) Determine whether to alarm based on the comparison between the predicted actual size and the corresponding set threshold, and adjust the printing process according to the alarm information; until all areas meet the requirements after printing, that is, no major defects appear and the crack area is less than the set threshold, the behavior monitoring during the construction process of the 3D printed bridge prefabricated box girder is completed.

[0057] The backend system used in the method includes the following components:

[0058] Processor: As the core component of the present invention, the processor is responsible for controlling and managing the operation of the entire system, including data acquisition, data processing, image recognition, alarm and other functions;

[0059] Memory: Memory can be used to store collected data and historical data for subsequent processing and analysis;

[0060] Database: The back-end system needs to use a database to store and manage collected data, historical data, analysis results and other information;

[0061] Network interface: The back-end system needs to use a network interface to exchange data and communicate with the front-end system and other external systems.

[0062] Compared with the existing technology, this system is highly customizable, and can adjust the printer nozzle movement speed and material discharge speed according to different printing materials and defect types, and provide feedback to the staff to adjust the wetness of the material. It has wide applicability and practicality. The scope of application of the present invention is wide, and it can be used for quality monitoring of concrete 3D printing process, and can also be used for quality monitoring in other 3D printing fields, such as metal 3D printing, medical 3D printing, aviation 3D and other fields. In practical applications, the present invention can provide a new idea and solution for quality monitoring of concrete 3D printing box girder process, and at the same time provide strong support and guarantee for the development of concrete 3D printing technology.

[0063] Based on the existing technical solutions, it can not only reduce labor costs, improve printing efficiency, and ensure the quality of printed products, but also make it possible to pursue larger-scale architectural-level 3D printing. Compared with existing technologies, it has great significance and good application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a structural schematic diagram of a 3D printed concrete bridge prefabricated box girder construction quality monitoring system based on deep learning according to an embodiment of the present invention.

[0065] Figure 2 It is a flow chart of the construction quality monitoring method of 3D printed concrete bridge prefabricated box girder based on deep learning of the present invention.

[0066] Figure 3 It is a schematic diagram of the conversion ratio between the monitoring lens pixels and the actual size of the present invention.

[0067] Figure 4 It is a monitoring flow chart of the construction quality monitoring system of a 3D printed concrete bridge prefabricated box girder based on deep learning of the present invention.

[0068] Figure 5 It is a schematic diagram of the structure of the improved YOLOv7 deep neural network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is described in detail below in conjunction with the accompanying drawings and embodiments. It should be noted that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0070] See also Figure 1 As shown, the 3D printing bridge prefabricated box girder construction quality monitoring system based on deep learning of the present invention includes:

[0071] The data (image) collection and processing module collects images of the printing area through an industrial camera, obtains multiple frames of images including defect information, and pre-processes the images;

[0072] The data labeling and model training module is connected to the data (image) collection and processing module, labels the defect type of the preprocessed defect image, measures the actual size of the actual frame of the labeled defect target in reality and the pixel size in all images, and constructs a convergence model for target detection and measurement. The convergence model includes a target detection algorithm and a conversion function module. The target detection algorithm is implemented using an improved YOLOv7 deep neural network;

[0073] An information calibration and storage module is connected to the data annotation and model training module, inputs the test set into the trained target detection algorithm, obtains the pixel width of the prediction box, uses the calibration algorithm to correct the conversion ratio M in the conversion function module, updates the conversion function module with the calibrated conversion ratio M', obtains the calibrated convergence model, and stores the corresponding monitoring information;

[0074] A defect detection and analysis module is connected to the information calibration and storage module, detects defect targets in the input data through the calibrated convergence model, obtains the predicted actual size of each defect, and sets a threshold to analyze the results;

[0075] The early warning and prompt processing module is connected to the defect detection and analysis module. It is used to display the measurement and early warning results after the results exceed the early warning threshold, and promptly feedback the defect information to the background system. The background system automatically reduces the movement speed of the printer nozzle, increases the discharge speed, and prompts relevant personnel to check the material status.

[0076] The background system is used to control the working state of the printer and receive the output results of the early warning and prompt processing module.

[0077] The method for monitoring the construction quality of 3D printed bridge prefabricated box beams based on deep learning in this invention is referred to Figure 2 As shown, the method comprises the following steps:

[0078] (1) using the data (image) collection and processing module to obtain defect images of the printing area during the construction process of the 3D printed bridge prefabricated box girder;

[0079] (2) annotating the defect type of the preprocessed defect image, measuring the actual size of the actual frame of the annotated defect target in reality and the pixel size in all images, and constructing a convergence model for target detection and measurement, wherein the convergence model includes a target detection algorithm and a conversion function module;

[0080] (3) Input the test set into the trained target detection algorithm to obtain the pixel width of the prediction box, and transmit the detection result to the information calibration and storage module for processing. Use the calibration algorithm to correct the conversion ratio M in the conversion function module, update the conversion function module with the calibrated conversion ratio M', obtain the calibrated convergence model, and store the corresponding monitoring information;

[0081] (4) Input the video image to be identified into the calibrated convergence model to detect defect targets, obtain the predicted actual size of each defect, and set a threshold to analyze the results;

[0082] (5) Determine whether to alarm based on the comparison between the predicted actual size and the corresponding threshold, and adjust the printing process according to the alarm information; until all areas meet the requirements after printing, that is, no major defects appear and the crack area is smaller than the set threshold, the behavior monitoring during the construction process of the 3D printed bridge prefabricated box girder is completed.

[0083] The defect images specifically include: broken defect images, crack defect images, bulge images, and collapse images during the construction process of the prefabricated box girder of the 3D printed concrete bridge.

[0084] The step (1) is specifically as follows:

[0085] (1.1) Using the data (image) collection and processing module, the defect images of the printing area during the construction of the 3D printed bridge prefabricated box girder are obtained, including the broken defect images, crack defect images, bulge images and collapse images;

[0086] (1.2) Convert the defect image into a standard format of 1280*1280 to obtain a standard format image;

[0087] (1.3) low-pass filtering the acquired standard format image using a Gaussian function to obtain a filtered image;

[0088] (1.4) Convert the acquired standard format image and the filtered image into the logarithmic domain and perform a difference (i.e., subtract the low-frequency components in the image) to obtain a reflection image in the logarithmic domain;

[0089] (1.5) Perform image summation on the pixel level on the logarithmic domain reflection image to obtain the result;

[0090] (1.6) Color restoration:

[0091] (1.6.1) At the channel level, for each channel, add the pixel values ​​of the channel in the original image to get the sum of the channel, calculate the maximum value of the sum of all channels, and use it as the denominator of the normalization factor. For the sum of each channel, divide it by the denominator of the normalization factor to get the normalization factor;

[0092] (1.6.2) Normalize the weight matrix (the weight matrix is ​​set manually according to different applications. Here, a uniform weight matrix is ​​used, that is, the weight of each channel is equal), convert it to the logarithmic domain, then multiply it by the nonlinear factor of color restoration (here taken as 2.0), and then divide it by the normalization factor to obtain the image color gain result.

[0093] (1.6.3) Recombine (multiply) the result obtained in (1.5) according to the weight matrix and the result of the image color gain to obtain the image result after color restoration;

[0094] (1.7) Image restoration: Multiply the color restored image result obtained in (1.6.3) by the gain of the image pixel value change range, and add the offset of the image pixel value change range to obtain the final result.

[0095] The gain of the image pixel value change range is a multiplication factor used to adjust the brightness and contrast of the image. It is applied to each pixel value of the image result after color restoration. The offset of the image pixel value change range is a constant value used to shift the image after the multiplication factor. It is added to each pixel value of the image result after color restoration. The final image restoration result is obtained by multiplying the multiplication factor by the image result after color restoration and adding the offset to the multiplication result.

[0096] The step (2) is specifically as follows:

[0097] (2.1) Annotate the areas that need to be identified in the preprocessed 3D printed concrete bridge prefabricated box girder images of broken defects, crack defects, bulges, and collapse;

[0098] (2.2) The annotated images are divided into training set and test set in a ratio of 7:3;

[0099] (2.3) Measure the actual size of the actual box of the defect target marked in the test set in reality and the pixel size in the picture, as the actual reference group;

[0100] (2.4) The labeled training set is read into the improved YOLOv7 deep neural network (target detection algorithm), and the target detection algorithm is trained for 300 epochs until the model converges. That is, when the model training gradient is close to 0, the training is stopped.

[0101] (2.5) Figure 3As shown, the convergence model has a built-in conversion function module, which is used to obtain the predicted actual size of the defect according to the pixel size of the predicted box using the conversion ratio M of the actual size of the actual box in reality and the pixel size in the picture. The camera of this application is fixed at a distance of 1 meter from the printing work surface;

[0102] The conversion formula in step (2.5) is:

[0103] 1) The actual size of the defect actual frame = the pixel size of the actual frame × M

[0104] 2)

[0105] a is the lens width pixel, which refers to the number of pixels in the width of the image captured by the camera lens; b is the lens length pixel, which refers to the number of pixels in the length of the image captured by the camera lens; L is the actual lens length, which refers to the actual length of the image; W is the actual lens width, which refers to the actual width of the image; M is the conversion ratio of the actual size to its corresponding pixel in the photo.

[0106] The step (3) is specifically as follows:

[0107] (3.1) Input the test set into the trained object detection algorithm to obtain the pixel width of the prediction box;

[0108] (3.2) Compare the pixel size of the actual detection target frame used for testing with the pixel width of the predicted frame, use the calibration algorithm to correct the conversion ratio M in the conversion function module, and update the conversion function module with the calibrated conversion ratio M'. When the mean average accuracy mAP reaches 94%, the calibrated converged model is obtained;

[0109] (3.3) Establish a storage module for transmission, storage and backup of results.

[0110] The specific steps of using the calibration algorithm in step (3.2) to correct the conversion ratio M in the transfer function module are:

[0111] 1) Use the following error function f(x) to measure the difference between the predicted box and the actual box:

[0112]

[0113] where p i is the pixel width of the i-th prediction box, is the actual width of the actual box of the i-th defect target, x is the conversion ratio calibration coefficient that needs to be optimized, and N is the total number of predicted boxes in all samples in the test set;

[0114] 2) Set the initial conversion ratio calibration factor x 0(Take the average of the actual widths of all samples in the test set - the average of the pixel widths of the predicted boxes divided by the average of the pixel widths of the predicted boxes);

[0115] 3) Use analytical methods to calculate the derivative of the error function f(x) and obtain:

[0116]

[0117]

[0118] Where f'(x), f"(x) are the first and second derivatives of f(x), respectively, and sgn(·) is the sign function. These derivatives are used to update the scaling calibration coefficients.

[0119] 4) Use the calibration algorithm to update the conversion ratio calibration coefficient. The iterative formula is:

[0120]

[0121] Among them, x k is the k-th iteration conversion ratio calibration coefficient, x k+1 is the conversion ratio calibration coefficient of the k+1th iteration; the conversion ratio calibration coefficient of the next iteration is obtained according to the iteration formula and is substituted into the error function f(x k+1 ) expression, we can obtain the error function f(x k+1 );

[0122] 5) Iterate the above formula several times until When it is close to 0, convergence is achieved and the optimal calibration coefficient X is obtained;

[0123] 6) The conversion ratio is corrected using the optimal calibration parameter X. The corrected conversion ratio is recorded as M', which is used to convert the predicted frame pixels to the size of the actual frame, that is, M'=M*X, and the actual size of the actual frame of the defect=the predicted frame pixel size×M'.

[0124] The step (4) is specifically as follows:

[0125] (4.1) The images of broken defects, crack defects, bulges and collapses in the prefabricated box girder concrete construction process to be identified are input into the improved YOLOv7 deep neural network, such as Figure 5 As shown;

[0126] (4.2) Using the calibrated convergence model, the model has a built-in conversion function module to obtain the length and width pixel values ​​of the prediction boxes of the broken defect image, crack defect image, bulge image and collapse image, and convert them to obtain the actual size range of the defect;

[0127] (4.3) The measurement parameter threshold is set to 5% of the ratio of the crack prediction frame area to the lens detection area, and the detected results are analyzed.

[0128] The parameter threshold in step (4.3), that is, the ratio of the crack prediction box area to the lens detection area is 5%. The value of 5% is the threshold obtained by training the model for 300 epochs. Fine-tune the anchor box (prior box) to form a prediction box that can accurately describe the crack location. The area of ​​this box is the crack prediction box area. The anchor box (prior box) divides the image into m*n areas, and then generates a series of boxes at the center of each grid according to the set aspect ratio.

[0129] The step (5) is specifically as follows:

[0130] (5.1) The system receives images for defect detection;

[0131] (5.2) If the image is identified as a crack defect image, check whether the ratio of the crack prediction box area to the lens detection area exceeds the threshold of 5%. If it does not exceed the threshold, print the material as qualified, and no alarm or mark will be made if it is qualified; if it exceeds the threshold, proceed to step (5.4);

[0132] (5.3) If the image is identified as a broken defect image, a collapsed defect image, or a bulging defect image, proceed to step (5.4);

[0133] (5.4) triggering the early warning module and transmitting the crack defect information area ratio or defect type to the prompt processing module;

[0134] (5.5) The processing module receives the defect information transmitted by the early warning module, wherein the defect information includes the defect type. If the defect type is a crack, it includes the crack area ratio. The defect information is then displayed and fed back to the back-end system to stop printing the material;

[0135] (5.6) The back-end system automatically slows down the printing speed and increases the pumping rate and checks the material status;

[0136] (5.7) If the image is not identified as a defect or the ratio of the crack prediction frame area to the lens detection area does not exceed the threshold, the material is deemed qualified and no alarm or marking is made.

[0137] If the early warning module is triggered in step (5.4), the system will follow the following sub-processes:

[0138] 1) The prompt processing module transmits the defect information to the back-end system;

[0139] 2) The back-end system analyzes the defect information, where the defect information is the defect type. If the defect type is a crack, it includes the crack area ratio, and determines the severity level of the defect. The steps for determining the severity level of the defect are as follows: if the level is identified as a crack defect image, then the ratio of the crack prediction frame area to the lens detection area exceeds 5%, which is a serious defect; if the ratio of the crack prediction frame area to the lens detection area does not exceed 5%, it is a defect that does not affect the structural function; if it is identified as a broken defect image, a collapsed defect image, or a bulging defect image, it is a serious defect;

[0140] 3) The back-end system automatically adjusts the printer parameters according to the severity of the defect and sends an alarm notification to relevant personnel;

[0141] 4) Relevant personnel conduct further inspections. If defects still occur after automatic parameter adjustment, appropriate measures are taken to resolve the defects, such as repairing or replacing equipment, adjusting the printing process, or replacing materials;

[0142] 5) The system will resume printing only after the defects are repaired and the back-end system material rheological parameters are adjusted, and the defects no longer occur during the printing process, where the printing process is a trial print of 1 meter in length.

[0143] The backend system includes the following components:

[0144] Processor: As the core component of the present invention, the processor is responsible for controlling and managing the operation of the entire system, including data acquisition, data processing, image recognition, alarm and other functions, and needs to have sufficient computing power and parallel processing capabilities to meet real-time requirements. The processor can be in different forms such as single-chip microcomputers, microprocessors, computers, etc. to meet the needs of different application scenarios;

[0145] Memory: Memory can be used to store collected data and historical data for subsequent processing and analysis. It has the characteristics of high speed, high reliability and scalability to meet the needs of long-term stable operation of the system;

[0146] Database: The back-end system needs to use a database to store and manage collected data, historical data, analysis results and other information;

[0147] Network interface: The back-end system needs to use a network interface to exchange and communicate data with the front-end system and other external systems. It has the characteristics of high speed, high stability and high security to ensure the reliability and security of data transmission.

[0148] The computer back-end system processor has the main feature that the processor is configured to execute computer executable instructions. When the computer executable instructions are executed by the processor, the various steps of the above-mentioned method for 3D printing bridge construction quality monitoring based on deep learning are implemented.

[0149] The main feature of the computer back-end system memory is that a computer program is stored thereon, and the computer program can be executed by a processor to implement the various steps of the above-mentioned deep learning-based 3D printing bridge construction quality monitoring method.

[0150] The computer backend system database, whose main feature is that the database is configured to store and manage computer application data, including various data types and structures, is applied to the various steps of the above-mentioned method for construction quality monitoring of 3D printed bridge prefabricated box girders based on deep learning.

[0151] The computer back-end system network interface has the main feature of realizing communication and data transmission between computers. The network interface can provide various communication protocols and data transmission methods to meet the communication and data transmission needs of different application scenarios and different needs, and is applied to each step of the above-mentioned deep learning-based 3D printed bridge prefabricated box girder construction quality monitoring method.

[0152] Please refer to Figure 4 As shown in the figure, a specific embodiment of the present invention is mainly used for quality monitoring during the construction process of 3D printed concrete bridge prefabricated box girders. The industrial camera installed on the 3D printing equipment is used to automatically identify defects during the construction process of 3D printed concrete bridge prefabricated box girders. For major defects such as breakage, bulging, and collapse, an automatic alarm is triggered once identified; for cracks, a quantitative analysis is performed, and an automatic alarm is triggered when the crack area is greater than a set threshold, thereby realizing the function of automatically adjusting the printing equipment parameters and performing material status inspection.

[0153] In a specific embodiment of the present invention, the technical solution is mainly implemented in the following ways:

[0154] Model training phase:

[0155] 1) Install an industrial camera on the printing equipment to obtain a video of the construction process of the 3D printed concrete bridge prefabricated box girder;

[0156] 2) Obtain the broken defect images, crack defect images, bulge images and collapse images in the process of 3D printing concrete bridge prefabricated box girder frame by frame from the specified position in the video;

[0157] 3) The acquired video images are processed and annotated frame by frame, and the actual size of the actual frame of the defect target annotated in the test set and the pixel size in all pictures are measured as the actual reference group. The data set is divided into a training set and a test set based on the images annotated with the actual size. The annotated training set of 3D printed concrete bridge prefabricated box girder broken defect images, crack defect images, bulge images, and collapse images are read into the improved YOLOv7 deep neural network (target detection algorithm) to train the target detection algorithm. The training model is trained for 300 epochs to reach convergence, that is, when the model training gradient is close to 0, the training is stopped;

[0158] Model calibration phase:

[0159] 1) Input the test set into the trained object detection algorithm to obtain the pixel width of the predicted box;

[0160] 2) Compare the actual width of the actual test box with the pixel width of the predicted box, use the calibration algorithm to correct the conversion ratio M in the conversion function module, and update the conversion function module with the calibrated conversion ratio M'. When the average accuracy mAP reaches 94%, the calibrated convergence model is obtained.

[0161] Implementation phase:

[0162] 1) The defect detection and analysis module receives images for defect detection;

[0163] 2) Input the image to be identified into the calibrated convergence model, use the target detection algorithm to obtain the length and width pixel values ​​of the above image prediction box, and convert it to obtain the actual size range of the defect (such as Figure 3 As shown), wherein the target detection algorithm adopts an improved YOLOv7 deep neural network;

[0164] 3) If the image is identified as a crack defect image, check whether the ratio of the crack prediction frame area to the lens detection area exceeds the threshold of 5% to determine whether to trigger the early warning module;

[0165] 4) If the image is identified as a broken defect image, a collapsed defect image or a bulging defect image, the early warning module is directly triggered and the defect information is passed to the prompt processing module, where the defect information is the defect type. If the defect type is a crack, it includes the crack area ratio;

[0166] 5) The processing module receives the defect information transmitted by the early warning module, where the defect information is the defect type. If the defect type is a crack, it includes the crack area ratio. The defect information is then displayed and fed back to the back-end system to stop printing the material;

[0167] 6) The back-end system automatically slows down the printing speed, increases the pumping rate, and checks the material status;

[0168] 7) If the image is not identified as a defect or the ratio of the crack prediction frame area to the lens detection area does not exceed the threshold, the material is deemed qualified, and no alarm or mark is given if it is qualified;

[0169] 8) Printing is completed until all areas meet the requirements and behavior monitoring is completed.

[0170] The traditional YOLOv7 neural network uses the SiLU activation function in the feedforward neural network. The improved YOLOv7 deep neural network in the present invention uses the DSiLU activation function in the feedforward neural network, that is, the derivative of the SiLU activation function, such as Figure 5 As shown, the computational efficiency is higher, it is suitable for large-scale deep neural networks, avoids the accumulation effect, improves the stability of training, and has a more stable gradient and better gradient fluidity.

[0171] The present invention replaces the BN (Batch Normalization) method of normalizing the input data in the neural network with the LN (Layer Normalization) method, such as Figure 5 As shown, the mean and variance of the features are calculated in each sample, independent of the batch size, and each feature dimension is normalized in each sample, making it robust to changes in data distribution.

[0172] One CBS is reduced in ELAN (Efficient Layer Aggregation Networks) and SPPCSPC respectively. 1 (Cross-Batch Memory Sampling-1) layer, CBS 1 The module convolution kernel k=3 and the step size s=1 improve the running speed of the entire network.

[0173] The improved YOLOv7 deep neural network includes a backbone network Backbone and a Head network. The input image is input into the backbone network Backbone. In the backbone network Backbone, after passing through CBS1, CBS2, CBS1, CBS2, ELAN, MP-1, ELAN, MP-1, ELAN, MP-1, ELAN, and then outputting SPPCSPC connected to the Head network. The SPPCSPC outputs result A through a CBS3 and UPsample. The output of the second-to-last ELAN is concatenated with result A through a CBS3 at the same time. The concatenated result is concatenated with the first ELAN-W, CBS3, and UPsample to output result B. The output of the second ELAN is concatenated with result B through a CBS3, and the concatenated result is concatenated with MP-2 and REP through the second ELAN-W.

[0174] The output of MP-2 is spliced ​​with the output of result A after the first ELAN-W processing, the spliced ​​output is processed by the third ELAN-W, then spliced ​​by another MP-2 and the output of SPPCSPC, and finally processed by the fourth ELAN-W;

[0175] The outputs of the second to fourth ELAN-W mentioned above are all connected to the CBM block through REP, and the outputs of the three CBM blocks are connected to each other to output a large pixel feature map as the output image of the final network.

[0176] The structures of CBS1, CBS2 and CBS3 are the same, each including a convolutional layer, an LN layer and a DSiLU function. 1 :k=3,s=1;CBS 2 :k=3,s=2;CBS 3 : k = 1, s = 1, where k is the convolution kernel size and s is the step size;

[0177] REP consists of a convolutional layer and an LN layer connected in series to form a branch. Two such branches are connected in parallel and then connected in parallel with a branch formed by an LN layer. The outputs of the three branches are added to obtain the output of REP.

[0178] MP-1 and MP-2 have the same structure, both of which include two branches. One branch is composed of the maximum pooling layer and CBS3, and the other branch is CBS 3 , CBS 2 The outputs of the two branches are concatenated to obtain the output of MP-1 or MP-2, where the number of hidden layers of MP-1 is i / 2, the number of hidden layers of MP-2 is i, and i represents the number of input nodes;

[0179] The structure of SPPCSPC is: data passes through CBS once 3 , CBS 1 , CBS 3 After the module, it is processed by three maximum pooling processes. The results of the three maximum pooling processes are the same as the last CBS 3 The output is spliced ​​and then passes through a CBS 3 Then with the input of SPPCSPC through a CBS 3 The processed output is directly spliced ​​and finally passed through a CBS 3 Then get the output of SPPCSPC;

[0180] The input of ELAN goes through a max pooling layer and CBS 3 Output result E, ELAN input passes through CBS in sequence at the same time 3 , CBS 1 , CBS 1 , CBS 1 Output the result F, and then compare it with the result F that only passes through CBS 3 , CBS 3 With two CBS 1 Get the results G and H, then concatenate E, F, G, and H together, and finally pass CBS 3 Get the output of ELAN;

[0181] The structure of ELAN-W is similar to that of ELAN. The difference is that there are four CBS1 in the F branch. The input of ELAN passes through CBS1 in sequence. 3 , CBS 1 , CBS 1 , CBS 1 , CBS 1 Output the result F, and then compare it with the result F that only passes through CBS 3 , CBS 3 With CBS 1 , CBS 3 With two CBS 1 , CBS 3 With three CBS 1 The results G, H, I, and J are obtained, and then E, F, G, H, I, and J are spliced ​​together, and finally the output of ELAN is obtained through CBS3;

[0182] The CBM block consists of a convolutional layer, a LN layer, and a sigmoid function.

[0183] Experimental results show that this optimization method can increase the network's operating speed by 7%-10%.

[0184] The traditional method is to input the image for training to obtain a convergent model. The calibrated convergent model obtained in this application is divided into two stages. The first is the model training stage, in which the acquired video images are processed and annotated frame by frame, and the actual size of the actual frame of the defect target annotated by the test set and the pixel size in all pictures are measured. As the actual reference group, the training set 3D printed concrete bridge prefabricated box girder broken defect images, crack defect images, bulge images and collapse images after the annotation are read into the improved YOLOv7 neural network (target detection algorithm), the target detection algorithm is trained, and the model is trained for 300 epochs to reach convergence, that is, when the model training gradient is close to 0, the training is stopped; the second is the model calibration stage, the processed test set is input into the convergent model to obtain the detection results; the actual parameters of the detection target used for the test are compared with the preliminary detection results, and the conversion ratio is corrected by the calibration algorithm. When the average accuracy mAP reaches 94%, the standard model is obtained. The convergence model obtained by the calibration of the two stages has a higher accuracy mAP, a faster model convergence speed, and a faster training time. Previous detection methods used older algorithms with poor performance, were prone to false positives and false negatives, and had poor real-time performance. This patent uses the latest and best performing YOLOv7 algorithm to greatly improve performance. The YOLOv7 algorithm is the fastest and most accurate algorithm among current target detection algorithms.

[0185] The invention distributes the annotated images into training sets and test sets in a ratio of 7:3, which not only ensures the amount of training data but also improves the generalization ability of the model. The training set is used to train the model, and the test set is used to evaluate the model performance under different parameter settings, so as to select the best parameter settings, and at the same time, the model can have better generalization ability.

[0186] The difference from other algorithms is that the image data of the present invention is trained and tested on relatively large pictures such as 1280*1280, and the mAP index is significantly improved at a larger resolution.

[0187] The lens pixels of industrial cameras are different from the actual lens range size. The algorithm model is equipped with a function that can convert the predicted frame pixels with the actual lens size, so as to obtain the correct size parameters of the defect length and width quickly and efficiently.

[0188] As one of the most important parts of a bridge, prefabricated box girders for concrete bridges need to meet very high requirements in terms of quality, strength, flatness, etc. Prefabricated box girders for concrete bridges constructed using 3D printing technology can be quickly built, maintained, and form strength.

[0189] The real-time monitoring system can provide accurate and real-time data analysis, thus helping engineers and technicians to promptly identify any factors that may affect the quality of the 3D printed concrete bridge prefabricated box girders, and take timely measures to ensure that the quality of the final 3D printed bridge prefabricated box girders meets the requirements.

[0190] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0191] It should be understood that each part of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0192] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0193] The present invention is based on a deep learning 3D printing bridge prefabricated box girder construction quality monitoring system and method. In the face of the current 3D printing heavily relying on manual monitoring, which consumes a lot of manpower and economic costs, manual monitoring of the 3D printing concrete process is prone to omissions or misjudgments. And manual monitoring cannot monitor places that exceed the height of the person's own field of vision, and cannot monitor the process quality of the box girder structure of the bridge. When defects occur during the printing process and are not promptly investigated and processed, the final printed structure will fail to form or cause major quality problems, and then the normal function of the bridge cannot be achieved, causing huge economic losses and safety risks. The technical solution uses an industrial camera installed on the 3D printing device to automatically identify defects in the construction process of the 3D printed concrete bridge box girder. For major defects such as broken, bulging, and collapse, an automatic alarm is issued once identified; for cracks, a quantitative analysis is performed, and an automatic alarm is issued when the crack area is greater than the set threshold, thereby realizing the automatic adjustment of the printing equipment parameters and the material status inspection function. Based on the existing technical solutions, it can not only reduce labor costs, improve printing efficiency, and ensure the quality of printed products, but also make it possible to pursue larger-scale architectural-level 3D printing. Compared with existing technologies, it has great significance and good application prospects.

[0194] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the principles and spirit of the present invention are included in the protection scope of the present invention.

[0195] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A method for monitoring the construction quality of 3D printed bridge prefabricated box girders based on deep learning, characterized in that: The steps of the monitoring method are: Obtain defect images of the printing area during the construction of the prefabricated box girder of the 3D printed bridge; The defect image is pre-processed and labeled with the defect type, and the actual size of the actual frame of the labeled defect target in reality and the pixel size in the image are measured; The labeled defect images are divided into a training set and a test set; Constructing a convergence model for target detection and measurement, the convergence model includes a target detection algorithm and a conversion function module, The target detection algorithm is used to obtain the pixel size of the prediction box in the defect image, and the conversion function module is used to obtain the predicted actual size of the defect according to the pixel size of the prediction box using the conversion ratio M between the actual size of the actual box and the pixel size; The target detection algorithm is trained using the training set, and the output is the pixel size of the prediction box. The predicted actual size of the defect is obtained through the conversion function module; Input the test set into the trained object detection algorithm to obtain the pixel width of the test set prediction box. The conversion ratio M in the conversion function module is corrected using a calibration algorithm, and the specific steps of the calibration algorithm are: 1) Use the following error function f(x) to measure the difference between the predicted box and the actual box: where p i is the pixel width of the i-th prediction box, is the actual width of the actual box of the i-th defect target, x is the conversion ratio calibration coefficient that needs to be optimized, and N is the total number of predicted boxes in all samples in the test set; 2) Take the average value of the actual width of all samples in the test set - the average value of the pixel width of the prediction box divided by the average value of the pixel width of the prediction box as the initial conversion ratio calibration coefficient x0; 3) Calculate the first and second order derivatives of the error function f(x) and obtain: where f'(x) and f”(x) are the first and second derivatives of f(x), respectively, and sgn(·) is the sign function; 4) Construct the iteration formula as follows: Among them, x k is the k-th iteration conversion ratio calibration coefficient, x k+1 is the conversion ratio calibration coefficient for the k+1th iteration; According to the iterative formula, the conversion ratio calibration coefficient for the next iteration is obtained and substituted into the error function f(x k+1 ) expression, we can obtain the error function f(x k+1 ), and the optimal calibration coefficient X is obtained through iteration; 5) Obtaining a calibrated conversion ratio M' according to M'=M*X, updating the conversion function module with the calibrated conversion ratio to obtain a calibrated convergence model; The video image to be identified is input into the calibrated convergence model to detect defect targets, and the actual size of each defect is predicted. The predicted actual size is compared with the corresponding threshold to determine whether to alarm, and the printing process is adjusted according to the alarm information; until the printing is completed and all areas meet the requirements, the behavior monitoring during the construction process of the 3D printed bridge prefabricated box girder is completed.

2. The method for monitoring the construction quality of a 3D printed bridge prefabricated box girder based on deep learning according to claim 1 is characterized in that: The target detection algorithm adopts an improved YOLOv7 neural network, the improved YOLOv7 neural network adopts an LN layer, and a DSiLU activation function is adopted in the feedforward neural network; a CBS1 layer is reduced in each of the ELAN and SPPCSPC parts.

3. The method for monitoring the construction quality of a 3D printed bridge prefabricated box girder based on deep learning according to claim 1 is characterized in that: The target detection algorithm adopts the improved YOLOv7 neural network and performs 300 epoch training to achieve convergence of the model; when the mean average accuracy mAP reaches 94%, the calibrated converged model is obtained.

4. The method for monitoring the construction quality of a 3D printed bridge prefabricated box girder based on deep learning according to claim 2 or 3, characterized in that: The improved YOLOv7 deep neural network includes a backbone network Backbone and a Head network. The input image is input into the backbone network Backbone. In the backbone network Backbone, after passing through CBS1, CBS2, CBS1, CBS2, ELAN, MP-1, ELAN, MP-1, ELAN, MP-1, ELAN, and then outputting SPPCSPC connected to the Head network. The SPPCSPC outputs result A through a CBS3 and UPsample. The output of the second-to-last ELAN is concatenated with result A through a CBS3 at the same time. The concatenated result is concatenated with the first ELAN-W, CBS3, and UPsample to output result B. The output of the second ELAN is concatenated with result B through a CBS3, and the concatenated result is concatenated with MP-2 and REP through the second ELAN-W. The output of MP-2 is spliced ​​with the output of result A after the first ELAN-W processing, the spliced ​​output is processed by the third ELAN-W, then spliced ​​by another MP-2 and the output of SPPCSPC, and finally processed by the fourth ELAN-W; The outputs of the second to fourth ELAN-Ws mentioned above are all connected to the CBM block through REP, and the outputs of the three CBM blocks are connected to each other to output a large pixel feature map as the output image of the final network; The structures of CBS1, CBS2 and CBS3 are the same, including a convolution layer, an LN layer and a DSiLU function. CBS1: k=3, s=1; CBS2: k=3, s=2; CBS3: k=1, s=1, where k is the convolution kernel size and s is the step size. REP consists of a convolutional layer and an LN layer connected in series to form a branch. Two such branches are connected in parallel and then connected in parallel with a branch formed by an LN layer. The outputs of the three branches are added to obtain the output of REP. MP-1 and MP-2 have the same structure, both of which include two branches. One branch is composed of the maximum pooling layer and CBS3, and the other branch is composed of CBS3 and CBS2. The outputs of the two branches are concatenated to obtain the output of MP-1 or MP-2, where the number of hidden layers of MP-1 is i / 2, and the number of hidden layers of MP-2 is i, where i represents the number of input nodes. The structure of SPPCSPC is as follows: the data passes through CBS3, CBS1, and CBS3 modules once and then undergoes three maximum pooling processes. The results of the three maximum pooling processes are concatenated with the output of the last CBS3. After that, the data passes through a CBS3 and then directly concatenates with the output of SPPCSPC after the input of a CBS3 process. Finally, the output of SPPCSPC is obtained after a CBS3 process. The input of ELAN passes through a maximum pooling layer and CBS3 to output the result E. The input of ELAN passes through CBS3, CBS1, CBS1, and CBS1 to output the result F. Then, the results G and H are obtained by passing through CBS3, CBS3 and two CBS1s respectively. Then, E, F, G, and H are concatenated together and finally the output of ELAN is obtained by passing through CBS3. The structure of ELAN-W is similar to that of ELAN, except that there are four CBS1s on the result F branch. The input of ELAN passes through CBS3, CBS1, CBS1, CBS1, and CBS1 in sequence to output the result F, and then passes through CBS3, CBS3 and CBS1, CBS3 and two CBS1s, and CBS3 and three CBS1s to obtain the results G, H, I, and J. Then, E, F, G, H, I, and J are concatenated together, and finally the output of ELAN is obtained through CBS3. The CBM block consists of a convolutional layer, a LN layer, and a sigmoid function.

5. The method for monitoring construction quality of 3D printed bridge prefabricated box beams based on deep learning according to claim 1, characterized in that: The defect images include a broken defect image, a crack defect image, a bulge image, and a collapse image.

6. The method for monitoring the construction quality of 3D printed bridge prefabricated box beams based on deep learning according to claim 1 is characterized in that: Set the measurement parameter threshold to 5% of the crack prediction frame area to the lens detection area. If it exceeds 5%, an alarm will be triggered and the printing process will be adjusted according to the alarm information. If it does not exceed the threshold or the image is not identified as a defect, the material will be printed as qualified. If it is qualified, no alarm will be triggered and no mark will be made. The crack defect information area ratio or defect type is transmitted to the prompt processing module; The processing module receives the defect information transmitted by the early warning module, wherein the defect information is the defect type. If the defect type is crack, it is the area ratio containing the crack. The defect information is then displayed and fed back to the back-end system to stop printing the material. The back-end system automatically slows down the printing speed, increases the pumping rate and checks the material status.

7. A 3D printed bridge prefabricated box girder construction quality monitoring system based on deep learning, characterized in that: include: The data collection and processing module uses an industrial camera to collect images of the printing area, obtain multiple frames of images including defect information, and pre-process the images; A data annotation and model training module is connected to the data collection and processing module, annotates the defect type of the preprocessed defect image, measures the actual size of the actual frame of the annotated defect target in reality and the pixel size in the picture, and constructs a convergence model for target detection and measurement. The convergence model includes a target detection algorithm and a conversion function module. The conversion function module is used to obtain the predicted actual size of the defect according to the pixel size of the predicted frame using the conversion ratio M of the actual size of the actual frame and the pixel size; An information calibration and storage module is connected to the data annotation and model training module, inputs the test set into the trained target detection algorithm, obtains the pixel width of the prediction box, uses the calibration algorithm to correct the conversion ratio M in the conversion function module, updates the conversion function module with the calibrated conversion ratio M', obtains the calibrated convergence model, and stores the corresponding monitoring information; A defect detection and analysis module is connected to the information calibration and storage module, detects defect targets in the input data through the calibrated convergence model, obtains the predicted actual size of each defect, and sets a threshold to analyze the results; The early warning and prompt processing module is connected to the defect detection and analysis module. It is used to display the measurement and early warning results after the results exceed the early warning threshold, and promptly feedback the defect information to the background system. The background system automatically reduces the movement speed of the printer nozzle, increases the discharge speed, and prompts relevant personnel to check the material status.

8. A method for monitoring the construction quality of 3D printed bridge prefabricated box girders based on deep learning, characterized in that: The monitoring method is implemented by the system according to claim 7, comprising the following steps: 1) Using the data collection and processing module, obtaining defect images of the printing area during the construction process of the 3D printed bridge prefabricated box girder; 2) Annotate the defect type of the preprocessed defect image, and measure the actual size of the actual frame of the marked defect target in reality and the pixel size in the picture to form a data set. The data set is divided into a training set and a test set. The training set is used to train a convergence model for target detection and measurement. The convergence model includes a target detection algorithm and a conversion function module; 3) Input the test set into the trained target detection algorithm to obtain the pixel width of the prediction box, and transmit the detection result to the information calibration and storage module for processing. Use the calibration algorithm to correct the conversion ratio M in the conversion function module, update the conversion function module with the calibrated conversion ratio M', obtain the calibrated convergence model, and store the corresponding monitoring information; 4) Input the video image to be identified into the calibrated convergence model to detect defect targets, obtain the predicted actual size of each defect, and set a threshold to analyze the results; 5) Determine whether to alarm based on the comparison between the predicted actual size and the corresponding set threshold, and adjust the printing process according to the alarm information; until all areas meet the requirements after printing, that is, no major defects appear and the crack area is less than the set threshold, the behavior monitoring during the construction process of the 3D printed bridge prefabricated box girder is completed.

9. The method for monitoring the construction quality of a 3D printed bridge prefabricated box girder based on deep learning according to claim 8, characterized in that: The backend system used in the method includes the following components: Processor: As the core component of the present invention, the processor is responsible for controlling and managing the operation of the entire system, including data acquisition, data processing, image recognition, alarm and other functions; Memory: Memory can be used to store collected data and historical data for subsequent processing and analysis; Database: The back-end system needs to use a database to store and manage collected data, historical data, analysis results and other information; Network interface: The back-end system needs to use a network interface to exchange data and communicate with the front-end system and other external systems.

Images