A feedback method for monitoring 3D printing defects using machine vision

Abstract

The application provides a feedback method for monitoring 3D printing defects by using machine vision, and belongs to the technical field of machine vision.The application estimates a point spread function in real time by using a mechanical arm vibration accelerometer signal through a blind deconvolution restoration algorithm, performs deblurring restoration on a printing surface image, inputs the restored image into a double-flow detection network, takes the depth discontinuity of a structured light three-dimensional depth map as a hard constraint to filter false positives caused by old mortar texture, outputs a defect candidate region set and a defect pixel proportion, inputs the above results into a time sequence driven physical constraint graph model to output a three-dimensional expansion probability graph, finally adjusts power distribution parameters of an artificial intelligence model according to a resource allocation index, and feeds back correction instructions to a printing control system, so that the technical problem of high false positive rate of defect detection caused by high aliasing of old mortar texture and microcracks in the two-dimensional image domain in the 3D printing process of recycled concrete is solved.

Description

Technical Field

[0001] This invention belongs to the field of machine vision technology, and more specifically, relates to a feedback method for monitoring 3D printing defects using machine vision. Background Technology

[0002] Recycled concrete 3D printing technology uses waste concrete aggregate as raw material and extrudes it layer by layer using a robotic arm, making it valuable for applications in the construction industrialization field. To monitor defects during the printing process, existing technologies typically use industrial cameras to capture images of the printed layer surface and combine them with two-dimensional convolutional neural networks to identify defects such as cracks, pores, and insufficient interlayer bonding. Some solutions introduce traditional image processing methods to extract grayscale gradient features to achieve online quality inspection.

[0003] However, the old mortar layer adhering to the surface of recycled concrete aggregate exhibits edge gradient characteristics highly similar to microcracks in two-dimensional grayscale images, making them almost indistinguishable in terms of texture direction, grayscale contrast, and local frequency components. When the printed layer contains a large number of exposed old aggregate surfaces, existing two-dimensional detection methods misclassify a large number of old mortar textures as cracks, leading to a significant increase in the false positive rate. Simultaneously, image blurring caused by robotic arm vibration further compresses the discriminative margin between sub-millimeter-level real defects and old mortar textures, causing a substantial decrease in the accuracy of two-dimensional detection frameworks based on fixed thresholds under these conditions.

[0004] In existing technologies, due to the high overlap of the gray-level gradient distribution of old mortar texture and microcracks on the surface of recycled concrete aggregate in the two-dimensional gray-level domain, pure two-dimensional image detection methods cannot distinguish between the two types of features using geometric depth information. This leads to a large accumulation of false positive detection results when the exposed area of ​​old aggregate is large during printing. In other words, existing technologies suffer from a high false positive rate in defect detection due to the high overlap of old mortar texture and microcracks in the two-dimensional image domain during the 3D printing of recycled concrete. Summary of the Invention

[0005] In view of this, the present invention provides a feedback method for monitoring 3D printing defects using machine vision, which can solve the technical problem in the prior art where the texture of old mortar and microcracks are highly mixed in the two-dimensional image domain during the 3D printing of recycled concrete, resulting in a high false positive rate for defect detection.

[0006] This invention is implemented as follows: This invention provides a feedback method for monitoring 3D printing defects using machine vision, comprising the following steps:

[0007] In the process of 3D printing recycled concrete, a camera is used to capture images of the printed surface of each layer, and the vibration accelerometer signal of the robotic arm is acquired simultaneously. The restored image is then processed by a blind deconvolution restoration algorithm.

[0008] The restored image and the vibration accelerometer signal of the robotic arm are input into the dual-stream detection network. One path extracts grayscale gradient features, and the other path fuses the three-dimensional depth map obtained by the structured light device. The depth discontinuity is used as a hard constraint to filter out false positive detection results caused by old mortar texture, and outputs a set of defect candidate regions and the corresponding defect pixel ratio.

[0009] The restored image, the set of defect candidate regions, and the hydration time after the current layer is printed are input into the temporal adaptive detection framework. The non-steady drift of image features caused by the hydration process is dynamically corrected through the conditional instance normalization layer. Elastic weight consolidation regularization is used to prevent catastrophic forgetting in incremental updates. The defect annotation results of each layer are output.

[0010] The crack propagation path prediction result is obtained by calculating the crack propagation path prediction result by using the crack propagation path tensor field tracking algorithm to obtain the coordinates and width distribution of the crack endpoints detected in the defect annotation results of each layer.

[0011] Input the defect labeling results, crack propagation path prediction results, hydration time, ambient temperature and humidity and flow meter data of each layer into the time-driven physical constraint graph model, and output the three-dimensional propagation probability graph of each layer of defects and the message propagation queue buffer length.

[0012] Based on the defect pixel ratio, hydration time, and message propagation queue buffer length, resource allocation indicators are calculated through a resource dynamic allocation function. The memory allocation and CUDA stream allocation parameters of the time-driven physical constraint graph model are dynamically adjusted according to the interval to which the resource allocation indicators belong. The three-dimensional extended probability graph and print path correction instructions are then fed back to the print control system.

[0013] Specifically, the blind deconvolution restoration algorithm establishes a physical mapping relationship between the robotic arm vibration accelerometer signal and the image blur kernel, estimates the point spread function of the current frame in real time based on the robotic arm vibration accelerometer signal, and uses the Richardson-Lucy iterative algorithm to perform deconvolution restoration on the degraded image, outputting the restored image.

[0014] The physical mapping relationship refers to the process of stimulating the robotic arm with different amplitudes within a preset frequency range when the printer is unloaded, simultaneously acquiring the vibration accelerometer signal of the robotic arm and the image of the calibration target, and obtaining the mapping parameters between the vibration amplitude and the fuzzy kernel radius by repeating the experiment and performing regression analysis on the data.

[0015] In the dual-stream detection network, the threshold for determining depth discontinuity is determined by statistically analyzing the quantiles of the depth gradient distribution of the two types of regions in a comparative measurement experiment between known real cracks and old mortar texture regions. The two features are fused element-wise in the fusion layer, and the candidate regions of old mortar texture are forcibly suppressed because the depth continuity criterion is not met.

[0016] The dual-stream detection network also introduces domain randomization data augmentation technology to generate simulated recycled aggregate samples to expand the training dataset. Domain randomization data augmentation technology refers to randomly sampling the color distribution, grayscale gradient magnitude, texture direction, and fragment geometry parameters of recycled aggregate in a simulation rendering environment to generate simulated training samples covering multiple aggregate morphologies.

[0017] The conditional instance normalization layer refers to superimposing a dynamic increment obtained by mapping the hydration time through a fully connected layer onto the affine transformation parameters of the standard instance normalization, so that the normalization behavior is adaptively adjusted with the hydration time; the hydration time is encoded to form a conditional vector, which is injected into each conditional instance normalization layer to dynamically adjust the mean and variance of the affine transformation parameters.

[0018] The elastic weight consolidation regularization refers to adding a weighted L2 penalty term to the loss function of the new task by calculating the diagonal elements of the Fisher information matrix of each network parameter at the end of the old task training. The larger the diagonal elements of the Fisher information matrix, the greater the corresponding penalty weight.

[0019] Specifically, the crack propagation path tensor field tracking algorithm uses the least squares method to invert the local crack endpoint coordinates and width distribution. Type stress intensity factor With local Type stress intensity factor A two-dimensional stress field gradient tensor field is constructed, and an improved fast travel method is used to iteratively predict the three-dimensional spatial propagation path of cracks with the direction of maximum principal stress as the propagation direction.

[0020] The improved fast-moving method refers to replacing the propagation velocity function with an anisotropic function determined by the direction of the maximum principal stress in the two-dimensional stress field gradient tensor field, based on the isotropic wavefront propagation framework of the standard fast-moving method, so that the propagation velocity in the direction of the maximum principal stress of the wavefront is higher than that in the vertical direction.

[0021] The time-driven physical constraint graph model uses the defect annotation results of each printing layer as graph nodes. The node feature vector is composed of visual feature vectors extracted by a lightweight convolutional neural network, sine and cosine encoded vectors of hydration time, and linear embedding vectors of environmental temperature and humidity and flow meter data. The weight of the directed edge between nodes is defined by the normalized product of the Euclidean distance and the time interval in the three-dimensional space between layers.

[0022] The graph convolution aggregation operation of the time-driven physical constraint graph model adopts a graph attention mechanism. The attention weights are superimposed on the physical constraint tensor on the learnable attention coefficients. The physical constraint tensor is mapped from the maximum principal stress direction field in the crack propagation path prediction result to the graph edge, and is used as a hard constraint prior to embed attention scores.

[0023] In the time-driven physical constraint graph model, when the crack confidence of any node exceeds the preset crack confidence threshold, a reverse message propagation path is triggered, which transmits the current layer's abnormal signal back to the historical layer nodes along the directed edge, and re-evaluates the danger level of early potential defect nodes.

[0024] The calculation formula for the resource dynamic allocation function is as follows: ,in The percentage of defective pixels. To serve as a reference for the proportion of defective pixels, For hydration time, For the maximum hydration monitoring time, The length of the message propagation queue. The maximum capacity of the message propagation queue. , , The weighting coefficients are and satisfy the following conditions: .

[0025] The training of the time-driven physical constraint graph model uses the class-balanced Focal loss function as the main loss function, and applies dynamic weights to amplify the loss gradient of rare defect categories; elastic weight consolidation regularization is used as an auxiliary loss; the training is divided into two stages: pre-training and online incremental update. The online incremental update only updates the parameters of the conditional instance normalization layer and the graph attention layer.

[0026] The typical range of the depth discontinuity determination threshold is 0.3–1.5 mm; the reference defect pixel ratio benchmark value The typical range is 0.02–0.08; the maximum hydration monitoring time The typical range is 15 to 30 minutes; the spatial resolution of the time-driven physical constraint graph model is set to 2 to 5 mm per grid, and the reverse message propagation performs a maximum of 5 rounds of iteration.

[0027] This invention constructs a dual-stream detection network to process the three-dimensional depth map and gray-level gradient features acquired by the structured light device in parallel. It uses depth discontinuity as a hard constraint to filter out false positive detection results caused by old mortar texture. This solves the technical problem of high false positive rate in defect detection caused by the high overlap of old mortar texture and microcracks in the two-dimensional image domain.

[0028] Real cracks appear as abrupt depth changes in a 3D depth map, while the area containing old mortar textures exhibits continuous depth, indicating a fundamental difference in their 3D geometry. The dual-stream detection network embeds a depth discontinuity criterion into the feature fusion layer, forcibly suppressing candidate regions of old mortar textures due to the failure to meet the depth continuity criterion. This physically eliminates the interference path of 2D grayscale aliasing on the detection results. Furthermore, the blind deconvolution restoration algorithm utilizes the robotic arm vibration accelerometer signal to estimate the point spread function in real time, performing deconvolution restoration on the degraded image. This restores the edge details of sub-millimeter-level defects, expands the resolvable margin between real defects and old mortar textures, and further reduces the false positive rate. The temporal adaptive detection framework compensates for image feature drift caused by the hydration process through a conditional instance normalization layer, ensuring the stability of detection accuracy throughout the entire printing timeline. In summary, this invention solves the technical problem mentioned in the background art: the high false positive rate in defect detection caused by the high aliasing of old mortar textures and microcracks in the 2D image domain during the 3D printing of recycled concrete. Attached Figure Description

[0029] Figure 1 This is a flowchart of the method of the present invention.

[0030] Figure 2 This is a comparison of the old mortar texture and the grayscale gradient amplitude distribution of the real cracks in the original and restored images.

[0031] Figure 3 The graph shows the dynamic increment of the defect pixel ratio and the normalized layer mean of the conditional instance for each layer as a function of hydration time.

[0032] Figure 4 This is a comparison image showing the superimposed results of the predicted propagation path of a typical through-crack and the measured crack location.

[0033] Figure 5 This is a schematic diagram illustrating how resource allocation indicators change with printing layer sequence and when gear switching occurs.

[0034] Figure 6 This is the spatial grid probability distribution diagram of the three-dimensional expansion probability map of the defect in layer 38. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below.

[0036] like Figure 1 The diagram shown is a flowchart of a feedback method for monitoring 3D printing defects using machine vision, provided by this invention. This method includes the following steps:

[0037] S01. In the process of 3D printing of recycled concrete, a camera is used to collect images of the printed surface of each layer, and the vibration accelerometer signal of the robotic arm is collected simultaneously. The restored image is obtained after calculation, and the vibration accelerometer signal of the robotic arm is collected simultaneously.

[0038] S02. Input the restored image and the vibration accelerometer signal of the robotic arm into a dual-stream detection network. One path extracts grayscale gradient features, and the other path fuses the three-dimensional depth map obtained by the structured light device. The depth discontinuity is used as a hard constraint to filter out false positive detection results caused by old mortar texture. Output the set of defect candidate regions and the corresponding defect pixel ratio.

[0039] S03. Input the restored image, the set of defect candidate regions, and the hydration time after the current layer is printed into the temporal adaptive detection framework. Dynamically correct the image feature non-steady-state drift caused by the hydration process through the conditional instance normalization layer. Use elastic weight consolidation regularization to prevent catastrophic forgetting in incremental updates. Output the defect annotation results for each layer.

[0040] S04. The crack propagation path prediction result is obtained by calculating the crack endpoint coordinates and width distribution detected in the defect annotation results;

[0041] S05. Input the defect annotation results, crack propagation path prediction results, hydration time, ambient temperature and humidity and flow meter data into the time-driven physical constraint graph model, and output the three-dimensional propagation probability graph of each layer of defects and the message propagation queue buffer length.

[0042] S06. Based on the defect pixel ratio, the hydration time, and the message propagation queue buffer length, calculate the resource allocation index through the resource dynamic allocation function, dynamically adjust the memory allocation and CUDA stream allocation parameters of the time-driven physical constraint graph model according to the interval to which the resource allocation index belongs, and feed back the three-dimensional extended probability graph and print path correction instructions to the print control system.

[0043] Specifically, the restored image is obtained by performing a blind deconvolution restoration algorithm on the printed surface image to perform deblurring preprocessing.

[0044] Specifically, the crack propagation path prediction result utilizes a crack propagation path tensor field tracking algorithm to construct a two-dimensional stress field tensor. Local type I stress intensity factors and local type II stress intensity factors are inverted through Williams expansion, and the crack front is propagated in the two-dimensional stress field tensor using an improved fast travel method. The crack propagation path in three-dimensional space is iteratively predicted, and the crack propagation path prediction result is finally obtained.

[0045] Among them, the time-driven physical constraint graph model defines the edge weights between nodes using the Euclidean distance and time interval in the three-dimensional space between layers, and integrates the maximum principal stress direction field in the crack propagation path prediction results as the hard constraint prior of the graph attention mechanism.

[0046] The principle of the blind deconvolution restoration algorithm is as follows: A physical mapping relationship is established between the robotic arm vibration accelerometer signal and the image blur kernel. The point spread function of the current frame is estimated in real time based on the robotic arm vibration accelerometer signal. The Richardson-Lucy iterative algorithm is then used to perform deconvolution restoration on the degraded image under the condition that the point spread function is known, restoring the edge details of sub-millimeter level defects and outputting the restored image. The physical mapping relationship is established as follows: Under the printer's idle state, the robotic arm is excited with different amplitudes within the frequency range of 5–20 Hz. The robotic arm vibration accelerometer signal and the calibration target image are simultaneously acquired. The blur kernel morphology corresponding to different vibration parameters is statistically analyzed. After no less than 30 sets of repeated experiments and regression analysis of the data, the physical mapping parameters between the vibration amplitude and the blur kernel radius are fitted to obtain the results. The blind deconvolution restoration algorithm brings the following technical benefits to the solution: it models the physical degradation process caused by vibration as a reversible linear convolution problem, and uses the real-time measurement information of the robotic arm vibration accelerometer signal to estimate the fuzzy kernel. This avoids the ill-posedness of traditional blind estimation methods when the degree of degradation is unknown, and significantly improves the sharpness of the image restoration result without sacrificing the frame rate. It provides a reliable image quality guarantee for subsequent sub-millimeter level defect feature extraction, and fundamentally resolves the engineering contradiction between high frame rate and high signal-to-noise ratio.

[0047] In the dual-stream detection network, depth discontinuity serves as a hard constraint to distinguish between real cracks and old mortar textures. Real cracks exhibit abrupt depth changes in the 3D depth map, while the areas containing old mortar textures have continuous depth. The two feature streams are fused element-wise in the fusion layer. Candidate regions for old mortar textures are forcibly suppressed because the depth continuity criterion is not met. Finally, the set of candidate defect regions and the proportion of defect pixels are output. The dual-stream detection network also incorporates domain randomization data augmentation technology to generate simulated recycled aggregate samples to expand the training dataset. The threshold for determining depth discontinuity is determined by statistically analyzing the quantiles of the depth gradient distribution in known real crack and old mortar texture areas during comparative measurement experiments, typically ranging from 0.3 to 1.5 mm. The technical benefits of the dual-stream detection network are as follows: embedding three-dimensional depth information as a strong physical-level discriminative criterion into the detection process allows the network to establish a correlation between material surface texture and geometry in the feature space, fundamentally solving the problem of high overlap between old mortar texture and microcracks in the two-dimensional grayscale domain. Simultaneously, the domain randomization data augmentation technology compensates for the insufficient number of labeled recycled concrete samples.

[0048] In the time-adaptive detection framework, the hydration time... After encoding, a conditional vector is formed and injected into each conditional instance normalization layer. The affine transformation parameters of the conditional instance normalization layer are dynamically adjusted so that the mean and variance of the feature normalization adapt to the hydration process, thereby compensating for the visual feature distribution drift of the same type of defect at different hydration times. The defect annotation results of each layer are then output. The hydration time is... The effective range is determined as follows: In the printing experiment, surface images are acquired at 1-minute intervals within 0-30 minutes after printing. The changes in the average grayscale value and texture roughness index over time are statistically analyzed to identify the time interval with the highest feature drift rate. After repeated verification through at least 5 independent printing experiments, the typical drift window is determined. The temporal adaptive detection framework brings the following technical benefits to the solution: It explicitly introduces the temporal information of the hydration physical process into the feature normalization mechanism, enabling the model to dynamically self-correct according to the material state. This avoids the problem of accuracy decline in the later stages of printing for fixed threshold detection models. At the same time, elastic weight consolidation regularization ensures the stability of the model during continuous online updates, keeping the detection accuracy consistent throughout the entire printing timeline.

[0049] The principle of the crack propagation path tensor field tracking algorithm is as follows: Based on linear elastic fracture mechanics, the Williams expansion describes the series expansion of the stress field near the crack endpoint. Using the crack endpoint coordinates and width distribution in the defect annotation results, the local type I stress intensity factor is inverted using the least squares method. With local type II stress intensity factor A two-dimensional stress field gradient tensor field is then constructed in the image coordinate system. The direction of the maximum principal stress at each point in the two-dimensional stress field gradient tensor field is the direction in which the crack is most likely to propagate. The improved fast travel method regards the crack leading edge as a wavefront, and in the two-dimensional stress field gradient tensor field, the direction of the maximum principal stress is used as the propagation direction. The wavefront is driven by an anisotropic velocity function to iteratively generate the crack propagation path prediction result. The crack propagation path prediction result is compared with the measured crack position in the next printed layer, and the stress field parameters are self-corrected by the path deviation. The Williams expansion truncation order and the initial values ​​of the stress field parameters are obtained as follows: image acquisition and strain gauge measurement are performed simultaneously on concrete fracture test specimens with no less than 20 sets of known stress boundary conditions. The inversion results are compared with the measured values ​​of the strain gauges, and the truncation order and regularization coefficient are adjusted until the mean square error converges, and finally the parameter value range is determined. The tensor field tracking algorithm for crack propagation paths brings the following technical advantages: it embeds the physical laws of fracture mechanics into the crack path prediction process, ensuring that the prediction results not only rely on image observations but are also constrained by the mechanical behavior of materials. This fundamentally solves the problem of path prediction distortion in pure image-based methods under complex stress states. Furthermore, it eliminates the need for finite element mesh generation, reducing computational complexity to [missing information]. ( (Image pixel count), suitable for online real-time processing; the path deviation-driven self-correction mechanism refines the stress field estimation gradually as the printing process progresses, improving the long-range prediction accuracy of cross-layer crack tracking.

[0050] The specific structure of the time-driven physical constraint graph model is as follows: the time-driven physical constraint graph model uses the defect annotation results of each printing layer as graph nodes. The node feature vector is composed of three parts: a visual feature vector (dimension 128) extracted by a lightweight convolutional neural network, the hydration time, and the time of the defect annotation. The network consists of sine and cosine encoded vectors, and linear embedding vectors of the ambient temperature and humidity and the flow meter data. The weights of directed edges between nodes are defined by the normalized product of the Euclidean distance and the time interval in the inter-layer three-dimensional space. The normalized baseline values ​​of the distance and interval are determined by statistically analyzing the mean of the inter-layer distance distribution from at least 50 printing experiments. The graph convolution aggregation operation adopts a graph attention mechanism. The attention weights are based on standard learnable attention coefficients with superimposed physical constraint tensors. The physical constraint tensors are mapped from the maximum principal stress direction field in the crack propagation path prediction results to the graph edges, serving as hard constraint prior embedding attention scores to force the aggregation direction to be consistent with the crack propagation direction predicted by fracture mechanics. The network consists of 6 graph convolution layers. The layered structure incorporates skip connections between adjacent layers to mitigate the oversmoothing problem in deep graph convolutions. These skip connections are implemented at the graph topology level, meaning that the source node features are linearly transformed and directly added to the aggregation result of the corresponding node in the target layer. When the crack confidence of any node exceeds a preset crack confidence threshold, a reverse message propagation path is triggered, transmitting the current layer's abnormal signal back along directed edges to historical layer nodes to reassess the danger level of early potential defect nodes. This reverse message propagation executes a maximum of 5 iterations. The final output layer is a node-by-node 3D extended probability vector, with dimensions consistent with the predefined spatial grid resolution, set to 2–5 mm per grid cell. The message propagation queue buffer length is also output. The memory allocation and CUDA stream allocation parameters are directly coupled with the output value of the resource dynamic allocation function. When the resource allocation index... When the data is in different intervals, the number of CUDA streams allocated to the feature extraction layer of the lightweight convolutional neural network and the size of the pre-allocated memory blocks for the graph convolutional layers are switched according to the levels specified by the resource dynamic allocation function. The lightweight convolutional neural network adopts a depthwise separable convolutional structure, with a total of 4 convolutional blocks. Each block contains a depthwise separable convolutional layer, a batch normalization layer, and a linear rectified activation function. The output feature map is compressed into a 128-dimensional visual feature vector by global average pooling and then fed into the graph nodes. The preset crack confidence threshold is determined by grid search on the validation set with a recall rate of not less than 95%. The upper limit of 5 iterations is determined after testing the message propagation convergence speed on the synthetic defect dataset. The time-driven physical constraint graph model brings the following technical benefits to the solution: By uniformly encoding the visual observations, hydration physical states, and material mechanics constraints of each layer of printing into the graph structure, the model can simultaneously perceive the spatial distribution pattern of defects, the temporal evolution trend, and the expansion direction driven by fracture mechanics. The coupling relationship between the three is adaptively fused in a learnable manner in the graph attention mechanism; the reverse message propagation mechanism enables the risk level of potential defects in historical layers to be dynamically corrected by new observations in the current layer, forming a cross-layer defect risk backtracking capability, which makes up for the fundamental defect that the independent detection method cannot perceive the cumulative effect of defects; the embedding of the physical constraint tensor makes the attention aggregation not only data-driven but also constrained by the laws of fracture mechanics, improving the model's generalization ability on rare defect categories with scarce samples.

[0051] The steps for establishing the training dataset for the time-driven physical constraint graph model specifically include: acquiring layer-by-layer printed surface images from no less than 200 recycled concrete 3D printing experiments, and simultaneously recording the hydration time. The environmental temperature and humidity, the flow meter data, and the robotic arm vibration accelerometer signal are used. Engineers with concrete structure testing qualifications manually annotate each layer of the image. Annotation categories include porosity, cracks, insufficient interlayer bonding, and alkali-aggregate reaction bulges. For alkali-aggregate reaction bulges, a physical constraint sample generation method based on a diffusion model is used to expand each category to at least 200 samples. During generation, real, low-sample defect images are used as a condition, constraining the porosity distribution and aggregate contour clarity of the generated images to conform to the material's physical statistical characteristics. After preprocessing all images using a blind deconvolution restoration algorithm, the training set, validation set, and test set are divided in an 8:1:1 ratio. During the division, stratified sampling is performed on a complete printing experiment basis to avoid frames from the same printing session spanning the training and test sets.

[0052] The specific steps of training the time-driven physical constraint graph model include: using a class-balanced Focal loss function as the main loss function, applying dynamic weights to amplify the loss gradient of rare defective classes, wherein the dynamic weights are determined through Bayesian optimization on the validation set with the goal of balancing the recall of each class; and using elastic weight consolidation regularization as an auxiliary loss, wherein the elastic weight consolidation regularization coefficients are obtained through grid search. to The range was determined; the optimizer used was AdamW, and the initial learning rate was determined through a learning rate warm-up experiment. to Within the defined range, the batch size is set to 4 to 8 based on the GPU memory capacity constraint. Training is divided into two stages: pre-training and online incremental update. Pre-training is completed on the full historical dataset. Online incremental update is triggered by the completion of each printing layer, and only the parameters of the conditional instance normalization layer and the graph attention layer are updated. During training, the harmonic mean of the recall rate of rare defect categories and the precision rate of common categories on the validation set is used as the early stopping indicator. Training is terminated when there is no improvement after 10 consecutive validation cycles.

[0053] The specific definition of the resource dynamic allocation function is: the resource allocation index The calculation formula is expressed as follows:

[0054] ;

[0055] in The proportion of defective pixels in the current layer. To serve as a reference for the proportion of defective pixels, The hydration time is mentioned. For the maximum hydration monitoring time, The length of the message propagation queue buffer. The maximum capacity of the message propagation queue. , , The weighting coefficients are and satisfy the following conditions: The weighting coefficients were determined by multiple linear regression after statistically analyzing the contribution ratio of each component to the computational load in at least 20 printing experiments; when At that time, the lightweight convolutional neural network feature extraction stage of the time-driven physical constraint graph model allocates one CUDA stream, the pre-allocated memory block size of the graph convolutional layer is set to 256MB, and the message propagation rounds are compressed to 3 rounds; when At that time, the lightweight convolutional neural network feature extraction stage allocates 2 CUDA streams, the pre-allocated memory block size of the graph convolutional layer is set to 512MB, and the message propagation rounds remain at 5 rounds; when At that time, the lightweight convolutional neural network feature extraction stage allocates 4 CUDA streams, the pre-allocated memory block size of the graph convolutional layer is expanded to 1024MB, all reverse message propagation paths are activated and their priority is raised to the highest CUDA stream scheduling level, and a deceleration command is triggered to the printing control system. The defect pixel ratio distribution between the defect-free layer and the defective layer is statistically analyzed in at least 30 printing experiments. The ratio value corresponding to the intersection of the two distributions is used to determine the defect level, with a typical range of 0.02 to 0.08. The upper limit of the water quality monitoring window is determined by the moment when the image feature drift in the printing experiment tends to stabilize, typically within the range of 15–30 minutes. This is calculated based on the memory budget of the print control system and the byte length of a single message.

[0056] The conditional instance normalization layer is a feature normalization method that superimposes the dynamic increment obtained by mapping the conditional vector through a fully connected layer onto the affine transformation parameters (scaling and offset coefficients) of the standard instance normalization, making the normalization behavior change with the hydration time. Adaptive adjustment.

[0057] Elastic weight consolidation regularization is a regularization method to prevent catastrophic forgetting. It calculates the diagonal elements of the Fisher information matrix of each network parameter at the end of training for the old task and adds a weighted L2 penalty term to the loss function of the new task to penalize the degree to which important parameters deviate from the optimal value of the old task. The larger the diagonal elements of the Fisher information matrix, the more important the parameter is to the old task, and the larger the corresponding penalty weight.

[0058] Among them, the diffusion model is a generative model that constructs a forward diffusion process by gradually adding Gaussian noise to real samples during the training phase, and learns an inverse denoising process to recover the target sample from random noise during the inference phase. When real few-sample defect images are used as input, the samples generated by the diffusion model are consistent with real defects in texture and shape. At the same time, the porosity distribution and aggregate outline clarity constraints ensure that the generated samples conform to the material physical and statistical properties of recycled concrete.

[0059] Among them, the category-balanced Focal loss function, based on the standard Focal loss function, multiplies the loss gradient of each category by a dynamic weight that is inversely proportional to the sample frequency of that category, so that rare defect categories can obtain a larger gradient contribution in backpropagation, thereby enabling the model to maintain sufficient detection sensitivity for low-frequency defect categories.

[0060] Among them, the domain randomization data augmentation technology involves randomly sampling the color distribution, grayscale gradient magnitude, texture direction, and fragment geometry parameters of recycled aggregates in a simulation rendering environment to generate simulation training samples covering multiple aggregate morphologies, thereby expanding the distribution coverage of training data and reducing model overfitting to specific acquisition environments.

[0061] The Williams expansion is a series expansion formula in linear elastic fracture mechanics that describes the stress field near the crack endpoint. Its expansion coefficients directly correspond to the stress intensity factor. By substituting the crack endpoint coordinates and width distribution into the boundary conditions of the expansion formula, the coefficients of each order are inverted using the least squares method, thereby obtaining the local type I stress intensity factor. With the aforementioned local type II stress intensity factor The estimated value.

[0062] The improved fast-moving method, based on the isotropic wavefront propagation framework of the standard fast-moving method, replaces the propagation velocity function with an anisotropic function determined by the direction of the maximum principal stress in the two-dimensional stress field gradient tensor. This makes the propagation velocity of the wavefront in the direction of the maximum principal stress higher than that in the vertical direction, thereby driving the crack propagation path prediction result to preferentially advance along the mechanically most probable direction, while maintaining the computational complexity at O(n). ,in This represents the number of pixels in the image.

[0063] Optionally, this solution also provides a computer-based approach to form a feedback system for monitoring defects in 3D-printed recycled concrete using machine vision. The computer is equipped with a readable storage medium that stores program instructions, which are used to execute the above-described method when the computer is run.

[0064] The specific implementation of step S01 is as follows: In the recycled concrete 3D printing system, an industrial camera is installed at the end of the robotic arm to acquire images of the surface after each layer of printing is completed at a fixed frame rate. Simultaneously, an accelerometer is fixed at the joint of the robotic arm to acquire triaxial vibration acceleration signals in real time. Because the robotic arm inevitably generates periodic vibrations during the extrusion process, the acquired printed surface images suffer from motion blur, severely affecting the discernibility of sub-millimeter-level defects. Therefore, a blind deconvolution restoration algorithm is used to preprocess the original images for deblurring. The core idea of ​​this algorithm is to model the image degradation caused by vibration as a linear convolution process, use the accelerometer signal to estimate the point spread function of the current frame in real time, and then use the Richardson-Lucy iterative algorithm to perform deconvolution restoration on the degraded image. The physical mapping relationship is established as follows: Under printer idle conditions, the robotic arm is excited with different amplitudes within the frequency range of 5–20 Hz, and vibration signals and calibration target images are acquired simultaneously. After no less than 30 sets of repeated experiments and regression analysis, the mapping parameters between the vibration amplitude and the blur kernel radius are fitted. The Richardson-Lucy iterative algorithm, given the point spread function, iteratively updates the image estimate using a maximum likelihood estimation framework. Each iteration uses the residual between the current estimate and the observed image for correction until convergence, ultimately outputting a restored image. This step outputs a restored image with clear sub-millimeter edge details and a synchronized robotic arm vibration accelerometer signal, providing a reliable image quality foundation for subsequent defect feature extraction.

[0065] The specific implementation of step S02 is as follows: The dual-stream detection network accepts the restored image and the 3D depth map as two inputs. The first input performs layer-by-layer convolution on the restored image to extract local gray-level gradient features and capture edge direction, amplitude, and local texture information. The second input receives the 3D depth map acquired by the structured light device, calculates the local depth gradient, and uses depth discontinuity as a hard constraint criterion. The two features are fused element-wise in the fusion layer. During fusion, the depth continuity of each candidate region is checked simultaneously: if the depth gradient of the candidate region is lower than the judgment threshold (typically ranging from 0.3 to 1.5 mm), it is judged as a depth continuous region, and the corresponding detection confidence is forcibly suppressed to zero, thereby filtering out false positives caused by old mortar texture. The judgment threshold is determined by statistically analyzing the quantiles of the depth gradient distribution of the two types of regions in a comparative measurement experiment of known real cracks and old mortar texture regions. To address the insufficient number of labeled recycled concrete samples, domain randomization data augmentation is introduced during network training. In the simulation rendering environment, aggregate color distribution, grayscale gradient magnitude, texture direction, and fragment geometry parameters are randomly sampled to generate simulated recycled aggregate samples to expand the training set. The network ultimately outputs a set of candidate defect regions and the corresponding defect pixel ratios. .

[0066] The specific implementation of step S03 is as follows: The temporal adaptive detection framework uses the restored image, the set of defect candidate regions, and the hydration time after the current layer is printed. For input. Hydration time After sine and cosine encoding, a conditional vector is formed and injected into each conditional instance normalization layer. The conditional instance normalization layer superimposes the dynamic increment obtained by mapping the conditional vector through a fully connected layer onto the affine transformation parameters (scaling and offset coefficients) of the standard instance normalization. This allows the mean and variance of the feature normalization to adaptively change with the hydration process, thereby compensating for the visual feature distribution drift of the same type of defect at different hydration times. Hydration time The effective range was determined by acquiring surface images at 1-minute intervals within 0–30 minutes after printing in printing experiments, statistically analyzing the changes in the mean grayscale value and texture roughness index over time, and verifying this through at least five independent printing experiments. To prevent catastrophic forgetting of historical knowledge during online incremental updates, the framework introduces elastic weight consolidation regularization as an auxiliary loss. This is achieved by calculating the diagonal elements of the Fisher information matrix of each network parameter at the end of training for the old task, and adding a weighted L2 penalty term to the loss function for the new task. Larger diagonal elements of the Fisher information matrix correspond to larger penalty weights. Training employs a class-balanced Focal loss function, dynamically amplifying the loss gradient for rare defect categories. The optimizer uses AdamW, and online incremental updates only update the parameters of the conditional instance normalization layer and the graph attention layer, triggered by the completion of each printing layer. The framework ultimately outputs the defect annotation results for each layer, with annotation categories including porosity, cracks, insufficient interlayer bonding, and alkali-aggregate reaction bulges.

[0067] The specific implementation of step S04 is as follows: The crack propagation path tensor field tracking algorithm takes the crack endpoint coordinates and width distribution from the defect annotation results of each layer as input. Based on the Williams expansion in linear elastic fracture mechanics, the stress field near the crack endpoint is expressed as a series expansion. The crack endpoint coordinates and width distribution are substituted into the boundary conditions using the least squares method to invert the local stress field. Type stress intensity factor With local Type stress intensity factor A two-dimensional stress field gradient tensor field in the image coordinate system is then constructed. The direction of the maximum principal stress at each point in the tensor field is the direction in which the crack is most likely to propagate at that point. Subsequently, an improved fast traversal method is adopted, treating the crack leading edge as a wavefront and replacing the propagation velocity function with an anisotropic function determined by the direction of the maximum principal stress, so that the propagation velocity in the direction of the maximum principal stress of the wavefront is higher than that in the vertical direction. The wavefront is iteratively advanced, and the crack propagation path prediction results are generated. The computational complexity is O(log n). ,in The image pixel count is used. The predicted crack propagation path is compared with the measured crack location in the next printed layer. The path deviation drives the stress field parameters to self-correct, gradually refining the cross-layer crack tracking accuracy. The Williams expansion truncation order and initial values ​​of stress field parameters are determined by simultaneously acquiring images and measuring strain gauges on at least 20 concrete fracture test specimens with known stress boundary conditions.

[0068] The specific implementation of step S05 is as follows: The time-driven physical constraint graph model uses the defect annotation results of each printing layer as graph nodes, and the node feature vector is composed of three parts: a 128-dimensional visual feature vector extracted by a lightweight convolutional neural network, hydration time, and so on. The network uses sine and cosine encoded vectors and linear embedding vectors of environmental temperature and humidity data and flow meter data. A lightweight convolutional neural network employs a depthwise separable convolutional structure with four convolutional blocks. Each block contains a depthwise separable convolutional layer, a batch normalization layer, and a linear rectified activation function. The output feature map is compressed into a 128-dimensional vector using global average pooling. The weights of directed edges between nodes are defined by the normalized product of the Euclidean distance and the time interval in the three-dimensional space between layers. The graph convolutional aggregation operation uses a graph attention mechanism. Attention weights are superimposed with physical constraint tensors on learnable attention coefficients. The physical constraint tensors are mapped from the maximum principal stress direction field in the crack propagation path prediction results to the graph edges, forcing the aggregation direction to be consistent with the crack propagation direction predicted by fracture mechanics. The network consists of six graph convolutional layers, with skip connections between adjacent layers to alleviate oversmoothing issues. When the crack confidence of any node exceeds a preset crack confidence threshold (determined by a grid search with a recall rate of no less than 95% on the validation set), a backpropagation path is triggered, executing a maximum of five iterations. The final output includes a node-by-node 3D extended probability map (spatial resolution of 2–5 mm per cell) and the message propagation queue buffer length. .

[0069] The specific implementation method of step S06 is: resource allocation index According to the formula Calculate the weighting coefficients. , , satisfy The contribution ratio of each component to the computational load was determined by multiple linear regression after statistically analyzing the contribution ratio of each component in at least 20 printing experiments. At that time, the lightweight convolutional neural network allocates one CUDA stream for the feature extraction stage, sets the pre-allocated memory block size for the graph convolutional layer to 256MB, and compresses the message propagation rounds to 3 rounds; when At that time, allocate 2 CUDA streams, set the pre-allocated memory block size to 512MB, and maintain the message propagation rounds at 5 rounds; when At that time, four CUDA streams are allocated, the pre-allocated memory block is expanded to 1024MB, all reverse message propagation paths are activated and their priority is raised to the highest CUDA stream scheduling level, and a deceleration command is triggered by the print control system. The 3D expansion probability map and print path correction command are fed back to the print control system via the interface to complete the closed-loop feedback control.

[0070] It should be noted that the key technologies of this invention include: based on a blind deconvolution restoration algorithm, the point spread function is estimated in real time using the vibration accelerometer signal of a robotic arm, and deconvolution restoration is performed on the degraded image. This ensures that image clarity does not depend on reducing the frame rate to achieve low noise, thus removing the engineering constraint between high frame rate acquisition and high signal-to-noise ratio, and providing a reliable image quality prerequisite for subsequent depth discrimination. The dual-stream detection network embeds three-dimensional geometric information as a strong physical discrimination criterion into the two-dimensional feature fusion process. The essential geometric difference between real cracks and old mortar texture in the depth domain is explicitly utilized, so that false positive suppression no longer relies on data-driven soft discrimination, but is enforced by physical constraints. This mechanism is particularly effective in cases where the exposed area of ​​old aggregate is large. The temporal adaptive detection framework explicitly introduces the temporal information of the hydration physical process into the feature normalization mechanism through a conditional instance normalization layer, enabling the detection model to dynamically self-correct according to the material state, avoiding the decrease in accuracy caused by feature drift in the later stages of printing using the fixed threshold method. The three key technologies mentioned above work together: image restoration enhances the resolution of depth discrimination, depth constraint filters false positives, and temporal correction ensures the temporal stability of detection accuracy. The three support each other and together realize high-precision online defect detection in the context of complex recycled concrete materials.

[0071] It should be noted that when using a time-driven physical constraint graph model for online incremental updates, there is a continuous knowledge conflict between the defect distribution knowledge accumulated in the historical printing layers and the new observations in the current layer. The update direction of the model parameters may deviate from the optimal solution of the historical task, leading to a rapid degradation in the ability to fit the historical data distribution. The reason for this technical problem is that during the gradient descent optimization process of deep neural network parameters, the gradient update of the new task directly overwrites the optimal parameter point of the old task. Since the gradient information of the old task is no longer visible during training for the new task, the network cannot simultaneously maintain the optimal solution for both tasks in the parameter space. This ultimately manifests as a rapid decrease in the detection sensitivity for historical defect categories. This phenomenon is particularly prominent for rare defect categories because their sample size is small, and the corresponding parameter memory traces are shallow and fragile, easily overwhelmed by the gradients of the new category during incremental updates. Common solutions to this technical problem include retaining all historical data and periodically retraining, or using a fixed learning rate decay strategy to slow down the parameter drift. However, fixed learning rate decay simultaneously reduces the model's adaptability to new observations, impairing the compensation effect for waterlogging feature drift. Furthermore, full retraining, triggered after each layer is printed, incurs a computational cost far exceeding the time window allowed by the printing cycle, failing to meet online real-time requirements. This invention effectively solves this technical problem. Elastic weight consolidation regularization quantifies the importance of each parameter to the old task by calculating the diagonal elements of the Fisher information matrix of each network parameter at the end of training for the old task. A weighted L2 penalty term proportional to this importance is added to the loss function for the new task, strongly constraining important parameters during incremental updates and limiting their deviation, while less important parameters can freely adapt to new observations. This mechanism allows the model to accept updates from new observations in the current layer while forming a protective barrier against the deficient knowledge accumulated in historical layers, maintaining the fitting ability of the old task in the parameter space without replaying historical data. By combining a selective incremental strategy that updates only the parameters of the conditional instance normalization layer and the graph attention layer, the range of parameters affected by each update is further narrowed. This ensures that the risk of catastrophic forgetting is kept within an acceptable range while maintaining the hydration adaptive capability, and keeps the detection accuracy stable throughout the continuous incremental updates of the entire printing sequence.

[0072] Specifically, the principle of this invention is:

[0073] The fundamental reason why this invention can solve the above-mentioned technical problems is that it introduces three-dimensional geometric information as a strong discrimination criterion at the physical level into the defect detection process, and makes up for the inherent defect of the inseparability of two-dimensional grayscale features by utilizing the essential difference in geometric characteristics between real cracks and old mortar textures in three-dimensional space.

[0074] In two-dimensional image space, both old mortar textures and microcracks exhibit abrupt changes in local gray-level gradients, with highly overlapping directional distributions, amplitude ranges, and spatial frequencies. Any classifier relying solely on two-dimensional gray-level features faces the problem of feature inseparability. This invention introduces structured light equipment to acquire three-dimensional depth maps, extending the detection problem from the two-dimensional gray-level domain to the three-dimensional geometric domain. Real cracks correspond to physical cracking on the surface of the printed object, inevitably exhibiting abrupt changes in local depth in the depth map; old mortar textures are merely differences in material color distribution on the aggregate surface, with continuous depth in the corresponding region in the depth map. The distinction boundary between the two at the three-dimensional geometric level is far clearer than in the two-dimensional gray-level domain. This physical fact provides a sufficient theoretical basis for hard-constraint filtering.

[0075] The dual-stream detection network embeds depth discontinuity determination into the element-wise weighted fusion layer, so that the final detection confidence of the candidate region must simultaneously satisfy the gray-scale gradient feature and the depth abrupt change criterion. The candidate region of old mortar texture is forced to be set to zero because the depth continuity criterion is not satisfied, which fundamentally cuts off the false positive accumulation path.

[0076] The blind deconvolution restoration algorithm establishes a physical mapping relationship between the robotic arm vibration accelerometer signal and the image blur kernel. It uses the vibration signal to estimate the point spread function in real time and performs deconvolution restoration on the degraded image through Richardson-Lucy iteration. After image restoration, the edge details of sub-millimeter-level real defects are restored, and the local depth gradient magnitude is increased, further enhancing the discriminative ability of the depth discontinuity criterion.

[0077] The temporal adaptive detection framework encodes hydration time into a conditional vector and injects it into a conditional instance normalization layer to dynamically adjust affine transformation parameters. This allows the mean and variance of feature normalization to adaptively change with the hydration process, compensating for the visual feature distribution drift of the same type of defect at different hydration times and ensuring the effectiveness of the depth discontinuity criterion throughout the entire printing timeline. The temporal-driven physical constraint graph model uses the maximum principal stress direction field from the crack propagation path prediction results as a hard constraint prior for the graph attention mechanism, subjecting cross-layer defect risk assessment to fracture mechanics constraints and further improving the physical consistency of the detection results. A complete technical chain is formed between the modules, from image restoration, feature extraction, 3D constraint filtering to temporal correction, effectively suppressing the false positive rate in cases where the exposed area of ​​recycled concrete aggregate is large.

[0078] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0079] The specific implementation of step S01 is as follows: During the 3D printing of recycled concrete, the camera acquires images of each printed surface layer at a fixed frame rate, and simultaneously acquires vibration accelerometer signals from the robotic arm. Then, a blind deconvolution restoration algorithm is used to preprocess the acquired printed surface images for deblurring, resulting in a restored image. The physical mapping relationship of the blind deconvolution restoration algorithm is established as follows: Under printer idle conditions, the robotic arm is excited with different amplitudes within a frequency range of 5–20 Hz. Simultaneously, vibration accelerometer signals from the robotic arm and images of the calibration target are acquired. The morphology of the blur kernel corresponding to different vibration parameters is statistically analyzed. After at least 30 sets of repeated experiments and regression analysis of the data, the physical mapping parameters between the vibration amplitude and the blur kernel radius are fitted. The formula for the mapping relationship between the vibration amplitude and the standard deviation of the blur kernel is expressed as follows:

[0080] ;

[0081] In the formula, The standard deviation of the fuzzy kernel (in pixels) is obtained from the regression of the physical mapping relationship. The regression intercept (unit: pixels). The regression slope (dimensionless). The vibration amplitude estimated in real time from the accelerometer signal (unit: ), The normalized reference value for vibration amplitude (unit: (Take the maximum excitation amplitude in the experiment). The regression error term (unit: pixels) is defined above, and these parameters were determined through regression analysis using at least 30 sets of repeated experiments. The estimation formula for the point spread function is as follows:

[0082] ;

[0083] In the formula, The estimated point spread function (unit: ), , Let be the pixel coordinates in the image coordinate system (unit: pixels). The integral of the above equation over the entire image domain is 1, which is dimensionally consistent. The restored image is restored by deconvolution of the degraded image using the Richardson-Lucy iterative algorithm. The iterative formula is expressed as follows:

[0084] ;

[0085] In the formula, For the first The estimated value of the restored image (dimensionless gray value) obtained in the next iteration. A degraded image of the printed surface (dimensionless grayscale values). It is a two-dimensional convolution operator. It is the conjugate flip of the point spread function. The iteration termination condition is that the mean square error of two adjacent iterations is less than the preset convergence threshold. Both sides of the equation are dimensionless gray values.

[0086] The specific implementation of step S02 is as follows: The restored image and the vibration accelerometer signal of the robotic arm are input into a dual-stream detection network. One stream extracts grayscale gradient features, while the other stream fuses the three-dimensional depth map obtained by the structured light device. Depth discontinuity is used as a hard constraint to filter out false positive detection results caused by old mortar texture. The network outputs a set of defect candidate regions and the corresponding defect pixel ratio. The formula for the defect pixel ratio is expressed as follows:

[0087] ;

[0088] In the formula, The proportion of defective pixels in the current layer (dimensionless). This represents the number of pixels (in pixels) in the current layer image that are identified as defects. Total number of pixels in the current layer image (in pixels). Threshold for determining depth discontinuity. The typical range is 0.3–1.5. The depth gradient distribution quantiles of the two types of regions were determined by statistically analyzing the distributions of known real cracks and old mortar textures in a comparative measurement experiment. The two features were then fused element-wise in the fusion layer, and the fusion formula is as follows:

[0089] ;

[0090] In the formula, For the fused feature map at location The value at the location (dimensionless). For grayscale gradient feature map at location The value at the location (dimensionless). Feature maps extracted from 3D depth maps at location The value at the location (dimensionless). and For learnable element-wise weights (dimensionless). For depth discontinuity masks, when the position The depth gradient magnitude at that location exceeds If the condition is met, select 1; otherwise, select 0. The candidate area for old mortar texture is determined by... However, due to forced suppression, both sides of the equation are dimensionless eigenvalues. Domain randomization data augmentation technology generates simulation training samples covering various aggregate morphologies by randomly sampling the color distribution, grayscale gradient magnitude, texture direction, and fragment geometry parameters of recycled aggregates in a simulation rendering environment, thereby expanding the distribution coverage of training data.

[0091] The specific implementation of step S03 is as follows: The restored image, the set of defect candidate regions, and the hydration time after the current layer is printed are combined. Input-time adaptive detection framework. The formula for the sine and cosine encoding vector of hydration time is expressed as follows:

[0092] ;

[0093] In the formula, The two-dimensional sine and cosine encoded vectors for hydration time (dimensionless). Hydration time (unit: ), Maximum hydration monitoring time (unit: The encoded vector range is Both sides of the equation are dimensionless values. The formula for calculating the dynamic affine transformation parameters of the conditional instance normalized layer is expressed as follows:

[0094] ;

[0095] In the formula, and These are the dynamic scaling factor and offset factor (both dimensionless) of the conditional instance normalization layer. and The basic scaling and offset factors (both dimensionless) are used for normalizing standard instances. and For the reason The dynamic increments obtained through mapping by the fully connected layer (all dimensionless) are represented by dimensionless normalized parameters on both sides of the equation. The formula for the elastic weight consolidation regularization auxiliary loss is as follows:

[0096] ;

[0097] In the formula, To reinforce the regularization loss (dimensionless) for elastic weights. The regularization coefficient (dimensionless, obtained through grid search) is... to (determined within the scope) For the current network Each parameter value (dimensionless). The first time the old task training ended The optimal value of each parameter (dimensionless). These are the diagonal elements (dimensionless) of the corresponding Fisher information matrix. A larger value indicates that the parameter is more important to the old task; both sides of the equation are dimensionless loss values. Hydration time. The effective range was determined by printing experiments with 1 For interval acquisition and printing, 0-30 The surface image inside is analyzed, and the curves of the change of the average gray level and texture roughness index over time are statistically analyzed to find the time interval with the largest feature drift rate. After repeated verification through no less than 5 sets of independent printing experiments, the typical drift window is determined.

[0098] The specific implementation of step S04 is as follows: The coordinates and width distribution of the crack endpoints detected in the defect annotation results are calculated using a crack propagation path tensor field tracking algorithm. First, the local type I stress intensity factor is inverted using Williams expansion. With local type II stress intensity factor The boundary condition equations in the Williams expansion are expressed as follows:

[0099] ;

[0100] In the formula, Stress components at the crack endpoint in polar coordinates (unit: ), Distance to the crack endpoint (unit: ), Polar angle (unit: ), The order of the unfolded truncation is determined by no less than 20 sets of concrete fracture tests. and For the first Order expansion coefficients (unit: ), and This is the angle function of the corresponding order (dimensionless). Model error term (unit: ),in (unit: ), (unit: Both sides of the equation are dimensionless. Substituting the crack endpoint coordinates and width distribution into the above equation, and using the least squares method to invert the coefficients of each order, the least squares objective function is expressed as follows:

[0101] ;

[0102] In the formula, The number of measurement points (dimensionless) in the neighborhood of the crack endpoint that participated in the inversion. For the first Stress observation values ​​(unit: ) obtained from the crack width distribution in the image at each measurement point. ), and The first The extreme diameter of each measurement point relative to the crack endpoint (unit: ) and polar angle (unit: The dimensions on both sides of the objective function are... Then, a two-dimensional stress field gradient tensor field in the image coordinate system is constructed. The direction of maximum principal stress at each pixel Calculated by the following formula:

[0103] ;

[0104] In the formula, For pixels The direction of the maximum principal stress at the location (unit: ), , , These are the points at that point. Directional normal stress, Directional normal stress and shear stress (unit: Both sides of the equation are angular quantities (unit: The improved fast travel method replaces the isotropic propagation velocity with an anisotropic velocity function, as expressed below:

[0105] ;

[0106] In the formula, For the crack front edge at position Along the direction The propagation speed (dimensionless normalized speed). Based on the propagation speed (dimensionless, empirically usually taken as 1). This is the anisotropy intensity coefficient (dimensionless, usually taken as 1 to 5). The direction of propagation at the crack front (unit: Both sides of the equation are dimensionless velocity values, and the computational complexity is O(n). ,in The image pixel count. Path bias-driven self-correction updates the root mean square deviation between the measured crack location and the predicted path in the next printed layer. and The estimated value.

[0107] The specific implementation of step S05 is as follows: The defect annotation results, crack propagation path prediction results, and hydration time are combined... The physical constraint graph model is driven by the timing of ambient temperature and humidity and flow meter data input. The feature vector of each printed layer node is composed of three parts, as expressed in the following formula:

[0108] ;

[0109] In the formula, For the first Feature vectors of layer nodes (dimensionless). This is a 128-dimensional visual feature vector (dimensionless) extracted by a lightweight convolutional neural network. For the first Layer corresponding hydration time The sine and cosine encoded vectors, i.e. (dimensionless) For the first Hydration time corresponding to the layer (unit: ), It is a linear embedding vector (dimensionless) of ambient temperature and humidity and flow meter data. This represents the vector concatenation operation. The formula for the linear embedding vector in the environment is expressed as follows:

[0110] ;

[0111] In the formula, For the first The layer's environment linear embedding vector (dimensionless). The weight matrix is ​​a learnable linear embedding (dimensionless). It is a bias vector (dimensionless). For the first Ambient temperature during layer printing (unit: ), Temperature normalized reference value (unit: The experience value is 25. ), For the first Ambient humidity during layer printing (unit: %). The normalized baseline value for humidity is 60% (empirical value). For the first Flow meter reading during layer printing (unit: ), The normalized baseline value for flow rate (unit: (Determined by the experimental statistical mean), all components of the input vector are dimensionless after normalization, and both sides of the equation are dimensionless vectors. The formula for defining the weight of directed edges between nodes is as follows:

[0112] ;

[0113] In the formula, For the first Layer and First Edge weights between layer nodes (dimensionless). Euclidean distance in three-dimensional space between layers (unit: ), Mean of inter-layer distance distribution (unit: (Determined through statistical analysis of at least 50 printing experiments). For the first Layer and First Time interval between layers (unit: ), The normalized reference value for the time interval (unit: (Determined by the experimental statistical mean), both sides of the equation are dimensionless values. The attention scoring formula for the attention mechanism is expressed as follows:

[0114] ;

[0115] In the formula, For the first Layer node to the first Attention score for layer nodes (dimensionless). This is a vector of learnable attention coefficients (dimensionless). It is a learnable linear transformation matrix (dimensionless). This represents vector concatenation. The physical constraint weights (dimensionless). The value of the physical constraint tensor (dimensionless) is the mapping of the maximum principal stress direction field from the crack propagation path prediction results to the graph edge. Both sides of the equation are dimensionless score values. The attention weight normalization formula is expressed as follows:

[0116] ;

[0117] In the formula, The normalized attention weights (dimensionless). For nodes The set of neighboring nodes, the node aggregation result is ,in For the first The aggregated feature vector (dimensionless) of each layer node. When the crack confidence of any node exceeds a preset crack confidence threshold, reverse message propagation is triggered, executing up to 5 iterations. The final output is the three-dimensional expansion probability vector of each layer node and the message propagation queue buffer length. The spatial resolution is set to 2-5 pixels per square. .

[0118] The specific implementation of step S06 is as follows: based on the defect pixel ratio hydration time and message propagation queue buffer length Resource allocation indicators are calculated using a dynamic resource allocation function, and the formula is expressed as follows:

[0119] ;

[0120] In the formula, This is a resource allocation indicator (dimensionless). The proportion of defective pixels in the current layer (dimensionless). The reference defect pixel ratio baseline value (dimensionless, typical range of 0.02 to 0.08, determined by statistically analyzing the defect pixel ratio distribution of defect-free and defective layers in no less than 30 printing experiments, and using the ratio value corresponding to the intersection of the two distributions) is used. Hydration time (unit: ), Maximum hydration monitoring time (unit: ), The length of the message propagation queue buffer (dimensionless). The maximum capacity of the message propagation queue (dimensionless). , , The weighting coefficients (all dimensionless) satisfy the following conditions: All terms have been normalized, and both sides of the equation are dimensionless values. The weighting coefficients are determined through multiple linear regression, as expressed in the following formula:

[0121] ;

[0122] In the formula, The design matrix (dimensionless, with rows representing the number of experiments and 3 columns) consists of the normalized values ​​of each component in at least 20 printing experiments. To obtain the normalized vector (dimensionless) of the measured load in the experiment, the solution is used for... , , Perform normalization to make it meet Both sides of the equation are dimensionless vectors. At that time, allocate 1 CUDA stream, and set the pre-allocated memory block size for the graph convolutional layer to 256. The number of message propagation rounds is compressed to 3; when At that time, allocate 2 CUDA streams, and set the pre-allocated memory block size to 512. The number of message propagation rounds remains at 5; when At that time, four CUDA streams are allocated, and the pre-allocated memory block size is expanded to 1024. All reverse message propagation is activated and its priority is raised to the highest scheduling level. At the same time, the deceleration command of the printing control system is triggered, and the three-dimensional extended probability map and printing path correction command are fed back to the printing control system.

[0123] To better understand and implement this invention, the following is a specific application scenario of the invention, Example 2: To verify the effect of the invention, the technicians built a test environment and constructed a complete online monitoring system for defects in 3D printing of recycled concrete. The system was used to test the printing of wall components on a batch of recycled aggregate (old mortar adhesion rate of about 42%). A total of 60 layers were printed, each layer was about 20mm thick, and the printing speed was set to 80mm / s.

[0124] The test environment consisted of an industrial line scan camera, a structured light depth sensor, a three-axis accelerometer, and an embedded GPU computing unit. The accelerometer was fixed at the third joint of the robotic arm, with a sampling rate of 1000Hz. The camera resolution was 3000×2000 pixels, and the structured light depth sensor had a depth resolution of 0.1mm. During the printing process, the main frequency of the robotic arm vibration accelerometer signal was concentrated between 8 and 12Hz, corresponding to an estimated blur kernel radius of approximately 1.2 to 1.8 pixels. The blind deconvolution restoration algorithm estimated the point spread function in real time using the aforementioned vibration signal, and then deblurred each frame of the original image through Richardson-Lucy iteration (with 15 iterations). The edge sharpness of the restored image was significantly improved compared to the original image, and the positioning error of the sub-millimeter crack endpoints was reduced from an average of 2.3 pixels to approximately 0.6 pixels.

[0125] like Figure 2 As shown, by comparing the grayscale gradient amplitude distribution of the old mortar texture and the real crack in the same area of ​​the original image and the restored image, it can be observed that the degree of overlap between the two types of areas in the grayscale domain after restoration does not disappear due to deblurring. This proves that reliable separation cannot be achieved by relying solely on two-dimensional grayscale information, further illustrating the necessity of introducing three-dimensional depth constraints.

[0126] The dual-flow detection network output 3847 defect candidate regions during 60-layer printing, of which 1204 were related to misjudgments of old mortar texture. After introducing a hard constraint on depth discontinuity (with a judgment threshold set to 0.8 mm), 1178 of the 1204 false positive candidate regions were forcibly suppressed, achieving a false positive suppression rate of 97.8%. The detection results for various defects are shown in Table 1.

[0127] Table 1 Statistical Table of Detection Results for Various Defects

[0128]

[0129] like Figure 3 As shown, the output of the temporal adaptive detection framework is statistically analyzed layer by layer, and the proportion of defect pixels in the defect annotation results of each layer is recorded. The changing trend of the printing layer order, and the dynamic increment of the mean in the conditional instance normalization layer with hydration time. The change curve. Experimental results show that within 0-8 minutes after printing (active hydration period), the average gray value of the image rises rapidly, the dynamic increment of the conditional instance normalization layer changes significantly, effectively compensating for feature drift, and reducing the fluctuation range of detection confidence of the same type of defect at different hydration times from an average of 0.18 before correction to an average of 0.04 after correction.

[0130] The crack propagation path tensor field tracking algorithm predicted the paths of 17 penetrating cracks appearing in 60 layers, and truncated to the third order inversion using Williams expansion. and After constructing a two-dimensional stress field gradient tensor, the improved fast-marching method drives the wavefront toward the direction of maximum principal stress using an anisotropic velocity function. For example... Figure 4 As shown, the predicted propagation path of a typical through-crack is superimposed with the measured crack location. The average spatial deviation between the predicted and measured paths is approximately 1.4 mm, which meets the accuracy requirements for printing path correction. The calculation time is approximately 0.3 seconds (image resolution). Approximately (pixels).

[0131] The time-driven physical constraint graph model performs graph convolution aggregation on 60 graph nodes. After embedding the maximum principal stress direction field as a hard constraint prior into the attention score, the model's attention weight for crack nodes with cross-layer propagation trends is increased by approximately 1.8 times compared to the unconstrained version. After triggering back message propagation at layer 38, the danger level of three early potential defect nodes in historical layers 31-37 is reassessed and increased, triggering the corresponding print path correction instruction. The output of the 3D expansion probability graph is shown in Table 2.

[0132] Table 2 Output Parameters of Typical Layer 3D Spread Probability Map

[0133]

[0134] The resource dynamic allocation function dynamically adjusted the CUDA stream allocation and video memory pre-allocation parameters during the printing of 60 layers. Taking layer 38 as an example, , min, Calculate the resource allocation index ,belong During this period, the system automatically switched to a configuration with 4 CUDA streams and 1024MB of pre-allocated video memory, and triggered a deceleration command in the print control system, reducing the print speed from 80mm / s to 52mm / s. For example... Figure 5 As shown, resource allocation indicators are displayed. The curve showing the change with the printing layer sequence and the switching times of the three settings. For example... Figure 6As shown, the probability distribution of the three-dimensional expansion probability map of the defect in layer 38 at each spatial grid location is displayed. The grid resolution is 3 mm per grid cell. The highest probability area is concentrated at the intersection of the interlayer insufficient bonding area and the main crack.

[0135] The technological advancements of this invention compared to traditional methods are reflected in the following aspects. Traditional two-dimensional image detection methods use gray-level gradient features as the sole criterion for discrimination. In situations where recycled concrete mortar textures and microcracks are highly overlapping, adjusting the classifier parameters cannot fundamentally separate the two types of features because the overlap occurs in the feature space, not the parameter space. Parameter adjustments can only trade off precision and recall, not improve both simultaneously. This invention introduces three-dimensional depth information as a strong physical-level discriminant, elevating the discrimination problem from the feature space to the geometric space. This transforms the distinction between mortar textures and real cracks from a vague statistical distribution difference to a clear geometric discontinuity, thus achieving a fundamental reduction in the false positive rate rather than parametric suppression. Furthermore, traditional online detection systems typically use fixed-parameter models, which cannot adapt to the time-varying nature of image features caused by the hydration process, leading to a continuous decline in detection accuracy in the later stages of printing. This invention explicitly embeds the temporal information of the hydration physical process into the feature normalization mechanism through a conditional instance normalization layer, enabling the model to dynamically self-correct according to the material state, and achieving temporal stability of detection accuracy without increasing human intervention.

[0136] It should be noted that the variables involved in this invention are explained in detail in Tables 3, 4, and 5.

[0137] Table 3. Variable Explanation Table (Part 1)

[0138]

[0139] Table 4. Variable Explanation Table (Part Two)

[0140]

[0141] Table 5. Variable Explanation Table (Part 3)

[0142]

[0143] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A feedback method for monitoring 3D printing defects using machine vision, characterized in that, Includes the following steps: In the process of 3D printing recycled concrete, a camera is used to capture images of the printed surface of each layer, and the vibration accelerometer signal of the robotic arm is acquired simultaneously. The restored image is then processed by a blind deconvolution restoration algorithm. The restored image and the vibration accelerometer signal of the robotic arm are input into the dual-stream detection network. One path extracts grayscale gradient features, and the other path fuses the three-dimensional depth map obtained by the structured light device. The depth discontinuity is used as a hard constraint to filter out false positive detection results caused by old mortar texture, and outputs a set of defect candidate regions and the corresponding defect pixel ratio. The restored image, the set of defect candidate regions, and the hydration time after the current layer is printed are input into the temporal adaptive detection framework. The non-steady drift of image features caused by the hydration process is dynamically corrected through the conditional instance normalization layer. Elastic weight consolidation regularization is used to prevent catastrophic forgetting in incremental updates. The defect annotation results of each layer are output. The crack propagation path prediction result is obtained by calculating the crack propagation path prediction result by using the crack propagation path tensor field tracking algorithm to obtain the coordinates and width distribution of the crack endpoints detected in the defect annotation results of each layer. Input the defect labeling results, crack propagation path prediction results, hydration time, ambient temperature and humidity and flow meter data of each layer into the time-driven physical constraint graph model, and output the three-dimensional propagation probability graph of each layer of defects and the message propagation queue buffer length. Based on the defect pixel ratio, hydration time, and message propagation queue buffer length, resource allocation indicators are calculated through a resource dynamic allocation function. The memory allocation and CUDA stream allocation parameters of the time-driven physical constraint graph model are dynamically adjusted according to the interval to which the resource allocation indicators belong. The three-dimensional extended probability graph and print path correction instructions are then fed back to the print control system.

2. The feedback method for monitoring 3D printing defects using machine vision according to claim 1, characterized in that, The blind deconvolution restoration algorithm specifically establishes a physical mapping relationship between the robotic arm vibration accelerometer signal and the image blur kernel, estimates the point spread function of the current frame in real time based on the robotic arm vibration accelerometer signal, and uses the Richardson-Lucy iterative algorithm to perform deconvolution restoration on the degraded image, outputting the restored image.

3. The feedback method for monitoring 3D printing defects using machine vision according to claim 2, characterized in that, The physical mapping relationship refers to the process of exciting the robotic arm with different amplitudes within a preset frequency range when the printer is unloaded, simultaneously acquiring the vibration accelerometer signal of the robotic arm and the image of the calibration target, and obtaining the mapping parameters between the vibration amplitude and the fuzzy kernel radius by repeating the experiment and performing regression analysis on the data.

4. The feedback method for monitoring 3D printing defects using machine vision according to claim 3, characterized in that, In the dual-stream detection network, the threshold for determining depth discontinuity is determined by statistically analyzing the quantiles of the depth gradient distribution of the two types of regions in a comparative measurement experiment between known real cracks and old mortar texture regions. The two features are fused element-wise in the fusion layer, and the candidate regions of old mortar texture are forcibly suppressed because the depth continuity criterion is not met.

5. The feedback method for monitoring 3D printing defects using machine vision according to claim 4, characterized in that, The dual-stream detection network also introduces domain randomization data augmentation technology to generate simulated recycled aggregate samples to expand the training dataset. Domain randomization data augmentation technology refers to randomly sampling the color distribution, grayscale gradient magnitude, texture direction and fragment geometry parameters of recycled aggregate in the simulation rendering environment to generate simulated training samples covering multiple aggregate morphologies.

6. The feedback method for monitoring 3D printing defects using machine vision according to claim 5, characterized in that, The conditional instance normalization layer refers to superimposing a dynamic increment obtained by mapping the hydration time through a fully connected layer onto the affine transformation parameters of the standard instance normalization, so that the normalization behavior is adaptively adjusted with the hydration time; the hydration time is encoded to form a conditional vector, which is injected into each conditional instance normalization layer to dynamically adjust the mean and variance of the affine transformation parameters.

7. The feedback method for monitoring 3D printing defects using machine vision according to claim 6, characterized in that, The elastic weight consolidation regularization refers to adding a weighted L2 penalty term to the loss function of the new task by calculating the diagonal elements of the Fisher information matrix of each network parameter at the end of the old task training. The larger the diagonal elements of the Fisher information matrix, the greater the corresponding penalty weight.

8. The feedback method for monitoring 3D printing defects using machine vision according to claim 7, characterized in that, The crack propagation path tensor field tracking algorithm specifically uses the crack endpoint coordinates and width distribution to invert the local area using the least squares method. Type stress intensity factor With local Type stress intensity factor A two-dimensional stress field gradient tensor field is constructed, and an improved fast travel method is used to iteratively predict the three-dimensional spatial propagation path of cracks with the direction of maximum principal stress as the propagation direction.

9. The feedback method for monitoring 3D printing defects using machine vision according to claim 8, characterized in that, The improved fast-moving method refers to replacing the propagation velocity function with an anisotropic function determined by the direction of the maximum principal stress in the two-dimensional stress field gradient tensor field, based on the isotropic wavefront propagation framework of the standard fast-moving method, so that the propagation velocity in the direction of the maximum principal stress of the wavefront is higher than that in the vertical direction.

10. The feedback method for monitoring 3D printing defects using machine vision according to claim 9, characterized in that, The time-driven physical constraint graph model uses the defect annotation results of each printing layer as graph nodes. The node feature vector is composed of visual feature vectors extracted by a lightweight convolutional neural network, sine and cosine encoded vectors of hydration time, and linear embedding vectors of environmental temperature and humidity and flow meter data. The weight of directed edges between nodes is defined by the normalized product of Euclidean distance and time interval in the three-dimensional space between layers.

Images