Concrete 3D printing defect detection device and method based on image analysis

Abstract

The invention provides a concrete 3D printing defect detection device and method based on image analysis, and the method comprises the steps: extracting a time sequence image block from a collected dynamic sequence image, and then extracting the spatial morphological characteristics of a concrete material strip in each frame of image block; determining a time sequence coherence feature sequence evolved by all the spatial morphological features in a time dimension, further determining a time sequence incoherence index representing the stability of the extrusion process, and splicing all the spatial morphological features, the time sequence coherence feature sequence and the time sequence incoherence index to obtain a splicing result; and further generating an extrusion stability comprehensive score of the concrete material strip, and when the extrusion stability comprehensive score is lower than a preset stability evaluation threshold value, triggering an extrusion abnormity alarm of a corresponding level. By adopting the scheme of the invention, the spatial morphological characteristics and the time evolution characteristics can be deeply fused from the dynamic sequence image so as to realize intelligent detection of the concrete 3D printing extrusion stability.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and in particular to a device and method for detecting defects in 3D printed concrete based on image analysis. Background Technology

[0002] Concrete 3D printing, as an emerging digital construction method, has shown broad application prospects in the construction field due to its advantages such as high degree of automation and ability to construct free-form surface structures. However, the industrial application of this method still faces many challenges. Among them, the stability of concrete material extrusion during the printing process is the key to ensuring the geometric accuracy and structural performance of the printed components. If abnormal conditions such as extrusion interruption, excessive material accumulation, or unevenness occur, it will directly lead to defects such as holes, cracks, or deformation in the printed layer, which will seriously affect the printing quality and structural safety.

[0003] Currently, monitoring the extrusion stability of 3D printed concrete mainly relies on manual visual inspection by operators and monitoring based on a single sensor. Manual visual inspection is highly dependent on the operator's experience and focus, resulting in issues such as strong subjectivity, lack of quantification, and low efficiency. During long, continuous printing processes, it is difficult for operators to maintain sustained attention and react promptly to momentary anomalies (such as minor material interruptions). While single-sensor monitoring achieves some degree of automation, it has significant limitations. Pressure sensors cannot directly sense the morphological changes of material strips after deposition, while laser displacement sensors typically only provide single-point, one-dimensional measurement information, making it difficult to comprehensively capture the complex morphological features of material strips in two-dimensional space (such as edge collapse, local bulges, etc.). More importantly, most existing methods are based on static or transient physical quantity measurements, and their technical principle is essentially sampling of point or instantaneous states, lacking temporal continuity analysis of the dynamic evolution trend of the extrusion process. Therefore, they cannot provide early warnings for slowly developing anomalies such as "imminent interruption" caused by the gradual depletion of material. Thus, how to deeply integrate spatial morphological features and temporal evolution features from dynamic sequence images to achieve intelligent detection of concrete 3D printing extrusion stability has become a challenge for the industry. Summary of the Invention

[0004] Based on this, this application provides an image analysis-based concrete 3D printing defect detection device and method for deeply fusing spatial morphological features and temporal evolution features from dynamic sequence images to achieve intelligent detection of concrete 3D printing extrusion stability.

[0005] In a first aspect, this application provides a method for evaluating the extrusion stability of concrete 3D printing based on image analysis, which is applied to a concrete 3D printing defect detection device. The method includes the following steps: Acquire dynamic sequence images of concrete material strips during the extrusion process; Image blocks of the target detection region are extracted from the dynamic sequence images to form time-series image blocks; The time-series image blocks are input into a convolutional neural network to extract the spatial morphological features of concrete material strips within each frame image block, thereby determining the temporal coherence feature sequence of all spatial morphological features evolving in the time dimension. Based on the multi-scale statistics of the time-series coherence feature sequence within the sliding time window, a time discontinuity index characterizing the stability of the extrusion process is determined, wherein the multi-scale statistics include statistics for capturing short-term fluctuations and statistics for capturing long-term trends. All spatial morphological features, the temporal coherence feature sequence, and the temporal discontinuity index are spliced ​​together and input into the attention fusion model to generate a comprehensive score for the extrusion stability of concrete material strips. When the overall extrusion stability score is lower than the preset stability assessment threshold, an extrusion anomaly alarm of the corresponding level is triggered.

[0006] In some embodiments, extracting image blocks of the target detection region from the dynamic sequence image to form a time-series image block specifically includes: The target detection area is located in each frame of image based on the real-time coordinates of the printing extrusion nozzle; The target detection area is subjected to size standardization and grayscale normalization. The processed image blocks are sorted according to the acquisition time to obtain time series image blocks.

[0007] In some embodiments, inputting the time-series image blocks into a convolutional neural network to extract the spatial morphological features of concrete material strips within each frame image block specifically includes: The time-series image blocks are input into a convolutional neural network; The primary image features of each frame of the time-series image block are extracted frame by frame through the convolutional layers of the convolutional neural network. The deep residual module in the convolutional neural network is used to perform multi-level abstraction of the primary features of each frame image block in the time series image block, so as to obtain a high-order feature map of each primary feature of the image representing the cross-sectional shape, contour integrity and surface morphology details of the concrete material strip. By using a global pooling layer, each high-order feature map is aggregated into a fixed-dimensional feature vector, thus obtaining the spatial morphological features of concrete material strips within each frame image block.

[0008] In some embodiments, determining the temporal coherence sequence of all spatial morphological features evolving in the time dimension specifically includes: Arrange all spatial morphological features in chronological order to form a spatial morphological feature sequence; The spatial morphological feature sequence is input into a long short-term memory network for temporal modeling. The hidden states of all time steps of the Long Short-Term Memory network are extracted to form the temporal coherence feature sequence.

[0009] In some embodiments, determining the temporal discontinuity index characterizing the stability of the extrusion process based on the multi-scale statistics of the temporal coherence feature sequence within the sliding time window specifically includes: A fixed-length window is slid sequentially across the temporal coherence feature sequence; Calculate the multi-scale statistics of the temporal coherence eigenvectors within each window; The multi-scale statistics are fused and mapped into a temporal discontinuity index characterizing the stability of the extrusion process.

[0010] In some embodiments, all spatial morphological features, the temporal coherence feature sequence, and the temporal discontinuity index are concatenated and input into an attention fusion model to generate a comprehensive score for the extrusion stability of concrete material strips. Specifically, this includes: The spatial morphological features and temporal continuity features of the concrete material strip at the current moment are obtained from all spatial morphological features and the sequence of temporal continuity features. The spatial morphological characteristics, temporal continuity characteristics, and temporal discontinuity index of the concrete material strip at the current moment are vectorized and concatenated. The concatenated feature vector is input into the attention fusion model to generate an element-wise attention weight vector corresponding to the spatial morphological features and the temporal coherence features at the current moment. The spatial morphological features and the temporal coherence features at the current moment are weighted element-wise using the corresponding element-wise attention weight vector; The weighted fusion feature vector obtained after weighting is input into the regression prediction layer, thereby obtaining the comprehensive score of the extrusion stability of the concrete material strip at the current moment.

[0011] In some embodiments, no action is taken when the overall extrusion stability score is greater than or equal to a preset stability assessment threshold.

[0012] Secondly, this application provides a concrete 3D printing defect detection device based on image analysis. The device includes a concrete 3D printing extrusion stability evaluation unit, which comprises: The acquisition module is used to acquire dynamic sequence images of concrete material strips during the extrusion process; The processing module is used to extract image blocks of the target detection region from the dynamic sequence image to form a time-series image block; The processing module is also used to input the time series image block into a convolutional neural network, extract the spatial morphological features of concrete material strips in each frame image block, and then determine the temporal coherence feature sequence of all spatial morphological features evolving in the time dimension. The processing module is further configured to determine a temporal discontinuity index characterizing the stability of the extrusion process based on the multi-scale statistics of the temporal coherence feature sequence within the sliding time window, wherein the multi-scale statistics include statistics for capturing short-term fluctuations and statistics for capturing long-term trends. The processing module is also used to splice together all spatial morphological features, the temporal coherence feature sequence and the temporal discontinuity index, and input them into the attention fusion model to generate a comprehensive score of the extrusion stability of concrete material strips. The execution module is used to trigger an extrusion anomaly alarm of the corresponding level when the overall extrusion stability score is lower than a preset stability assessment threshold.

[0013] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described image analysis-based concrete 3D printing extrusion stability assessment method.

[0014] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described image analysis-based concrete 3D printing extrusion stability assessment method.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: The image analysis-based concrete 3D printing defect detection device and method provided in this application firstly extracts image blocks of the target detection area from the acquired dynamic sequence images to form time-series image blocks. These time-series image blocks are then input into a convolutional neural network to extract the spatial morphological features of concrete material strips within each frame image block. This process determines the temporal coherence feature sequence of all spatial morphological features evolving over time. This step automatically extracts discriminative spatial morphological features from the dynamic sequence and further captures the long-term evolution of these features over time, thereby improving the comprehensiveness of the extrusion process state description and the accuracy of the dynamic characterization. Secondly, based on the multi-scale statistics of the temporal coherence feature sequence within a sliding time window, a temporal discontinuity index characterizing the stability of the extrusion process is determined. The multi-scale statistics include statistics for capturing short-term fluctuations and statistics for capturing long-term trends. This step simultaneously extracts multi-scale fluctuation information such as short-term abrupt changes and long-term drifts from the temporal coherence features and integrates... A comprehensive instability quantification index is generated, thereby improving the detection sensitivity and coverage of various abnormal working conditions, such as instantaneous blockage and slow interruption. Then, all spatial morphological features, the temporal coherence feature sequence, and the temporal discontinuity index are spliced ​​together and input into the attention fusion model to generate a comprehensive extrusion stability score for concrete material strips. This step can adaptively adjust the contribution weights of spatial and temporal features based on the current context information and perform intelligent fusion, thereby improving the reliability of the final stability score. Finally, when the comprehensive extrusion stability score is lower than a preset stability assessment threshold, an extrusion anomaly alarm of the corresponding level is triggered. This step can intelligently judge based on continuous multi-frame scores and graded thresholds and trigger a graded response from early warning to emergency shutdown, thereby improving the accuracy and real-time performance of the alarm system and effectively avoiding false alarms and missed alarms. In summary, the solution of this application can deeply fuse spatial morphological features and temporal evolution features from dynamic sequence images to achieve intelligent detection of extrusion stability in concrete 3D printing. Attached Figure Description

[0016] Figure 1 This is an exemplary flowchart of an image analysis-based method for assessing the extrusion stability of 3D printed concrete, according to some embodiments of this application. Figure 2 This is a schematic diagram illustrating an application scenario of a concrete 3D printing extrusion stability assessment data processing system according to some embodiments of this application; Figure 3 This is a schematic flowchart illustrating the extraction of spatial morphological features according to some embodiments of this application; Figure 4 This is a structural schematic diagram of a concrete 3D printing extrusion stability evaluation unit according to some embodiments of this application; Figure 5 This is a schematic diagram of the structure of a computer device for implementing an image analysis-based method for evaluating the extrusion stability of concrete 3D printing, according to some embodiments of this application. Detailed Implementation

[0017] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0018] refer to Figure 1 The figure is an exemplary flowchart of an image analysis-based method for assessing the extrusion stability of concrete 3D printing, according to some embodiments of this application. This image analysis-based method for assessing the extrusion stability of concrete 3D printing mainly includes the following steps: In step 101, dynamic sequence images of concrete material strips are acquired during the extrusion process.

[0019] In specific implementation, the following method can be adopted: First, an industrial CMOS camera is fixedly installed 500-800mm in front of the concrete 3D printing extrusion nozzle, so that the optical axis of the industrial CMOS camera forms an angle of 30-45° with the printing platform to fully capture the outline of the material strip, and a ring LED diffuse light source is configured to provide uniform illumination and eliminate dynamic shadows; then, the camera is used to acquire 1280×720 pixel RGB images at a frame rate of 25-30fps, and the image data is transmitted to the industrial control computer in real time through the GigE Vision interface, and then the built-in image acquisition program is cached into a continuous video frame sequence according to the timestamp order to obtain a dynamic sequence image. Other methods can also be used in other embodiments, and this application does not limit them.

[0020] It should be noted that the camera trigger and printhead motion control signals must be synchronized to ensure the timing consistency between image acquisition and the extrusion process.

[0021] Additionally, it should be noted that the dynamic sequence images in this application refer to a series of images continuously acquired in chronological order during the concrete 3D printing extrusion process, and their purpose is to fully record the continuous dynamic changes of concrete material strips during the deposition process.

[0022] In some embodiments, reference Figure 2As shown in the figure, this figure is a schematic diagram of the application scenario of the concrete 3D printing extrusion stability assessment data processing system according to some embodiments of this application. The figure includes three main components: acquisition device, server and data storage device. The acquisition device is responsible for collecting dynamic sequence images of concrete material strips during the extrusion process and sending the collected dynamic sequence images to the server through a communication network. The concrete 3D printing extrusion stability assessment data processing system runs on the server. The server stores the processing results in the data storage device and visualizes them.

[0023] In step 102, image blocks of the target detection region are extracted from the dynamic sequence image to form a time-series image block.

[0024] In some embodiments, extracting image patches of the target detection region from the dynamic sequence images to form time-series image patches can be achieved by the following steps: The target detection area is located in each frame of image based on the real-time coordinates of the printing extrusion nozzle; The target detection area is subjected to size standardization and grayscale normalization. The processed image blocks are sorted according to the acquisition time to obtain time series image blocks.

[0025] In specific implementation, the target detection area in each frame of image can be located based on the real-time coordinates of the printing extrusion nozzle. This can be achieved by: using an industrial control computer to read the three-dimensional spatial coordinates (X, Y, Z) of the printing extrusion nozzle from the 3D printer's motion control card in real time; and using the intrinsic and extrinsic parameters (rotation matrix and translation vector) pre-calibrated using the Zhang Zhengyou calibration method, calculating the pixel coordinates of the nozzle tip in the current image frame through coordinate mapping; using these pixel coordinates as the center, extending upwards, leftwards, and rightwards by 150 pixels each, defining a rectangular area of ​​fixed size 300×300 pixels. This rectangular area is determined as the target detection area covering the extruded material strip and its adjacent background. The extrinsic parameters are determined through a one-time hand-eye calibration during system initialization to ensure accurate transformation from the machine coordinate system to the image coordinate system. Other methods can be used in other embodiments, and this application does not limit this approach.

[0026] In specific implementation, the size standardization and grayscale normalization of the target detection area can be achieved in the following way: First, the extracted 300×300 pixel RGB image block is uniformly scaled to a preset standard size of 224×224 pixels using a bilinear interpolation algorithm to adapt to the input requirements of the subsequent convolutional neural network; then, the size-standardized RGB image is converted into a single-channel grayscale image, and linear normalization is performed using a method based on the minimum and maximum grayscale values ​​of the image to map the grayscale value of each pixel to the [0, 1] interval, thereby eliminating the influence of changes in illumination intensity. Other methods can also be used in other embodiments, and this application does not limit them.

[0027] In specific implementation, the processed image blocks are sorted according to the acquisition time order to obtain time-series image blocks. This can be achieved by: attaching a precise timestamp (with millisecond precision) obtained from the image acquisition system to each frame of processed 224×224 pixel grayscale image block; then, arranging consecutive pixel grayscale image blocks in ascending order of timestamps; finally, stacking these ordered image blocks to form a four-dimensional tensor with dimensions [T, 224, 224, 1]. This four-dimensional tensor is the time-series image block, which is used as the direct input to the subsequent convolutional neural network. Other methods can also be used in other embodiments, and this application does not limit them.

[0028] In step 103, the time-series image blocks are input into a convolutional neural network to extract the spatial morphological features of concrete material strips within each frame image block, thereby determining the temporal coherence feature sequence of all spatial morphological features evolving in the time dimension.

[0029] In some embodiments, reference Figure 3 As shown in the figure, this is a flowchart illustrating the extraction of spatial morphological features in some embodiments of this application. In this embodiment, the spatial morphological features of concrete material strips within each frame image block are extracted by inputting the time-series image block into a convolutional neural network using the following steps: In step 1031, the time-series image block is input into a convolutional neural network; In step 1032, the primary image features of each frame image block within the time series image block are extracted frame by frame through the convolutional layer of the convolutional neural network; In step 1033, the deep residual module in the convolutional neural network is used to perform multi-level abstraction on the primary features of each frame image block in the time series image block, so as to obtain a high-order feature map of each primary feature representing the cross-sectional shape, contour integrity and surface morphology details of the concrete material strip. In step 1034, each high-order feature map is aggregated into a fixed-dimensional feature vector through a global pooling layer to obtain the spatial morphological features of concrete material strips within each frame image block.

[0030] It should be noted that the primary image features in this application refer to the basic visual patterns extracted from a single frame image. Their function is to characterize the basic constituent elements of the concrete material strip at the pixel level, such as edges, corners, and primary textures. The higher-order feature map refers to the feature representation obtained by deep abstraction and fusion of the primary image features. Its function is to comprehensively describe the macroscopic morphological attributes of the concrete material strip from a semantic level, such as cross-sectional shape, contour integrity, and surface morphological details. The spatial morphological features comprehensively describe the quantitative vector of the geometric shape and visual appearance of the concrete material strip at a single point in time. This spatial morphological feature condenses the static morphological information of the material strip at a specific moment and is a key basis for assessing whether its current state is normal.

[0031] In a specific implementation, the input of the time-series image block to the convolutional neural network can be achieved in the following way: the tensor of the time-series image block with dimensions [T, 224, 224, 1] is input frame by frame in time step order into a convolutional neural network with a pre-trained ResNet-18 as the backbone.

[0032] In specific implementation, the primary image features of each frame of the time-series image block can be extracted frame by frame through the convolutional layer in the convolutional neural network. This can be achieved as follows: each frame of grayscale image block with a size of 224×224 pixels is input into a convolutional layer with a kernel size of 7×7 and a stride of 2. This convolutional layer uses 64 filters to perform two-dimensional convolution operations, and then performs nonlinear transformation through the ReLU activation function. After downsampling through a max pooling layer with a kernel size of 3×3 and a stride of 2, the feature map tensor of each frame of grayscale image block is finally output as the primary image features for subsequent deep abstraction processing. The feature map tensor contains the responses of material strip edges, corners and basic textures. In other embodiments, other convolutional kernel sizes, strides or activation functions can also be used, which are not limited in this application.

[0033] In specific implementation, the deep residual module in the convolutional neural network is used to perform multi-level abstraction of the primary features of each frame image block within the time-series image block, obtaining high-order feature maps representing the cross-sectional shape, contour integrity, and surface morphology details of the concrete material strips. This can be achieved in the following way: each primary image feature is used as input and fed into a deep residual module constructed with a pre-trained ResNet-18 network backbone. This deep residual module performs forward propagation through its four layers (layer1 to layer4), where layer1 contains two residual blocks, layer2 contains two residual blocks, and so on. Layer 3 contains two residual blocks, and layer 4 contains two residual blocks. Each residual block contains two 3×3 convolutional layers and a shortcut connection across layers. Through this structure, semantic information from low order to high order is extracted and fused step by step, and finally a high-order feature map that can comprehensively represent the cross-sectional shape, contour integrity and surface morphology details of concrete material is output. The spatial dimension of the high-order feature map is 7×7 and the number of channels is 512, thus obtaining the high-order feature map corresponding to the primary features of each image. In other embodiments, deeper or shallower network structures such as ResNet-34 and ResNet-50 can also be used to implement this, and this application does not limit it.

[0034] In specific implementation, each high-order feature map is aggregated into a fixed-dimensional feature vector through a global pooling layer. The spatial morphological features of concrete material strips within each frame image block can be obtained in the following way: For each high-order feature map, the high-order feature map is input into a global average pooling layer. This global average pooling layer calculates the arithmetic mean of all 49 spatial locations (7×7) of the high-order feature map in each channel, compresses the two-dimensional information of each channel into a scalar value, and thus aggregates the entire feature map into a fixed-dimensional one-dimensional feature vector containing 512 elements. This one-dimensional feature vector is the spatial morphological feature used for subsequent temporal analysis and fusion, thereby obtaining the spatial morphological features of concrete material strips within each frame image. In other embodiments, global max pooling or other methods can also be used to achieve this, but this application does not limit it.

[0035] In some embodiments, determining the temporal coherence sequence of all spatial morphological features evolving in the time dimension can be achieved by the following steps: Arrange all spatial morphological features in chronological order to form a spatial morphological feature sequence; The spatial morphological feature sequence is input into a long short-term memory network for temporal modeling. The hidden states of all time steps of the Long Short-Term Memory network are extracted to form the temporal coherence feature sequence.

[0036] It should be noted that the temporal coherence feature sequence in this application is a sequence of feature vectors arranged in chronological order, where each vector depicts the evolution pattern and dynamic trend of the spatial morphology of concrete material strips between adjacent time points. The overall function of this temporal coherence feature sequence is to capture and record the continuity and change pattern of the extrusion process in the time dimension.

[0037] In specific implementation, arranging all spatial morphological features in chronological order to form a spatial morphological feature sequence can be achieved in the following way: the one-dimensional feature vector of all 512 elements output by the convolution part of the convolutional neural network after processing T consecutive frames of images is arranged in the order of the acquisition timestamps of the corresponding image frames, where T is, for example, 16, forming a two-dimensional tensor with dimensions [16, 512]. This two-dimensional tensor is the spatial morphological feature sequence. In other embodiments, the sequence length T can also take other values ​​such as 32 or 64, which is not limited in this application.

[0038] In specific implementation, the spatial morphological feature sequence can be input into a Long Short-Term Memory (LSTM) network for temporal modeling in the following way: the spatial morphological feature sequence with dimensions [16, 512] is input into a two-layer stacked LSTM network contained in the LSTM network. The stacked LSTM network is LSTM, and the hidden state dimension of the LSTM is set to 128. It performs cyclic processing on the 512-dimensional feature vector dependency relationship from frame 1 to frame 16 in the spatial morphological feature sequence through internal input gate, forget gate and output gate mechanism, thereby modeling the dynamic evolution law of the material strip morphological features during the extrusion process. In other embodiments, other recurrent neural network structures such as gated recurrent units or unidirectional LSTM can also be used. This application does not limit this.

[0039] In specific implementation, the hidden states of all time steps of the Long Short-Term Memory Network are extracted to form the temporal coherence feature sequence in the following way: after the two stacked LSTM layers process the entire 16-frame sequence, the hidden state vectors output by the second layer LSTM at all 16 time steps are obtained; each hidden state vector is a 128-dimensional dense vector. These 16 vectors are arranged in chronological order to form a two-dimensional tensor with dimensions [16, 128]. This tensor is the temporal coherence feature sequence, which completely records the dynamic evolution information of the material strip shape at each moment during the extrusion process. In other embodiments, the hidden states of the first layer LSTM can also be extracted or multiple hidden states can be spliced ​​together. This application does not limit this.

[0040] It should be noted that the above steps can automatically extract discriminative spatial morphological features from the dynamic sequence and further capture the long-term laws of the evolution of these spatial morphological features over time, thereby improving the comprehensiveness of the state description of the extrusion process and the accuracy of the dynamic characterization.

[0041] In step 104, a temporal discontinuity index characterizing the stability of the extrusion process is determined based on the multi-scale statistics of the temporal coherence feature sequence within the sliding time window, wherein the multi-scale statistics include statistics for capturing short-term fluctuations and statistics for capturing long-term trends.

[0042] In some embodiments, determining the temporal discontinuity index characterizing the stability of the extrusion process based on the multi-scale statistics of the temporal coherence feature sequence within the sliding time window can be achieved by the following steps: A fixed-length window is slid sequentially across the temporal coherence feature sequence; Calculate the multi-scale statistics of the temporal coherence eigenvectors within each window; The multi-scale statistics are fused and mapped into a temporal discontinuity index characterizing the stability of the extrusion process.

[0043] It should be noted that the temporal discontinuity index in this application is a scalar value that measures the degree of instability or intensity of abnormal fluctuations in the concrete extrusion process over time. The higher the value of the temporal discontinuity index, the more discontinuous and unstable the dynamic behavior of the extrusion process is.

[0044] In specific implementation, the fixed-length sliding window on the temporal coherence feature sequence can be implemented in the following way: the continuous 128-dimensional temporal coherence feature sequence output by the LSTM layer is sampled using a sliding window of length 10. The sliding step of the window is 1 time step each time, that is, 10 consecutive 128-dimensional feature vectors are extracted in the interval [t-9, t] of the sequence to form a window data, where t is the current time step. As t increases from 10 to the total length of the sequence, a series of consecutive overlapping window sequences are generated for local feature analysis. Other methods can also be used in other embodiments, and this application does not limit them.

[0045] In specific implementation, the multi-scale statistics include: the average variance of each feature dimension within the window, the average range of each feature dimension, the average absolute value of the linear fitting slope of the time series values ​​of each feature dimension, and the Euclidean distance between the average feature vectors of the beginning and end segments of the window. As a preferred embodiment, the multi-scale statistics for calculating the time series coherence feature vectors within each window can be implemented in the following way: for each of the 10 128-dimensional feature vectors within the window, calculate the statistics reflecting short-term fluctuations; more specifically, calculate the statistics for each of the 10 128-dimensional feature vectors within the window. The mean of the variance and the mean of the ranges of the 10 128-dimensional eigenvectors within the window are calculated. At the same time, a statistic reflecting the long-term trend is calculated. More specifically, a univariate linear regression is performed on the 10 continuous values ​​in each feature dimension, the slope is obtained and its absolute value is taken, and then the arithmetic mean of the absolute values ​​of the slopes of all dimensions is calculated, as well as the Euclidean distance between the mean of the first 3 feature vectors and the mean of the last 3 feature vectors in the window. Finally, four statistical scalars with clear physical meaning are obtained, which together constitute the multi-scale statistic. Other methods can be used in other embodiments, and this application does not limit them.

[0046] In specific implementation, the fusion and mapping of the multi-scale statistics into a temporal discontinuity index characterizing the stability of the extrusion process can be achieved in the following way: the four statistical scalars are concatenated into a 4-dimensional feature vector, which is then input into a single-layer fully connected neural network. This single-layer fully connected neural network contains 4 input nodes and 1 output node. Multi-feature fusion is achieved through linear transformation of the weight matrix and the bias term. The output value is then compressed to the 0 to 1 range by the Sigmoid activation function, and finally the temporal discontinuity index characterizing the stability of the extrusion process is obtained. The value of the temporal discontinuity index is negatively correlated with the extrusion stability. Other methods can also be used in other embodiments, and this application does not limit them.

[0047] It should be noted that the above steps can simultaneously extract multi-scale fluctuation information such as short-term mutations and long-term drifts from the temporal continuity characteristics, and fuse them to generate a comprehensive instability quantification index, thereby improving the detection sensitivity and coverage of various abnormal operating conditions, such as instantaneous blockages and slow interruptions.

[0048] In step 105, all spatial morphological features, the temporal coherence feature sequence, and the temporal discontinuity index are spliced ​​together and input into the attention fusion model to generate a comprehensive score for the extrusion stability of concrete material strips.

[0049] In some embodiments, the following steps can be used to concatenate all spatial morphological features, the temporal coherence feature sequence, and the temporal discontinuity index, and input them into an attention fusion model to generate a comprehensive score for the extrusion stability of concrete material strips: The spatial morphological features and temporal continuity features of the concrete material strip at the current moment are obtained from all spatial morphological features and the sequence of temporal continuity features. The spatial morphological characteristics, temporal continuity characteristics, and temporal discontinuity index of the concrete material strip at the current moment are vectorized and concatenated. The concatenated feature vector is input into the attention fusion model to generate an element-wise attention weight vector corresponding to the spatial morphological features and the temporal coherence features at the current moment. The spatial morphological features and the temporal coherence features at the current moment are weighted element-wise using the corresponding element-wise attention weight vector; The weighted fusion feature vector obtained after weighting is input into the regression prediction layer, thereby obtaining the comprehensive score of the extrusion stability of the concrete material strip at the current moment.

[0050] It should be noted that the comprehensive extrusion stability score in this application is a quantitative indicator used to comprehensively evaluate the health and stability of the concrete 3D printing extrusion process at the current moment. This comprehensive extrusion stability score is generated by integrating multi-dimensional information such as spatial morphology and temporal evolution. The higher the score value, the more stable and reliable the extrusion process is. More specifically, the closer the score value is to 1, the more stable the extrusion process is, and the closer it is to 0, the higher the risk of abnormality.

[0051] In specific implementation, the spatial morphological features and temporal continuity features of the concrete material strip at the current moment can be obtained from all spatial morphological features and the temporal continuity feature sequence in the following ways: The spatial morphological features and temporal continuity features of the concrete material strip at the current moment can be directly extracted from all spatial morphological features and the temporal continuity feature sequence; alternatively, when processing continuous data streams, the image block corresponding to the current time step can be processed by the convolutional part of a convolutional neural network, and the 512-dimensional spatial morphological features at the current moment can be directly obtained from the output of its global average pooling layer. Simultaneously, the 128-dimensional hidden state vector corresponding to the current time step can be extracted from the hidden state sequence maintained by the LSTM layer of the Long Short-Term Memory network as the temporal continuity feature. It should be noted that, at the same time, based on the temporal position of the current time step, the latest temporal discontinuity index calculation result is obtained from the real-time updated sliding window cache as the temporal discontinuity index at the current moment. It is necessary to ensure that these three features correspond to the same extrusion state through a timestamp synchronization mechanism. Other methods can be used in other embodiments, and this application does not limit this.

[0052] In specific implementation, the spatial morphological features, temporal continuity features, and temporal discontinuity index of the concrete material strip at the current moment can be vectorized by the following method: the 512-dimensional spatial morphological feature vector, the 128-dimensional temporal continuity feature vector, and the 1-dimensional temporal discontinuity index corresponding to the current time step are concatenated in a first-to-last order to form a joint feature vector containing 641 feature elements. It should be noted that the temporal discontinuity index is converted from a scalar to a 1-dimensional vector through a dimension expansion operation before concatenation to ensure the consistency of all input features in the data structure. Other methods can also be used in other embodiments, and this application does not limit them.

[0053] In specific implementation, the concatenated feature vector is input into the attention fusion model to generate an element-wise attention weight vector corresponding to the spatial morphological features and the temporal coherence features at the current moment. This can be achieved in the following way: the 641-dimensional joint feature vector is input into an attention fusion model consisting of two fully connected layers. The first layer of this attention fusion model is a hidden layer with 64 neurons and uses the ReLU activation function; the second layer is an output layer with 640 neurons and uses the Sigmoid activation function; the network finally outputs a 640-dimensional attention weight vector A=[a1, a2, ..., a640], where the first 512 weight elements a1...a512 correspond to 512-dimensional spatial morphological features, and the last 128 weight elements a513...a640 correspond to 128-dimensional temporal coherence features. Other methods can also be used in other embodiments, and this application does not limit them.

[0054] In specific implementation, the element-wise weighting of the spatial morphological features and the temporal coherence features at the current moment using the corresponding element-wise attention weight vector can be achieved in the following way: the first 512 dimensions [a1...a512] of the 640-dimensional attention weight vector A are multiplied element-wise with the 512-dimensional spatial morphological feature vector; at the same time, the last 128 dimensions [a513...a640] of the weight vector A are multiplied element-wise with the 128-dimensional temporal coherence feature vector; then, the two weighted feature vectors are concatenated along the feature dimensions to form a 640-dimensional weighted fusion feature vector. Other methods can also be used in other embodiments, and this application does not limit them.

[0055] In specific implementation, the weighted fusion feature vector obtained after weighting is input into the regression prediction layer to obtain the comprehensive score of extrusion stability of the concrete material strip at the current time. This can be achieved by the following method: the 640-dimensional weighted fusion feature vector is input into the regression prediction layer, which is a fully connected layer with one output neuron. The sigmoid activation function is used to map the linear output value to between 0 and 1, and finally the comprehensive score of extrusion stability at the current time is obtained. Other methods can be used in other embodiments, and this application does not limit them.

[0056] It should be noted that the above steps can adaptively adjust the contribution weights of spatial and temporal features based on the current context information and perform intelligent fusion, thereby improving the reliability of the final stability score.

[0057] In step 106, when the overall extrusion stability score is lower than the preset stability assessment threshold, an extrusion anomaly alarm of the corresponding level is triggered.

[0058] In specific implementation, when the overall extrusion stability score is lower than the preset stability assessment threshold, triggering the corresponding level of extrusion anomaly alarm can be achieved in the following way: Multiple stability assessment thresholds are preset in the industrial control computer. When the overall extrusion stability score is below 0.3 for three consecutive frames, a "serious anomaly" alarm is triggered, sending an emergency stop signal to the 3D printer's control system through the industrial control computer's digital output interface. Simultaneously, a red alarm indicator is displayed on the on-site operation interface, and a high-frequency audible and visual alarm is activated. When the score is between 0.3 and 0.6 for five consecutive frames, a "warning" alarm is triggered, displaying a yellow warning indicator on the on-site operation interface and recording the current image data and the overall extrusion stability score to the anomaly database. Simultaneously, a warning message is sent to the remote monitoring terminal through the network interface. After the alarm is triggered, the system continuously monitors score changes. Only when the score recovers to above 0.6 for ten consecutive frames is the alarm state deactivated, and normal operation is restored. Other methods can be used in other embodiments, and this application does not limit them.

[0059] In addition, in some embodiments, no action is taken when the overall extrusion stability score is greater than or equal to a preset stability evaluation threshold.

[0060] It should be noted that the stability assessment threshold in this application is a pre-set critical value for judging whether the comprehensive extrusion stability score has entered an abnormal state. By comparing the real-time comprehensive extrusion stability score with this stability assessment threshold, the normal and abnormal working ranges can be objectively defined, thereby triggering different levels of monitoring and intervention measures. As a preferred embodiment, the stability assessment threshold can be preset in the following way: the threshold can be set based on the statistical analysis results of the comprehensive extrusion stability score in historical normal printing processes and typical abnormal cases. Specifically, this includes: collecting a large amount of score data corresponding to normal and stable extrusion periods and taking its distribution... The 5th percentile is used as the lower limit of the "early warning" threshold, such as 0.6. If the score is consistently below this value but above the 1st percentile, an early warning is triggered. At the same time, score data of serious abnormal cases caused by material interruption and blockage are collected, and the 1st percentile of their distribution is used as the lower limit of the "serious abnormality" threshold, such as 0.3. When the score is consistently below this value, a serious alarm is triggered. The threshold setting also needs to be combined with the specific printing material characteristics and printing speed parameters, and the optimal value is obtained through offline experimental calibration. The threshold configuration interface is provided in the system human-machine interface, allowing operators to fine-tune the thresholds of each level according to the actual working conditions. Other methods can also be used in other embodiments, and this application does not limit them.

[0061] It should be noted that the above steps can be based on intelligent judgment of continuous multi-frame scores and grading thresholds, and trigger a graded response from early warning to emergency shutdown, thereby improving the accuracy and real-time performance of the alarm system and effectively avoiding false alarms and missed alarms.

[0062] It should be noted that, in some embodiments, the convolutional neural network, the long short-term memory network, the attention fusion model, and the regression prediction layer need to be jointly trained end-to-end. Specifically, this can be achieved as follows: Prepare a dataset of dynamic sequence images of concrete extrusion containing normal and various abnormal working conditions, and label each time-series image block with a true stability score between 0 and 1, where 1 represents completely stable and 0 represents completely abnormal; use mean squared error as the loss function, the calculation formula of which is... ,in, The overall score for the stability of the model predictions. To provide a true stability score, N is the total number of time-series image patches in the dynamic sequence image dataset; the model is optimized using the Adam optimizer, with an initial learning rate set to... The batch size is set to 32, and the training cycle is no less than 100 rounds.

[0063] Furthermore, in another aspect of this application, in some embodiments, this application provides a concrete 3D printing defect detection device based on image analysis, the device including a concrete 3D printing extrusion stability assessment unit, with reference to... Figure 4The figure is a schematic diagram of the structure of a concrete 3D printing extrusion stability evaluation unit according to some embodiments of this application. The concrete 3D printing extrusion stability evaluation unit includes: a data acquisition module 401, a processing module 402, and an execution module 403, which are described below: Acquisition module 401, in this application, is mainly used to acquire dynamic sequence images of concrete material strips during the extrusion process; Processing module 402, in this application, is mainly used to extract image blocks of the target detection region from the dynamic sequence image to form a time series image block; The processing module 402 described in this application is further used to input the time series image block into a convolutional neural network, extract the spatial morphological features of concrete material strips in each frame image block, and then determine the temporal coherence feature sequence of all spatial morphological features evolving in the time dimension. The processing module 402 described in this application is further configured to determine a temporal discontinuity index characterizing the stability of the extrusion process based on the multi-scale statistics of the temporal coherence feature sequence within the sliding time window, wherein the multi-scale statistics include statistics for capturing short-term fluctuations and statistics for capturing long-term trends. The processing module 402 described in this application is also used to splice together all spatial morphological features, the temporal coherence feature sequence and the temporal discontinuity index, and input them into the attention fusion model to generate a comprehensive score for the extrusion stability of concrete material strips. The execution module 403 in this application is mainly used to trigger the corresponding level of extrusion anomaly alarm when the comprehensive score of extrusion stability is lower than the preset stability assessment threshold.

[0064] Each module in the aforementioned concrete 3D printing extrusion stability evaluation unit can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0065] In another embodiment, this application provides a computer device, which may be a server, and its internal structure diagram may be as follows. Figure 5As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores image-analysis-based concrete 3D printing extrusion stability evaluation data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an image-analysis-based concrete 3D printing extrusion stability evaluation method.

[0066] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0067] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above embodiment of the image analysis-based concrete 3D printing extrusion stability assessment method.

[0068] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps described in the embodiment of the image analysis-based concrete 3D printing extrusion stability assessment method.

[0069] In one embodiment, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps described in the embodiment of the image analysis-based concrete 3D printing extrusion stability assessment method.

[0070] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0071] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0072] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for evaluating the extrusion stability of concrete 3D printing based on image analysis, applied to a concrete 3D printing defect detection device, characterized in that, The method includes the following steps: Acquire dynamic sequence images of concrete material strips during the extrusion process; Image blocks of the target detection region are extracted from the dynamic sequence images to form time-series image blocks; The time-series image blocks are input into a convolutional neural network to extract the spatial morphological features of concrete material strips within each frame image block, thereby determining the temporal coherence feature sequence of all spatial morphological features evolving in the time dimension. Based on the multi-scale statistics of the time-series coherence feature sequence within the sliding time window, a time discontinuity index characterizing the stability of the extrusion process is determined, wherein the multi-scale statistics include statistics for capturing short-term fluctuations and statistics for capturing long-term trends. All spatial morphological features, the temporal coherence feature sequence, and the temporal discontinuity index are spliced ​​together and input into the attention fusion model to generate a comprehensive score for the extrusion stability of concrete material strips. When the overall extrusion stability score is lower than the preset stability assessment threshold, an extrusion anomaly alarm of the corresponding level is triggered.

2. The method as described in claim 1, characterized in that, Extracting image patches of the target detection region from the dynamic sequence images to form a time-series image patch specifically includes: The target detection area is located in each frame of image based on the real-time coordinates of the printing extrusion nozzle; The target detection area is subjected to size standardization and grayscale normalization. The processed image blocks are sorted according to the acquisition time to obtain time series image blocks.

3. The method as described in claim 1, characterized in that, The time-series image blocks are input into a convolutional neural network to extract the spatial morphological features of concrete material strips within each frame image block, specifically including: The time-series image blocks are input into a convolutional neural network; The primary image features of each frame of the time-series image block are extracted frame by frame through the convolutional layers of the convolutional neural network. The deep residual module in the convolutional neural network is used to perform multi-level abstraction of the primary features of each frame image block in the time series image block, so as to obtain a high-order feature map of each primary feature of the image representing the cross-sectional shape, contour integrity and surface morphology details of the concrete material strip. By using a global pooling layer, each high-order feature map is aggregated into a fixed-dimensional feature vector, thus obtaining the spatial morphological features of concrete material strips within each frame image block.

4. The method as described in claim 1, characterized in that, Determining the temporal coherence sequence of all spatial morphological features in their evolution over time specifically includes: Arrange all spatial morphological features in chronological order to form a spatial morphological feature sequence; The spatial morphological feature sequence is input into a long short-term memory network for temporal modeling. The hidden states of all time steps of the Long Short-Term Memory network are extracted to form the temporal coherence feature sequence.

5. The method as described in claim 1, characterized in that, The determination of the temporal discontinuity index characterizing the stability of the extrusion process based on the multi-scale statistics of the temporal coherence feature sequence within the sliding time window specifically includes: A fixed-length window is slid sequentially across the temporal coherence feature sequence; Calculate the multi-scale statistics of the temporal coherence eigenvectors within each window; The multi-scale statistics are fused and mapped into a temporal discontinuity index characterizing the stability of the extrusion process.

6. The method as described in claim 1, characterized in that, All spatial morphological features, the temporal coherence feature sequence, and the temporal discontinuity index are concatenated and input into the attention fusion model to generate a comprehensive score for the extrusion stability of concrete material strips. Specifically, this includes: The spatial morphological features and temporal continuity features of the concrete material strip at the current moment are obtained from all spatial morphological features and the sequence of temporal continuity features. The spatial morphological characteristics, temporal continuity characteristics, and temporal discontinuity index of the concrete material strip at the current moment are vectorized and concatenated. The concatenated feature vector is input into the attention fusion model to generate an element-wise attention weight vector corresponding to the spatial morphological features and the temporal coherence features at the current moment. The spatial morphological features and the temporal coherence features at the current moment are weighted element-wise using the corresponding element-wise attention weight vector; The weighted fusion feature vector obtained after weighting is input into the regression prediction layer, thereby obtaining the comprehensive score of the extrusion stability of the concrete material strip at the current moment.

7. The method as described in claim 1, characterized in that, Specifically, it includes: No action is taken when the overall extrusion stability score is greater than or equal to the preset stability evaluation threshold.

8. A concrete 3D printing defect detection device based on image analysis, the device comprising a concrete 3D printing extrusion stability assessment unit, characterized in that, The concrete 3D printing extrusion stability assessment unit includes: The acquisition module is used to acquire dynamic sequence images of concrete material strips during the extrusion process; The processing module is used to extract image blocks of the target detection region from the dynamic sequence image to form a time-series image block; The processing module is also used to input the time series image block into a convolutional neural network, extract the spatial morphological features of concrete material strips in each frame image block, and then determine the temporal coherence feature sequence of all spatial morphological features evolving in the time dimension. The processing module is further configured to determine a temporal discontinuity index characterizing the stability of the extrusion process based on the multi-scale statistics of the temporal coherence feature sequence within the sliding time window, wherein the multi-scale statistics include statistics for capturing short-term fluctuations and statistics for capturing long-term trends. The processing module is also used to splice together all spatial morphological features, the temporal coherence feature sequence and the temporal discontinuity index, and input them into the attention fusion model to generate a comprehensive score of the extrusion stability of concrete material strips. The execution module is used to trigger an extrusion anomaly alarm of the corresponding level when the overall extrusion stability score is lower than a preset stability assessment threshold.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the image analysis-based concrete 3D printing extrusion stability assessment method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the image analysis-based concrete 3D printing extrusion stability assessment method as described in any one of claims 1 to 7.

Images