3D printing extrusion monitoring method and system based on continuous image analysis

Abstract

The invention discloses a 3D printing extrusion monitoring method and system based on continuous image analysis. The method comprises the following steps: acquiring image data of a printing head of a 3D printing platform at a plurality of continuous time points; according to the image data, a printing model part of the printing head is determined based on an image recognition algorithm; according to the image data, the discharging flow corresponding to the printing head is calculated based on a continuous image recognition algorithm; and according to the printing model part and the discharging flow, the flow abnormal condition corresponding to the printing head is analyzed. Therefore, accurate flow anomaly detection based on image analysis can be realized, the stability and the printing quality of the 3D printing process are improved, and the printing defect risk caused by flow anomaly is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a 3D printing extrusion monitoring method and system based on continuous image analysis. BACKGROUND

[0002] With the wide application of 3D printing technology in the field of high-precision manufacturing, enterprises and users pay more and more attention to improving the printing quality and stability by precisely monitoring the flow of the print head. The existing technology usually collects the flow sensor data of the print head or the image data at a single time point, uses simple threshold analysis or manual inspection method to monitor the discharge state, and adjusts the printing parameters based on fixed rules to ensure the smoothness of the printing process. The existing solution is too dependent on the flow sensor and cannot adapt to the printing equipment without a flow sensor. The scheme of identifying the single time point image data lacks dynamic analysis of continuous time point image data and accurate identification of the printing model part, and it is difficult to accurately calculate the discharge flow and detect flow abnormalities. The commonly used static monitoring strategy cannot adapt to complex printing scenarios, resulting in insufficient accuracy of flow abnormality detection, which easily causes printing defects or material waste, and limits the stability and product quality of the 3D printing process. It can be seen that the existing technology has defects and needs to be solved. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a 3D printing extrusion monitoring method and system based on continuous image analysis, which can realize accurate flow abnormality detection based on image analysis, improve the stability and printing quality of the 3D printing process, and reduce the risk of printing defects caused by flow abnormalities.

[0004] In order to solve the above technical problems, the present application discloses a 3D printing extrusion monitoring method based on continuous image analysis, which comprises: acquiring image data of a print head of a 3D printing platform at multiple continuous time points; determining a printing model part of the print head based on an image recognition algorithm according to the image data; calculating the discharge flow corresponding to the print head based on a continuous image recognition algorithm according to the image data; analyzing the flow abnormality situation corresponding to the print head according to the printing model part and the discharge flow.

[0005] As an optional implementation, in the first aspect of the present application, the determination of the printing model part of the print head based on the image recognition algorithm according to the image data comprises: inputting each image data into a trained joint image recognition model to obtain the predicted model part corresponding to each image data; clustering all the image data based on the predicted model part, to obtain a data set; determining the mode of the predicted model part corresponding to all the image data in the data set as the printing model part of the printing head.

[0006] As an optional implementation, in the first aspect of the present application, the joint image recognition model comprises an image segmentation model, an image recognition model and a comprehensive prediction model; the image segmentation model is used to segment and determine the model part image in the image data; the image recognition model is used to predict the part type and the corresponding prediction probability corresponding to each model part image; and the comprehensive prediction model is used to comprehensively determine the part type corresponding to each model part image to determine the corresponding predicted model part.

[0007] As an optional implementation, in the first aspect of the present application, the comprehensive prediction model is specifically used to perform the following steps: For each model part image, the average value of the part distance between the part type corresponding to the model part image and the part types corresponding to all other model part images is calculated to obtain a part distance parameter corresponding to the model part image; the part distance is the distance between the corresponding positions of the two part types in the preset model data; The product of the prediction probability corresponding to the model part image and the part distance parameter is calculated to obtain a reliability parameter corresponding to the model part image; The model part images with all the reliability parameters greater than a preset parameter threshold are screened out to obtain an image set; The mode of the part type corresponding to all the model part images in the image set is determined as the corresponding predicted model part.

[0008] As an optional implementation, in the first aspect of the present application, the clustering of all the image data based on the predicted model part to obtain a data set comprises: The target function is set to the maximum total number of the image data in the data set; The constraint condition comprises: The distance between the part positions of the predicted model parts of any two image data in the data set is less than a preset first distance threshold; The image similarity between any two image data in the data set is greater than a preset first similarity threshold; The distance between the part positions of the predicted model parts between each image data in the data set and any image data not in the data set is greater than a preset second distance threshold; the second distance threshold is greater than the first distance threshold. an image similarity between each of the image data in the data set and any of the image data in the non-data set is less than a preset second similarity threshold value; the second similarity threshold value is less than the first similarity threshold value; based on a dynamic programming algorithm, iteratively performing clustering calculation on all the image data according to the objective function and the constraint condition until optimization, to obtain a data set.

[0009] As an optional implementation, in the first aspect of the present application, the calculation of the discharge flow corresponding to the print head according to the image data based on a continuous image recognition algorithm comprises: inputting each of the image data into a trained flow recognition model to obtain a predicted discharge flow of the print head corresponding to each of the image data; sorting all the predicted discharge flows according to the time points corresponding to the image data from early to late to obtain a flow sequence; inputting the flow sequence into a trained LSTM neural network to obtain the discharge flow corresponding to the print head; the LSTM neural network is trained by a training data set comprising a plurality of training flow sequences and corresponding stable flow labels.

[0010] As an optional implementation, in the first aspect of the present application, the print model part is an outer wall pattern part, an inner wall pattern part, an outer structure part, an inner structure part, a support structure part, an enclosing structure part, a hollow part or an epitaxial structure part.

[0011] As an optional implementation, in the first aspect of the present application, the analysis of the flow abnormality of the print head corresponding to the print model part and the discharge flow comprises: determining a reference flow corresponding to the print model part in preset printing plan data; calculating a flow difference value between the discharge flow and the reference flow; when the flow difference value is greater than a preset difference threshold value, determining that the print head has a flow abnormality.

[0012] The second aspect of the embodiment of the present application discloses a 3D printing extrusion monitoring system based on continuous image analysis, which comprises: an acquisition module configured to acquire image data of a print head of a 3D printing platform at a plurality of continuous time points; a determination module configured to determine a print model part of the print head based on an image recognition algorithm according to the image data; a calculation module configured to calculate a discharge flow corresponding to the print head based on a continuous image recognition algorithm according to the image data. an analysis module configured to analyze, according to the printing model part and the discharge flow, a flow abnormality of the print head.

[0013] As an optional implementation, in the second aspect of the present application, the determination module determines the specific printing model part of the print head based on an image recognition algorithm according to the image data, including: inputting each of the image data into a trained joint image recognition model to obtain a predicted model part corresponding to each of the image data; performing clustering on all the image data based on the predicted model part to obtain a data set; determining a mode item of the predicted model part corresponding to all the image data in the data set as the printing model part of the print head.

[0014] As an optional implementation, in the second aspect of the present application, the joint image recognition model includes an image segmentation model, an image recognition model and a comprehensive prediction model; the image segmentation model is configured to segment and determine a model part image in the image data; the image recognition model is configured to predict a part type and a corresponding prediction probability corresponding to each of the model part images; and the comprehensive prediction model is configured to comprehensively determine a corresponding predicted model part based on the part type corresponding to each of the model part images.

[0015] As an optional implementation, in the second aspect of the present application, the comprehensive prediction model is specifically configured to perform the following steps: for each of the model part images, calculate an average value of a part distance between the part type corresponding to the model part image and the part types corresponding to all other model part images to obtain a part distance parameter corresponding to the model part image; the part distance is a distance between corresponding positions of two part types in preset model data; calculate a product of the prediction probability corresponding to the model part image and the part distance parameter to obtain a reliability parameter corresponding to the model part image; screen out model part images with a reliability parameter greater than a preset parameter threshold value to obtain an image set; determine a mode item of the part type corresponding to all the model part images in the image set as the corresponding predicted model part.

[0016] As an optional implementation, in the second aspect of the present application, the determination module performs clustering on all the image data based on the predicted model part to obtain a data set, including: The target function is set to maximize the total number of the image data in the data set; The constraint condition includes: The distance between the part positions of the prediction model parts of any two of the image data in the data set is less than a preset first distance threshold; The image similarity between any two of the image data in the data set is greater than a preset first similarity threshold; The distance between the part positions of the prediction model parts between each of the image data in the data set and any of the image data not in the data set is greater than a preset second distance threshold; the second distance threshold is greater than the first distance threshold; The image similarity between each of the image data in the data set and any of the image data not in the data set is less than a preset second similarity threshold; the second similarity threshold is less than the first similarity threshold; Based on a dynamic programming algorithm, the image data are iteratively clustered according to the target function and the constraint condition until optimization, to obtain the data set.

[0017] As an optional implementation, in the second aspect of the present application, the calculation module calculates the specific manner of the discharge flow of the print head corresponding to the image data based on a continuous image recognition algorithm, which includes: Each of the image data is input into a trained flow recognition model to obtain the predicted discharge flow of the print head corresponding to each of the image data; All of the predicted discharge flows are sorted according to the time points corresponding to the image data from early to late to obtain a flow sequence; The flow sequence is input into a trained LSTM neural network to obtain the discharge flow of the print head; the LSTM neural network is trained by a training data set including a plurality of training flow sequences and corresponding stable flow labels.

[0018] As an optional implementation, in the second aspect of the present application, the print model part is an outer wall pattern part, an inner wall pattern part, an outer structure part, an inner structure part, a support structure part, an enclosing structure part, a hollow part or an epitaxial structure part.

[0019] As an optional implementation, in the second aspect of the present application, the analysis module analyzes the specific manner of the flow abnormality of the print head corresponding to the print model part and the discharge flow, which includes: The reference flow corresponding to the print model part is determined in the preset printing plan data; calculating a flow difference between the discharge flow and the reference flow; determining that the print head has a flow abnormality when the flow difference is greater than a preset difference threshold.

[0020] A third aspect of the present application discloses another 3D printing extrusion monitoring system based on continuous image analysis, comprising: a memory storing executable program codes; a processor coupled with the memory; The processor invokes the executable program codes stored in the memory to execute part or all of the steps of the 3D printing extrusion monitoring method based on continuous image analysis disclosed in the first aspect of the present application.

[0021] A fourth aspect of the present application discloses a computer storage medium storing computer instructions, which when invoked, are used to execute part or all of the steps of the 3D printing extrusion monitoring method based on continuous image analysis disclosed in the first aspect of the present application.

[0022] Compared with the prior art, the embodiments of the present application have the following beneficial effects: The present application can realize accurate flow abnormality detection based on image analysis, improve the stability of the 3D printing process and the printing quality, and reduce the risk of printing defects caused by flow abnormalities. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0024] Figure 1 is a flow diagram of a 3D printing extrusion monitoring method based on continuous image analysis disclosed in the embodiments of the present application.

[0025] Figure 2 is a structural diagram of a 3D printing extrusion monitoring system based on continuous image analysis disclosed in the embodiments of the present application.

[0026] Figure 3 is a structural diagram of another 3D printing extrusion monitoring system based on continuous image analysis disclosed in the embodiments of the present application. DETAILED DESCRIPTION

[0027] In order to better understand the technical scheme of the present application, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0028] The terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish different objects, rather than to describe a particular order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product, or apparatus.

[0029] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0030] The present application discloses a 3D printing extrusion monitoring method and system based on continuous image analysis. By acquiring image data of the printing head of the 3D printing platform at multiple continuous time points, the printing model part is determined based on the image recognition algorithm, and the discharge flow is calculated using the continuous image recognition algorithm. According to the printing model part and the discharge flow, the abnormal flow condition is analyzed, so as to realize accurate flow abnormality detection based on image analysis, improve the stability and printing quality of the 3D printing process, and reduce the risk of printing defects caused by flow abnormalities. The following will be described in detail.

[0031] Embodiment one Please refer to Figure 1 , Figure 1 is a flowchart of a 3D printing extrusion monitoring method based on continuous image analysis according to an embodiment of the present application. Among them, Figure 1 The described 3D printing extrusion monitoring method based on continuous image analysis can be applied in a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). For example, Figure 1As shown, the 3D printing extrusion monitoring method based on continuous image analysis can include the following operations: 101. Obtain image data of a print head of a 3D printing platform at multiple continuous time points.

[0032] Optionally, the image data can be RGB images, grayscale images, depth images, or infrared images, which are not limited by the present application.

[0033] Optionally, the continuous time points can be millisecond, second, or frame time intervals, which are not limited by the present application.

[0034] Optionally, the image data can be obtained based on a high-speed camera, a laser scanner, or a stereo vision system, which are not limited by the present application.

[0035] 102. Determine the print model part of the print head based on image recognition algorithms according to the image data. 103. Calculate the discharge flow corresponding to the print head based on continuous image recognition algorithms according to the image data. 104. Analyze the flow abnormality of the print head according to the print model part and the discharge flow.

[0036] Optionally, the flow abnormality can be excessive flow, insufficient flow, or unstable flow, which are not limited by the present application.

[0037] Optionally, the analysis process can be based on threshold comparison, statistical analysis, or machine learning classification model, which are not limited by the present application.

[0038] Optionally, the analysis of the flow abnormality can be dynamically adjusted in combination with the printing environment or task progress, which are not limited by the present application.

[0039] As can be seen, the above embodiments of the application obtain image data of a print head of a 3D printing platform at multiple continuous time points, determine the print model part based on image recognition algorithms, calculate the discharge flow using continuous image recognition algorithms, and analyze the flow abnormality according to the print model part and the discharge flow, thereby enabling precise flow abnormality detection based on image analysis, improving the stability and printing quality of the 3D printing process, and reducing the risk of printing defects caused by flow abnormalities.

[0040] As an optional embodiment, in the above steps, determining the print model part of the print head based on image recognition algorithms according to the image data includes: inputting each image data into a trained joint image recognition model to obtain a predicted model part corresponding to each image data; clustering all image data based on the predicted model part to obtain a data set; The mode item of the predicted model part corresponding to all image data in the data set is determined as the printing model part of the print head.

[0041] Optionally, the clustering can be based on a K-means clustering, hierarchical clustering or density clustering algorithm, and the application is not limited in this regard.

[0042] As can be seen, through the above optional embodiments, by inputting each frame of image data into the trained joint image recognition model to predict the model part, clustering the image data based on the predicted model part to form a data set, and taking the mode item of the predicted model part in the set as the printing model part of the print head, the precise printing model part determination based on joint recognition and clustering analysis is realized, the accuracy of 3D printing flow anomaly detection and the printing quality are improved, and the risk of defects caused by part misjudgment is reduced.

[0043] As an optional embodiment, in the above steps, the joint image recognition model includes an image segmentation model, an image recognition model and a comprehensive prediction model; the image segmentation model is used to segment and determine the model part image in the image data; the image recognition model is used to predict the part type and the corresponding prediction probability corresponding to each model part image; and the comprehensive prediction model is used to determine the corresponding prediction model part by comprehensively predicting the part type corresponding to each model part image.

[0044] Optionally, the image segmentation model can be a U-Net model, a Mask R-CNN model or a DeepLab model, and the application is not limited in this regard.

[0045] Optionally, the image recognition model can be a ResNet model, an EfficientNet model or a VisionTransformer model, and the application is not limited in this regard.

[0046] Optionally, the comprehensive prediction model can be a fully connected network, a decision tree model or a probability fusion model, and the application is not limited in this regard.

[0047] Optionally, the image segmentation model can be a U-Net model, a Mask R-CNN model or a DeepLab model, and the application is not limited in this regard.

[0048] Optionally, the image recognition model can be a ResNet model, an EfficientNet model or a VisionTransformer model, and the application is not limited in this regard.

[0049] Optionally, the comprehensive prediction model can be a fully connected network, a decision tree model or a probability fusion model, and the application is not limited in this regard.

[0050] It can be seen that by the above optional embodiments, the functional model constitution of the joint image recognition model is defined to realize more accurate print site recognition, assist in realizing accurate flow abnormality detection based on image analysis, improve the stability and print quality of the 3D printing process, and reduce the risk of print defects caused by flow abnormality.

[0051] As an optional embodiment, in the above step, the comprehensive prediction model is specifically used to perform the following steps: For each model part image, the average value of the site distance between the site type corresponding to the model part image and the site types corresponding to all other model part images is calculated to obtain a site distance parameter corresponding to the model part image; optionally, the site distance is the distance between the corresponding positions of two site types in the preset model data; The product of the prediction probability corresponding to the model part image and the site distance parameter is calculated to obtain a reliability parameter corresponding to the model part image; All model part images with reliability parameters greater than a preset parameter threshold are screened out to obtain an image set; The mode item of the site type corresponding to all model part images in the image set is determined as the corresponding predicted model site.

[0052] Optionally, the preset model data can be a three-dimensional model file, a CAD design file or point cloud data, which is not limited by the application.

[0053] Optionally, the calculation of the site distance parameter can be optimized in combination with the geometric characteristics or spatial distribution of the site type, which is not limited by the application.

[0054] Optionally, the parameter threshold can be a fixed threshold, a dynamic threshold or a threshold adjusted based on the scene, which is not limited by the application.

[0055] Optionally, the screening process can be based on threshold filtering, sorting algorithm or priority selection, which is not limited by the application.

[0056] Optionally, the determination of the mode item can be based on frequency statistics, weighted voting or maximum likelihood estimation, which is not limited by the application.

[0057] It can be seen that, through the above optional embodiments, the part distance parameter is obtained by calculating the average value of the distance between the part type of each frame model part image and the part type of all other model part images in the corresponding position in the preset model data, the product of the image prediction probability and the part distance parameter is calculated as the reliability parameter, the model part images with reliability parameters exceeding the preset threshold are screened out to form an image set, and the mode item of the part type in the set is taken as the predicted model part, so as to realize accurate printing model part determination based on part distance and probability weighting, improve the accuracy of 3D printing flow anomaly detection and printing quality, and reduce the risk of defects caused by part misjudgment.

[0058] As an optional embodiment, in the above step, the image data is clustered based on the predicted model part to obtain a data set, including: The target function is set to maximize the total number of image data in the data set; The constraint condition includes: The distance between the part positions of the predicted model parts of any two image data in the data set is less than a preset first distance threshold; The image similarity between any two image data in the data set is greater than a preset first similarity threshold; The distance between the part positions of the predicted model parts between each image data in the data set and any image data not in the data set is greater than a preset second distance threshold; optionally, the second distance threshold is greater than the first distance threshold; The image similarity between each image data in the data set and any image data not in the data set is less than a preset second similarity threshold; optionally, the second similarity threshold is less than the first similarity threshold; Based on the dynamic programming algorithm, the target function and the constraint condition are used to iteratively cluster and calculate all image data until the optimum is obtained, and the data set is obtained.

[0059] Optionally, the first distance threshold and the second distance threshold can be set based on the model geometric size or printing accuracy, and the present application is not limited.

[0060] Optionally, the image similarity can be calculated based on the SSIM algorithm, cosine similarity or feature vector matching, and the present application is not limited.

[0061] It can be seen that through the above optional embodiments, by setting the target function to maximize the number of image data in the data set, and ensuring that the distance between the part positions of any two image data in the set is less than the first distance threshold, the image similarity is greater than the first similarity threshold, the distance between the part positions of the image data in the set and the image data outside the set is greater than the second distance threshold, and the image similarity is less than the second similarity threshold, the optimal data set is obtained by iterative clustering algorithm based on dynamic programming algorithm, thereby realizing accurate image data clustering based on part distance and image similarity, improving the accuracy of 3D printing model part recognition and the efficiency of flow anomaly detection, and reducing the risk of misjudgment.

[0062] As an optional embodiment, in the above step, the discharge flow corresponding to the print head is calculated based on the image data and a continuous image recognition algorithm, which includes: Each image data is input into the trained flow recognition model to obtain the predicted discharge flow of the print head corresponding to each image data; All predicted discharge flows are sorted according to the time points corresponding to the image data from early to late to obtain a flow sequence; The flow sequence is input into the trained LSTM neural network to obtain the discharge flow corresponding to the print head; the LSTM neural network is trained by a training data set including a plurality of training flow sequences and corresponding stable flow labels.

[0063] Optionally, the LSTM neural network can include a single-layer LSTM, a multi-layer LSTM, or a bidirectional LSTM, which is not limited by the present application.

[0064] Optionally, the training data set can include historical printing data, simulated flow data, or real-time feedback data, which is not limited by the present application.

[0065] It can be seen that through the above optional embodiments, by inputting each frame of image data into the trained flow recognition model to predict the discharge flow of the print head, sorting the image data time points from early to late to form a flow sequence, and inputting the trained LSTM neural network to determine the discharge flow, accurate discharge flow prediction based on image recognition and time series analysis is realized, the accuracy of 3D printing platform flow anomaly detection and printing stability are improved, and the risk of printing defects caused by flow anomalies is reduced.

[0066] As an optional embodiment, in the above step, the print model part is an outer wall pattern part, an inner wall pattern part, an outer structure part, an inner structure part, a support structure part, a surrounding structure part, a hollow part, or an epitaxial structure part.

[0067] It can be seen that by the above optional embodiments, the part type of the printing model part is limited to comprehensively represent the structural features of the printing model part, to assist in realizing accurate flow abnormality detection based on image analysis, to improve the stability and printing quality of the 3D printing process, and to reduce the risk of printing defects caused by flow abnormality.

[0068] As an optional embodiment, in the above step, the flow abnormality of the print head is analyzed according to the printing model part and the discharge flow, including: determining the reference flow corresponding to the printing model part in the preset printing plan data; calculating the flow difference between the discharge flow and the reference flow; when the flow difference is greater than a preset difference threshold, determining that the print head has a flow abnormality.

[0069] Optionally, the difference threshold can be a fixed threshold, a dynamic threshold, or a threshold adjusted based on a printing task, which is not limited by the present application.

[0070] Optionally, the determination of the flow abnormality can be based on threshold judgment, an abnormality detection model, or rule reasoning, which is not limited by the present application.

[0071] It can be seen that by the above optional embodiments, by determining the reference flow corresponding to the printing model part in the preset printing plan data, calculating the flow difference between the discharge flow and the reference flow, and determining that the print head has a flow abnormality when the difference exceeds the preset threshold, accurate flow abnormality detection based on plan data comparison is realized, the printing quality and stability of the 3D printing platform are improved, and the risk of printing defects caused by flow abnormality is reduced.

[0072] Embodiment two Please refer to Figure 2 , Figure 2 is a structure diagram of a 3D printing extrusion monitoring system based on continuous image analysis disclosed by the embodiments of the present application. Among them, Figure 2 The 3D printing extrusion monitoring system based on continuous image analysis described can be applied in a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). As Figure 2 shown, the 3D printing extrusion monitoring system based on continuous image analysis can include: The acquisition module 201 is configured to acquire image data of the print head of the 3D printing platform at a plurality of continuous time points.

[0073] The determination module 202 is configured to determine the printing model part of the print head based on an image recognition algorithm according to the image data. The computing module 203 is configured to calculate the discharge flow corresponding to the print head according to the image data based on a continuous image recognition algorithm. The analyzing module 204 is configured to analyze the flow abnormality of the print head according to the print model part and the discharge flow.

[0074] It can be seen that the above embodiments of the application can obtain image data of the print head of the 3D printing platform at multiple continuous time points, determine the print model part based on the image recognition algorithm, calculate the discharge flow by using the continuous image recognition algorithm, and analyze the flow abnormality according to the print model part and the discharge flow, so as to realize accurate flow abnormality detection based on image analysis, improve the stability of the 3D printing process and the printing quality, and reduce the risk of printing defects caused by flow abnormality.

[0075] As an optional embodiment, the specific manner in which the determining module determines the print model part of the print head according to the image data based on the image recognition algorithm comprises: inputting each image data into the trained joint image recognition model to obtain a predicted model part corresponding to each image data; performing clustering on all the image data based on the predicted model part to obtain a data set; determining the mode of the predicted model parts corresponding to all the image data in the data set as the print model part of the print head.

[0076] It can be seen that, by the above optional embodiment, each frame of image data is input into the trained joint image recognition model to obtain a predicted model part, the image data is clustered based on the predicted model part to form a data set, and the mode of the predicted model part in the data set is taken as the print model part of the print head, so as to realize accurate determination of the print model part based on joint recognition and clustering analysis, improve the accuracy of 3D printing flow abnormality detection and the printing quality, and reduce the risk of defects caused by misjudgment of the part.

[0077] As an optional embodiment, the joint image recognition model comprises an image segmentation model, an image recognition model, and a comprehensive prediction model; the image segmentation model is configured to segment and determine a model part image in the image data; the image recognition model is configured to predict a part type and a corresponding prediction probability corresponding to each model part image; and the comprehensive prediction model is configured to comprehensively predict the part type corresponding to each model part image to determine a corresponding predicted model part.

[0078] It can be seen that, by the above optional embodiment, the function model of the joint image recognition model is defined to realize more accurate print part recognition, assist in realizing accurate flow abnormality detection based on image analysis, improve the stability of the 3D printing process and the printing quality, and reduce the risk of printing defects caused by flow abnormality.

[0079] As an optional embodiment, the comprehensive prediction model is specifically configured to perform the following steps: For each model part image, an average value of the part distance between the part type corresponding to the model part image and the part types corresponding to all other model part images is calculated to obtain a part distance parameter corresponding to the model part image; optionally, the part distance is the distance between the corresponding positions of the two part types in the preset model data; The product of the prediction probability corresponding to the model part image and the part distance parameter is calculated to obtain a reliability parameter corresponding to the model part image; Model part images with reliability parameters greater than a preset parameter threshold are screened out to obtain an image set; The mode item of the part type corresponding to all model part images in the image set is determined as the corresponding prediction model part.

[0080] As can be seen, through the above optional embodiment, the part distance parameter is obtained by calculating the average value of the distance between the part type of each frame model part image and the part types of all other model part images at the corresponding positions in the preset model data, the product of the prediction probability and the part distance parameter is calculated as the reliability parameter, the model part images with reliability parameters greater than the preset threshold are screened out to form the image set, and the mode item of the part type in the set is taken as the prediction model part, thereby realizing accurate printing model part determination based on part distance and probability weighting, improving the accuracy of 3D printing flow anomaly detection and printing quality, and reducing the risk of defects caused by part misjudgment.

[0081] As an optional embodiment, the determining module clusters all image data based on the prediction model part to obtain a data set in the following specific manner: The target function is set to maximize the total number of image data in the data set; The constraint conditions include: The distance between the part positions of the prediction model parts of any two image data in the data set is less than a preset first distance threshold; The image similarity between any two image data in the data set is greater than a preset first similarity threshold; The distance between the part positions of the prediction model parts between each image data in the data set and any image data not in the data set is greater than a preset second distance threshold; optionally, the second distance threshold is greater than the first distance threshold; The image similarity between each image data in the data set and any image data not in the data set is less than a preset second similarity threshold; optionally, the second similarity threshold is less than the first similarity threshold; Based on the dynamic programming algorithm, all image data are iteratively clustered according to the objective function and the constraint condition until the optimal data set is obtained.

[0082] It can be seen that, through the above optional embodiments, by setting the objective function to maximize the number of image data in the data set, and by setting the constraint condition to ensure that the position distance of any two image data in the set is less than the first distance threshold, the image similarity is greater than the first similarity threshold, the position distance of the image data in the set and outside the set is greater than the second distance threshold, and the image similarity is less than the second similarity threshold, the optimal data set is obtained based on the dynamic programming algorithm, thereby realizing accurate image data clustering based on part distance and image similarity, improving the accuracy of 3D printing model part recognition and the efficiency of flow anomaly detection, and reducing the risk of misjudgment.

[0083] As an optional embodiment, the calculation module calculates the specific manner of the discharge flow corresponding to the print head based on the image data based on a continuous image recognition algorithm, which includes: Each image data is input into the trained flow recognition model to obtain the predicted discharge flow of the print head corresponding to each image data; All predicted discharge flows are sorted according to the time points corresponding to the image data from early to late to obtain a flow sequence; The flow sequence is input into the trained LSTM neural network to obtain the discharge flow corresponding to the print head; the LSTM neural network is trained by a training data set including a plurality of training flow sequences and corresponding stable flow labels.

[0084] It can be seen that, through the above optional embodiments, by inputting each frame of image data into the trained flow recognition model to predict the discharge flow of the print head, sorting the image data time points from early to late to form a flow sequence, and inputting the trained LSTM neural network to determine the discharge flow, accurate discharge flow prediction based on image recognition and time series analysis is realized, the accuracy of 3D printing platform flow anomaly detection and printing stability are improved, and the risk of printing defects caused by flow anomalies is reduced.

[0085] As an optional embodiment, the print model part is an outer wall pattern part, an inner wall pattern part, an outer structure part, an inner structure part, a support structure part, a surrounding structure part, a hollow part, or an epitaxial structure part.

[0086] It can be seen that, through the above optional embodiments, the part type of the print model part is limited to fully represent the structural characteristics of the print model part, to assist in realizing accurate flow anomaly detection based on image analysis, to improve the stability of the 3D printing process and the printing quality, and to reduce the risk of printing defects caused by flow anomalies.

[0087] As an optional embodiment, the analysis module analyzes the specific manner of the flow abnormality of the print head according to the print model part and the discharge flow, including: determining the reference flow corresponding to the print model part in the preset printing plan data; calculating the flow difference between the discharge flow and the reference flow; when the flow difference is greater than the preset difference threshold, determining that the print head has a flow abnormality.

[0088] As can be seen, through the above optional embodiment, by determining the reference flow corresponding to the print model part in the preset printing plan data, calculating the flow difference between the discharge flow and the reference flow, and determining that the print head has a flow abnormality when the difference exceeds the preset threshold, precise flow abnormality detection based on plan data comparison is realized, the printing quality and stability of the 3D printing platform are improved, and the risk of printing defects caused by flow abnormality is reduced.

[0089] Embodiment three Please refer to Figure 3 , Figure 3 is another 3D printing extrusion monitoring system based on continuous image analysis disclosed in the embodiments of the present application. Figure 3 The 3D printing extrusion monitoring system based on continuous image analysis described is applied in a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). As Figure 3 shown, the 3D printing extrusion monitoring system based on continuous image analysis can include: a memory 301 storing executable program codes; a processor 302 coupled with the memory 301; wherein the processor 302 invokes the executable program codes stored in the memory 301, for executing the steps of the 3D printing extrusion monitoring method based on continuous image analysis described in embodiment one.

[0090] Embodiment four The embodiments of the present application disclose a computer readable storage medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to execute the steps of the 3D printing extrusion monitoring method based on continuous image analysis described in embodiment one.

[0091] Embodiment five The embodiments of the present application disclose a computer program product, which includes a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the steps of the 3D printing extrusion monitoring method based on continuous image analysis described in embodiment one.

[0092] The above-described embodiments of the present specification are described with reference to particular embodiments. Other embodiments are within the scope of the following claims. In some cases, actions recited in the claims can be performed in a different order and still achieve desirable results. Additionally, the processes depicted in the accompanying figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0093] The systems, apparatuses, modules, or units illustrated by the above-described embodiments can be specifically realized by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0094] For the convenience of description, the above apparatuses are described in various units by functions respectively when described. Of course, the functions of each unit can be realized in the same or multiple software and / or hardware when implementing the present specification.

[0095] Those skilled in the art will understand that the embodiments of the present specification can be provided as a method, a system, or a computer program product. Therefore, the embodiments of the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] The present specification is described with reference to flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present specification. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a machine that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions of one or more flows and / or blocks Figure 1 The functions of one or more flows and / or blocks

[0097] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0099] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0100] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer readable media.

[0101] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0102] It should also be noted that the terms "comprising," "including," and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a... " does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0103] The specification can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of the program code means for executing steps of the methods disclosed herein. The particular sequence of steps and methods described in this specification is not necessarily required to be executed in any specific order, unless otherwise constrained by a particular implementation.

[0104] Various embodiments of the present specification are described in progressive manner, and the same or similar parts between various embodiments can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0105] Finally, it should be noted that the 3D printing extrusion monitoring method and system based on continuous image analysis disclosed in the embodiments of the present application are only the preferred embodiments of the present application, and are only used to illustrate the technical solutions of the present application, but not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent ones. The modification or replacement does not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for monitoring 3D printing extrusion based on continuous image analysis, characterized in that, The method includes: Acquire image data of the print head of the 3D printing platform at multiple consecutive time points; Based on the image data and using an image recognition algorithm, the printing model part of the print head is determined; Based on the image data, the output flow rate corresponding to the print head is calculated using a continuous image recognition algorithm; Based on the printed model location and the output flow rate, analyze the abnormal flow rate corresponding to the print head.

2. The 3D printing extrusion monitoring method based on continuous image analysis according to claim 1, characterized in that, The step of determining the printing model portion of the print head based on the image data and an image recognition algorithm includes: Each of the image data is input into the trained joint image recognition model to obtain the predicted model part corresponding to each of the image data; Based on the predicted model, all the image data are clustered to obtain a dataset; The mode of the predicted model part corresponding to all the image data in the dataset is determined as the printing model part of the print head.

3. The 3D printing extrusion monitoring method based on continuous image analysis according to claim 2, characterized in that, The joint image recognition model includes an image segmentation model, an image recognition model, and a comprehensive prediction model; the image segmentation model is used to segment and determine the model portion of the image data; the image recognition model is used to predict the type of the corresponding part of each model portion and the corresponding prediction probability; The comprehensive prediction model is used to integrate the part types corresponding to each model part image to determine the corresponding prediction model part.

4. The 3D printing extrusion monitoring method based on continuous image analysis according to claim 3, characterized in that, The comprehensive prediction model is specifically used to perform the following steps: For each of the model partial images, the average of the part distances between the part type corresponding to the model partial image and the part types corresponding to all other model partial images is calculated to obtain the part distance parameter corresponding to the model partial image; the part distance is the distance between the corresponding positions of the two part types in the preset model data; The reliability parameter corresponding to the partial image of the model is obtained by multiplying the predicted probability corresponding to the partial image of the model with the distance parameter of the part. Filter out all model images whose reliability parameters are greater than a preset parameter threshold to obtain an image set; The mode of the region type corresponding to all model partial images in the image set is determined as the corresponding predicted model region.

5. The 3D printing extrusion monitoring method based on continuous image analysis according to claim 3, characterized in that, The process of clustering all the image data based on the predicted model parts to obtain a dataset includes: The objective function is set to maximize the total number of image data in the dataset. The constraints include: The distance between the location of the predicted model part of any two image data in the dataset is less than a preset first distance threshold; The image similarity between any two image data in the dataset is greater than a preset first similarity threshold; The distance between the location of the predicted model part and each image data in the dataset and any image data outside the dataset is greater than a preset second distance threshold; the second distance threshold is greater than the first distance threshold; The image similarity between each image data in the dataset and any image data outside the dataset is less than a preset second similarity threshold; the second similarity threshold is less than the first similarity threshold; Based on the dynamic programming algorithm, according to the objective function and the constraints, all the image data are iteratively clustered until the optimal result is obtained, thus obtaining the data set.

6. The 3D printing extrusion monitoring method based on continuous image analysis according to claim 1, characterized in that, The step of calculating the output flow rate corresponding to the printhead based on the image data and a continuous image recognition algorithm includes: Each of the aforementioned image data is input into the trained flow recognition model to obtain the predicted output flow rate of the printhead corresponding to each of the aforementioned image data. All the predicted discharge flow rates are sorted from morning to night according to the time points corresponding to the image data to obtain the flow rate sequence; The flow rate sequence is input into a trained LSTM neural network to obtain the output flow rate corresponding to the printhead; the LSTM neural network is trained using a training dataset that includes multiple training flow rate sequences and corresponding stable flow rate labels.

7. The 3D printing extrusion monitoring method based on continuous image analysis according to claim 1, characterized in that, The printed model parts are the outer wall pattern part, the inner wall pattern part, the outer structure part, the inner structure part, the supporting structure part, the surrounding structure part, the hollow part, or the extension structure part.

8. The 3D printing extrusion monitoring method based on continuous image analysis according to claim 1, characterized in that, The step of analyzing the abnormal flow rate of the print head based on the printed model part and the output flow rate includes: Determine the reference flow rate corresponding to the printed model part from the preset print plan data; Calculate the flow difference between the discharge flow rate and the reference flow rate; When the flow difference is greater than a preset difference threshold, it is determined that there is an abnormal flow in the printhead.

9. A 3D printing extrusion monitoring system based on continuous image analysis, characterized in that, The system includes: The acquisition module is used to acquire image data of the print head of the 3D printing platform at multiple consecutive time points; The determining module is used to determine the printing model part of the print head based on the image data and an image recognition algorithm; The calculation module is used to calculate the output flow rate corresponding to the print head based on the image data and a continuous image recognition algorithm; The analysis module is used to analyze the abnormal flow conditions of the print head based on the printed model part and the output flow rate.

10. A 3D printing extrusion monitoring system based on continuous image analysis, characterized in that, The system includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the 3D printing extrusion monitoring method based on continuous image analysis as described in any one of claims 1-8.

Images