3D printing anomaly detection method and system based on image processing

Abstract

The invention discloses a 3D printing anomaly detection method and system based on image processing. The method comprises the steps of image acquisition, image preprocessing, 3D printing anomaly detection model construction, 3D printing anomaly detection model parameter search and 3D printing real-time anomaly detection. The invention belongs to the technical field of printing anomaly detection, and particularly relates to a 3D printing anomaly detection method and system based on image processing, according to the scheme, a simulated anomaly mask is generated, an anomaly image foreground is calculated, the anomaly image foreground is combined with a normal image to generate a pseudo-anomaly image, and information of different scales is synthesized to calculate multi-scale aggregation printing features. Setting an exception threshold value to perform exception judgment; individual golden proportions are introduced to divide individuals into explorers and followers, the explorers randomly select dimensions for mutation to complete position updating, the followers calculate scores of all the individuals and select the individual with the highest score to update the positions of the followers, and the performance and accuracy of the 3D printing anomaly detection model are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of printing anomaly detection, and specifically refers to a 3D printing anomaly detection method and system based on image processing. Background Art

[0002] The 3D printing anomaly detection method is to conduct in-depth analysis and processing of the images collected during the 3D printing process, and use image processing algorithms and technologies to identify abnormal situations, so as to avoid the production of unqualified printed products and improve the quality and yield of 3D printing. However, the existing 3D printing anomaly detection methods have the problem of difficulty in dealing with complex and changeable printing anomalies and difficulty in effectively extracting complex 3D printing features, resulting in reduced accuracy during anomaly detection and reduced production efficiency; the existing 3D printing anomaly detection methods are not widely adaptable to different printing detections, and are difficult to be universally applicable to all 3D printing anomaly detection needs, resulting in low practicality and detection accuracy. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a 3D printing anomaly detection method and system based on image processing. In view of the problems that the existing 3D printing anomaly detection methods are difficult to deal with complex and changeable printing anomalies and difficult to effectively extract complex 3D printing features, resulting in reduced accuracy during anomaly detection and reduced production efficiency, the present solution generates a simulated anomaly mask and calculates the foreground of the abnormal image, and combines it with the normal image to generate a pseudo-abnormal image with simulated anomalies, thereby increasing the diversity of training data and alleviating the problem of insufficient adaptability to complex and changeable printing anomalies; calculates preliminary aggregate features and intermediate aggregate features, and calculates multi-scale aggregate printing features by integrating information of different scales, thereby improving the model's learning ability for complex 3D printing features; calculates the mean and covariance of multi-scale features, and sets an abnormal threshold for abnormality judgment , ensuring printing quality and production efficiency, while also improving the stability and reliability of the system; the existing 3D printing anomaly detection methods are not widely adaptable to different printing detections, and are difficult to be universally applicable to all 3D printing anomaly detection needs, resulting in low practicality and detection accuracy. This scheme introduces the individual golden ratio to divide individuals into explorers and followers. Explorers randomly select dimensions for mutation to complete position updates, which increases the diversity of search and avoids falling into local optimality; followers calculate the score of each individual based on the global optimal individual and weighted coefficient, and select the individual with the highest score to update the position of the follower, making the search more directional and reasonable, and better adaptable to different 3D printing scenarios; the optimal parameters are determined based on the fitness threshold to improve the performance and accuracy of the 3D printing anomaly detection model.

[0004] The technical solution adopted by the present invention is as follows: The present invention provides a 3D printing anomaly detection method based on image processing, the method comprising the following steps:

[0005] Step S1: image acquisition;

[0006] Step S2: image preprocessing;

[0007] Step S3: constructing a 3D printing anomaly detection model;

[0008] Step S4: 3D printing anomaly detection model parameter search;

[0009] Step S5: 3D printing real-time anomaly detection.

[0010] Furthermore, in step S1, the data collection is to collect historical 3D printing process images.

[0011] Furthermore, in step S2, the data preprocessing is to perform image denoising, image enhancement, image annotation and data set construction on the collected 3D printing process images; image denoising is to use a Gaussian filter to remove high-frequency noise in the 3D printing process images; image enhancement is to use histogram equalization to improve the contrast of the 3D printing process images; image annotation is to annotate the printing status of the 3D printing process images, and the printing status includes a normal state and an abnormal state; and constructing a data set is to construct a training data set and a test data set based on the 3D printing process images after image denoising, image enhancement and image annotation.

[0012] Furthermore, in step S3, the construction of the 3D printing anomaly detection model specifically includes the following steps:

[0013] Step S31: generating a pseudo abnormal image, including the following steps:

[0014] Step S311: Generate a simulated abnormal mask, generate a random two-dimensional Perlin noise and randomly rotate it, pre-set a binarization threshold, binarize the two-dimensional Perlin noise, obtain the original abnormal mask, and binarize the 3D printing process image in a normal state based on the binarization threshold to obtain the foreground mask of the normal image, and multiply the original abnormal mask and the foreground mask to obtain the simulated abnormal mask Q M ;

[0015] Step S312: Calculate the abnormal image foreground, and multiply the 3D printing process image in the abnormal state by the simulated abnormal mask element by element to obtain the abnormal image foreground. The formula used is as follows:

[0016] ;

[0017] In the formula, I Z is the abnormal image foreground, and are the 3D printing process images in abnormal and normal states, β is the balance factor, is the element-wise multiplication symbol;

[0018] Step S313: Generate, simulate exception mask Q M Perform the inverse operation to get the abnormal inverse mask , the 3D printing process image in normal state and the abnormal inverse mask Multiply element by element to get the normal image, then add it to the abnormal image foreground to get the pseudo abnormal image, mark its printing state as abnormal state, and add it to the training data set. The formula used is as follows:

[0019] ;

[0020] In the formula, I P It is a pseudo-abnormal image;

[0021] Step S32: extracting multi-scale aggregated printing features, including the following steps:

[0022] Step S321: Calculate the preliminary aggregated features, obtain the intermediate layer features of each 3D printing process image through the feature extractor ResNet18, upsample the multi-semantic low-resolution features of the second and third layers to match the resolution of the first layer, and then merge them through channel splicing to obtain the preliminary aggregated feature map. The formula used is as follows:

[0023] ;

[0024] Where f1 is the initial aggregate feature map, , and They are the feature maps of the 1st, 2nd and 3rd layers, UP s1 (·) and UP s2 (·)are and Upsampled to The same resolution, ω 1 ,ω 2 and ω 3 are the first weight, the second weight and the third weight respectively. , and The ratios are (W, H), (W / 2, H / 2) and (W / 4, H / 4), Concat(·) is the concatenation function, W and H are the width and height of the feature map respectively;

[0025] Step S322: Calculate the intermediate aggregate features, perform 1×1 point-by-point convolution on the preliminary aggregate features, and combine the coordinate attention mechanism to highlight the key spatial position information to obtain the intermediate aggregate feature map. The formula used is as follows:

[0026] ;

[0027] Where f2 is the intermediate aggregate feature map, CA is the coordinate attention mechanism, and PWC is a 1×1 point-by-point convolution;

[0028] Step S323: Calculate the multi-scale aggregated print features, combine the preliminary aggregated features and the intermediate aggregated features, and process them using two 3×3 convolutions to obtain a multi-scale aggregated print feature map. The formula used is as follows:

[0029] ;

[0030] In the formula, f mul is a multi-scale aggregated print feature map, It is the element-by-element addition symbol, CONV1 and CONV2 are two 3×3 convolutions;

[0031] Step S33: Designing and printing anomaly scores, including the following steps:

[0032] Step S331: Calculate the mean and covariance matrix of each location (i, j) in the multi-scale aggregated printing feature map of all 3D printing process images in a normal state, using the following formula:

[0033] ;

[0034] ;

[0035] In the formula, and is the mean and covariance matrix at (i, j), i and j are the horizontal and vertical coordinate position indexes in the image, i∈[1, W], j∈[1, H], A is the number of 3D printing process images in normal state, a is the index of 3D printing process images in normal state, is the ath normal state 3D printing process image, yes Its multi-scale aggregation prints the value of the feature map at (i, j), T is the transposition operation, γ is the regularization factor, E is the identity matrix, and γE is used as the regularization constraint;

[0036] Step S332: Anomaly detection, extract the multi-scale aggregated printing features of the 3D printing process image to be detected, calculate the printing anomaly score of each location (i, j), set the anomaly threshold, if the printing anomaly score is greater than the anomaly threshold, it means that the location is abnormal, otherwise the location is normal, and output the detection result of the model. The formula used is as follows:

[0037] ;

[0038] In the formula, is the bth 3D printing process image to be detected, b is the index of the 3D printing process image to be detected, yes Its multi-scale aggregate print feature map at (i, j) value, yes The print anomaly score of yes The inverse of.

[0039] Furthermore, in step S4, the 3D printing anomaly detection model parameter search specifically includes the following steps:

[0040] Step S41: Initialize individual positions, and adjust the balance factor β and the first weight ω in the 3D printing anomaly detection model. 1 , the second weight ω 2 , the third weight ω 3 A parameter search space is established with the regularization factor γ, C individual positions are randomly initialized in the parameter search space, and the individual positions are used as representatives of the parameters of the 3D printing anomaly detection model. The cross entropy loss function of the 3D printing anomaly detection model trained based on the individual positions on the test data set is used as the fitness value of the individual.

[0041] Step S42: Individual division: All individuals are arranged in ascending order based on the size of the fitness value, and the individual golden ratio h is pre-set. individuals act as explorers and the remaining individuals act as followers, where is rounded down, and max(·) is the maximum value function;

[0042] Step S43: Explorer position update, randomly selected from R dimensions The dimensions are mutated, and the formula used is as follows:

[0043] ;

[0044] In the formula, and are the positions of the rth dimension of the vth explorer in the gth and g+1th searches, respectively. v is the explorer index, R is the number of dimensions in the parameter search space, r is the randomly selected dimension index, g is the search count index, and g max is the maximum number of searches, q1 is the first random number in [0, 1], is the position of a randomly selected explorer in the rth dimension during the gth search, and u is an adjustment factor in [-1, 1];

[0045] Step S44: Follower position update, calculate the Euclidean distance between each individual and the global optimal individual, and normalize the fitness value and Euclidean distance corresponding to each individual, calculate the score of each individual according to the weighted coefficient, and select the individual with the highest score to update the position of the follower. The formula used is as follows:

[0046] ;

[0047] ;

[0048] In the formula, and are the positions of the wth follower at the gth and g+1th searches, respectively, φ is the weighting coefficient, w is the follower index, q2 is the second random number in [0, 1], and are the normalized fitness value and normalized Euclidean distance corresponding to the cth individual in the gth search, is the score of the cth individual in the gth search, is the position of the individual with the highest score in the g-th search, and the global optimal individual is the individual with the smallest fitness value among all individuals;

[0049] Step S45: The optimal parameters are determined, the fitness threshold is pre-set, and the fitness value of the individual is updated. When the fitness value corresponding to the global optimal individual is lower than the fitness threshold, the model parameters represented by the position of the global optimal individual are the optimal parameters, and a 3D printing anomaly detection model is constructed based on the optimal parameters; otherwise, if the maximum number of searches is reached, return to step S41 to reinitialize the individual position; otherwise, add 1 to the number of searches and return to step S42 to continue the iterative search.

[0050] Furthermore, in step S5, the 3D printing real-time anomaly detection is to collect real-time 3D printing process images, perform image preprocessing, and then input them into a 3D printing anomaly detection model constructed based on optimal parameters for processing. Based on the detection results output by the model, the printing status of the real-time 3D printing process image is obtained to complete the 3D printing anomaly detection.

[0051] The 3D printing anomaly detection system based on image processing provided by the present invention includes an image acquisition module, an image preprocessing module, a 3D printing anomaly detection model building module, a 3D printing anomaly detection model parameter search module and a 3D printing real-time anomaly detection module;

[0052] The image acquisition module acquires historical 3D printing process images and sends the data to the image preprocessing module;

[0053] The image preprocessing module performs image denoising, image enhancement, image annotation and data set construction on the collected 3D printing process images, and sends the data to the 3D printing anomaly detection model construction module;

[0054] The module for constructing a 3D printing anomaly detection model generates a simulated anomaly mask and calculates the foreground of the abnormal image, combines it with the normal image to generate a pseudo-abnormal image with simulated anomalies, calculates preliminary aggregation features and intermediate aggregation features, calculates multi-scale aggregation printing features by integrating information of different scales, calculates the mean and covariance of the multi-scale features, sets an abnormal threshold for abnormal judgment, and sends the data to the 3D printing anomaly detection model parameter search module;

[0055] The 3D printing anomaly detection model parameter search module introduces the individual golden ratio to divide individuals into explorers and followers. The explorers randomly select dimensions for mutation to complete position update, and the followers calculate the score of each individual according to the global optimal individual and weighted coefficient, select the individual with the highest score to update the position of the follower, determine the optimal parameters based on the fitness threshold, and send the data to the 3D printing real-time anomaly detection module;

[0056] The 3D printing real-time anomaly detection module obtains the printing status of the real-time 3D printing process image based on the detection result output by the 3D printing anomaly detection model.

[0057] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0058] (1) In view of the problems that existing 3D printing anomaly detection methods are difficult to deal with complex and changeable printing anomalies and difficult to effectively extract complex 3D printing features, resulting in reduced accuracy in anomaly detection and reduced production efficiency, this scheme generates simulated anomaly masks and calculates the foreground of abnormal images, and combines them with normal images to generate pseudo-abnormal images with simulated anomalies, thereby increasing the diversity of training data, alleviating the problem of insufficient adaptability to complex and changeable printing anomalies, and reducing the missed detection rate; calculates preliminary aggregate features and intermediate aggregate features, and calculates multi-scale aggregate printing features by integrating information at different scales, focusing on key information, and improving the model's learning ability for complex 3D printing features; calculates the mean and covariance of multi-scale features, sets anomaly thresholds for anomaly judgment, making the detection process more scientific, timely detecting anomalies, ensuring printing quality and production efficiency, and also improving the stability and reliability of the system.

[0059] (2) In view of the fact that the existing 3D printing anomaly detection methods are not adaptable enough to different printing detections and are difficult to be universally applied to all 3D printing anomaly detection needs, resulting in low practicality and detection accuracy, this scheme introduces the individual golden ratio to divide individuals into explorers and followers. Explorers randomly select dimensions for mutation to complete position updates, which increases the diversity of search and avoids falling into local optimality, helping to find better model parameters, thereby improving the detection accuracy and adaptability; followers calculate the score of each individual based on the global optimal individual and weighting coefficient, and select the individual with the highest score to update the position of the follower, making the search more directional and reasonable, and better adapting to different 3D printing scenarios, reducing false alarm and missed alarm rates; the optimal parameters are determined based on the fitness threshold to ensure that the optimal parameters that meet the requirements are obtained, thereby improving the performance and accuracy of the 3D printing anomaly detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A schematic diagram of the process of the 3D printing anomaly detection method based on image processing provided by the present invention;

[0061] Figure 2 A schematic diagram of a 3D printing anomaly detection system based on image processing provided by the present invention;

[0062] Figure 3 is a schematic flow chart of step S3;

[0063] Figure 4 It is a schematic diagram of the process of step S4.

[0064] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0066] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0067] Example 1, see Figure 1 The present invention provides a 3D printing anomaly detection method based on image processing, which comprises the following steps:

[0068] Step S1: Image acquisition, collecting historical 3D printing process images;

[0069] Step S2: image preprocessing, image denoising, image enhancement, image annotation and data set construction for the collected 3D printing process images;

[0070] Step S3: construct a 3D printing anomaly detection model, generate a simulated anomaly mask and calculate the foreground of the abnormal image, combine it with the normal image to generate a pseudo-abnormal image with simulated anomalies, calculate preliminary aggregation features and intermediate aggregation features, calculate multi-scale aggregation printing features by integrating information at different scales, calculate the mean and covariance of multi-scale features, and set an abnormal threshold for abnormality judgment;

[0071] Step S4: 3D printing anomaly detection model parameter search, introduce the individual golden ratio to divide individuals into explorers and followers, explorers randomly select dimensions for mutation to complete position update, followers calculate the score of each individual according to the global optimal individual and weighted coefficient, select the individual with the highest score to update the position of the follower, and determine the optimal parameters based on the fitness threshold;

[0072] Step S5: 3D printing real-time anomaly detection, based on the detection results output by the 3D printing anomaly detection model, obtain the printing status of the real-time 3D printing process image.

[0073] Example 2, see Figure 1This embodiment is based on the above embodiment. In step S2, image denoising is to remove high-frequency noise in the 3D printing process image by using a Gaussian filter; image enhancement is to improve the contrast of the 3D printing process image by using histogram equalization; image annotation is to annotate the printing status of the 3D printing process image, and the printing status includes a normal state and an abnormal state; constructing a data set is to construct a training data set and a test data set based on the 3D printing process image after image denoising, image enhancement and image annotation.

[0074] Example 3, see Figure 1 and Figure 3 This embodiment is based on the above embodiment. In step S3, building a 3D printing anomaly detection model specifically includes the following steps:

[0075] Step S31: generating a pseudo abnormal image, including the following steps:

[0076] Step S311: Generate a simulated abnormal mask, generate a random two-dimensional Perlin noise and randomly rotate it, pre-set a binarization threshold, binarize the two-dimensional Perlin noise, obtain the original abnormal mask, and binarize the 3D printing process image in a normal state based on the binarization threshold to obtain the foreground mask of the normal image, and multiply the original abnormal mask and the foreground mask to obtain the simulated abnormal mask Q M ,simulating an anomaly mask with a certain randomness and authenticity will help to subsequently generate a pseudo anomaly image that is closer to the actual situation;

[0077] Step S312: Calculate the abnormal image foreground, and multiply the 3D printing process image in the abnormal state by the simulated abnormal mask element by element to obtain the abnormal image foreground. The formula used is as follows:

[0078] ;

[0079] In the formula, I Z is the abnormal image foreground, and are the 3D printing process images in abnormal and normal states, β is the balance factor, is the element-wise multiplication symbol;

[0080] Step S313: Generate, simulate exception mask Q M Perform the inverse operation to get the abnormal inverse mask , the 3D printing process image in normal state and the abnormal inverse mask Multiply element by element to get the normal image, then add it to the abnormal image foreground to get the pseudo abnormal image, mark its printing state as abnormal state, and add it to the training data set. The generated pseudo abnormal image is more realistic and reliable, enriching the training data set, which is conducive to improving the model's ability to detect abnormal situations; the formula used is as follows:

[0081] ;

[0082] In the formula, I P It is a pseudo-abnormal image;

[0083] Step S32: extracting multi-scale aggregated printing features, including the following steps:

[0084] Step S321: Calculate the preliminary aggregated features, obtain the intermediate layer features of each 3D printing process image through the feature extractor ResNet18, upsample the multi-semantic low-resolution features of the second and third layers to match the resolution of the first layer, and then merge them through channel splicing to obtain the preliminary aggregated feature map. The formula used is as follows:

[0085] ;

[0086] Where f1 is the initial aggregate feature map, , and They are the feature maps of the 1st, 2nd and 3rd layers, UP s1 (·) and UP s2 (·)are and Upsampled to The same resolution, ω 1 ,ω 2 and ω 3 are the first weight, the second weight and the third weight respectively. , and The ratios are (W, H), (W / 2, H / 2) and (W / 4, H / 4), Concat(·) is the concatenation function, W and H are the width and height of the feature map respectively;

[0087] Step S322: Calculate the intermediate aggregate features, perform 1×1 point-by-point convolution on the preliminary aggregate features, and combine the coordinate attention mechanism to highlight the key spatial position information to obtain the intermediate aggregate feature map. The introduction of the coordinate attention mechanism can more accurately capture the features of the key positions and improve the detection accuracy of the model. The formula used is as follows:

[0088] ;

[0089] Where f2 is the intermediate aggregate feature map, CA is the coordinate attention mechanism, and PWC is a 1×1 point-by-point convolution;

[0090] Step S323: Calculate the multi-scale aggregated printing features, combine the preliminary aggregated features and the intermediate aggregated features, and use two 3×3 convolutions to process them to obtain a multi-scale aggregated printing feature map, fuse features at different levels, obtain more comprehensive information, and improve the model's ability to represent printing features; the formula used is as follows:

[0091] ;

[0092] In the formula, f mul is a multi-scale aggregated print feature map, It is the element-by-element addition symbol, CONV1 and CONV2 are two 3×3 convolutions;

[0093] Step S33: Designing and printing anomaly scores, including the following steps:

[0094] Step S331: Calculate the mean and covariance matrix of each location (i, j) in the multi-scale aggregated printing feature map of all 3D printing process images in a normal state to provide a reference standard for subsequent abnormality detection; the formula used is as follows:

[0095] ;

[0096] ;

[0097] In the formula, and is the mean and covariance matrix at (i, j), i and j are the horizontal and vertical coordinate position indexes in the image, i∈[1, W], j∈[1, H], A is the number of 3D printing process images in normal state, a is the index of 3D printing process images in normal state, is the ath normal state 3D printing process image, yes Its multi-scale aggregation prints the value of the feature map at (i, j), T is the transposition operation, γ is the regularization factor, E is the identity matrix, and γE is used as the regularization constraint;

[0098] Step S332: Anomaly detection, extract the multi-scale aggregated printing features of the 3D printing process image to be detected, calculate the printing anomaly score of each location (i, j), set the anomaly threshold, if the printing anomaly score is greater than the anomaly threshold, it means that the location is abnormal, otherwise the location is normal, and output the detection result of the model, accurately detect the abnormal area, and improve the detection performance and accuracy of the model; the formula used is as follows:

[0099] ;

[0100] In the formula, is the bth 3D printing process image to be detected, b is the index of the 3D printing process image to be detected, yes Its multi-scale aggregate print feature map at (i, j) value, yes The print anomaly score of yes The inverse of.

[0101] By performing the above operations, in view of the problems that the existing 3D printing anomaly detection methods are difficult to deal with complex and changeable printing anomalies and difficult to effectively extract complex 3D printing features, resulting in reduced accuracy in anomaly detection and reduced production efficiency, this solution generates a simulated anomaly mask and calculates the foreground of the abnormal image, and combines it with the normal image to generate a pseudo-abnormal image with simulated anomalies, thereby increasing the diversity of training data, alleviating the problem of insufficient adaptability to complex and changeable printing anomalies, and reducing the missed detection rate; calculates preliminary aggregate features and intermediate aggregate features, calculates multi-scale aggregate printing features by integrating information at different scales, focuses on key information, and improves the model's learning ability for complex 3D printing features; calculates the mean and covariance of multi-scale features, sets anomaly thresholds for anomaly judgment, makes the detection process more scientific, detects anomalies in a timely manner, ensures printing quality and production efficiency, and also improves the stability and reliability of the system.

[0102] Example 4, see Figure 1 and Figure 4 This embodiment is based on the above embodiment. In step S4, the 3D printing anomaly detection model parameter search specifically includes the following steps:

[0103] Step S41: Initialize individual positions, and adjust the balance factor β and the first weight ω in the 3D printing anomaly detection model. 1 , the second weight ω 2 , the third weight ω 3 A parameter search space is established with the regularization factor γ, C individual positions are randomly initialized in the parameter search space, and the individual positions are used as representatives of the parameters of the 3D printing anomaly detection model. The cross entropy loss function of the 3D printing anomaly detection model trained based on the individual positions on the test data set is used as the fitness value of the individual.

[0104] Step S42: Individual division: All individuals are arranged in ascending order based on the size of the fitness value, and the individual golden ratio h is pre-set. individuals act as explorers and the remaining individuals act as followers, where is rounded down, and max(·) is the maximum value function, which can distinguish explorers with better fitness and other followers in order to implement different position update strategies;

[0105] Step S43: Explorer position update, randomly selected from R dimensions The dimensions are mutated to increase the diversity of explorer positions and exploration capabilities, which helps to find better parameter combinations; the formula used is as follows:

[0106] ;

[0107] In the formula, and are the positions of the rth dimension of the vth explorer in the gth and g+1th searches, respectively. v is the explorer index, R is the number of dimensions in the parameter search space, r is the randomly selected dimension index, g is the search count index, and g max is the maximum number of searches, q1 is the first random number in [0, 1], is the position of a randomly selected explorer in the rth dimension during the gth search, and u is an adjustment factor in [-1, 1];

[0108] Step S44: Update the follower position, calculate the Euclidean distance between each individual and the global optimal individual, and normalize the fitness value and Euclidean distance corresponding to each individual, calculate the score of each individual according to the weighted coefficient, select the individual with the highest score to update the position of the follower, and encourage the follower to move closer to the better individual, thereby improving the overall search efficiency and effect; the formula used is as follows:

[0109] ;

[0110] ;

[0111] In the formula, and are the positions of the wth follower at the gth and g+1th searches, respectively, φ is the weighting coefficient, w is the follower index, q2 is the second random number in [0, 1], and are the normalized fitness value and normalized Euclidean distance corresponding to the cth individual in the gth search, is the score of the cth individual in the gth search, is the position of the individual with the highest score in the g-th search, and the global optimal individual is the individual with the smallest fitness value among all individuals;

[0112] Step S45: The optimal parameters are determined, the fitness threshold is pre-set, and the fitness value of the individual is updated. When the fitness value corresponding to the global optimal individual is lower than the fitness threshold, the model parameters represented by the position of the global optimal individual are the optimal parameters, and a 3D printing anomaly detection model is constructed based on the optimal parameters; otherwise, if the maximum number of searches is reached, return to step S41 to reinitialize the individual position; otherwise, add 1 to the number of searches and return to step S42 to continue the iterative search.

[0113] By performing the above operations, in view of the fact that the existing 3D printing anomaly detection methods are not widely adaptable to different printing detections and are difficult to be universally applicable to all 3D printing anomaly detection needs, resulting in low practicality and detection accuracy, this scheme introduces the individual golden ratio to divide individuals into explorers and followers. Explorers randomly select dimensions for mutation to complete position updates, which increases the diversity of search and avoids falling into local optimality, which helps to find better model parameters, thereby improving the accuracy and adaptability of detection; followers calculate the score of each individual based on the global optimal individual and weighting coefficient, and select the individual with the highest score to update the position of the follower, making the search more directional and reasonable, and better adapting to different 3D printing scenarios, reducing false alarm and omission rates; the optimal parameters are determined based on the fitness threshold to ensure that the optimal parameters that meet the requirements are obtained, thereby improving the performance and accuracy of the 3D printing anomaly detection model.

[0114] Example 5, see Figure 1 This embodiment is based on the above embodiment. In step S5, the real-time anomaly detection of 3D printing is to collect real-time 3D printing process images, perform image preprocessing, and then input them into a 3D printing anomaly detection model constructed based on optimal parameters for processing. Based on the detection results output by the model, the printing status of the real-time 3D printing process image is obtained to complete the 3D printing anomaly detection.

[0115] Example 6, see Figure 2 , this embodiment is based on the above embodiment, and the 3D printing anomaly detection system based on image processing provided by the present invention includes an image acquisition module, an image preprocessing module, a 3D printing anomaly detection model building module, a 3D printing anomaly detection model parameter search module and a 3D printing real-time anomaly detection module;

[0116] The image acquisition module acquires historical 3D printing process images and sends the data to the image preprocessing module;

[0117] The image preprocessing module performs image denoising, image enhancement, image annotation and data set construction on the collected 3D printing process images, and sends the data to the 3D printing anomaly detection model construction module;

[0118] The module for constructing a 3D printing anomaly detection model generates a simulated anomaly mask and calculates the foreground of the abnormal image, combines it with the normal image to generate a pseudo-abnormal image with simulated anomalies, calculates preliminary aggregation features and intermediate aggregation features, calculates multi-scale aggregation printing features by integrating information of different scales, calculates the mean and covariance of the multi-scale features, sets an abnormal threshold for abnormal judgment, and sends the data to the 3D printing anomaly detection model parameter search module;

[0119] The 3D printing anomaly detection model parameter search module introduces the individual golden ratio to divide individuals into explorers and followers. The explorers randomly select dimensions for mutation to complete position update, and the followers calculate the score of each individual according to the global optimal individual and weighted coefficient, select the individual with the highest score to update the position of the follower, determine the optimal parameters based on the fitness threshold, and send the data to the 3D printing real-time anomaly detection module;

[0120] The 3D printing real-time anomaly detection module obtains the printing status of the real-time 3D printing process image based on the detection result output by the 3D printing anomaly detection model.

[0121] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0122] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

[0123] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A 3D printing anomaly detection method based on image processing, characterized in that: The method comprises the following steps: Step S1: Image acquisition, collecting historical 3D printing process images; Step S2: image preprocessing; Step S3: construct a 3D printing anomaly detection model, generate a simulated anomaly mask and calculate the foreground of the abnormal image, combine it with the normal image to generate a pseudo-abnormal image with simulated anomalies, calculate preliminary aggregation features and intermediate aggregation features, calculate multi-scale aggregation printing features by integrating information at different scales, calculate the mean and covariance of multi-scale features, and set an abnormal threshold for abnormality judgment; Step S4: 3D printing anomaly detection model parameter search, introduce the individual golden ratio to divide individuals into explorers and followers, explorers randomly select dimensions for mutation to complete position update, followers calculate the score of each individual according to the global optimal individual and weighted coefficient, select the individual with the highest score to update the position of the follower, and determine the optimal parameters based on the fitness threshold; Step S5: 3D printing real-time anomaly detection.

2. The 3D printing anomaly detection method based on image processing according to claim 1, characterized in that: In step S3, the construction of the 3D printing anomaly detection model specifically includes the following steps: Step S31: generating a pseudo abnormal image; Step S32: extracting multi-scale aggregated printing features, including the following steps: Step S321: Calculate the preliminary aggregated features, obtain the intermediate layer features of each 3D printing process image through the feature extractor ResNet18, upsample the multi-semantic low-resolution features of the second and third layers to match the resolution of the first layer, and then merge them through channel splicing to obtain the preliminary aggregated feature map. The formula used is as follows: ； Where f1 is the initial aggregate feature map, , and They are the feature maps of the 1st, 2nd and 3rd layers respectively, UP s1 (·) and UP s2 (·)are and Upsampled to The same resolution, ω 1 ,ω 2 and ω 3 are the first weight, the second weight and the third weight respectively. , and The ratios are (W, H), (W / 2, H / 2) and (W / 4, H / 4), Concat(·) is the concatenation function, W and H are the width and height of the feature map respectively; Step S322: Calculate the intermediate aggregate features, perform 1×1 point-by-point convolution on the preliminary aggregate features, and combine the coordinate attention mechanism to highlight the key spatial position information to obtain the intermediate aggregate feature map. The formula used is as follows: ； Where f2 is the intermediate aggregate feature map, CA is the coordinate attention mechanism, and PWC is a 1×1 point-by-point convolution; Step S323: Calculate the multi-scale aggregated print features, combine the preliminary aggregated features and the intermediate aggregated features, and process them using two 3×3 convolutions to obtain a multi-scale aggregated print feature map. The formula used is as follows: ； In the formula, f mul is a multi-scale aggregated print feature map, It is the element-by-element addition symbol, CONV1 and CONV2 are two 3×3 convolutions; Step S33: Design and print anomaly scores.

3. The 3D printing anomaly detection method based on image processing according to claim 2, characterized in that: In step S31, the generating of the pseudo abnormal image specifically comprises the following steps: Step S311: Generate a simulated abnormal mask, generate a random two-dimensional Perlin noise and randomly rotate it, pre-set a binarization threshold, binarize the two-dimensional Perlin noise, obtain the original abnormal mask, and binarize the 3D printing process image in a normal state based on the binarization threshold to obtain the foreground mask of the normal image, and multiply the original abnormal mask and the foreground mask to obtain the simulated abnormal mask Q M ; Step S312: Calculate the abnormal image foreground, and multiply the 3D printing process image in the abnormal state by the simulated abnormal mask element by element to obtain the abnormal image foreground. The formula used is as follows: ； In the formula, I Z is the abnormal image foreground, and are the 3D printing process images in abnormal and normal states, β is the balance factor, is the element-wise multiplication symbol; Step S313: Generate, simulate exception mask Q M Perform the inverse operation to get the abnormal inverse mask , the 3D printing process image in normal state and the abnormal inverse mask Multiply element by element to get the normal image, then add it to the abnormal image foreground to get the pseudo abnormal image, mark its printing state as abnormal state, and add it to the training data set. The formula used is as follows: ； In the formula, I P It is a pseudo-abnormal image.

4. The 3D printing anomaly detection method based on image processing according to claim 2, characterized in that: In step S33, the design of the printing anomaly score specifically includes the following steps: Step S331: Calculate the mean and covariance matrix of each location (i, j) in the multi-scale aggregated printing feature map of all 3D printing process images in a normal state, using the following formula: ； ； In the formula, and is the mean and covariance matrix at (i, j), i and j are the horizontal and vertical coordinate position indexes in the image, A is the number of 3D printing process images in normal state, a is the index of 3D printing process images in normal state, is the ath normal state 3D printing process image, yes Its multi-scale aggregation prints the value of the feature map at (i, j), T is the transpose operation, γ is the regularization factor, and E is the unit matrix; Step S332: Anomaly detection, extract the multi-scale aggregated printing features of the 3D printing process image to be detected, calculate the printing anomaly score of each location (i, j), set the anomaly threshold, if the printing anomaly score is greater than the anomaly threshold, it means that the location is abnormal, otherwise the location is normal, and output the detection result of the model. The formula used is as follows: ； In the formula, is the bth 3D printing process image to be detected, b is the index of the 3D printing process image to be detected, yes Its multi-scale aggregate print feature map at (i, j) value, yes The print anomaly score of yes The inverse of.

5. The 3D printing anomaly detection method based on image processing according to claim 1, characterized in that: In step S4, the 3D printing anomaly detection model parameter search specifically includes the following steps: Step S41: Initialize individual positions, and adjust the balance factor β and the first weight ω in the 3D printing anomaly detection model. 1 , the second weight ω 2 , the third weight ω 3 A parameter search space is established with the regularization factor γ, C individual positions are randomly initialized in the parameter search space, and the individual positions are used as representatives of the parameters of the 3D printing anomaly detection model. The cross entropy loss function of the 3D printing anomaly detection model trained based on the individual positions on the test data set is used as the fitness value of the individual. Step S42: Individual division: All individuals are arranged in ascending order based on the size of the fitness value, and the individual golden ratio h is pre-set. individuals act as explorers and the remaining individuals act as followers, where is rounded down, and max(·) is the maximum value function; Step S43: Explorer position update, randomly selected from R dimensions The dimensions are mutated, and the formula used is as follows: ； In the formula, and are the positions of the rth dimension of the vth explorer in the gth and g+1th searches, respectively. v is the explorer index, R is the number of dimensions in the parameter search space, r is the randomly selected dimension index, g is the search count index, and g max is the maximum number of searches, q1 is the first random number, is the position of a randomly selected explorer in the rth dimension during the gth search, and u is the adjustment factor; Step S44: Follower position update, calculate the Euclidean distance between each individual and the global optimal individual, and normalize the fitness value and Euclidean distance corresponding to each individual, calculate the score of each individual according to the weighted coefficient, and select the individual with the highest score to update the position of the follower. The formula used is as follows: ； ； In the formula, and are the positions of the wth follower at the gth and g+1th searches, φ is the weighting coefficient, w is the follower index, q2 is the second random number, and are the normalized fitness value and normalized Euclidean distance corresponding to the cth individual in the gth search, is the score of the cth individual in the gth search, is the individual position with the highest score in the g-th search; Step S45: The optimal parameters are determined, the fitness threshold is pre-set, and the fitness value of the individual is updated. When the fitness value corresponding to the global optimal individual is lower than the fitness threshold, the model parameters represented by the position of the global optimal individual are the optimal parameters, and a 3D printing anomaly detection model is constructed based on the optimal parameters; otherwise, if the maximum number of searches is reached, return to step S41 to reinitialize the individual position; otherwise, add 1 to the number of searches and return to step S42 to continue the iterative search.

6. The 3D printing anomaly detection method based on image processing according to claim 1, characterized in that: In step S5, the 3D printing real-time anomaly detection is to collect real-time 3D printing process images, perform image preprocessing, and input them into a 3D printing anomaly detection model constructed based on optimal parameters for processing. Based on the detection results output by the model, the printing status of the real-time 3D printing process image is obtained to complete the 3D printing anomaly detection.

7. The 3D printing anomaly detection method based on image processing according to claim 1, characterized in that: In step S2, the image preprocessing is to perform image denoising, image enhancement, image annotation and data set construction on the collected 3D printing process images; image denoising is to use a Gaussian filter to remove high-frequency noise in the 3D printing process images; image enhancement is to use histogram equalization to improve the contrast of the 3D printing process images; image annotation is to annotate the printing status of the 3D printing process images, and the printing status includes a normal state and an abnormal state; constructing a data set is to construct a training data set and a test data set based on the 3D printing process images after image denoising, image enhancement and image annotation.

8. A 3D printing anomaly detection system based on image processing, used to implement the 3D printing anomaly detection method based on image processing as described in any one of claims 1 to 7, characterized in that: It includes an image acquisition module, an image preprocessing module, a 3D printing anomaly detection model building module, a 3D printing anomaly detection model parameter search module and a 3D printing real-time anomaly detection module; The image acquisition module acquires historical 3D printing process images and sends the data to the image preprocessing module; The image preprocessing module performs image denoising, image enhancement, image annotation and data set construction on the collected 3D printing process images, and sends the data to the 3D printing anomaly detection model construction module; The module for constructing a 3D printing anomaly detection model generates a simulated anomaly mask and calculates the foreground of the abnormal image, combines it with the normal image to generate a pseudo-abnormal image with simulated anomalies, calculates preliminary aggregation features and intermediate aggregation features, calculates multi-scale aggregation printing features by integrating information of different scales, calculates the mean and covariance of the multi-scale features, sets an abnormal threshold for abnormal judgment, and sends the data to the 3D printing anomaly detection model parameter search module; The 3D printing anomaly detection model parameter search module introduces the individual golden ratio to divide individuals into explorers and followers. The explorers randomly select dimensions for mutation to complete position update, and the followers calculate the score of each individual according to the global optimal individual and weighted coefficient, select the individual with the highest score to update the position of the follower, determine the optimal parameters based on the fitness threshold, and send the data to the 3D printing real-time anomaly detection module; The 3D printing real-time anomaly detection module obtains the printing status of the real-time 3D printing process image based on the detection result output by the 3D printing anomaly detection model.

Images