A modeling method for target image fusion monitoring model of continuous fiber composite material 3D printing curing molding

Abstract

The present invention provides a modeling method for a target image fusion monitoring model for 3D printing and curing molding of continuous fiber composite materials, which specifically includes the following steps: step 1, obtaining an infrared image and a visible light image from a set data set, and aligning the infrared image and the visible light image; step 2, based on the result of the registration of the infrared image and the visible light image, using domain adaptation to process the visible light image to obtain a preprocessed visible light image; step 3, generating a control chart based on the preprocessed visible light image; step 4, constructing an image fusion network model, and using the control chart for training to obtain a final model.

Description

Technical Field

[0001] The present invention relates to the field of 3D printing monitoring, and in particular to a modeling method for a target image fusion monitoring model for 3D printing and curing molding of continuous fiber composite materials. Background Art

[0002] Infrared and visible light images play an important role in composite material defect monitoring. Infrared images are acquired based on the thermal radiation emitted by an object. While they are less affected by lighting conditions, background information is often missing from infrared images. In contrast, visible light images contain more texture information, but are highly susceptible to the effects of the imaging environment and lighting conditions. To address these issues, infrared and visible light image fusion technology has been proposed. This technology fuses infrared and visible light image pairs into a single image that incorporates texture information from the visible light image and thermal radiation details from the infrared, making it more accessible to both human observation and computer analysis.

[0003] For continuous fiber-reinforced composite parts, the performance of these parts is significantly anisotropic due to the directional nature of fiber placement. Therefore, the performance of these parts can be controlled by designing the fiber placement path. However, during the additive manufacturing process, especially in corners and arcs, the fiber printing quality is often unsatisfactory, often manifesting as fiber misalignment and fiber wear. This means that the fibers in the part are not printed according to the designed path or break during printing, which directly affects the quality and performance of the part and limits the widespread application of additive manufacturing technology for continuous fiber-reinforced composites.

[0004] To address this issue, numerous researchers and engineers have dedicated themselves to finding the optimal combination of process parameters. However, in additive manufacturing, process parameters are strongly coupled, and temperature fields are complex and variable. A single set of process parameters is insufficient to adapt to diverse manufacturing environments and conditions. Currently, there is a lack of methods for online monitoring of fiber printing quality in the additive manufacturing industry for continuous fiber-reinforced composites. Summary of the Invention

[0005] The main purpose of the present invention is to provide a modeling method for a target image fusion monitoring model of continuous fiber composite material 3D printing curing molding, which is used to establish a fusion model.

[0006] To achieve the above objectives, the present invention adopts a technical solution: a modeling method for a target image fusion monitoring model for 3D printing and curing of continuous fiber composite materials, which specifically includes the following steps:

[0007] Step 1: Obtain infrared images and visible light images from the set data set, and align the infrared images and visible light images;

[0008] Step 2: Based on the registration results of the infrared image and the visible light image, domain adaptation is used to process the visible light image to obtain a preprocessed visible light image;

[0009] Step 3: generating a control chart based on the preprocessed visible light image;

[0010] Step 4: Build an image fusion network model and use the control chart for training to obtain the final model.

[0011] Preferably, step one specifically includes the following steps:

[0012] Step 11, set point P(X w ,Y w ,Z w ) in the infrared thermal imager pixel coordinate system and the visible light camera pixel coordinate system are respectively expressed as X1(x1,y1) and X2(x2,y2). The infrared thermal imager is used to capture the infrared image, and the visible light camera is used to capture the visible light image. Assume that the plane is located at Z w =0;

[0013] Step 12: Obtain the mapping of the infrared thermal imager, which can be expressed as:

[0014]

[0015] Where s1 is the scale factor, and the intrinsic matrix of the infrared thermal imager is x0 and y0 are the coordinates of the origin of the plane in the infrared thermal imager pixel coordinate system. The extrinsic matrix of the infrared thermal imager is [r1r2t] (3×3) =E1, r1, r2, t are the external parameters of the infrared camera, f x 、f y ,γ is the internal parameter of the infrared camera;

[0016] Step 13: Obtain the mapping of the visible light camera, which can be expressed as:

[0017]

[0018] Where s2 is the scale factor, the extrinsic matrix of the visible light camera is [r1′r2′t′]=E2, r1′, r2′, t are the extrinsic parameters of the visible light camera, and the intrinsic matrix of the visible light camera is f x ′、f y ′ and γ′ are the internal parameters of the visible light camera;

[0019] Step 14: The mapping between infrared image point pairs and visible light image point pairs can be expressed as:

[0020]

[0021] In the formula It is a 3*3 homography matrix. The H matrix represents the mapping relationship between the point pairs of the infrared image and the point pairs of the visible light image.

[0022] Preferably, step 2 specifically includes the following steps:

[0023] Step 21: Replace the corresponding low-frequency portion of the visible light image with the low-frequency portion of the amplitude component of the infrared image. The low-frequency portion is the portion whose amplitude is less than a predetermined value. When replacing, the coordinates of the low-frequency portion of the amplitude component of the infrared image must first be converted into corresponding coordinates in the visible light image using a mapping relationship H.

[0024] Step 22: Use 255-I1 and I2 as the input of DA, and obtain the output of DA through inverse Fourier transform, which is expressed as:

[0025]

[0026] in, is the visible light image obtained by inverse Fourier transform, that is, the preprocessed visible light image, I1 is the pixel of the low-frequency part in the amplitude component of the infrared image, I2 is the pixel of the low-frequency part in the visible light image, F A () is the amplitude component, F P () is the phase component, M α is a resizable mask defined at the center of the amplitude component, M α (h,w)=l (h,w)∈[-αh：αh，-αW：αW] , h and w are the length and width of the mask respectively, and α is a hyperparameter indicating that the mask size is (0.01).

[0027] Preferably, step three specifically includes the following steps:

[0028] Step 31: Remove the In the environment part, only the image of the 3D printed part is retained;

[0029] Step 32: Since two images from two consecutive layers may have extremely high similarity, this similarity is autocorrelation and needs to be eliminated. Therefore, an ARIMA filter is used to eliminate the autocorrelation of the image obtained in step 31.

[0030] Step 33: Compare the filtered image obtained in step 32 with the standard image to obtain the difference between the filtered image and the standard image to determine whether the process is under control. In a 3D printed product, different product parts correspond to different standard images, and the same product parts correspond to the same standard image. The standard image is obtained by photographing a uniform layer in the 3D printing process. The uniform layer refers to a printed layer that meets the requirements during the 3D printing process.

[0031] Step 34: If the filtered image is under control, the filtered image is added to the standard image to form a new standard image, and similar adjacent layers are clustered according to the distribution of the standard image, and a control chart is generated for the standard image corresponding to each isomorphic cluster.

[0032] Preferably, the step 4 specifically includes the following steps:

[0033] Step 41: Construct an image fusion network model. The network fusion model includes a generator and two discriminators. The generator consists of an encoder, a feature generator, and a decoder. The encoder has three convolution kernels of 3x3, 5x5, and 7x7, which are used to extract multi-receptive fields and multi-scale depth features of the input image. The receptive fields and depth features extracted by the three convolution kernels are spliced ​​and fed to the feature generator. The feature generator extracts features of the input image. The decoder is used to generate a fused image based on the features extracted by the feature generator.

[0034] Step 42: inputting the control chart into the image fusion network model for training;

[0035] Step 43: Use the loss function to calculate the loss value during the training process, optimize the gradient according to the loss value and backpropagate, and update the parameters in the generative adversarial network;

[0036] Step 44: Repeat steps 42 and 43 until all control charts are input, and obtain the final image fusion network model.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The modeling method of the present invention fuses infrared images and visible light images as training data to train the model. The obtained model can comprehensively consider the features in the two images, and the established model can better monitor the 3D printing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0040] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are merely examples, and those skilled in the art may conceive of other obvious variations.

[0041] like Figure 1 As shown, a modeling method for a target image fusion monitoring model for 3D printing and curing of continuous fiber composite materials specifically includes the following steps:

[0042] Step 1: Obtain infrared images and visible light images from the set data set and align the infrared images and visible light images. The specific steps include the following:

[0043] Step 11, set point P(X w ,Y w ,Z w ) in the infrared thermal imager pixel coordinate system and the visible light camera pixel coordinate system are respectively expressed as X1(x1,y1) and X2(x2,y2). The infrared thermal imager is used to capture the infrared image, and the visible light camera is used to capture the visible light image. Assume that the plane is located at Z w =0;

[0044] Step 12: Obtain the mapping of the infrared thermal imager, which can be expressed as:

[0045]

[0046] Where s1 is the scale factor, and the intrinsic matrix of the infrared thermal imager is x0 and y0 are the coordinates of the origin of the plane in the infrared thermal imager pixel coordinate system. The extrinsic matrix of the infrared thermal imager is [r1r2t] (3×3) =E1, r1, r2, t are the external parameters of the infrared camera, f x 、f y ,γ is the internal parameter of the infrared camera;

[0047] Step 13: Obtain the mapping of the visible light camera, which can be expressed as:

[0048]

[0049] Where s2 is the scale factor, the extrinsic matrix of the visible light camera is [r1′r2′t′]=E2, r1′, r2′, t are the extrinsic parameters of the visible light camera, and the intrinsic matrix of the visible light camera is f x ′、f y ′ and γ′ are the internal parameters of the visible light camera;

[0050] Step 14: The mapping between infrared image point pairs and visible light image point pairs can be expressed as:

[0051] In the formula It is a 3*3 homography matrix. The H matrix represents the mapping relationship between the point pairs of the infrared image and the point pairs of the visible light image.

[0052] Step 15: When the relative positions of the infrared thermal imager, the visible light camera, and the plane are fixed, H is a constant matrix representing the mapping relationship between the point pairs of the infrared image and the visible light image. The spatial registration of the infrared image and the visible light image can be achieved by solving H. The unique solution of H is calculated using the least squares method and four pairs of points, and then substituted back into formula (3) to obtain the registration result.

[0053] Step 2: Use domain adaptation (DA) to preprocess the visible light image to reduce the distribution difference between the infrared and visible light image pairs and reduce the impact of background differences on the similarity measurement of infrared and visible light images. The specific steps include the following:

[0054] Step 21: To model the background distribution of the visible light image similar to the infrared image, replace the corresponding low-frequency portion of the visible light image with the low-frequency portion of the amplitude component of the infrared image. The low-frequency portion is the portion whose amplitude is less than a predetermined value. During the replacement, the coordinates of the low-frequency portion of the amplitude component of the infrared image must first be converted into corresponding coordinates in the visible light image using a mapping relationship H.

[0055] Step 22: Since dark-toned surface defects often exhibit higher absorbance, resulting in the infrared image appearing white, 255-I1 and I2 are used as the input of DA, and the output of DA is obtained through inverse Fourier transform, which is expressed as:

[0056]

[0057] in, is the visible light image obtained by inverse Fourier transform, which is also the output result of DA. I1 is the pixel of the low-frequency part in the amplitude component of the infrared image, I2 is the pixel of the low-frequency part in the visible light image, and F A () is the amplitude component, F P () is the phase component, M α is a resizable mask defined at the center of the amplitude component, M α (h,w)=l (h,w)∈[-αh：αh，-αW：αW] , h and w are the length and width of the mask respectively, and α is a hyperparameter indicating that the mask size is (0.01).

[0058] Step 3: Based on the result from step 2 Generate a control chart, including the following steps:

[0059] Step 31: Remove the In the environment part, only the image of the 3D printed part is retained;

[0060] Step 32: Since two images from two consecutive layers may have extremely high similarity, this similarity is autocorrelation and needs to be eliminated. Therefore, an ARIMA filter is used to eliminate the autocorrelation of the image obtained in step 31.

[0061] Step 33: Compare the filtered image obtained in step 32 with the standard image to obtain the difference between the filtered image and the standard image to determine whether the process is under control. In a 3D printed product, different product parts correspond to different standard images, and the same product parts correspond to the same standard image. The standard image is obtained by photographing a uniform layer in the 3D printing process. The uniform layer refers to a printed layer that meets the requirements during the 3D printing process.

[0062] Step 34: If the filtered image is under control, the filtered image is added to the standard image to form a new standard image, and similar adjacent layers are clustered according to the distribution of the standard image, and a control chart is generated for the standard image corresponding to each isomorphic cluster.

[0063] Step 4: Obtain the image fusion network model, which specifically includes the following steps:

[0064] Step 41: Construct an image fusion network model. The network fusion model includes a generator and two discriminators. The generator consists of an encoder, a feature generator, and a decoder. The encoder has three convolution kernels of 3x3, 5x5, and 7x7, which are used to extract multi-receptive fields and multi-scale depth features of the input image. The receptive fields and depth features extracted by the three convolution kernels are spliced ​​and fed to the feature generator. The feature generator extracts features of the input image. The decoder is used to generate a fused image based on the features extracted by the feature generator.

[0065] Step 42: inputting the control chart into the image fusion network model for training;

[0066] Step 43: Calculate the loss value during the training process using a loss function, optimize the gradient based on the loss value and perform backpropagation to update the parameters in the generative adversarial network; the loss function uses the loss function in the prior art;

[0067] Step 44: Repeat steps 42 and 43 until all control charts are input, and obtain the final image fusion network model.

[0068] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A modeling method for a target image fusion monitoring model for 3D printing and curing of continuous fiber composite materials, characterized in that: The specific steps include: Step 1: Obtain infrared images and visible light images from the set data set, and align the infrared images and visible light images; Step 2: Based on the registration results of the infrared image and the visible light image, domain adaptation is used to process the visible light image to obtain a preprocessed visible light image; Step 3: generating a control chart based on the preprocessed visible light image; Step 4: construct an image fusion network model and use the control chart for training to obtain the final model; The step 2 specifically includes the following steps: Step 21: Replace the corresponding low-frequency portion of the visible light image with the low-frequency portion of the amplitude component of the infrared image. The low-frequency portion is the portion whose amplitude is less than a predetermined value. When replacing, the coordinates of the low-frequency portion of the amplitude component of the infrared image must first be converted into corresponding coordinates in the visible light image using a mapping relationship H. Step 22, 255- and As the input of DA, the output of DA is obtained by inverse Fourier transform, which is expressed as: ， in, is the visible light image obtained by inverse Fourier transform, that is, the preprocessed visible light image, is the pixel of the low-frequency part in the amplitude component of the infrared image, is the pixel of the low-frequency part in the visible light image, is the amplitude component, is the phase component, is a resizable mask defined at the center of the amplitude component, , h and w are the length and width of the mask respectively, is a hyperparameter, indicating that the mask size is 0.01; Step 3 specifically includes the following steps: Step 31: Remove the In the environment part, only the image of the 3D printed part is retained; Step 32: Since two images from two consecutive layers may have extremely high similarity, this similarity is autocorrelation and needs to be eliminated. Therefore, an ARIMA filter is used to eliminate the autocorrelation of the image obtained in step 31. Step 33: Compare the filtered image obtained in step 32 with the standard image to obtain the difference between the filtered image and the standard image to determine whether the process is under control. In a 3D printed product, different product parts correspond to different standard images, and the same product parts correspond to the same standard image. The standard image is obtained by photographing a uniform layer in the 3D printing process. The uniform layer refers to a printed layer that meets the requirements during the 3D printing process. Step 34: If the filtered image is under control, the filtered image is added to the standard image to form a new standard image, and similar adjacent layers are clustered according to the distribution of the standard image, and a control chart is generated for the standard image corresponding to each isomorphic cluster.

2. The modeling method of a target image fusion monitoring model for 3D printing and curing of continuous fiber composite materials according to claim 1, characterized in that: Step 1 specifically includes the following steps: Step 11, set point P ( , , ) in the infrared thermal imager pixel coordinate system and the visible light camera pixel coordinate system are respectively expressed as ( , )and ( , ), the infrared thermal imager is used to capture the infrared image, and the visible light camera is used to capture the visible light image. Assume that the plane is located at ; Step 12: Obtain the mapping of the infrared thermal imager, expressed as: （1） ， In the formula is the scale factor, and the intrinsic matrix of the infrared thermal imager is , and is the coordinate value of the origin of the plane in the infrared thermal imager pixel coordinate system, and the extrinsic matrix of the infrared thermal imager is , r1, r2, t are the external parameters of the infrared camera, 、 、 is the internal reference of the infrared camera; Step 13: Obtain the mapping of the visible light camera, expressed as: （2）， in is the scale factor, and the extrinsic matrix of the visible light camera is , 、 , t is the external parameter of the visible light camera, and the intrinsic matrix of the visible light camera is ; 、 、 is the internal reference of the visible light camera; Step 14: The mapping between infrared image point pairs and visible light image point pairs is expressed as: （3）， In the formula It is a 3*3 homography matrix. The H matrix represents the mapping relationship between the point pairs of the infrared image and the point pairs of the visible light image.

3. The modeling method of a target image fusion monitoring model for 3D printing and curing of continuous fiber composite materials according to claim 1, characterized in that: The step 4 specifically includes the following steps: Step 41: Construct an image fusion network model. The network fusion model includes a generator and two discriminators. The generator consists of an encoder, a feature generator, and a decoder. The encoder has three convolution kernels of 3x3, 5x5, and 7x7, which are used to extract multi-receptive fields and multi-scale depth features of the input image. The receptive fields and depth features extracted by the three convolution kernels are spliced ​​and fed to the feature generator. The feature generator extracts features of the input image. The decoder is used to generate a fused image based on the features extracted by the feature generator. Step 42: inputting the control chart into the image fusion network model for training; Step 43: Use the loss function to calculate the loss value during the training process, optimize the gradient according to the loss value and backpropagate, and update the parameters in the generative adversarial network; Step 44: Repeat steps 42 and 43 until all control charts are input, and obtain the final image fusion network model.

Images