Modeling method of continuous fiber composite 3D printing curing molding target image fusion monitoring model

Abstract

The invention provides a modeling method of a continuous fiber composite 3D printing curing molding target image fusion monitoring model, and the method specifically comprises the following steps: 1, obtaining an infrared image and a visible light image from a set data set, and carrying out the registration of the infrared image and the visible light image; 2, processing the visible light image by adopting domain self-adaption according to a registration result of the infrared image and the visible light image to obtain a pre-processed visible light image; 3, generating a control chart according to the pre-processed visible light image; and 4, constructing an image fusion network model, and training by adopting the control chart to obtain a final model.

Description

Technical Field

[0001] The present invention relates to the field of 3D printing monitoring, and in particular to a modeling method of a target image fusion monitoring model for 3D printing and curing of continuous fiber composite materials. Background Art

[0002] Infrared images and visible light images play an important role in the field of composite material defect monitoring. Infrared images are obtained based on the thermal radiation emitted by objects. Although they are less affected by lighting conditions, the background information in infrared images is often missing. In contrast, visible light images contain more texture information, but they are very susceptible to the imaging environment and lighting conditions. In order to solve the above problems, infrared and visible light image fusion technology is proposed. Infrared and visible light image fusion technology can fuse infrared and visible light image pairs into one image. The fused image contains texture information from visible light images and thermal radiation details from infrared, which is more conducive to human observation and computer analysis.

[0003] For continuous fiber reinforced composite parts, since fiber placement is directional, its performance has obvious anisotropy, so the performance of the parts can be controlled by designing the fiber placement path. However, during the additive manufacturing process, especially in corners and arc areas, the printing quality of the fibers is often unsatisfactory, usually manifested as fiber misalignment and fiber wear. This means that the fibers in the part are not printed according to the designed path or break during printing, which directly affects the quality and performance of the parts and limits the widespread application of additive manufacturing technology for continuous fiber reinforced composites.

[0004] In order to solve this problem, many scholars and engineers are committed to finding the optimal combination of process parameters. However, for additive manufacturing, there is a strong coupling between the process parameters, the temperature field is complex and changeable, and a set of process parameters is not enough to adapt to various manufacturing environments and conditions. At present, there is still a lack of methods for online monitoring of fiber printing quality in the field of continuous fiber reinforced composite additive manufacturing. Summary of the invention

[0005] The main purpose of the present invention is to provide a modeling method for a continuous fiber composite material 3D printing curing molding target image fusion monitoring model, which is used to establish a fusion model.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is: a modeling method of a target image fusion monitoring model for 3D printing and curing of continuous fiber composite materials, which specifically includes the following steps:

[0007] Step 1: Obtain infrared images and visible light images from a set data set, and align the infrared images and visible light images;

[0008] Step 2: According to the result of the registration of the infrared image and the visible light image, the visible light image is processed by domain adaptation to obtain a preprocessed visible light image;

[0009] Step 3: Generate a control chart based on the preprocessed visible light image;

[0010] Step 4: construct an image fusion network model and use the control chart for training to obtain the final model.

[0001] Preferably, step one specifically includes the following steps:

[0002] Step 11: Set point P(X w ,Y w ,Z w ) in the infrared thermal imager pixel coordinate system and the visible light camera pixel coordinate system are respectively represented as X1(x1, y1) and X2(x2, y2). The infrared thermal imager is used to capture the infrared image, and the visible light camera is used to capture the visible light image. Assume that the plane is located at Z w =0;

[0003] Step 12, obtaining the mapping of the infrared thermal imager, can be expressed as:

[0004]

[0005] Where s1 is the scale factor, and the intrinsic matrix of the infrared thermal imager is x0 and y0 are the coordinate values ​​of the origin of the plane in the infrared thermal imager pixel coordinate system. The external matrix of the infrared thermal imager is [r1r2t] (3×3) =E1, r1, r2, t are the external parameters of the infrared camera, f x 、f y ,γ is the internal parameter of the infrared camera;

[0006] Step 13, obtaining the mapping of the visible light camera, can be expressed as:

[0007]

[0008] Where s2 is the scale factor, the extrinsic matrix of the visible light camera is [r1′r2′t′]=E2, r1′, r2′, t are the extrinsic parameters of the visible light camera, and the intrinsic matrix of the visible light camera is f x ′、f y ′ and γ′ are the internal parameters of the visible light camera;

[0009] Step 14: The mapping of infrared image point pairs to visible light image point pairs can be expressed as:

[0010] In the formula It is a 3*3 homography matrix. The H matrix represents the mapping relationship between the point pairs of the infrared image and the point pairs of the visible light image.

[0011] Preferably, step 2 specifically includes the following steps:

[0012] Step 21, replacing the corresponding low-frequency part in the visible light image with the low-frequency part in the amplitude component of the infrared image, where the low-frequency part is the part whose amplitude is less than a predetermined value. When replacing, the coordinates of the low-frequency part in the amplitude component of the infrared image need to be converted into corresponding coordinates in the visible light image through a mapping relationship H;

[0013] Step 22: Use 255-I1 and I2 as the input of DA, and obtain the output of DA through inverse Fourier transform, which is expressed as:

[0014]

[0015] in, is the visible light image obtained by inverse Fourier transform, that is, the preprocessed visible light image, I1 is the pixel of the low-frequency part in the amplitude component of the infrared image, I2 is the pixel of the low-frequency part in the visible light image, F A () is the amplitude component, F P () is the phase component, M α is a resizable mask defined at the center of the amplitude component, M α (h,w)=l (h,w)∈[-αh：αh，-αW：αW] , h and w are the length and width of the mask respectively, and α is a hyperparameter, indicating that the mask size is (0.01).

[0001] Preferably, step three specifically includes the following steps:

[0002] Step 31: Remove the In the environment part, only the image of the part obtained by 3D printing is retained;

[0003] Step 32: Since two images from two consecutive layers may have extremely high similarity, this similarity is autocorrelation and needs to be eliminated. Therefore, an ARIMA filter is used to eliminate the autocorrelation of the image obtained in step 31.

[0004] Step 33: compare the filtered image obtained in step 32 with the standard image to obtain the difference between the filtered image and the standard image to determine whether the process is under control. In the 3D printed product, different product parts correspond to different standard images, and the same product parts correspond to the same standard image. The standard image is obtained by photographing the uniform layer in 3D printing. The uniform layer refers to the printing layer that meets the requirements in the 3D printing process.

[0005] Step 34: If the filtered image is under control, the filtered image is added to the standard image to form a new standard image, and similar adjacent layers are clustered according to the distribution of the standard image, and a control chart is generated for the standard image corresponding to each isomorphic cluster.

[0006] Preferably, the step 4 specifically includes the following steps:

[0007] Step 41, constructing an image fusion network model, wherein the network fusion model includes a generator and two discriminators, wherein the generator is composed of an encoder, a feature generator and a decoder, wherein the encoder has three convolution kernels of 3x3, 5x5 and 7x7 for extracting multi-receptive fields and multi-scale depth features of the input image, and the receptive fields and depth features extracted by the three convolution kernels are concatenated and fed to the feature generator, which extracts features of the input image, and the decoder is used to generate a fused image according to the features extracted by the feature generator;

[0008] Step 42: input the control chart into the image fusion network model for training;

[0009] Step 43: Use the loss function to calculate the loss value in the training process, optimize the gradient according to the loss value and back propagate, and update the parameters in the generative adversarial network;

[0010] Step 44, repeat step 42-step 43 until all control charts are input and the final image fusion network model is obtained.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] The modeling method of the present invention fuses infrared images and visible light images as training data to train the model. The obtained model can comprehensively consider the features in the two images, and the established model can better monitor the 3D printing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0014] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0015] like Figure 1 As shown, a modeling method of a target image fusion monitoring model for 3D printing curing molding of continuous fiber composite materials specifically includes the following steps:

[0016] Step 1: Obtain infrared images and visible light images from the set data set, and align the infrared images and visible light images, which specifically includes the following steps:

[0017] Step 11: Set point P(X w ,Y w ,Z w ) in the infrared thermal imager pixel coordinate system and the visible light camera pixel coordinate system are respectively represented as X1(x1, y1) and X2(x2, y2). The infrared thermal imager is used to capture the infrared image, and the visible light camera is used to capture the visible light image. Assume that the plane is located at Z w =0;

[0018] Step 12, obtaining the mapping of the infrared thermal imager, can be expressed as:

[0019]

[0020] Where s1 is the scale factor, and the intrinsic matrix of the infrared thermal imager is x0 and y0 are the coordinate values ​​of the origin of the plane in the infrared thermal imager pixel coordinate system. The external matrix of the infrared thermal imager is [r1r2t] (3×3) =E1, r1, r2, t are the external parameters of the infrared camera, f x 、f y ,γ is the internal parameter of the infrared camera;

[0021] Step 13, obtaining the mapping of the visible light camera, can be expressed as:

[0022]

[0023] Where s2 is the scale factor, the extrinsic matrix of the visible light camera is [r1′r2′t′]=E2, r1′, r2′, t are the extrinsic parameters of the visible light camera, and the intrinsic matrix of the visible light camera is f x ′、f y ′ and γ′ are the internal parameters of the visible light camera;

[0024] Step 14: The mapping of infrared image point pairs to visible light image point pairs can be expressed as:

[0025] In the formula It is a 3*3 homography matrix. The H matrix represents the mapping relationship between the point pairs of the infrared image and the point pairs of the visible light image.

[0026] Step 15, when the relative positions of the infrared thermal imager, the visible light camera and the plane are fixed, H is a constant matrix, which represents the mapping relationship between the point pairs of the infrared image and the visible light image. The spatial registration of the infrared image and the visible light image can be achieved by solving H. The unique solution of H is calculated using the least squares method and four pairs of points, and it is substituted back into formula (3) to obtain the registration result.

[0027] Step 2: Use domain adaptation (DA) to preprocess the visible light image to reduce the distribution difference between the infrared and visible light image pairs and reduce the impact of background differences on the similarity measurement of infrared and visible light images. The specific steps include the following:

[0028] Step 21, in order to model the background distribution of the visible light image similar to the infrared image, the low-frequency part in the amplitude component of the infrared image is used to replace the corresponding low-frequency part in the visible light image, where the low-frequency part is a part whose amplitude is less than a predetermined value. When replacing, the coordinates of the low-frequency part in the amplitude component of the infrared image need to be converted into corresponding coordinates in the visible light image through a mapping relationship H;

[0029] Step 22: Since dark-toned surface defects often show higher absorbance, resulting in the infrared image appearing white, 255-I1 and I2 are used as the input of DA, and the output of DA is obtained by inverse Fourier transform, which is expressed as:

[0030]

[0031] in, is the visible light image obtained by inverse Fourier transform, which is also the output result of DA. I1 is the pixel of the low-frequency part in the amplitude component of the infrared image, and I2 is the pixel of the low-frequency part in the visible light image. A () is the amplitude component, F P () is the phase component, M α is a resizable mask defined at the center of the amplitude component, M α (h,w)=l (h,w)∈[-αh：αh，-αW：αW] , h and w are the length and width of the mask respectively, and α is a hyperparameter, indicating that the mask size is (0.01).

[0032] Step 3: Based on the result obtained in step 2 Generate a control chart, including the following steps:

[0033] Step 31: Remove the In the environment part, only the image of the part obtained by 3D printing is retained;

[0034] Step 32: Since two images from two consecutive layers may have extremely high similarity, this similarity is autocorrelation and needs to be eliminated. Therefore, an ARIMA filter is used to eliminate the autocorrelation of the image obtained in step 31.

[0035] Step 33: compare the filtered image obtained in step 32 with the standard image to obtain the difference between the filtered image and the standard image to determine whether the process is under control. In the 3D printed product, different product parts correspond to different standard images, and the same product parts correspond to the same standard image. The standard image is obtained by photographing the uniform layer in 3D printing. The uniform layer refers to the printing layer that meets the requirements in the 3D printing process.

[0036] Step 34: If the filtered image is under control, the filtered image is added to the standard image to form a new standard image, and similar adjacent layers are clustered according to the distribution of the standard image, and a control chart is generated for the standard image corresponding to each isomorphic cluster.

[0037] Step 4: Obtain the image fusion network model, which specifically includes the following steps:

[0038] Step 41, constructing an image fusion network model, wherein the network fusion model includes a generator and two discriminators, wherein the generator is composed of an encoder, a feature generator and a decoder, wherein the encoder has three convolution kernels of 3x3, 5x5 and 7x7 for extracting multi-receptive fields and multi-scale depth features of the input image, and the receptive fields and depth features extracted by the three convolution kernels are concatenated and fed to the feature generator, which extracts features of the input image, and the decoder is used to generate a fused image according to the features extracted by the feature generator;

[0039] Step 42: input the control chart into the image fusion network model for training;

[0040] Step 43: using a loss function to calculate the loss value in the training process, optimizing the gradient according to the loss value and back-propagating, and updating the parameters in the generative adversarial network; the loss function uses the loss function in the prior art;

[0041] Step 44, repeat step 42-step 43 until all control charts are input and the final image fusion network model is obtained.

[0042] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A modeling method for a target image fusion monitoring model for 3D printing and curing of continuous fiber composite materials, characterized in that: The specific steps include: Step 1: Obtain an infrared image and a visible light image from a set data set, and align the infrared image and the visible light image; Step 2: According to the result of the registration of the infrared image and the visible light image, the visible light image is processed by domain adaptation to obtain a preprocessed visible light image; Step 3: Generate a control chart based on the preprocessed visible light image; Step 4: construct an image fusion network model and use the control chart for training to obtain the final model.

2. The modeling method of a target image fusion monitoring model for 3D printing and curing of continuous fiber composite materials according to claim 1 is characterized in that: Step 1 specifically includes the following steps: Step 11: Set point P(X w ,Y w ,Z w ) in the infrared thermal imager pixel coordinate system and the visible light camera pixel coordinate system are respectively represented as X1(x1, y1) and X2(x2, y2). The infrared thermal imager is used to capture the infrared image, and the visible light camera is used to capture the visible light image. Assume that the plane is located at Z w =0; Step 12: Obtain the mapping of the infrared thermal imager, expressed as: Where s1 is the scale factor, and the intrinsic matrix of the infrared thermal imager is x0 and y0 are the coordinate values ​​of the origin of the plane in the infrared thermal imager pixel coordinate system. The external matrix of the infrared thermal imager is [r1 r2 t] (3×3) =E1, r1, r2, t are the external parameters of the infrared camera, f x 、f y ,γ is the internal parameter of the infrared camera; Step 13: Obtain the mapping of the visible light camera, expressed as: Where s2 is the scale factor, the extrinsic matrix of the visible light camera is [r1′r2′t′]=E2, r1′, r2′, t are the extrinsic parameters of the visible light camera, and the intrinsic matrix of the visible light camera is is the internal reference of the visible light camera; Step 14: The mapping between infrared image point pairs and visible light image point pairs is expressed as: In the formula It is a 3*3 homography matrix. The H matrix represents the mapping relationship between the point pairs of the infrared image and the point pairs of the visible light image.

3. The modeling method of a target image fusion monitoring model for 3D printing and curing of continuous fiber composite materials according to claim 2 is characterized in that: Step 2 specifically includes the following steps: Step 21, replacing the corresponding low-frequency part in the visible light image with the low-frequency part in the amplitude component of the infrared image, where the low-frequency part is the part whose amplitude is less than a predetermined value. When replacing, the coordinates of the low-frequency part in the amplitude component of the infrared image need to be converted into corresponding coordinates in the visible light image through a mapping relationship H; Step 22: Use 255-I1 and I2 as the input of DA, and obtain the output of DA through inverse Fourier transform, which is expressed as: in, is the visible light image obtained by inverse Fourier transform, that is, the preprocessed visible light image, I1 is the pixel of the low-frequency part in the amplitude component of the infrared image, I2 is the pixel of the low-frequency part in the visible light image, F A ( ) is the amplitude component, F P ( ) is the phase component, M α is a resizable mask defined at the center of the amplitude component, M α (h,w)=l (h,w)∈[-αh：αh，-αW：αW] , h and w are the length and width of the mask respectively, and α is a hyperparameter, indicating that the mask size is (0.01).

4. The modeling method of a target image fusion monitoring model for 3D printing and curing of continuous fiber composite materials according to claim 3 is characterized in that: Step 3 specifically includes the following steps: Step 31: Remove the In the environment part, only the image of the part obtained by 3D printing is retained; Step 32: Since two images from two consecutive layers may have extremely high similarity, this similarity is autocorrelation and needs to be eliminated. Therefore, an ARIMA filter is used to eliminate the autocorrelation of the image obtained in step 31. Step 33: compare the filtered image obtained in step 32 with the standard image to obtain the difference between the filtered image and the standard image to determine whether the process is under control. In the 3D printed product, different product parts correspond to different standard images, and the same product parts correspond to the same standard image. The standard image is obtained by photographing the uniform layer in 3D printing. The uniform layer refers to the printing layer that meets the requirements in the 3D printing process. Step 34: If the filtered image is under control, the filtered image is added to the standard image to form a new standard image, and similar adjacent layers are clustered according to the distribution of the standard image, and a control chart is generated for the standard image corresponding to each isomorphic cluster.

5. The modeling method of a target image fusion monitoring model for 3D printing and curing of continuous fiber composite materials according to claim 4, characterized in that: The step 4 specifically includes the following steps: Step 41, constructing an image fusion network model, wherein the network fusion model includes a generator and two discriminators, wherein the generator is composed of an encoder, a feature generator and a decoder, wherein the encoder has three convolution kernels of 3x3, 5x5 and 7x7 for extracting multi-receptive fields and multi-scale depth features of the input image, and the receptive fields and depth features extracted by the three convolution kernels are concatenated and fed to the feature generator, which extracts features of the input image, and the decoder is used to generate a fused image according to the features extracted by the feature generator; Step 42: input the control chart into the image fusion network model for training; Step 43: Use the loss function to calculate the loss value in the training process, optimize the gradient according to the loss value and back propagate, and update the parameters in the generative adversarial network; Step 44, repeat step 42-step 43 until all control charts are input and the final image fusion network model is obtained.

Images