Woven preform surface parameter detection method based on deep learning

Abstract

The invention discloses a deep learning-based woven preform surface parameter detection method, which comprises the steps of collecting and constructing a woven preform image set, training a U-Net model until a loss function converges, verifying the U-Net model by using a test set, comparing an output yarn edge with an artificial mark, and finishing training of the U-Net model when MIoU is higher than a preset value. Inputting a woven preform surface image collected in real time into the trained U-Net model to output a yarn edge recognition result, removing noise of the recognition result by using image morphological processing, connecting edge breakpoints, fitting a parallelogram yarn unit by using contour extraction and an adaptive DP algorithm, and obtaining a parallelogram yarn unit; and obtaining the weaving angle, the pitch and the yarn width of the preform through geometric operation. The method for detecting the surface parameters of the braided preform based on deep learning has the characteristics of intelligence, high efficiency and accuracy, and can greatly improve the efficiency and reliability of a composite material braiding process.

Description

Technical Field

[0001] The present invention relates to the field of parameter detection of composite material braided preforms, and in particular to a method for detecting surface parameters of braided preforms based on deep learning. Background Art

[0002] Braided composite materials have shown great application potential in aerospace, automobile manufacturing, national defense and military industries due to their excellent specific strength and modulus, outstanding fatigue resistance, excellent impact resistance and high damage tolerance. Braiding angle, pitch and yarn width are important surface parameters of braided preforms, which are directly related to key performance indicators such as fiber volume fraction, mechanical response and fatigue durability of composite materials.

[0003] At present, the measurement of surface parameters such as braiding angle, pitch and yarn width mainly relies on manual operation, which is arduous, time-consuming and has limited measurement accuracy. Some studies have tried to use machine vision and image processing technology to realize the automatic measurement of surface parameters of braided preforms, but the algorithm efficiency is low and cannot meet the real-time detection requirements of surface parameters in the production process. Therefore, it is urgent to develop a high-precision and high-efficiency real-time detection method to monitor the surface parameters of preforms during the production process to ensure that the forming quality of the braided preforms meets the performance requirements. Summary of the invention

[0004] The purpose of the present invention is to provide a method for detecting the surface parameters of a braided preform based on deep learning, so as to solve the problem of how to automatically detect the surface parameters of the preform.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0006] A method for detecting surface parameters of a braided preform based on deep learning comprises the following steps:

[0007] (1) collecting a number of woven preform images to form an image set, marking the edges of the yarn parts in the woven preform images, and dividing the image set into a training set, a verification set, and a test set;

[0008] (2) Input the training set into the U-Net model for training until the loss function converges, use the test set to verify the U-Net model, compare the output preform surface yarn edge with the manual mark, and complete the training of the U-Net model when the MIoU is higher than the preset value, otherwise repeat step (2);

[0009] (3) Inputting the surface image of the woven preform to be identified into the trained U-Net model to output the yarn edge recognition result;

[0010] (4) removing noise from the yarn edge recognition result obtained in step (3) by image morphological processing and connecting edge breakpoints in the yarn edge;

[0011] (5) The image obtained in step (4) is contour extracted and the parallelogram yarn unit is fitted by an adaptive DP algorithm to calculate the preform braiding angle, pitch and yarn width.

[0012] Furthermore, the weaving angle range of the woven preform image described in step (1) is 25°-75°, the pitch range is 4-8cm, the carbon fiber used for weaving is one or more of 3k, 6k, 12k and 24k, and the woven preform structure is one or more of two-dimensional biaxial, two-dimensional triaxial, three-dimensional four-directional or three-dimensional five-directional.

[0013] Furthermore, the data augmentation method in step (1) includes one or more of rotation, cropping, splicing and grayscale, and the training set, validation set and test set are divided into 3:1:1.

[0014] Furthermore, the training process in step (2) is specifically as follows: an image in the training set is input into the U-Net model, a feature map is obtained through the convolution layer and pooling layer of the model, and features of different scales of the image are captured. Then, the feature map is used to restore the spatial resolution through upsampling and jump connection, and a segmentation probability map consistent with the size of the input image is generated. The probability map is used to represent the probability of each pixel belonging to the yarn edge or background. By setting a threshold δ, pixels with a segmentation probability greater than δ are judged as yarn edges, and the remaining pixels are judged as image backgrounds. The category label of each pixel after judgment is saved and the loss function value is calculated; the batch size and the number of iterations are set, and the remaining images in the training set are input in turn. The above process is repeated, and the loss function is gradually converged through iterative optimization.

[0015] Furthermore, the loss function L is:

[0016]

[0017] Among them, TP is the number of pixels that are both predicted and actually yarn edges, FP is the number of pixels that are predicted to be yarn edges but are actually image backgrounds, FN is the number of pixels that are predicted to be image backgrounds but are actually yarn edges, and p t is the predicted probability of the model for each category. s is a constant used to avoid the situation where the denominator is 0; β is a hyperparameter used to adjust the weight between TP and FN; γ is a focusing parameter for adjusting the weight of easy-to-classify samples; α is a weight factor for balancing positive and negative samples, which is used to deal with the problem of class imbalance.

[0018] Furthermore, the MIoU is:

[0019]

[0020] Furthermore, the image morphological processing in step (4) is specifically as follows: first, the yarn edge recognition result output in step (2) is subjected to a corrosion algorithm to eliminate noise; for image A and structural element B, the corrosion algorithm is expressed as:

[0021]

[0022] Among them, B z represents the translation of the structural element B at position z.

[0023] After eliminating the noise, the expansion algorithm is used to find the yarn edge breakpoints; for image A and structural element B, the expansion algorithm is expressed as:

[0024]

[0025] After the yarn edge breakpoint Q1 is identified, its connection with the edge point Q2 is completed by two-dimensional linear interpolation; the expression of the two-dimensional linear interpolation is:

[0026] Q=tQ1+(1-t)Q2,0≤t≤1.

[0027] Furthermore, the step (5) of fitting the parallelogram yarn unit by the adaptive DP algorithm is specifically as follows: calculating the arc length of the yarn edge contour output in the step (4), determining the initial DP algorithm fitting accuracy ε according to the arc length, performing quadrilateral fitting on the contour for multiple iterations, and if the fitting result satisfies the characteristics of the parallelogram, outputting the vertex coordinates of the quadrilateral; if unsuccessful, adjusting the fitting accuracy ε and refitting until the fitting is successful or the number of iterations is exceeded. The fitting accuracy ε of the adaptive DP algorithm is calculated as follows:

[0028] ε=μ n-1 kC,n≤N

[0029] Among them, μ is the iteration coefficient, n is the number of iterations, k is the approximate accuracy scaling factor, C is the contour circumference, and N is the iteration limit.

[0030] Furthermore, the preform braiding angle θ is obtained by geometric calculation in step (5): t , pitch p t and yarn width w t The specific calculation formula is:

[0031]

[0032]

[0033]

[0034] Among them, (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the vertex coordinates of the parallelogram approximation unit, arranged in clockwise order from top to bottom, P1 is the short side vector of the parallelogram approximation unit, and P2 is the long side vector of the parallelogram approximation unit.

[0035] Compared with the prior art, the present invention has the following beneficial effects: the surface parameter detection method of the braided preform based on deep learning of the present invention realizes the edge detection of the braided preform yarn through the U-Net model, removes the recognition noise through morphological processing, compensates for the edge breakpoints, and uses contour extraction and adaptive DP algorithm to fit the parallelogram yarn unit and calculate the braiding angle, pitch and yarn width, so as to realize the automatic measurement of the surface parameters of the preform with high efficiency and high accuracy, which can meet the detection requirements and can be applied to the real-time detection of the yarn edge of the braided preform in the production process. Compared with the traditional manual measurement method, the surface parameter detection method of the braided preform based on deep learning of the present invention reduces the interference of human subjectivity, improves the intelligent measurement level in the preform production process, reduces the work intensity of the quality inspectors, realizes high-precision detection and real-time feedback, and can effectively reduce the surface parameter unevenness or mutation problems in the production process of the braided preform, and ensures that the fiber volume fraction, mechanical response and fatigue durability of the final formed composite component meet the requirements of the use of high-precision fields.

[0036] The present invention also provides a computer device for implementing the above detection method, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method when executing the computer program. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of the real-time detection method of the surface parameters of a braided preform based on deep learning of the present invention;

[0038] Figure 2 This is a schematic diagram of the edge detection result of a braided preform using the U-Net model in the present invention;

[0039] Figure 3 This is a schematic diagram of the result of image morphological processing in the present invention;

[0040] Figure 4 It is a schematic diagram of yarn unit recognition and parameter calculation results of the preform surface image in the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention will be further described below in conjunction with the accompanying drawings.

[0042] Aiming at the problems of low efficiency and high cost caused by the heavy reliance on manual labor in the detection of surface parameters of traditional composite material braided preforms, the present invention provides a method for detecting surface parameters of braided preforms based on deep learning, such as Figure 1 , specifically including the following steps:

[0043] (1) Images of composite woven preforms with various surface parameters are collected by industrial cameras. The image sets are formed by data augmentation methods. The edges of the yarns in the image sets are marked and the image sets are divided into training set, validation set and test set.

[0044] The original image of the composite material woven preform contains a variety of surface parameters, among which the weaving angle range is 25°-75°, the pitch range is 4-8cm, the carbon fiber used for weaving is one or more of 3k, 6k, 12k and 24k, and the woven preform structure contained in the image set is one or more of two-dimensional two-axis, two-dimensional three-axis, three-dimensional four-way and three-dimensional five-way. After obtaining the original image of the composite material woven preform, the data is expanded by data augmentation methods such as rotation, cropping, splicing and graying, and finally 1120 image data are obtained to form a composite material woven preform image set, and the data set is divided into a training set, a validation set and a test set according to a ratio of 3:1:1.

[0045] (2) The manually labeled training set is input into the U-Net model for training until the loss function converges. The U-Net model is verified using the test set. The output yarn edge of the preform surface is compared with the manual label. When the MIoU is higher than 85%, the training of the U-Net model is completed, otherwise repeat step (2).

[0046] The U-Net model training process in the present invention is specifically as follows: an image in the training set is input into the U-Net model, a feature map is obtained through the convolution layer and pooling layer of the model, different scale features of the image are captured, and then the feature map is used to restore the spatial resolution through upsampling and jump connection, and a segmentation probability map consistent with the size of the input image is generated. The probability map is used to represent the probability that each pixel belongs to the edge of the yarn or the background. By setting a threshold δ, the pixel points with a segmentation probability greater than δ are judged as the edge of the yarn, and the remaining pixel points are judged as the image background, and their category labels are saved and the loss function value is calculated. In order to improve the training efficiency, the batch size and the number of iterations are set, the remaining images in the training set are input in turn, and the above process is repeated. Through iterative optimization, the loss function is gradually converged.

[0047] The loss function in the present invention is composed of the Dice coefficient loss function and the Focal loss function. The Dice coefficient loss function can be used to measure the similarity of two samples and is suitable for evaluating the performance of image segmentation tasks. The Focal loss function is a loss function used to solve the problem of class imbalance. It modifies the standard cross entropy loss function, reduces the weight of easy-to-classify samples, and increases the weight of difficult-to-classify samples. The loss function L involved in the present invention is:

[0048]

[0049] Among them, TP is the number of pixels that are predicted and actually positive, FP is the number of pixels that are predicted to be positive but actually negative, FN is the number of pixels that are predicted to be negative but actually positive, and p t is the predicted probability of the model for each category. s is a constant used to avoid the situation where the denominator is 0; β is a hyperparameter used to adjust the weight between TP and FN; γ is a focusing parameter for adjusting the weight of easy-to-classify samples; α is a weight factor for balancing positive and negative samples, which is used to deal with the problem of class imbalance.

[0050] In the present invention, MIoU is used to evaluate the training effect of the U-Net model. MIoU is an important indicator for measuring the performance of semantic segmentation algorithms. The model performance is measured by calculating the ratio between the intersection and union of the predicted results and the true labels. The MIoU involved in the present invention is:

[0051]

[0052] (3) The surface image of the woven preform captured by the industrial camera in real time is input into the trained U-Net model to output the yarn edge recognition result. The recognition result is as follows: Figure 2 As shown;

[0053] (4) removing noise from the yarn edge recognition result obtained in step (3) by image morphological processing and connecting edge breakpoints;

[0054] The image morphological processing in the present invention is specifically as follows: first, the yarn edge recognition result output in step (2) is subjected to an erosion algorithm to eliminate noise. The erosion algorithm is an operation that reduces or erodes the yarn edge area of ​​an image. It reduces pixels on the yarn edge through a structural element, causing the edge to shrink inward, thereby eliminating small isolated noise. For image A and structural element B, erosion can be expressed as:

[0055]

[0056] Among them, B z represents the translation of the structural element B at position z.

[0057] After eliminating the noise, the dilation algorithm is used to find the yarn edge breakpoints. The dilation algorithm is an operation to expand or dilate the yarn edge area of ​​the image. It adds pixels to the yarn edge through the structural element so that the edge expands outward. The background pixel coordinates that are repeatedly filled by different structural elements after dilation are recorded as the yarn edge breakpoints. For image A and structural element B, the dilation can be expressed as:

[0058]

[0059] After identifying the yarn edge breakpoint Q1, its connection with the edge point Q2 can be completed by two-dimensional linear interpolation. The expression of two-dimensional linear interpolation is:

[0060] Q=tQ1+(1-t)Q2,0≤t≤1

[0061] The result of image morphological processing is as follows Figure 3 shown.

[0062] (5) Extract the yarn edge contour from the image obtained in step (4) and fit the parallelogram yarn unit through the adaptive DP algorithm to obtain its four-side vectors.

[0063] The present invention uses the adaptive DP algorithm to fit the parallelogram yarn unit specifically as follows: calculate the arc length of the yarn edge contour output in step (4), determine the initial DP algorithm fitting accuracy ε according to the arc length, perform quadrilateral fitting on the contour in multiple iterations, and if the fitting result meets the characteristics of the parallelogram, output the vertex coordinates of the quadrilateral; if unsuccessful, adjust the fitting accuracy ε and refit until the fitting is successful or exceeds the iteration number limit N. The fitting accuracy ε calculation formula of the adaptive DP algorithm involved in the present invention is:

[0064] ε=μ n-1 kC,n≤N

[0065] Among them, μ is the iteration coefficient, n is the number of iterations, k is the approximate accuracy scaling factor, and C is the contour circumference.

[0066] After obtaining the parallelogram yarn unit, the preform braiding angle θ is obtained through geometric calculation t , pitch p t and yarn width w t , the calculation formula is:

[0067]

[0068]

[0069]

[0070] Among them, (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the vertex coordinates of the parallelogram approximation unit, arranged in clockwise order from top to bottom, P1 is the short side vector of the parallelogram approximation unit, and P2 is the long side vector of the parallelogram approximation unit.

[0071] Figure 4 The figure is an example of yarn unit recognition and parameter calculation results of a partial preform surface image obtained by the method of the present invention.

[0072] The braiding angle θ is obtained by measuring the preform samples P1, P2 and P3 using the method of the present invention. t , pitch p t and yarn width w t , and the manually measured braiding angle θ m , pitch p m and yarn width w m The measurement error is obtained by comparison, and the measurement error is shown in Table 1. The measurement results show that the braiding angle recognition error e obtained by this method is θ ≤1.2%, pitch recognition error e p ≤2.0%, yarn width recognition error e w ≤2.4%, and the response time of a single image is calculated as t≤0.035s.

[0073] Table 1 Measurement results and measurement errors

[0074]

[0075] In summary, the present invention realizes high-efficiency and high-accuracy automated measurement of preform surface parameters, reduces the influence of subjective factors, can detect the preform weaving production process in real time, improves the level of intelligent measurement in the preform production process, reduces the workload of quality inspectors, realizes high-precision detection and real-time feedback, and can effectively reduce the problem of surface parameter instability during the preform weaving process, ensuring that the mechanical properties of the final composite material components meet the application requirements of high-performance fields.

[0076] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A method for detecting surface parameters of a braided preform based on deep learning, characterized in that: The following steps are involved: (1) collecting a number of woven preform images to form an image set, marking the edges of the yarn parts in the woven preform images, and dividing the image set into a training set, a verification set, and a test set; (2) Input the training set into the U-Net model for training until the loss function converges, use the test set to verify the U-Net model, compare the output preform surface yarn edge with the manual mark, and complete the training of the U-Net model when the MIoU is higher than the preset value, otherwise repeat step (2); (3) Inputting the surface image of the woven preform to be identified into the trained U-Net model to output the yarn edge recognition result; (4) removing noise from the yarn edge recognition result obtained in step (3) by image morphological processing and connecting edge breakpoints in the yarn edge; (5) The image obtained in step (4) is contour extracted and the parallelogram yarn unit is fitted by an adaptive DP algorithm to calculate the preform braiding angle, pitch and yarn width.

2. According to the method for detecting surface parameters of a braided preform based on deep learning in claim 1, it is characterized in that: The braiding angle range of the braided preform image described in step (1) is 25°-75°, the pitch range is 4-8cm, the carbon fiber used for weaving is one or more of 3k, 6k, 12k and 24k, and the braided preform structure is one or more of two-dimensional biaxial, two-dimensional triaxial, three-dimensional four-directional or three-dimensional five-directional.

3. According to the method for detecting surface parameters of a braided preform based on deep learning in claim 1, it is characterized in that: The data augmentation method in step (1) includes one or more of rotation, cropping, splicing and grayscale, and the training set, validation set and test set are divided into 3:1:

1.

4. The method for detecting surface parameters of a braided preform based on deep learning according to claim 1, characterized in that: The training process in step (2) is as follows: an image in the training set is input into the U-Net model, a feature map is obtained through the convolution layer and pooling layer of the model, and features of different scales of the image are captured. Then, the feature map is used to restore the spatial resolution through upsampling and jump connection, and a segmentation probability map consistent with the size of the input image is generated. The probability map is used to represent the probability that each pixel belongs to the yarn edge or the background. By setting a threshold δ, pixels with a segmentation probability greater than δ are judged as yarn edges, and the remaining pixels are judged as image backgrounds. The category label of each pixel after judgment is saved and the loss function value is calculated; the batch size and the number of iterations are set, and the remaining images in the training set are input in turn. The above process is repeated, and the loss function is gradually converged through iterative optimization.

5. The method for detecting surface parameters of a braided preform based on deep learning according to claim 4, characterized in that: The loss function L is: Among them, TP is the number of pixels that are both predicted and actually yarn edges, FP is the number of pixels that are predicted to be yarn edges but are actually image backgrounds, FN is the number of pixels that are predicted to be image backgrounds but are actually yarn edges, and p t is the predicted probability of the model for each category; s is a constant used to avoid the situation where the denominator is 0; β is a hyperparameter used to adjust the weight between TP and FN; γ is a focusing parameter for adjusting the weight of easy-to-classify samples; α is a weight factor for balancing positive and negative samples, which is used to deal with the problem of category imbalance.

6. The method for detecting surface parameters of a braided preform based on deep learning according to claim 5, characterized in that: The MIoU is:

7. The method for detecting surface parameters of a braided preform based on deep learning according to claim 1, characterized in that: The image morphological processing in step (4) is specifically as follows: first, the yarn edge recognition result output in step (2) is subjected to a corrosion algorithm to eliminate noise; for image A and structural element B, the corrosion algorithm is expressed as: Among them, B z represents the translation of the structural element B at position z; After eliminating the noise, the expansion algorithm is used to find the yarn edge breakpoints; for image A and structural element B, the expansion algorithm is expressed as: After the yarn edge breakpoint Q1 is identified, its connection with the edge point Q2 is completed by two-dimensional linear interpolation; the expression of the two-dimensional linear interpolation is: Q=tQ1+(1-t)Q2,0≤t≤1.

8. The method for detecting surface parameters of a braided preform based on deep learning according to claim 1, characterized in that: The step (5) of fitting the parallelogram yarn unit by the adaptive DP algorithm is specifically as follows: calculating the arc length of the yarn edge contour output in step (4), determining the initial DP algorithm fitting accuracy ε according to the arc length, performing quadrilateral fitting on the contour for multiple iterations, and if the fitting result satisfies the characteristics of the parallelogram, outputting the vertex coordinates of the quadrilateral; if unsuccessful, adjusting the fitting accuracy ε and refitting until the fitting is successful or the iteration number limit N is exceeded; the fitting accuracy ε of the adaptive DP algorithm is calculated as follows: e=m n-1 kC,n≤N Among them, μ is the iteration coefficient, n is the number of iterations, k is the approximate accuracy scaling factor, C is the contour circumference, and N is the iteration limit.

9. The method for detecting surface parameters of a braided preform based on deep learning according to claim 1, characterized in that: The preform braiding angle θ is obtained by geometric calculation in step (5): t , pitch p t and yarn width w t The specific calculation formula is: Among them, (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the vertex coordinates of the parallelogram approximation unit, arranged in clockwise order from top to bottom, P1 is the short side vector of the parallelogram approximation unit, and P2 is the long side vector of the parallelogram approximation unit.

10. A computer device for implementing the detection method according to any one of claims 1 to 9, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the steps of the method are implemented when the processor executes the computer program.

Images