A woven preform surface parameter detection method based on deep learning

Abstract

The application discloses a kind of based on deep learning's woven preform surface parameter detection method, comprising: collection and construction woven preform image set, train U-Net model until loss function converges, use test set to verify U-Net model, when MIoU is higher than preset value, the yarn edge of output is compared with artificial mark, complete U-Net model training.Real-time acquisition of woven preform surface image is input into trained U-Net model and the yarn edge recognition result of output is identified, utilize image morphology processing to remove the noise of recognition result and connect edge breakpoint, utilize contour extraction and adaptive DP algorithm to fit out parallelogram yarn unit, and obtain preform weaving angle, pitch and yarn width by geometric operation.The woven preform surface parameter detection method based on deep learning of the application has the characteristics of intelligence, high efficiency and accuracy, and can greatly improve the efficiency and reliability of composite material weaving process.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of composite material braided preform parameter detection, in particular to a braided preform surface parameter detection method based on deep learning. BACKGROUND

[0002] Braided composites have great application potential in aerospace, automobile manufacturing, national defense and military industry, etc. due to their excellent specific strength and specific modulus, outstanding fatigue resistance, excellent impact resistance and high damage tolerance. Braiding angle, pitch and yarn width are important surface parameters of braided preforms, which are directly related to the fiber volume fraction, mechanical response and fatigue durability of composites and other key performance indicators.

[0003] Currently, the measurement of surface parameters such as braiding angle, pitch and yarn width mainly relies on manual operation, which is time-consuming and has limited measurement accuracy. Some studies have attempted to use machine vision and image processing technology to realize the automatic measurement of braided preform surface parameters, but the algorithm efficiency is low and cannot meet the real-time detection requirements of surface parameters in the production process. Therefore, it is urgent to develop a high-precision and high-efficiency real-time detection method to monitor the surface parameters of preforms in the production process and ensure that the forming quality of braided preforms meets the performance requirements. SUMMARY

[0004] The purpose of the present application is to provide a braided preform surface parameter detection method based on deep learning, which solves the problem of how to automatically detect the surface parameters of preforms.

[0005] To achieve the above technical purpose, the present application adopts the following technical solutions:

[0006] A braided preform surface parameter detection method based on deep learning, comprising the following steps:

[0007] (1) Collect a number of braided preform images to form an image set, mark the edges of the yarn part in the braided preform image, and divide the image set into a training set, a validation set and a test set;

[0008] (2) input the training set into the U-Net model for training until the loss function converges, use the test set to verify the U-Net model, compare the output preform surface yarn edge with the manual marking, when the MIoU is higher than the preset value, the training of the U-Net model is completed, otherwise repeat step (2);

[0009] (3) input the braided preform surface image to be identified into the trained U-Net model to output the yarn edge recognition result;

[0010] (4) The yarn edge recognition result obtained in step (3) is processed by image morphology to remove noise of the recognition result and connect edge breakpoints in the yarn edge;

[0011] (5) The image obtained in step (4) is subjected to contour extraction and a parallelogram yarn unit is fitted by an adaptive DP algorithm, so that the braiding angle, pitch and yarn width of the preform are calculated.

[0012] Further, the braiding angle of the braided preform image in step (1) ranges from 25° to 75°, the pitch ranges from 4 to 8 cm, the carbon fiber used for braiding is one or more of 3k, 6k, 12k and 24k, and the braided preform structure is one or more of two-dimensional two-axis, two-dimensional three-axis, three-dimensional four-direction or three-dimensional five-direction.

[0013] Further, the data augmentation method in step (1) includes one or more of rotation, cropping, splicing and grayscale, and the training set, validation set and test set are divided according to 3:1:1.

[0014] Further, the training process in step (2) is as follows: one image in the training set is input into the U-Net model, the feature map is obtained through the convolution layer and the pooling layer of the model, and the different scale features of the image are captured, then the spatial resolution is restored by upsampling and jump connection using the feature map, and a segmentation probability map consistent with the size of the input image is generated, which is used to represent the probability of each pixel belonging to the yarn edge or the background, by setting a threshold δ, the pixel points with a segmentation probability greater than δ are judged as yarn edges, and the remaining pixel points are judged as image backgrounds, the class labels of each judged pixel point are saved and the loss function value is calculated; the batch size and the number of iterations are set, the remaining images in the training set are input in turn, and the above process is repeatedly executed, and the loss function is gradually converged through iterative optimization.

[0015] Further, the loss function L is:

[0016]

[0017] Where TP is the number of pixels that are both predicted and actual yarn edges, FP is the number of pixels that are predicted to be yarn edges but are actual image backgrounds, FN is the number of pixels that are predicted to be image backgrounds but are actual yarn edges, p t is the prediction probability of the model for each class. s is a constant to avoid the case where the denominator is 0; β is a hyperparameter to adjust the weight between TP and FN; γ is a focusing parameter to adjust the weight of easy-to-classify samples; α is a weight factor to balance positive and negative samples, used to deal with the class imbalance problem.

[0018] Further, the MIoU is:

[0019]

[0020] Further, the image morphological processing in step (4) is specifically: firstly, the yarn edge recognition result output in step (2) is eliminated of noise through an erosion algorithm; for image A and structural element B, the erosion algorithm is expressed as:

[0021]

[0022] Wherein, B z represents the translation of structural element B at position z.

[0023] After eliminating noise, an expansion algorithm is used to find the yarn edge breakpoint; for image A and structural element B, the expansion algorithm is expressed as:

[0024]

[0025] After recognizing the yarn edge breakpoint Q1, the connection of Q1 and edge point Q2 is completed through two-dimensional linear interpolation; the expression of the two-dimensional linear interpolation is:

[0026] Q = tQ1 + (1-t)Q2, 0≤t≤1.

[0027] Further, the fitting of parallelogram yarn unit through the adaptive DP algorithm in step (5) is specifically: the arc length of the yarn edge contour output in step (4) is calculated, the initial DP algorithm fitting precision ε is determined according to the arc length, the contour is fitted through quadrilateral multiple times, if the fitting result meets the characteristics of the parallelogram, the coordinates of the vertex of the parallelogram are output; if it is not successful, the fitting precision ε is adjusted for re-fitting until the fitting is successful or the iteration number limit N is exceeded; the fitting precision ε calculation formula of the adaptive DP algorithm is:

[0028] ε = μ n-1 kC,n≤N

[0029] Wherein, μ is the iteration coefficient, n is the iteration number, k is the approximate precision proportion factor, C is the contour perimeter, and N is the iteration number limit.

[0030] Further, the specific calculation formula of the preform braiding angle θ t , the pitch p t and the yarn width w t obtained through geometric operation in step (5) is:

[0031]

[0032]

[0033]

[0034] Wherein, (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the vertex coordinates of the parallelogram approximation unit, arranged in order from top to bottom clockwise, P1 is the short edge vector of the parallelogram approximation unit, and P2 is the long edge vector of the parallelogram approximation unit.

[0035] Compared with the prior art, the present application has the following beneficial effects: the weaving preform surface parameter detection method based on deep learning of the present application realizes the weaving preform yarn edge detection through the U-Net model, removes the recognition noise through morphological processing, makes up the edge breakpoint, and realizes the high-efficiency and high-accuracy preform surface parameter automatic measurement by using the contour extraction and adaptive DP algorithm to fit the parallelogram yarn unit and calculate the weaving angle, pitch and yarn width, which can meet the detection requirements and can be applied to the real-time detection of the weaving preform yarn edge in the production process. Compared with the traditional manual measurement method, the weaving preform surface parameter detection method based on deep learning of the present application reduces the interference of human subjectivity, improves the intelligent measurement level in the preform production process, reduces the work intensity of the quality inspection personnel, realizes the high-precision detection and real-time feedback, can effectively reduce the surface parameter unevenness or mutation problem in the weaving preform production process, and ensures that the fiber volume fraction, mechanical response and fatigue durability and other performances of the finally formed composite component meet the use requirements of high-precision and high-tech fields.

[0036] The present application also provides a computer device for realizing the above detection method, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, and the processor realizes the steps of the method when executing the computer program. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The flowchart of the weaving preform surface parameter real-time detection method based on deep learning of the present application;

[0038] Figure 2 The weaving preform edge detection result schematic diagram adopted in the present application;

[0039] Figure 3 The result schematic diagram of image morphological processing adopted in the present application;

[0040] Figure 4 The yarn unit recognition and parameter calculation result schematic diagram of the preform surface image in the present application. DETAILED DESCRIPTION

[0041] The technical solutions of the present application will be further described below in combination with the drawings.

[0042] In view of the problems of low efficiency and high cost caused by the fact that the detection of the surface parameters of the traditional composite braided preform largely relies on manual work, the application provides a braided preform surface parameter detection method based on deep learning, which specifically comprises the following steps: Figure 1

[0043] (1) Collect the images of the composite braided preform containing various surface parameters through an industrial camera, form an image set through a data augmentation method, mark the edges of the yarn in the image set, and divide the image set into a training set, a validation set and a test set.

[0044] The original image of the composite braided preform contains various surface parameters, wherein the braiding angle ranges from 25° to 75°, the pitch ranges from 4 cm to 8 cm, the carbon fiber used for braiding is one or more of 3k, 6k, 12k and 24k, and the structure of the braided preform contained in the image set is one or several of two-dimensional two-axis, two-dimensional three-axis, three-dimensional four-direction and three-dimensional five-direction. After obtaining the original image of the composite braided preform, the data is expanded through rotation, cropping, splicing and grayscale, etc. Data augmentation method, finally get 1120 image data, form a composite braided preform image set, and divide the data set into a training set, a validation set and a test set according to 3:1:1.

[0045] (2) Input the training set marked by artificial into the U-Net model for training until the loss function converges, use the test set to verify the U-Net model, compare the output preform surface yarn edge with the artificial marking, when the MIoU is higher than 85%, the training of the U-Net model is completed, otherwise repeat step (2).

[0046] The U-Net model training process in the application is as follows: input one image in the training set into the U-Net model, obtain feature maps through the convolution layer and the pooling layer of the model, capture different scale features of the image, then restore the spatial resolution of the feature maps through upsampling and skip connection, generate a segmentation probability map consistent with the size of the input image, the probability map is used to represent the probability of each pixel belonging to the yarn edge or the background, set a threshold value δ, judge the pixel points with a segmentation probability greater than δ as the yarn edge, and the remaining pixel points are judged as the image background, save the class labels and calculate the loss function value. In order to improve the training efficiency, set the batch size and the iteration number, input the remaining images in the training set in turn, repeat the above process, and gradually converge the loss function through iteration optimization.

[0047] ​The loss function in the application is composed of a Dice coefficient loss function and a Focal loss function. The Dice coefficient loss function can be used to measure the similarity of two samples and is suitable for evaluating the performance of image segmentation tasks. The Focal loss function is a loss function used to solve the class imbalance problem. It modifies the standard cross-entropy loss function, reduces the weight of easy-to-classify samples, and increases the weight of difficult-to-classify samples. The loss function L involved in the application is:

[0048]

[0049] where TP is the number of pixels that are both predicted and actual positive, FP is the number of pixels that are predicted positive and actual negative, FN is the number of pixels that are predicted negative and actual positive, p t is the predicted probability of the model for each class. s is a constant to avoid the case where the denominator is 0; β is a hyperparameter to adjust the weight between TP and FN; γ is a focusing parameter to adjust the weight of easy-to-classify samples; α is a weight factor to balance positive and negative samples, used to handle the class imbalance problem.

[0050] In the application, MIoU is used to evaluate the training effect of the U-Net model. MIoU is an important indicator to measure the performance of semantic segmentation algorithms. It measures the performance of the model by calculating the ratio of the intersection and union between the predicted results and the true labels. The MIoU involved in the application is:

[0051]

[0052] (3) Input the woven preform surface image collected by the industrial camera in real time into the trained U-Net model to output the yarn edge recognition result. The recognition result is as shown in Figure 2 ;

[0053] (4) Remove noise and connect edge breakpoints of the yarn edge recognition result obtained in step (3) through image morphological processing;

[0054] In the application, the image morphological processing is as follows: first, the yarn edge recognition result output in step (2) is processed through an erosion algorithm to eliminate noise. The erosion algorithm is an operation that shrinks or erodes the yarn edge region of an image. It reduces the pixels on the yarn edge by a structure element, making the edge shrink inward, so it can eliminate small isolated noise. For image A and structure element B, erosion can be represented as:

[0055]

[0056] where B z represents the translation of structure element B at position z.

[0057] After the noise is eliminated, the yarn edge breakpoint is searched by using the expansion algorithm, the expansion algorithm is an operation of expanding or dilating the yarn edge area of the image, pixels on the yarn edge are increased by a structure element, so that the edge is expanded outward, and the background pixel coordinates repeatedly filled by different structure elements after the expansion are recorded, that is, the yarn edge breakpoint.

[0058]

[0059] After the yarn edge breakpoint Q1 is identified, the connection between Q1 and the edge point Q2 can be completed by two-dimensional linear interpolation, and the expression of two-dimensional linear interpolation is:

[0060] Q=tQ1+(1-t)Q2,0≤t≤1

[0061] The result of the image morphological processing is shown in Figure 3 .

[0062] (5) the yarn edge contour of the image obtained in step (4) is extracted, and a parallelogram yarn unit is fitted by using the adaptive DP algorithm, so that the four edge vectors are obtained.

[0063] In the present application, the parallelogram yarn unit is fitted by using the adaptive DP algorithm, specifically: the arc length of the yarn edge contour output in step (4) is calculated, the initial DP algorithm fitting precision ε is determined according to the arc length, the contour is fitted by quadrilaterals for multiple iterations, if the fitting result meets the characteristics of the parallelogram, the coordinates of the vertices of the parallelogram are output; if it is not successful, the fitting precision ε is adjusted for re-fitting, until the fitting is successful or the iteration number limit N is exceeded. The fitting precision ε calculation formula of the adaptive DP algorithm involved in the present application is:

[0064] ε=μ n-1 kC,n≤N

[0065] Wherein, μ is the iteration coefficient, n is the iteration number, k is the approximate precision proportion factor, and C is the contour perimeter.

[0066] After the parallelogram yarn unit is obtained, the preform braiding angle θ t , the pitch p t and the yarn width w t are obtained by geometric operation, and the calculation formula is:

[0067]

[0068]

[0069]

[0070] Wherein, (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the vertex coordinates of the parallelogram approximation unit, sequentially arranged in a clockwise direction from top to bottom, P1 is the short edge vector of the parallelogram approximation unit, and P2 is the long edge vector of the parallelogram approximation unit.

[0071] Figure 4 The yarn unit identification and parameter calculation results of the surface image of the partial preform obtained by the method are shown in the following table.

[0072] The preform samples P1, P2 and P3 are measured by the method, and the weaving angles θ t , pitches p t and yarn widths w t obtained are compared with the manually measured weaving angles θ m , pitches p m and yarn widths w m , and the measurement errors are shown in Table 1. The measurement results show that the identification error e θ of the weaving angle is ≤1.2%, the identification error e p of the pitch is ≤2.0%, the identification error e w of the yarn width is ≤2.4%, and the response time t of calculating a single picture is ≤0.035s.

[0073] Table 1 Measurement results and measurement errors

[0074]

[0075] In summary, the present application realizes efficient and accurate automatic measurement of preform surface parameters, reduces the influence of subjective factors, can detect the preform weaving production process in real time, improves the intelligent measurement level in the preform production process, reduces the work intensity of quality inspection personnel, realizes high-precision detection and real-time feedback, can effectively reduce the surface parameter instability problem in the preform weaving process, and ensures that the mechanical properties of the final composite material component meet the application requirements of high-performance fields.

[0076] The above is only a preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solution falling within the scope of the present application shall be considered as falling within the protection scope of the present application. It should be noted that for ordinary skilled persons in the art, some improvements and refinements without departing from the principles of the present application shall be considered as falling within the protection scope of the present application.

Claims

1. A method for detecting surface parameters of woven prefabricated structures based on deep learning, characterized in that, Includes the following steps: (1) Collect several images of woven prefabricated bodies to form an image set, mark the edges of the yarn parts in the images of the woven prefabricated bodies, and divide the image set into a training set, a validation set and a test set; the weaving angle of the images of the woven prefabricated bodies is 25°-75°, the pitch is 4-8cm, the carbon fiber used for weaving is one or more of 3k, 6k, 12k and 24k, and the structure of the woven prefabricated body is one or more of two-dimensional biaxial, two-dimensional triaxial, three-dimensional four-dimensional or three-dimensional five-dimensional; (2) Input the training set into the U-Net model for training until the loss function converges. Use the test set to verify the U-Net model. Compare the output of the yarn edge on the surface of the preform with the manual label. When the MIoU is higher than the preset value, the training of the U-Net model is completed; otherwise, repeat step (2). The loss function is: ； Where L is the loss function, TP is the number of pixels that are both predicted and actually represent yarn edges, FP is the number of pixels that are predicted as yarn edges but actually represent the image background, FN is the number of pixels that are predicted as image background but actually represent yarn edges, and p t y is the predicted probability of the model for each class; s is a constant used to avoid the case where the denominator is 0; β is a hyperparameter used to adjust the weights between TP and FN; γ is a focusing parameter that adjusts the weights of easily classified samples; α is a weighting factor that balances positive and negative samples and is used to deal with class imbalance problems. (3) Input the surface image of the woven prefabricated body to be identified into the trained U-Net model and output the yarn edge recognition result; (4) Remove noise from the yarn edge recognition results obtained in step (3) by image morphology processing and connect the edge breakpoints in the yarn edge. The image morphological processing specifically involves: firstly, eliminating noise from the yarn edge recognition results output in step (2) using an erosion algorithm; for image A and structuring element B, the erosion algorithm is expressed as: ； Among them, B z This indicates the translation of structuring element B at position z; After noise removal, a dilation algorithm is used to find the yarn edge breakpoints; for image A and structuring element B, the dilation algorithm is expressed as: ； After identifying the yarn edge breakpoint Q1, its connection to edge point Q2 is completed using two-dimensional linear interpolation; the expression for the two-dimensional linear interpolation is: ； (5) Extract the contour of the image obtained in step (4) and fit the parallelogram yarn unit using the adaptive DP algorithm to calculate the prefab weaving angle, pitch and yarn width; calculate the prefab weaving angle θ. t Pitch p t and yarn width w t The specific calculation formula is as follows: ； ； ； Where (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the vertex coordinates of the parallelogram approximation element, arranged in clockwise order from top to bottom, P1 is the short side vector of the parallelogram approximation element, and P2 is the long side vector of the parallelogram approximation element.

2. The method for detecting surface parameters of woven preforms based on deep learning according to claim 1, characterized in that, Step (1) also includes data augmentation, wherein the data augmentation method includes one or more of rotation, cropping, splicing and grayscale conversion, and the training set, validation set and test set are divided in a 3:1:1 ratio.

3. The method for detecting surface parameters of woven preforms based on deep learning according to claim 1, characterized in that, The training process in step (2) is as follows: an image from the training set is input into the U-Net model. The model obtains feature maps through convolutional and pooling layers to capture features at different scales of the image. Then, the spatial resolution is restored using the feature maps through upsampling and skip connections to generate a segmentation probability map with the same size as the input image. This segmentation probability map is used to represent the probability that each pixel belongs to the yarn edge or the background. By setting a threshold δ, pixels with a segmentation probability greater than δ are judged as yarn edges, and the remaining pixels are judged as the image background. The category label of each judged pixel is saved and the loss function value is calculated. The batch size and number of iterations are set, and the remaining images in the training set are input in sequence. The above process is repeated. Through iterative optimization, the loss function gradually converges.

4. The method for detecting surface parameters of woven preforms based on deep learning according to claim 1, characterized in that, The MIoU is: 。 5. The method for detecting surface parameters of woven preforms based on deep learning according to claim 1, characterized in that, The specific steps of fitting the parallelogram yarn unit using the adaptive DP algorithm in step (5) are as follows: Calculate the arc length of the yarn edge contour output in step (4), determine the initial DP algorithm fitting accuracy ε based on the arc length, iterate the contour multiple times to fit the quadrilateral, and if the fitting result satisfies the characteristics of a parallelogram, output the vertex coordinates of the quadrilateral; if unsuccessful, adjust the fitting accuracy ε and refit until the fitting is successful or the iteration limit N is exceeded; the formula for calculating the fitting accuracy ε of the adaptive DP algorithm is: ； Where μ is the iteration coefficient, n is the number of iterations, k is the approximation accuracy scaling factor, C is the contour perimeter, and N is the iteration limit.

6. A computer device for implementing the detection method according to any one of claims 1 to 5, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method.

Images